2024-09-21T01:59:08.586 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T01:59:08.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T01:59:08.687 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913409' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJlILFpLgG9S3GWq0Ng/yZ+JmIXVebw+08zEikgf3Xc4S9cqFlQSoiNsKIDweVwBpI4G/bx/2SdHBuBYzGguHsY= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaV1+dBiNRWxho5lhJmqrQEXx5I7IHF9ub/o3sl8ooRLE5iMay/PfRAYAG7VwqyHQTJ/UVLQHgRmvkXAL7Jd3o= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T01:59:08.687 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T01:59:08.688 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T01:59:08.688 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T01:59:08.690 INFO:teuthology.task.internal:Checking packages... 2024-09-21T01:59:08.710 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T01:59:08.710 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T01:59:08.710 INFO:teuthology.packaging:ref: None 2024-09-21T01:59:08.710 INFO:teuthology.packaging:tag: None 2024-09-21T01:59:08.710 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:59:08.710 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:59:08.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:59:08.854 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120-g18432218-1jammy 2024-09-21T01:59:08.855 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T01:59:08.868 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T01:59:08.868 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T01:59:08.903 INFO:teuthology.task.internal:Saving configuration 2024-09-21T01:59:08.920 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T01:59:08.926 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T01:59:08.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:53:35.350453', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGFedzWYSBBMJvmxtqfTKE7/So5N0NyDtpjAG3DCdjqi'} 2024-09-21T01:59:08.969 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 01:53:35.352414', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJALC4BwxG0DryO4/UfysGZFxh7LMT1lBdZymgUXlk1L'} 2024-09-21T01:59:08.969 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T01:59:08.976 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T01:59:08.976 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T01:59:08.976 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T01:59:09.031 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1022530160>, signals=[15]) 2024-09-21T01:59:09.032 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T01:59:09.039 INFO:teuthology.task.internal:Opening connections... 2024-09-21T01:59:09.039 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T01:59:09.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:59:09.121 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T01:59:09.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:59:09.201 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T01:59:09.207 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-09-21T01:59:09.212 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-09-21T01:59:09.212 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-09-21T01:59:09.258 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="22.04" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=jammy 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T01:59:09.259 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=jammy 2024-09-21T01:59:09.260 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-09-21T01:59:09.277 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-09-21T01:59:09.282 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-09-21T01:59:09.282 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Ubuntu" 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="22.04" 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_CODENAME=jammy 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:ID=ubuntu 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE=debian 2024-09-21T01:59:09.330 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T01:59:09.331 INFO:teuthology.orchestra.run.smithi191.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T01:59:09.331 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T01:59:09.331 INFO:teuthology.orchestra.run.smithi191.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T01:59:09.331 INFO:teuthology.orchestra.run.smithi191.stdout:UBUNTU_CODENAME=jammy 2024-09-21T01:59:09.331 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-09-21T01:59:09.352 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T01:59:09.361 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T01:59:09.366 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T01:59:09.367 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:59:09.369 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-09-21T01:59:09.377 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T01:59:09.383 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T01:59:09.384 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:59:09.417 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-09-21T01:59:09.433 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T01:59:09.486 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T01:59:09.500 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T01:59:09.500 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T01:59:09.500 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:59:09.501 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:59:09.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:59:09.505 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-09-21T01:59:09.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:59:09.506 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-21T01:59:09.550 INFO:teuthology.orchestra.run.smithi191.stdout:5.15.0-56-generic 2024-09-21T01:59:09.550 INFO:teuthology.task.kernel:Running kernel on smithi191: 5.15.0-56-generic 2024-09-21T01:59:09.550 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get clean 2024-09-21T01:59:09.555 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-56-generic 2024-09-21T01:59:09.555 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.15.0-56-generic 2024-09-21T01:59:09.556 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-09-21T01:59:09.669 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2024-09-21T01:59:09.698 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T01:59:09.829 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T01:59:09.829 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T01:59:09.829 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T01:59:09.845 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T01:59:09.854 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T01:59:09.897 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T01:59:09.922 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T01:59:09.957 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T01:59:10.075 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T01:59:10.080 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T01:59:10.146 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T01:59:10.161 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T01:59:10.184 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T01:59:10.189 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T01:59:10.191 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T01:59:10.191 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T01:59:10.192 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T01:59:10.200 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T01:59:10.204 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T01:59:10.204 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T01:59:10.206 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T01:59:10.216 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T01:59:10.220 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T01:59:10.220 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T01:59:10.224 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T01:59:10.231 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T01:59:10.232 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T01:59:10.233 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T01:59:10.234 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T01:59:10.239 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T01:59:10.239 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T01:59:10.245 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T01:59:10.256 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T01:59:10.256 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T01:59:10.256 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T01:59:10.257 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T01:59:10.257 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T01:59:10.260 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T01:59:10.260 INFO:teuthology.orchestra.run.smithi191.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T01:59:10.261 INFO:teuthology.orchestra.run.smithi191.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T01:59:10.286 INFO:teuthology.orchestra.run.smithi191.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T01:59:10.289 INFO:teuthology.orchestra.run.smithi191.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T01:59:10.289 INFO:teuthology.orchestra.run.smithi191.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T01:59:10.294 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T01:59:10.296 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T01:59:10.296 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T01:59:10.297 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T01:59:10.297 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T01:59:10.298 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T01:59:10.298 INFO:teuthology.orchestra.run.smithi191.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T01:59:10.298 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T01:59:10.299 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T01:59:10.299 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T01:59:10.303 INFO:teuthology.orchestra.run.smithi191.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T01:59:10.305 INFO:teuthology.orchestra.run.smithi191.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T01:59:10.305 INFO:teuthology.orchestra.run.smithi191.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T01:59:10.305 INFO:teuthology.orchestra.run.smithi191.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T01:59:10.312 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T01:59:10.324 INFO:teuthology.orchestra.run.smithi191.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T01:59:10.324 INFO:teuthology.orchestra.run.smithi191.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T01:59:10.324 INFO:teuthology.orchestra.run.smithi191.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T01:59:10.324 INFO:teuthology.orchestra.run.smithi191.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T01:59:10.326 INFO:teuthology.orchestra.run.smithi191.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T01:59:10.327 INFO:teuthology.orchestra.run.smithi191.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T01:59:10.327 INFO:teuthology.orchestra.run.smithi191.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T01:59:10.327 INFO:teuthology.orchestra.run.smithi191.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T01:59:10.329 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T01:59:10.330 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T01:59:10.330 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T01:59:10.330 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T01:59:10.343 INFO:teuthology.orchestra.run.smithi191.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T01:59:10.344 INFO:teuthology.orchestra.run.smithi191.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T01:59:10.400 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T01:59:10.685 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T01:59:10.859 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T01:59:10.872 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T01:59:10.872 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T01:59:10.874 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T01:59:10.958 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T01:59:10.966 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T01:59:10.967 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T01:59:10.984 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T01:59:11.018 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T01:59:11.023 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T01:59:11.024 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T01:59:11.024 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T01:59:11.025 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T01:59:11.104 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T01:59:16.727 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 16.3 MB in 2s (7,363 kB/s) 2024-09-21T01:59:16.796 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 16.3 MB in 2s (7,520 kB/s) 2024-09-21T01:59:18.194 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T01:59:18.210 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T01:59:18.214 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T01:59:18.227 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T01:59:18.281 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T01:59:18.290 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T01:59:18.500 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-21T01:59:18.501 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-21T01:59:18.501 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T01:59:18.501 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T01:59:18.676 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T01:59:18.677 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T01:59:18.677 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T01:59:18.677 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T01:59:18.677 INFO:teuthology.orchestra.run.smithi191.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T01:59:18.677 INFO:teuthology.orchestra.run.smithi191.stdout: python2.7-minimal 2024-09-21T01:59:18.678 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T01:59:18.678 INFO:teuthology.orchestra.run.smithi191.stdout:The following additional packages will be installed: 2024-09-21T01:59:18.679 INFO:teuthology.orchestra.run.smithi191.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:59:18.679 INFO:teuthology.orchestra.run.smithi191.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T01:59:18.679 INFO:teuthology.orchestra.run.smithi191.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: python2.7-minimal 2024-09-21T01:59:18.680 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi191.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi191.stdout:Recommended packages: 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi191.stdout: thermald 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:59:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T01:59:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T01:59:18.683 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T01:59:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T01:59:18.683 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-09-21T01:59:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: thermald 2024-09-21T01:59:18.727 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2024-09-21T01:59:18.727 INFO:teuthology.orchestra.run.smithi191.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:59:18.727 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T01:59:18.728 INFO:teuthology.orchestra.run.smithi191.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T01:59:18.729 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages will be upgraded: 2024-09-21T01:59:18.730 INFO:teuthology.orchestra.run.smithi191.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T01:59:18.731 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T01:59:18.731 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T01:59:18.731 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T01:59:18.731 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T01:59:18.733 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2024-09-21T01:59:18.733 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T01:59:18.783 INFO:teuthology.orchestra.run.smithi170.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T01:59:18.783 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 113 MB of archives. 2024-09-21T01:59:18.783 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T01:59:18.783 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T01:59:18.909 INFO:teuthology.orchestra.run.smithi191.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T01:59:18.909 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 113 MB of archives. 2024-09-21T01:59:18.909 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T01:59:18.909 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T01:59:19.221 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T01:59:19.331 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T01:59:20.143 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T01:59:20.143 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T01:59:20.143 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T01:59:20.249 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T01:59:20.262 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T01:59:20.284 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T01:59:20.654 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T01:59:20.658 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 113 MB in 2s (73.4 MB/s) 2024-09-21T01:59:20.819 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T01:59:22.835 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T01:59:22.835 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T01:59:22.836 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T01:59:22.840 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T01:59:22.845 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:22.908 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T01:59:23.451 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T01:59:23.849 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2024-09-21T01:59:24.026 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T01:59:25.293 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T01:59:25.314 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:25.393 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:25.854 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T01:59:25.874 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:25.921 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:26.242 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T01:59:26.246 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:26.473 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:28.581 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T01:59:28.602 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:28.685 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:29.159 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T01:59:29.180 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:29.218 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:32.314 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:59:32.414 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:59:32.781 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:59:32.909 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:59:33.220 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T01:59:33.243 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T01:59:33.287 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:59:36.547 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:59:36.646 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:59:36.963 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:59:37.084 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:59:37.397 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T01:59:37.419 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T01:59:37.455 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:59:38.916 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T01:59:38.954 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:38.999 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:41.637 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:59:41.761 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:59:42.094 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:59:42.216 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:42.342 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T01:59:42.471 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:43.215 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T01:59:43.252 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T01:59:43.290 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:43.787 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:59:43.788 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T01:59:43.788 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:59:43.788 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T01:59:43.939 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:46.224 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T01:59:46.350 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T01:59:46.476 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:47.281 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T01:59:47.395 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T01:59:47.711 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T01:59:47.833 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:47.959 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T01:59:48.088 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:48.319 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:48.381 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T01:59:48.381 INFO:teuthology.orchestra.run.smithi170.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T01:59:49.425 INFO:teuthology.orchestra.run.smithi191.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T01:59:49.425 INFO:teuthology.orchestra.run.smithi191.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T01:59:49.425 INFO:teuthology.orchestra.run.smithi191.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T01:59:49.426 INFO:teuthology.orchestra.run.smithi191.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T01:59:49.573 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:50.966 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T01:59:51.092 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T01:59:51.218 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:53.496 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T01:59:53.546 INFO:teuthology.orchestra.run.smithi191.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T01:59:53.546 INFO:teuthology.orchestra.run.smithi191.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T02:00:02.743 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T02:00:02.743 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub' 2024-09-21T02:00:02.756 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T02:00:02.812 INFO:teuthology.orchestra.run.smithi170.stdout:Generating grub configuration file ... 2024-09-21T02:00:03.125 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T02:00:03.139 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T02:00:03.523 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T02:00:03.528 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T02:00:03.591 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T02:00:03.597 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T02:00:03.773 INFO:teuthology.orchestra.run.smithi170.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T02:00:03.773 INFO:teuthology.orchestra.run.smithi170.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T02:00:03.774 INFO:teuthology.orchestra.run.smithi170.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T02:00:03.819 INFO:teuthology.orchestra.run.smithi170.stdout:done 2024-09-21T02:00:04.293 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-09-21T02:00:04.311 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.122.122 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-09-21T02:00:04.312 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-09-21T02:00:04.313 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-09-21T02:00:04.313 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-09-21T02:00:04.313 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T02:00:04.313 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T02:00:04.313 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T02:00:04.313 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T02:00:04.313 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T02:00:04.313 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-09-21T02:00:04.501 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T02:00:07.547 INFO:teuthology.orchestra.run.smithi191.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T02:00:07.547 INFO:teuthology.orchestra.run.smithi191.stdout:Sourcing file `/etc/default/grub' 2024-09-21T02:00:07.568 INFO:teuthology.orchestra.run.smithi191.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T02:00:07.596 INFO:teuthology.orchestra.run.smithi191.stdout:Generating grub configuration file ... 2024-09-21T02:00:07.834 INFO:teuthology.orchestra.run.smithi191.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T02:00:07.850 INFO:teuthology.orchestra.run.smithi191.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T02:00:08.153 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T02:00:08.157 INFO:teuthology.orchestra.run.smithi191.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T02:00:08.163 INFO:teuthology.orchestra.run.smithi191.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T02:00:08.227 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T02:00:08.228 INFO:teuthology.orchestra.run.smithi191.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T02:00:08.233 INFO:teuthology.orchestra.run.smithi191.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T02:00:08.244 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T02:00:08.262 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T02:00:08.396 INFO:teuthology.orchestra.run.smithi191.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T02:00:08.396 INFO:teuthology.orchestra.run.smithi191.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T02:00:08.396 INFO:teuthology.orchestra.run.smithi191.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T02:00:08.430 INFO:teuthology.orchestra.run.smithi191.stdout:done 2024-09-21T02:00:08.870 DEBUG:teuthology.orchestra.run.smithi191:> dpkg -s linux-image-generic 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Package: linux-image-generic 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Status: install ok installed 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Priority: optional 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Section: kernel 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Installed-Size: 21 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: amd64 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Source: linux-meta 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Version: 5.15.0.122.122 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T02:00:08.891 INFO:teuthology.orchestra.run.smithi191.stdout:Recommends: thermald 2024-09-21T02:00:08.892 INFO:teuthology.orchestra.run.smithi191.stdout:Description: Generic Linux kernel image 2024-09-21T02:00:08.892 INFO:teuthology.orchestra.run.smithi191.stdout: This package will always depend on the latest generic kernel image 2024-09-21T02:00:08.892 INFO:teuthology.orchestra.run.smithi191.stdout: available. 2024-09-21T02:00:08.892 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T02:00:08.892 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T02:00:08.893 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T02:00:08.893 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T02:00:08.893 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T02:00:08.893 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get clean 2024-09-21T02:00:09.046 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2024-09-21T02:00:09.523 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T02:00:09.542 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T02:00:09.597 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T02:00:09.783 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T02:00:09.784 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:00:09.955 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:00:09.995 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T02:00:09.997 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T02:00:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.122.122 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-09-21T02:00:10.055 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-09-21T02:00:10.055 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-09-21T02:00:10.103 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.NRNMHrnvHe 2024-09-21T02:00:10.103 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub/grub.cfg /tmp/tmp.NRNMHrnvHe 2024-09-21T02:00:10.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.NRNMHrnvHe 2024-09-21T02:00:10.288 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.NRNMHrnvHe is 10KB 2024-09-21T02:00:10.300 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.NRNMHrnvHe 2024-09-21T02:00:10.305 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T02:00:10.359 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:00:10.359 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T02:00:10.359 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T02:00:10.439 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T02:00:10.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo update-grub 2024-09-21T02:00:11.317 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T02:00:11.390 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T02:00:11.469 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T02:00:11.598 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub' 2024-09-21T02:00:11.599 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T02:00:11.602 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-09-21T02:00:11.758 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T02:00:11.783 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T02:00:11.790 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T02:00:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T02:00:12.076 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T02:00:12.118 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T02:00:12.123 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T02:00:12.259 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T02:00:12.259 INFO:teuthology.orchestra.run.smithi170.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T02:00:12.260 INFO:teuthology.orchestra.run.smithi170.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T02:00:12.282 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-09-21T02:00:12.291 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-09-21T02:00:12.989 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T02:00:13.009 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T02:00:13.071 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T02:00:13.283 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-21T02:00:13.284 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-21T02:00:13.462 INFO:teuthology.orchestra.run.smithi191.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T02:00:13.462 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:00:13.462 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:00:13.463 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:00:13.463 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:00:13.463 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:00:13.463 INFO:teuthology.orchestra.run.smithi191.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:00:13.463 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:00:13.504 INFO:teuthology.orchestra.run.smithi191.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T02:00:13.505 DEBUG:teuthology.orchestra.run.smithi191:> dpkg -s linux-image-generic 2024-09-21T02:00:13.521 INFO:teuthology.orchestra.run.smithi191.stdout:Package: linux-image-generic 2024-09-21T02:00:13.521 INFO:teuthology.orchestra.run.smithi191.stdout:Status: install ok installed 2024-09-21T02:00:13.521 INFO:teuthology.orchestra.run.smithi191.stdout:Priority: optional 2024-09-21T02:00:13.521 INFO:teuthology.orchestra.run.smithi191.stdout:Section: kernel 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Installed-Size: 21 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: amd64 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Source: linux-meta 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Version: 5.15.0.122.122 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Recommends: thermald 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout:Description: Generic Linux kernel image 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout: This package will always depend on the latest generic kernel image 2024-09-21T02:00:13.522 INFO:teuthology.orchestra.run.smithi191.stdout: available. 2024-09-21T02:00:13.522 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-09-21T02:00:13.565 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.chr0o7sOra 2024-09-21T02:00:13.566 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub/grub.cfg /tmp/tmp.chr0o7sOra 2024-09-21T02:00:13.621 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.chr0o7sOra 2024-09-21T02:00:13.751 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.chr0o7sOra is 10KB 2024-09-21T02:00:13.764 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.chr0o7sOra 2024-09-21T02:00:13.769 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T02:00:13.824 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:00:13.825 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T02:00:13.825 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T02:00:13.907 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T02:00:13.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo update-grub 2024-09-21T02:00:15.186 INFO:teuthology.orchestra.run.smithi191.stderr:Sourcing file `/etc/default/grub' 2024-09-21T02:00:15.187 INFO:teuthology.orchestra.run.smithi191.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T02:00:15.190 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-09-21T02:00:15.384 INFO:teuthology.orchestra.run.smithi191.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T02:00:15.389 INFO:teuthology.orchestra.run.smithi191.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T02:00:15.639 INFO:teuthology.orchestra.run.smithi191.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T02:00:15.644 INFO:teuthology.orchestra.run.smithi191.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T02:00:15.683 INFO:teuthology.orchestra.run.smithi191.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T02:00:15.687 INFO:teuthology.orchestra.run.smithi191.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T02:00:15.813 INFO:teuthology.orchestra.run.smithi191.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T02:00:15.814 INFO:teuthology.orchestra.run.smithi191.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T02:00:15.814 INFO:teuthology.orchestra.run.smithi191.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T02:00:15.824 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-09-21T02:00:15.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-09-21T02:00:42.318 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T02:00:42.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:00:42.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:00:45.842 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T02:00:45.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:00:45.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:00.657 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T02:01:04.239 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-09-21T02:01:09.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:01:09.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:12.719 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T02:01:13.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:01:13.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:16.303 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-09-21T02:01:24.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:01:24.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:27.791 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T02:01:28.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:01:28.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:42.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:01:42.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:01:45.870 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T02:01:46.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-09-21T02:02:01.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:02:01.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:02:01.958 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-21T02:02:02.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:02:02.500 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T02:02:02.500 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-09-21T02:02:02.546 INFO:teuthology.orchestra.run.smithi191.stdout:5.15.0-122-generic 2024-09-21T02:02:02.547 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T02:02:02.547 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T02:02:02.547 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T02:02:03.547 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T02:02:03.548 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T02:02:03.630 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2024-09-21T02:02:03.649 DEBUG:teuthology.parallel:result is None 2024-09-21T02:02:03.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:02:03.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:02:04.240 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-21T02:02:04.799 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:02:04.799 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T02:02:04.800 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-21T02:02:04.844 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-122-generic 2024-09-21T02:02:04.845 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T02:02:04.845 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T02:02:04.845 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T02:02:05.846 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T02:02:05.846 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T02:02:05.917 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-09-21T02:02:05.942 DEBUG:teuthology.parallel:result is None 2024-09-21T02:02:05.942 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T02:02:05.949 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T02:02:05.949 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:02:05.952 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:02:05.957 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T02:02:05.964 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T02:02:05.974 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T02:02:05.974 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:02:05.998 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:02:06.030 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T02:02:06.037 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T02:02:06.037 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:02:06.058 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:02:06.076 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:02:06.082 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:02:06.084 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:02:06.090 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:02:06.092 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T02:02:06.099 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T02:02:06.099 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:02:06.129 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:02:06.146 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T02:02:06.159 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T02:02:06.159 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:02:06.189 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:02:06.195 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:02:06.240 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:02:06.288 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:02:06.289 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:02:06.387 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:02:06.393 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:02:06.439 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:02:06.439 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:02:06.504 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-09-21T02:02:06.507 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-09-21T02:02:06.592 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T02:02:06.599 INFO:teuthology.task.internal:Starting timer... 2024-09-21T02:02:06.599 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T02:02:06.613 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T02:02:06.628 DEBUG:teuthology.task.selinux:Excluding smithi170: OS 'ubuntu' does not support SELinux 2024-09-21T02:02:06.628 DEBUG:teuthology.task.selinux:Excluding smithi191: OS 'ubuntu' does not support SELinux 2024-09-21T02:02:06.628 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T02:02:06.628 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T02:02:06.628 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T02:02:06.628 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T02:02:06.638 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T02:02:08.468 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T02:02:08.468 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi170.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T02:09:36.607 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi170.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-09-21T02:09:36.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:09:36.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:09:36.767 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-21T02:09:36.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T02:09:36.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:09:36.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:09:36.939 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-21T02:09:37.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-21T02:09:37.016 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T02:09:37.030 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T02:09:37.030 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:09:37.030 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:09:37.039 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:09:37.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Command line: ntpd -gq 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: ---------------------------------------------------- 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: corporation. Support and training for ntp-4 are 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: available at https://www.nwtime.org/support 2024-09-21T02:09:37.067 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: ---------------------------------------------------- 2024-09-21T02:09:37.068 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: proto: precision = 0.046 usec (-24) 2024-09-21T02:09:37.068 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: basedate set to 2022-02-04 2024-09-21T02:09:37.068 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: gps base set to 2022-02-06 (week 2196) 2024-09-21T02:09:37.068 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:09:37.068 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:09:37.069 INFO:teuthology.orchestra.run.smithi170.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:09:37.069 INFO:teuthology.orchestra.run.smithi170.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listen normally on 3 enp3s0f1 172.21.15.170:123 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listen normally on 4 lo [::1]:123 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7255%5]:123 2024-09-21T02:09:37.070 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:37 ntpd[18476]: Listening on routing socket on fd #22 for interface updates 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Command line: ntpd -gq 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: ---------------------------------------------------- 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: corporation. Support and training for ntp-4 are 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: available at https://www.nwtime.org/support 2024-09-21T02:09:37.089 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: ---------------------------------------------------- 2024-09-21T02:09:37.090 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: proto: precision = 0.063 usec (-24) 2024-09-21T02:09:37.090 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: basedate set to 2022-02-04 2024-09-21T02:09:37.090 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: gps base set to 2022-02-06 (week 2196) 2024-09-21T02:09:37.090 INFO:teuthology.orchestra.run.smithi191.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:09:37.091 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T02:09:37.091 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:09:37.091 INFO:teuthology.orchestra.run.smithi191.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listen normally on 3 enp3s0f1 172.21.15.191:123 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listen normally on 4 lo [::1]:123 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7211%5]:123 2024-09-21T02:09:37.092 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:37 ntpd[18489]: Listening on routing socket on fd #22 for interface updates 2024-09-21T02:09:38.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:38.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:38 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:38.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:38.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:38.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:38.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:38.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:38.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:38 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:39.067 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:39 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:39 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:39 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:39 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:39 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:39.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:39 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:40 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:40.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:40.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:40.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:40.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:40.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:40 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:41.067 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:41 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:41 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:41 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:41 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:41 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:41 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:41.089 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:41.089 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:41 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:42 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:42.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:42.092 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:42 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:43.067 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:43 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:43.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:43 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:43.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:43 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:43.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:43 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:43.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:43 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:43.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:43 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:44.068 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:44.069 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 02:09:44 ntpd[18476]: ntpd: time slew +0.000344 s 2024-09-21T02:09:44.069 INFO:teuthology.orchestra.run.smithi170.stdout:ntpd: time slew +0.000344s 2024-09-21T02:09:44.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T02:09:44.069 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 02:09:44 ntpd[18476]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T02:09:44.090 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T02:09:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T02:09:44.091 INFO:teuthology.orchestra.run.smithi191.stdout:21 Sep 02:09:44 ntpd[18489]: ntpd: time slew -0.001439 s 2024-09-21T02:09:44.091 INFO:teuthology.orchestra.run.smithi191.stdout:ntpd: time slew -0.001439s 2024-09-21T02:09:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T02:09:44.091 INFO:teuthology.orchestra.run.smithi191.stderr:21 Sep 02:09:44 ntpd[18489]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T02:09:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:09:44.127 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-09-21T02:09:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.144 INFO:teuthology.orchestra.run.smithi191.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:09:44.145 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================== 2024-09-21T02:09:44.145 INFO:teuthology.orchestra.run.smithi191.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.145 INFO:teuthology.orchestra.run.smithi191.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.145 INFO:teuthology.orchestra.run.smithi191.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.145 INFO:teuthology.orchestra.run.smithi191.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T02:09:44.146 INFO:teuthology.run_tasks:Running task install... 2024-09-21T02:09:44.158 DEBUG:teuthology.task.install:project ceph 2024-09-21T02:09:44.158 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T02:09:44.158 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T02:09:44.158 INFO:teuthology.task.install:Using flavor: default 2024-09-21T02:09:44.163 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T02:09:44.163 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T02:09:44.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-key list | grep Ceph 2024-09-21T02:09:44.164 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-key list | grep Ceph 2024-09-21T02:09:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T02:09:44.244 INFO:teuthology.orchestra.run.smithi191.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T02:09:44.255 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T02:09:44.255 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T02:09:44.256 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-21T02:09:44.256 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-21T02:09:44.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:09:44.271 INFO:teuthology.orchestra.run.smithi191.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T02:09:44.271 INFO:teuthology.orchestra.run.smithi191.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T02:09:44.271 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-21T02:09:44.271 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-21T02:09:44.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:09:44.401 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default/ 2024-09-21T02:09:44.401 INFO:teuthology.task.install.deb:Package version is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:09:44.408 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default/ 2024-09-21T02:09:44.408 INFO:teuthology.task.install.deb:Package version is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:09:44.518 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:09:44.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T02:09:44.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2024-09-21T02:09:44.558 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:09:44.559 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T02:09:44.572 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T02:09:44.701 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T02:09:44.709 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T02:09:44.727 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T02:09:44.740 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T02:09:44.754 INFO:teuthology.orchestra.run.smithi191.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T02:09:44.775 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-21T02:09:44.793 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T02:09:44.797 INFO:teuthology.orchestra.run.smithi191.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T02:09:44.814 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-21T02:09:44.818 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,432 B] 2024-09-21T02:09:44.836 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T02:09:44.839 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T02:09:44.847 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-21T02:09:44.857 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,432 B] 2024-09-21T02:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T02:09:44.885 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-21T02:09:44.942 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T02:09:45.023 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T02:09:45.054 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 28.6 kB in 0s (83.7 kB/s) 2024-09-21T02:09:45.161 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 28.6 kB in 0s (69.3 kB/s) 2024-09-21T02:09:46.047 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T02:09:46.064 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-5120-g18432218-1jammy cephadm=19.3.0-5120-g18432218-1jammy ceph-mds=19.3.0-5120-g18432218-1jammy ceph-mgr=19.3.0-5120-g18432218-1jammy ceph-common=19.3.0-5120-g18432218-1jammy ceph-fuse=19.3.0-5120-g18432218-1jammy ceph-test=19.3.0-5120-g18432218-1jammy ceph-volume=19.3.0-5120-g18432218-1jammy radosgw=19.3.0-5120-g18432218-1jammy python3-rados=19.3.0-5120-g18432218-1jammy python3-rgw=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy python3-rbd=19.3.0-5120-g18432218-1jammy libcephfs2=19.3.0-5120-g18432218-1jammy libcephfs-dev=19.3.0-5120-g18432218-1jammy librados2=19.3.0-5120-g18432218-1jammy librbd1=19.3.0-5120-g18432218-1jammy rbd-fuse=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy cephfs-shell=19.3.0-5120-g18432218-1jammy cephfs-top=19.3.0-5120-g18432218-1jammy cephfs-mirror=19.3.0-5120-g18432218-1jammy 2024-09-21T02:09:46.127 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T02:09:46.128 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T02:09:46.143 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-5120-g18432218-1jammy cephadm=19.3.0-5120-g18432218-1jammy ceph-mds=19.3.0-5120-g18432218-1jammy ceph-mgr=19.3.0-5120-g18432218-1jammy ceph-common=19.3.0-5120-g18432218-1jammy ceph-fuse=19.3.0-5120-g18432218-1jammy ceph-test=19.3.0-5120-g18432218-1jammy ceph-volume=19.3.0-5120-g18432218-1jammy radosgw=19.3.0-5120-g18432218-1jammy python3-rados=19.3.0-5120-g18432218-1jammy python3-rgw=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy python3-rbd=19.3.0-5120-g18432218-1jammy libcephfs2=19.3.0-5120-g18432218-1jammy libcephfs-dev=19.3.0-5120-g18432218-1jammy librados2=19.3.0-5120-g18432218-1jammy librbd1=19.3.0-5120-g18432218-1jammy rbd-fuse=19.3.0-5120-g18432218-1jammy python3-cephfs=19.3.0-5120-g18432218-1jammy cephfs-shell=19.3.0-5120-g18432218-1jammy cephfs-top=19.3.0-5120-g18432218-1jammy cephfs-mirror=19.3.0-5120-g18432218-1jammy 2024-09-21T02:09:46.205 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T02:09:46.333 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-21T02:09:46.334 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-21T02:09:46.417 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T02:09:46.417 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T02:09:46.536 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:09:46.536 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:09:46.536 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:09:46.536 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:09:46.536 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:09:46.537 INFO:teuthology.orchestra.run.smithi191.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:09:46.537 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:09:46.537 INFO:teuthology.orchestra.run.smithi191.stdout:The following additional packages will be installed: 2024-09-21T02:09:46.537 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T02:09:46.538 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T02:09:46.538 INFO:teuthology.orchestra.run.smithi191.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:09:46.538 INFO:teuthology.orchestra.run.smithi191.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T02:09:46.538 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T02:09:46.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2024-09-21T02:09:46.540 INFO:teuthology.orchestra.run.smithi191.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout:Recommended packages: 2024-09-21T02:09:46.541 INFO:teuthology.orchestra.run.smithi191.stdout: btrfs-tools 2024-09-21T02:09:46.618 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:09:46.618 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:09:46.618 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:09:46.618 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:09:46.619 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:09:46.619 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:09:46.619 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:09:46.619 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T02:09:46.619 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T02:09:46.620 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T02:09:46.620 INFO:teuthology.orchestra.run.smithi170.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:09:46.620 INFO:teuthology.orchestra.run.smithi170.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T02:09:46.620 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T02:09:46.621 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-21T02:09:46.622 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-09-21T02:09:46.623 INFO:teuthology.orchestra.run.smithi170.stdout: btrfs-tools 2024-09-21T02:09:46.649 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2024-09-21T02:09:46.649 INFO:teuthology.orchestra.run.smithi191.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T02:09:46.649 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T02:09:46.649 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-21T02:09:46.651 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T02:09:46.652 INFO:teuthology.orchestra.run.smithi191.stdout: socat unzip xmlstarlet zip 2024-09-21T02:09:46.732 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T02:09:46.732 INFO:teuthology.orchestra.run.smithi170.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T02:09:46.732 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T02:09:46.732 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T02:09:46.734 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T02:09:46.735 INFO:teuthology.orchestra.run.smithi170.stdout: socat unzip xmlstarlet zip 2024-09-21T02:09:46.792 INFO:teuthology.orchestra.run.smithi191.stdout:0 upgraded, 125 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T02:09:46.792 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 239 MB of archives. 2024-09-21T02:09:46.793 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-21T02:09:46.793 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-5120-g18432218-1jammy [3,591 kB] 2024-09-21T02:09:46.794 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 125 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T02:09:46.794 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 239 MB of archives. 2024-09-21T02:09:46.794 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-21T02:09:46.794 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-21T02:09:46.813 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T02:09:46.823 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-21T02:09:46.840 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-21T02:09:46.863 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T02:09:46.878 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-5120-g18432218-1jammy [3,591 kB] 2024-09-21T02:09:46.884 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T02:09:46.886 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T02:09:46.898 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T02:09:46.923 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T02:09:46.944 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T02:09:46.947 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T02:09:46.955 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T02:09:46.958 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-21T02:09:46.959 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T02:09:46.960 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T02:09:46.967 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T02:09:46.968 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-21T02:09:46.969 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T02:09:46.971 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T02:09:46.972 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T02:09:46.974 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-5120-g18432218-1jammy [3,370 kB] 2024-09-21T02:09:46.980 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T02:09:46.982 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T02:09:46.985 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T02:09:46.986 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T02:09:46.987 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T02:09:46.997 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T02:09:46.999 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T02:09:46.999 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T02:09:46.999 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T02:09:46.999 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T02:09:47.015 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T02:09:47.015 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T02:09:47.015 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T02:09:47.017 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T02:09:47.018 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T02:09:47.031 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T02:09:47.032 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T02:09:47.032 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T02:09:47.033 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T02:09:47.033 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T02:09:47.040 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-5120-g18432218-1jammy [733 kB] 2024-09-21T02:09:47.048 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T02:09:47.049 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T02:09:47.050 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T02:09:47.055 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T02:09:47.055 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T02:09:47.065 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T02:09:47.066 INFO:teuthology.orchestra.run.smithi170.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T02:09:47.067 INFO:teuthology.orchestra.run.smithi170.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T02:09:47.069 INFO:teuthology.orchestra.run.smithi170.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T02:09:47.070 INFO:teuthology.orchestra.run.smithi170.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T02:09:47.075 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-5120-g18432218-1jammy [356 kB] 2024-09-21T02:09:47.082 INFO:teuthology.orchestra.run.smithi170.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T02:09:47.082 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-5120-g18432218-1jammy [32.8 kB] 2024-09-21T02:09:47.083 INFO:teuthology.orchestra.run.smithi170.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T02:09:47.083 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-5120-g18432218-1jammy [180 kB] 2024-09-21T02:09:47.085 INFO:teuthology.orchestra.run.smithi170.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T02:09:47.086 INFO:teuthology.orchestra.run.smithi170.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T02:09:47.086 INFO:teuthology.orchestra.run.smithi170.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T02:09:47.087 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-5120-g18432218-1jammy [72.9 kB] 2024-09-21T02:09:47.090 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-5120-g18432218-1jammy [340 kB] 2024-09-21T02:09:47.095 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-5120-g18432218-1jammy [8,845 kB] 2024-09-21T02:09:47.099 INFO:teuthology.orchestra.run.smithi170.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T02:09:47.145 INFO:teuthology.orchestra.run.smithi170.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-5120-g18432218-1jammy [3,370 kB] 2024-09-21T02:09:47.150 INFO:teuthology.orchestra.run.smithi170.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T02:09:47.150 INFO:teuthology.orchestra.run.smithi170.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T02:09:47.151 INFO:teuthology.orchestra.run.smithi170.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T02:09:47.152 INFO:teuthology.orchestra.run.smithi170.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T02:09:47.153 INFO:teuthology.orchestra.run.smithi170.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T02:09:47.164 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T02:09:47.172 INFO:teuthology.orchestra.run.smithi170.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T02:09:47.173 INFO:teuthology.orchestra.run.smithi170.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T02:09:47.173 INFO:teuthology.orchestra.run.smithi170.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T02:09:47.188 INFO:teuthology.orchestra.run.smithi170.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T02:09:47.188 INFO:teuthology.orchestra.run.smithi170.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T02:09:47.188 INFO:teuthology.orchestra.run.smithi170.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T02:09:47.189 INFO:teuthology.orchestra.run.smithi170.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T02:09:47.189 INFO:teuthology.orchestra.run.smithi170.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T02:09:47.190 INFO:teuthology.orchestra.run.smithi170.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T02:09:47.203 INFO:teuthology.orchestra.run.smithi170.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T02:09:47.203 INFO:teuthology.orchestra.run.smithi170.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T02:09:47.205 INFO:teuthology.orchestra.run.smithi170.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T02:09:47.219 INFO:teuthology.orchestra.run.smithi170.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T02:09:47.220 INFO:teuthology.orchestra.run.smithi170.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T02:09:47.221 INFO:teuthology.orchestra.run.smithi170.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T02:09:47.223 INFO:teuthology.orchestra.run.smithi170.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T02:09:47.226 INFO:teuthology.orchestra.run.smithi170.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-21T02:09:47.226 INFO:teuthology.orchestra.run.smithi170.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T02:09:47.227 INFO:teuthology.orchestra.run.smithi170.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T02:09:47.237 INFO:teuthology.orchestra.run.smithi170.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T02:09:47.240 INFO:teuthology.orchestra.run.smithi170.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T02:09:47.241 INFO:teuthology.orchestra.run.smithi170.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T02:09:47.259 INFO:teuthology.orchestra.run.smithi170.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T02:09:47.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T02:09:47.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T02:09:47.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T02:09:47.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T02:09:47.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T02:09:47.261 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T02:09:47.263 INFO:teuthology.orchestra.run.smithi170.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T02:09:47.270 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T02:09:47.271 INFO:teuthology.orchestra.run.smithi170.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T02:09:47.271 INFO:teuthology.orchestra.run.smithi170.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T02:09:47.273 INFO:teuthology.orchestra.run.smithi170.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T02:09:47.274 INFO:teuthology.orchestra.run.smithi170.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-5120-g18432218-1jammy [733 kB] 2024-09-21T02:09:47.287 INFO:teuthology.orchestra.run.smithi170.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T02:09:47.287 INFO:teuthology.orchestra.run.smithi170.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-5120-g18432218-1jammy [356 kB] 2024-09-21T02:09:47.295 INFO:teuthology.orchestra.run.smithi170.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-5120-g18432218-1jammy [32.8 kB] 2024-09-21T02:09:47.296 INFO:teuthology.orchestra.run.smithi170.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-5120-g18432218-1jammy [180 kB] 2024-09-21T02:09:47.300 INFO:teuthology.orchestra.run.smithi170.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-5120-g18432218-1jammy [72.9 kB] 2024-09-21T02:09:47.301 INFO:teuthology.orchestra.run.smithi170.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-5120-g18432218-1jammy [340 kB] 2024-09-21T02:09:47.319 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T02:09:47.339 INFO:teuthology.orchestra.run.smithi170.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-5120-g18432218-1jammy [8,845 kB] 2024-09-21T02:09:47.350 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-5120-g18432218-1jammy [112 kB] 2024-09-21T02:09:47.353 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-5120-g18432218-1jammy [478 kB] 2024-09-21T02:09:47.368 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-5120-g18432218-1jammy [29.1 MB] 2024-09-21T02:09:47.495 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T02:09:47.504 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T02:09:47.531 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T02:09:47.540 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-21T02:09:47.542 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T02:09:47.543 INFO:teuthology.orchestra.run.smithi191.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T02:09:47.561 INFO:teuthology.orchestra.run.smithi191.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-21T02:09:47.565 INFO:teuthology.orchestra.run.smithi191.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-21T02:09:47.569 INFO:teuthology.orchestra.run.smithi191.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-21T02:09:47.578 INFO:teuthology.orchestra.run.smithi191.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-21T02:09:47.654 INFO:teuthology.orchestra.run.smithi191.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-21T02:09:47.658 INFO:teuthology.orchestra.run.smithi191.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-21T02:09:47.663 INFO:teuthology.orchestra.run.smithi191.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-21T02:09:47.667 INFO:teuthology.orchestra.run.smithi191.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T02:09:47.668 INFO:teuthology.orchestra.run.smithi191.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T02:09:47.669 INFO:teuthology.orchestra.run.smithi191.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T02:09:47.671 INFO:teuthology.orchestra.run.smithi191.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T02:09:47.671 INFO:teuthology.orchestra.run.smithi191.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T02:09:47.672 INFO:teuthology.orchestra.run.smithi191.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T02:09:47.724 INFO:teuthology.orchestra.run.smithi170.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-5120-g18432218-1jammy [112 kB] 2024-09-21T02:09:47.726 INFO:teuthology.orchestra.run.smithi170.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-5120-g18432218-1jammy [478 kB] 2024-09-21T02:09:47.736 INFO:teuthology.orchestra.run.smithi170.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-5120-g18432218-1jammy [29.1 MB] 2024-09-21T02:09:47.741 INFO:teuthology.orchestra.run.smithi191.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T02:09:47.754 INFO:teuthology.orchestra.run.smithi191.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T02:09:47.755 INFO:teuthology.orchestra.run.smithi191.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T02:09:47.755 INFO:teuthology.orchestra.run.smithi191.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T02:09:47.762 INFO:teuthology.orchestra.run.smithi191.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T02:09:47.763 INFO:teuthology.orchestra.run.smithi191.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T02:09:47.823 INFO:teuthology.orchestra.run.smithi191.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T02:09:47.824 INFO:teuthology.orchestra.run.smithi191.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T02:09:47.826 INFO:teuthology.orchestra.run.smithi191.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T02:09:47.826 INFO:teuthology.orchestra.run.smithi191.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T02:09:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T02:09:47.907 INFO:teuthology.orchestra.run.smithi191.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T02:09:47.908 INFO:teuthology.orchestra.run.smithi191.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T02:09:47.909 INFO:teuthology.orchestra.run.smithi191.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T02:09:47.920 INFO:teuthology.orchestra.run.smithi191.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T02:09:47.985 INFO:teuthology.orchestra.run.smithi191.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T02:09:47.986 INFO:teuthology.orchestra.run.smithi191.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T02:09:47.987 INFO:teuthology.orchestra.run.smithi191.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T02:09:47.989 INFO:teuthology.orchestra.run.smithi191.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T02:09:47.993 INFO:teuthology.orchestra.run.smithi191.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T02:09:48.066 INFO:teuthology.orchestra.run.smithi191.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T02:09:48.069 INFO:teuthology.orchestra.run.smithi191.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T02:09:48.071 INFO:teuthology.orchestra.run.smithi191.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T02:09:48.076 INFO:teuthology.orchestra.run.smithi191.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T02:09:48.077 INFO:teuthology.orchestra.run.smithi191.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T02:09:48.148 INFO:teuthology.orchestra.run.smithi191.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T02:09:48.152 INFO:teuthology.orchestra.run.smithi191.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T02:09:48.372 INFO:teuthology.orchestra.run.smithi191.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T02:09:48.466 INFO:teuthology.orchestra.run.smithi191.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T02:09:48.469 INFO:teuthology.orchestra.run.smithi191.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-5120-g18432218-1jammy [5,023 kB] 2024-09-21T02:09:48.469 INFO:teuthology.orchestra.run.smithi191.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T02:09:48.470 INFO:teuthology.orchestra.run.smithi191.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T02:09:48.473 INFO:teuthology.orchestra.run.smithi191.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T02:09:48.518 INFO:teuthology.orchestra.run.smithi191.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T02:09:48.522 INFO:teuthology.orchestra.run.smithi191.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T02:09:48.523 INFO:teuthology.orchestra.run.smithi191.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T02:09:48.547 INFO:teuthology.orchestra.run.smithi191.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-5120-g18432218-1jammy [252 kB] 2024-09-21T02:09:48.549 INFO:teuthology.orchestra.run.smithi191.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-5120-g18432218-1jammy [126 kB] 2024-09-21T02:09:48.551 INFO:teuthology.orchestra.run.smithi191.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-5120-g18432218-1jammy [1,552 kB] 2024-09-21T02:09:48.560 INFO:teuthology.orchestra.run.smithi191.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T02:09:48.561 INFO:teuthology.orchestra.run.smithi191.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T02:09:48.598 INFO:teuthology.orchestra.run.smithi191.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-5120-g18432218-1jammy [6,198 kB] 2024-09-21T02:09:48.631 INFO:teuthology.orchestra.run.smithi191.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T02:09:48.632 INFO:teuthology.orchestra.run.smithi191.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T02:09:48.633 INFO:teuthology.orchestra.run.smithi191.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T02:09:48.634 INFO:teuthology.orchestra.run.smithi191.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T02:09:48.635 INFO:teuthology.orchestra.run.smithi191.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T02:09:48.635 INFO:teuthology.orchestra.run.smithi170.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-5120-g18432218-1jammy [5,023 kB] 2024-09-21T02:09:48.636 INFO:teuthology.orchestra.run.smithi191.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T02:09:48.643 INFO:teuthology.orchestra.run.smithi191.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T02:09:48.712 INFO:teuthology.orchestra.run.smithi191.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T02:09:48.716 INFO:teuthology.orchestra.run.smithi191.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T02:09:48.717 INFO:teuthology.orchestra.run.smithi191.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T02:09:48.720 INFO:teuthology.orchestra.run.smithi170.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-5120-g18432218-1jammy [252 kB] 2024-09-21T02:09:48.732 INFO:teuthology.orchestra.run.smithi170.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-5120-g18432218-1jammy [126 kB] 2024-09-21T02:09:48.734 INFO:teuthology.orchestra.run.smithi170.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-5120-g18432218-1jammy [1,552 kB] 2024-09-21T02:09:48.783 INFO:teuthology.orchestra.run.smithi170.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-5120-g18432218-1jammy [6,198 kB] 2024-09-21T02:09:48.786 INFO:teuthology.orchestra.run.smithi191.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-5120-g18432218-1jammy [22.9 MB] 2024-09-21T02:09:48.794 INFO:teuthology.orchestra.run.smithi191.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T02:09:48.800 INFO:teuthology.orchestra.run.smithi191.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-21T02:09:48.801 INFO:teuthology.orchestra.run.smithi191.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-21T02:09:48.803 INFO:teuthology.orchestra.run.smithi191.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-21T02:09:48.805 INFO:teuthology.orchestra.run.smithi191.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T02:09:48.816 INFO:teuthology.orchestra.run.smithi191.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-21T02:09:48.818 INFO:teuthology.orchestra.run.smithi191.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T02:09:48.875 INFO:teuthology.orchestra.run.smithi191.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T02:09:48.875 INFO:teuthology.orchestra.run.smithi191.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T02:09:48.877 INFO:teuthology.orchestra.run.smithi191.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T02:09:48.956 INFO:teuthology.orchestra.run.smithi191.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T02:09:48.960 INFO:teuthology.orchestra.run.smithi191.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T02:09:48.962 INFO:teuthology.orchestra.run.smithi191.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T02:09:48.980 INFO:teuthology.orchestra.run.smithi191.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T02:09:48.981 INFO:teuthology.orchestra.run.smithi191.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T02:09:48.982 INFO:teuthology.orchestra.run.smithi191.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T02:09:48.987 INFO:teuthology.orchestra.run.smithi191.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T02:09:49.007 INFO:teuthology.orchestra.run.smithi170.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-5120-g18432218-1jammy [22.9 MB] 2024-09-21T02:09:49.038 INFO:teuthology.orchestra.run.smithi191.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T02:09:49.581 INFO:teuthology.orchestra.run.smithi191.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-5120-g18432218-1jammy [14.0 kB] 2024-09-21T02:09:49.582 INFO:teuthology.orchestra.run.smithi191.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-5120-g18432218-1jammy [865 kB] 2024-09-21T02:09:49.597 INFO:teuthology.orchestra.run.smithi191.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-5120-g18432218-1jammy [2,390 kB] 2024-09-21T02:09:49.634 INFO:teuthology.orchestra.run.smithi191.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-5120-g18432218-1jammy [813 kB] 2024-09-21T02:09:49.653 INFO:teuthology.orchestra.run.smithi191.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-5120-g18432218-1jammy [160 kB] 2024-09-21T02:09:49.657 INFO:teuthology.orchestra.run.smithi191.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-5120-g18432218-1jammy [61.5 MB] 2024-09-21T02:09:49.847 INFO:teuthology.orchestra.run.smithi170.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-5120-g18432218-1jammy [14.0 kB] 2024-09-21T02:09:49.847 INFO:teuthology.orchestra.run.smithi170.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-5120-g18432218-1jammy [865 kB] 2024-09-21T02:09:49.863 INFO:teuthology.orchestra.run.smithi170.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-5120-g18432218-1jammy [2,390 kB] 2024-09-21T02:09:49.902 INFO:teuthology.orchestra.run.smithi170.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-5120-g18432218-1jammy [813 kB] 2024-09-21T02:09:49.923 INFO:teuthology.orchestra.run.smithi170.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-5120-g18432218-1jammy [160 kB] 2024-09-21T02:09:49.927 INFO:teuthology.orchestra.run.smithi170.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-5120-g18432218-1jammy [61.5 MB] 2024-09-21T02:09:51.926 INFO:teuthology.orchestra.run.smithi191.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-5120-g18432218-1jammy [8,625 kB] 2024-09-21T02:09:52.035 INFO:teuthology.orchestra.run.smithi191.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-5120-g18432218-1jammy [14.2 kB] 2024-09-21T02:09:52.035 INFO:teuthology.orchestra.run.smithi191.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-5120-g18432218-1jammy [40.9 MB] 2024-09-21T02:09:52.189 INFO:teuthology.orchestra.run.smithi170.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-5120-g18432218-1jammy [8,625 kB] 2024-09-21T02:09:52.400 INFO:teuthology.orchestra.run.smithi170.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-5120-g18432218-1jammy [14.2 kB] 2024-09-21T02:09:52.400 INFO:teuthology.orchestra.run.smithi170.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-5120-g18432218-1jammy [40.9 MB] 2024-09-21T02:09:53.275 INFO:teuthology.orchestra.run.smithi191.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-5120-g18432218-1jammy [130 kB] 2024-09-21T02:09:53.277 INFO:teuthology.orchestra.run.smithi191.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-5120-g18432218-1jammy [237 kB] 2024-09-21T02:09:53.282 INFO:teuthology.orchestra.run.smithi191.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-5120-g18432218-1jammy [34.0 kB] 2024-09-21T02:09:53.283 INFO:teuthology.orchestra.run.smithi191.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-5120-g18432218-1jammy [23.6 kB] 2024-09-21T02:09:53.284 INFO:teuthology.orchestra.run.smithi191.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-5120-g18432218-1jammy [39.0 kB] 2024-09-21T02:09:53.285 INFO:teuthology.orchestra.run.smithi191.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-5120-g18432218-1jammy [17.6 MB] 2024-09-21T02:09:53.834 INFO:teuthology.orchestra.run.smithi191.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-5120-g18432218-1jammy [93.1 kB] 2024-09-21T02:09:53.928 INFO:teuthology.orchestra.run.smithi170.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-5120-g18432218-1jammy [130 kB] 2024-09-21T02:09:53.930 INFO:teuthology.orchestra.run.smithi170.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-5120-g18432218-1jammy [237 kB] 2024-09-21T02:09:53.935 INFO:teuthology.orchestra.run.smithi170.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-5120-g18432218-1jammy [34.0 kB] 2024-09-21T02:09:53.935 INFO:teuthology.orchestra.run.smithi170.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-5120-g18432218-1jammy [23.6 kB] 2024-09-21T02:09:53.937 INFO:teuthology.orchestra.run.smithi170.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-5120-g18432218-1jammy [39.0 kB] 2024-09-21T02:09:53.937 INFO:teuthology.orchestra.run.smithi170.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-5120-g18432218-1jammy [17.6 MB] 2024-09-21T02:09:54.268 INFO:teuthology.orchestra.run.smithi170.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-5120-g18432218-1jammy [93.1 kB] 2024-09-21T02:09:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 239 MB in 7s (33.4 MB/s) 2024-09-21T02:09:54.599 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-21T02:09:54.687 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T02:09:54.693 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-21T02:09:54.731 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-21T02:09:54.900 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 239 MB in 8s (31.8 MB/s) 2024-09-21T02:09:55.042 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-21T02:09:55.059 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T02:09:55.085 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T02:09:55.126 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:09:55.132 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T02:09:55.137 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-21T02:09:55.182 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-21T02:09:55.446 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cmd2. 2024-09-21T02:09:55.468 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T02:09:55.472 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-21T02:09:55.494 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T02:09:55.513 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-21T02:09:55.527 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:09:55.813 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cmd2. 2024-09-21T02:09:55.828 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-21T02:09:55.842 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T02:09:55.862 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-21T02:09:55.868 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T02:09:55.942 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:09:56.165 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T02:09:56.191 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T02:09:56.272 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T02:09:56.274 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:09:56.298 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T02:09:56.338 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:09:56.628 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T02:09:56.654 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T02:09:56.676 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T02:09:56.694 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:09:56.702 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T02:09:56.742 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:09:57.039 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T02:09:57.065 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T02:09:57.105 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:09:57.164 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T02:09:57.190 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:09:57.239 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:09:57.467 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T02:09:57.494 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:09:57.534 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:09:57.718 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T02:09:57.744 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:09:57.818 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:09:57.879 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T02:09:57.905 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:09:57.937 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:09:58.205 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T02:09:58.231 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:09:58.297 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:09:58.298 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T02:09:58.324 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T02:09:58.365 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:09:58.710 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T02:09:58.736 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T02:09:58.743 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T02:09:58.769 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T02:09:58.784 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:09:58.810 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:09:59.096 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librados2. 2024-09-21T02:09:59.111 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../010-librados2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:09:59.122 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librados2. 2024-09-21T02:09:59.148 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../010-librados2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:09:59.179 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:09:59.188 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:09:59.692 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libnbd0. 2024-09-21T02:09:59.719 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-21T02:09:59.751 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-21T02:09:59.774 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libnbd0. 2024-09-21T02:09:59.800 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-21T02:09:59.832 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-21T02:10:00.046 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librbd1. 2024-09-21T02:10:00.074 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../012-librbd1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:00.112 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:00.118 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librbd1. 2024-09-21T02:10:00.145 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../012-librbd1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:00.185 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:00.575 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T02:10:00.601 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:00.641 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:00.663 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T02:10:00.689 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:00.729 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:00.895 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rados. 2024-09-21T02:10:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../014-python3-rados_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:00.953 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:01.015 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rados. 2024-09-21T02:10:01.041 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../014-python3-rados_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:01.081 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:01.273 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T02:10:01.300 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:01.340 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:01.359 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T02:10:01.386 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:01.417 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:01.576 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T02:10:01.603 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:01.643 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:01.653 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T02:10:01.669 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:01.712 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:01.963 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T02:10:01.990 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:02.030 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:02.031 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T02:10:02.057 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:02.097 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:02.342 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T02:10:02.369 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T02:10:02.409 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T02:10:02.425 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T02:10:02.441 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T02:10:02.476 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T02:10:02.654 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T02:10:02.680 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:02.720 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T02:10:02.721 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:02.735 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:02.778 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:03.075 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T02:10:03.101 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T02:10:03.131 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T02:10:03.141 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:10:03.157 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T02:10:03.197 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:10:03.420 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T02:10:03.446 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T02:10:03.478 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:10:03.483 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-21T02:10:03.498 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-21T02:10:03.558 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:10:03.749 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T02:10:03.764 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T02:10:03.807 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:10:03.845 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-21T02:10:03.871 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-21T02:10:03.903 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:10:04.144 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package lua5.1. 2024-09-21T02:10:04.171 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T02:10:04.211 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:10:04.214 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua5.1. 2024-09-21T02:10:04.240 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-21T02:10:04.281 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:10:04.515 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package lua-any. 2024-09-21T02:10:04.530 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-21T02:10:04.573 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T02:10:04.609 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-any. 2024-09-21T02:10:04.635 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-21T02:10:04.676 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-21T02:10:04.827 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package zip. 2024-09-21T02:10:04.842 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-21T02:10:04.885 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T02:10:04.911 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package zip. 2024-09-21T02:10:04.927 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-21T02:10:04.961 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking zip (3.0-12build2) ... 2024-09-21T02:10:05.206 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package unzip. 2024-09-21T02:10:05.233 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T02:10:05.265 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:10:05.342 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package unzip. 2024-09-21T02:10:05.368 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-21T02:10:05.516 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:10:05.644 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package luarocks. 2024-09-21T02:10:05.670 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T02:10:05.710 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:10:05.854 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package luarocks. 2024-09-21T02:10:05.880 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-21T02:10:05.920 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:10:06.118 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librgw2. 2024-09-21T02:10:06.144 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../028-librgw2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:06.184 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:06.347 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librgw2. 2024-09-21T02:10:06.373 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../028-librgw2_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:06.414 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:06.912 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T02:10:06.927 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:06.961 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:07.125 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T02:10:07.140 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:07.183 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:07.256 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T02:10:07.282 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T02:10:07.323 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:10:07.504 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T02:10:07.530 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T02:10:07.570 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:10:07.659 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T02:10:07.685 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:07.728 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:07.874 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T02:10:07.890 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:07.924 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:08.073 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-common. 2024-09-21T02:10:08.099 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../032-ceph-common_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:08.139 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:08.329 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-common. 2024-09-21T02:10:08.358 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../032-ceph-common_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:08.395 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:10.020 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-base. 2024-09-21T02:10:10.038 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../033-ceph-base_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:10.086 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-base. 2024-09-21T02:10:10.087 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:10.112 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../033-ceph-base_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:10.154 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:10.641 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T02:10:10.667 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T02:10:10.700 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T02:10:10.700 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:10:10.715 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T02:10:10.750 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:10:10.993 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T02:10:11.020 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T02:10:11.028 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T02:10:11.055 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T02:10:11.060 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:10:11.095 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:10:11.405 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T02:10:11.424 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T02:10:11.434 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T02:10:11.450 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T02:10:11.471 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:10:11.490 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:10:11.774 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T02:10:11.794 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T02:10:11.801 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T02:10:11.821 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T02:10:11.841 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:10:11.861 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:10:12.102 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T02:10:12.128 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T02:10:12.139 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T02:10:12.160 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:10:12.166 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T02:10:12.206 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:10:12.472 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T02:10:12.498 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T02:10:12.510 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T02:10:12.536 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T02:10:12.538 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T02:10:12.577 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T02:10:12.841 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-portend. 2024-09-21T02:10:12.863 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-portend. 2024-09-21T02:10:12.867 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-21T02:10:12.879 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-21T02:10:12.908 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T02:10:12.913 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T02:10:13.194 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T02:10:13.209 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T02:10:13.220 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T02:10:13.235 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T02:10:13.261 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T02:10:13.276 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T02:10:13.522 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T02:10:13.545 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T02:10:13.563 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T02:10:13.580 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:10:13.590 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T02:10:13.629 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:10:13.909 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T02:10:13.935 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T02:10:13.975 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T02:10:13.984 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T02:10:14.010 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T02:10:14.051 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T02:10:14.271 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T02:10:14.298 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T02:10:14.337 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T02:10:14.388 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T02:10:14.414 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T02:10:14.455 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T02:10:14.640 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T02:10:14.666 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T02:10:14.707 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:10:14.742 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T02:10:14.768 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T02:10:14.808 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:10:15.068 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-mako. 2024-09-21T02:10:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T02:10:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:10:15.137 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-mako. 2024-09-21T02:10:15.164 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T02:10:15.204 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:10:15.438 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T02:10:15.464 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T02:10:15.505 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:10:15.533 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T02:10:15.560 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T02:10:15.608 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:10:15.783 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T02:10:15.809 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T02:10:15.849 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:10:16.129 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T02:10:16.155 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T02:10:16.195 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:10:16.210 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webob. 2024-09-21T02:10:16.237 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T02:10:16.285 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:10:16.490 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webob. 2024-09-21T02:10:16.517 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T02:10:16.565 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:10:16.697 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T02:10:16.723 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T02:10:16.766 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:10:16.895 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T02:10:16.921 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T02:10:16.956 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:10:17.083 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T02:10:17.109 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T02:10:17.150 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:10:17.256 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T02:10:17.283 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T02:10:17.323 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:10:17.452 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-paste. 2024-09-21T02:10:17.479 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T02:10:17.519 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:10:17.627 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-paste. 2024-09-21T02:10:17.654 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T02:10:17.693 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:10:17.839 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T02:10:17.866 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T02:10:17.906 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:10:18.014 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T02:10:18.030 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T02:10:18.073 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:10:18.150 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T02:10:18.177 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T02:10:18.208 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:10:18.401 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T02:10:18.428 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T02:10:18.459 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:10:18.495 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T02:10:18.521 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T02:10:18.561 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T02:10:18.755 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T02:10:18.782 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T02:10:18.822 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T02:10:18.906 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T02:10:18.932 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T02:10:18.973 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:10:19.142 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T02:10:19.169 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T02:10:19.209 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:10:19.319 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T02:10:19.346 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T02:10:19.386 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T02:10:19.557 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T02:10:19.583 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T02:10:19.623 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T02:10:19.697 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-idna. 2024-09-21T02:10:19.724 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T02:10:19.764 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:10:19.944 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-idna. 2024-09-21T02:10:19.970 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T02:10:20.010 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:10:21.143 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T02:10:21.169 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T02:10:21.206 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T02:10:21.209 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:10:21.233 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T02:10:21.272 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:10:21.521 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests. 2024-09-21T02:10:21.547 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T02:10:21.577 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-requests. 2024-09-21T02:10:21.587 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:10:21.603 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T02:10:21.643 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:10:21.865 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T02:10:21.892 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T02:10:21.924 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:10:21.939 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T02:10:21.965 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T02:10:22.005 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:10:22.218 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T02:10:22.245 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:22.276 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T02:10:22.285 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:22.292 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:22.351 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:22.673 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T02:10:22.700 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:22.724 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T02:10:22.751 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:22.756 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:22.782 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:23.076 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T02:10:23.095 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T02:10:23.102 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T02:10:23.121 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T02:10:23.142 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:10:23.161 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:10:23.429 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T02:10:23.455 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T02:10:23.465 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T02:10:23.487 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T02:10:23.491 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T02:10:23.531 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T02:10:23.824 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T02:10:23.850 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:23.870 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T02:10:23.890 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:23.896 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:23.936 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:24.318 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T02:10:24.331 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T02:10:24.345 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:24.358 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:24.385 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:24.397 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:24.948 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T02:10:24.960 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T02:10:24.974 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:24.987 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:25.027 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:25.032 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:26.151 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph. 2024-09-21T02:10:26.154 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph. 2024-09-21T02:10:26.166 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../069-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:26.169 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../069-ceph_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:26.209 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:26.212 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:26.528 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T02:10:26.555 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:26.556 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T02:10:26.583 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:26.603 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:26.622 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:26.948 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T02:10:26.974 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:27.001 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T02:10:27.006 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:27.027 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:27.067 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:27.401 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephadm. 2024-09-21T02:10:27.428 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../072-cephadm_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:27.428 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephadm. 2024-09-21T02:10:27.455 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../072-cephadm_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:27.467 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:27.486 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:27.795 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T02:10:27.797 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T02:10:27.822 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T02:10:27.824 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T02:10:27.864 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T02:10:27.870 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T02:10:28.158 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T02:10:28.185 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:10:28.197 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T02:10:28.216 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:10:28.220 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:10:28.264 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:10:29.196 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-babel. 2024-09-21T02:10:29.223 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:10:29.261 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-babel. 2024-09-21T02:10:29.262 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:10:29.288 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T02:10:29.327 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:10:29.582 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T02:10:29.609 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T02:10:29.648 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:10:29.672 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T02:10:29.699 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T02:10:29.739 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:10:29.985 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T02:10:30.013 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:30.043 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:30.092 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T02:10:30.119 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:30.158 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:30.372 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T02:10:30.398 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T02:10:30.438 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T02:10:30.537 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T02:10:30.564 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T02:10:30.604 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T02:10:30.741 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-routes. 2024-09-21T02:10:30.767 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T02:10:30.807 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:10:31.023 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-routes. 2024-09-21T02:10:31.050 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T02:10:31.090 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:10:31.102 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T02:10:31.115 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:31.152 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:31.427 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T02:10:31.454 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:31.493 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:34.131 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T02:10:34.148 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T02:10:34.181 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:10:34.340 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T02:10:34.357 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T02:10:34.406 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:10:34.776 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T02:10:34.803 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T02:10:34.877 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T02:10:34.893 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T02:10:34.928 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:10:35.013 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:10:35.282 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T02:10:35.310 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T02:10:35.348 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:10:35.584 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T02:10:35.611 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T02:10:35.626 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T02:10:35.650 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:10:35.654 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T02:10:35.692 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:10:35.961 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T02:10:35.988 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T02:10:36.028 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:10:36.334 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T02:10:36.363 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:36.401 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:36.787 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T02:10:36.815 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:36.862 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:37.354 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T02:10:37.382 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T02:10:37.420 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T02:10:37.707 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T02:10:37.734 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T02:10:37.773 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:10:37.857 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T02:10:37.885 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T02:10:37.923 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T02:10:38.110 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T02:10:38.137 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T02:10:38.176 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:10:38.201 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T02:10:38.217 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T02:10:38.251 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:10:38.487 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T02:10:38.515 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-21T02:10:38.530 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T02:10:38.554 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T02:10:38.557 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T02:10:38.597 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:10:38.866 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T02:10:38.874 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T02:10:38.894 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-21T02:10:38.902 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T02:10:38.932 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T02:10:38.940 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T02:10:39.253 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T02:10:39.261 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T02:10:39.280 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T02:10:39.288 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T02:10:39.319 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T02:10:39.327 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:10:39.638 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T02:10:39.648 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T02:10:39.666 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T02:10:39.675 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T02:10:39.705 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T02:10:39.714 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:10:39.976 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T02:10:40.003 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T02:10:40.016 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T02:10:40.034 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T02:10:40.044 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T02:10:40.083 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:10:40.346 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T02:10:40.364 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T02:10:40.404 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:10:40.813 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T02:10:40.842 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:40.880 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:41.207 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T02:10:41.235 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T02:10:41.251 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T02:10:41.274 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:10:41.279 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:41.318 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:41.760 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T02:10:41.788 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:10:41.813 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T02:10:41.835 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:10:41.840 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T02:10:41.879 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:10:42.055 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package jq. 2024-09-21T02:10:42.082 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:10:42.121 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T02:10:42.174 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T02:10:42.203 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:10:42.232 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:10:42.374 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package socat. 2024-09-21T02:10:42.391 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T02:10:42.432 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:10:42.501 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package jq. 2024-09-21T02:10:42.526 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T02:10:42.568 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T02:10:42.743 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T02:10:42.771 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T02:10:42.826 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:10:42.829 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package socat. 2024-09-21T02:10:42.858 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T02:10:42.896 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:10:43.172 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-test. 2024-09-21T02:10:43.189 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../100-ceph-test_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:43.224 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T02:10:43.230 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:43.252 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T02:10:43.291 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:10:43.653 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-test. 2024-09-21T02:10:43.670 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../100-ceph-test_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:43.711 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:45.584 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T02:10:45.613 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:45.651 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:45.800 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T02:10:45.827 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:45.866 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:46.038 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-21T02:10:46.067 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:46.104 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:46.262 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-21T02:10:46.290 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:46.328 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:46.366 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-colorama. 2024-09-21T02:10:46.394 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-21T02:10:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-21T02:10:46.581 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-colorama. 2024-09-21T02:10:46.598 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-21T02:10:46.631 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-21T02:10:46.668 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephfs-shell. 2024-09-21T02:10:46.684 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:46.726 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:46.875 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephfs-shell. 2024-09-21T02:10:46.892 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:46.934 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:47.012 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephfs-top. 2024-09-21T02:10:47.029 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:47.070 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:47.195 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephfs-top. 2024-09-21T02:10:47.223 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-5120-g18432218-1jammy_all.deb ... 2024-09-21T02:10:47.261 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:47.323 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T02:10:47.351 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:47.389 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:47.522 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T02:10:47.539 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:47.581 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:47.700 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T02:10:47.728 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T02:10:47.767 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:10:47.867 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-21T02:10:47.895 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-21T02:10:47.925 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:10:48.063 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T02:10:48.079 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T02:10:48.121 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:10:48.240 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-21T02:10:48.268 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-21T02:10:48.306 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:10:48.483 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T02:10:48.511 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T02:10:48.549 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:10:48.644 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T02:10:48.672 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T02:10:48.710 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:10:48.963 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package pkg-config. 2024-09-21T02:10:48.991 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T02:10:49.029 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:10:49.091 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package pkg-config. 2024-09-21T02:10:49.119 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-21T02:10:49.149 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:10:49.291 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T02:10:49.319 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T02:10:49.357 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:10:49.385 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T02:10:49.401 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T02:10:49.443 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:10:49.769 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T02:10:49.796 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T02:10:49.843 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T02:10:49.854 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T02:10:49.871 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T02:10:49.913 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T02:10:50.130 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T02:10:50.157 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T02:10:50.196 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T02:10:50.216 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T02:10:50.244 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T02:10:50.282 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T02:10:50.516 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T02:10:50.544 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T02:10:50.582 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:10:50.603 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T02:10:50.630 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T02:10:50.670 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:10:50.893 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T02:10:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T02:10:50.960 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T02:10:50.980 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T02:10:51.008 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T02:10:51.047 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T02:10:51.313 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-py. 2024-09-21T02:10:51.341 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-21T02:10:51.379 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T02:10:51.433 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-py. 2024-09-21T02:10:51.461 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-21T02:10:51.508 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T02:10:51.724 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T02:10:51.752 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T02:10:51.790 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:10:51.828 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T02:10:51.856 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T02:10:51.895 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:10:52.261 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T02:10:52.290 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T02:10:52.324 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T02:10:52.327 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:10:52.352 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T02:10:52.382 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:10:52.588 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-toml. 2024-09-21T02:10:52.605 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-21T02:10:52.638 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T02:10:52.668 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-toml. 2024-09-21T02:10:52.696 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-21T02:10:52.735 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T02:10:52.933 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T02:10:53.059 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T02:10:53.060 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T02:10:53.060 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:10:53.066 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T02:10:53.104 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:10:53.336 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T02:10:53.365 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T02:10:53.402 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:10:53.483 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T02:10:53.511 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T02:10:53.549 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:10:53.680 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T02:10:53.709 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T02:10:53.747 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:10:53.819 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T02:10:53.835 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T02:10:53.877 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:10:54.492 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package radosgw. 2024-09-21T02:10:54.520 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../123-radosgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:54.558 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:54.606 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package radosgw. 2024-09-21T02:10:54.635 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../123-radosgw_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:54.664 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:55.711 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T02:10:55.727 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:55.769 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:55.892 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T02:10:55.908 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-5120-g18432218-1jammy_amd64.deb ... 2024-09-21T02:10:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:56.074 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T02:10:56.264 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T02:10:56.324 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:10:56.450 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:10:56.645 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T02:10:56.714 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T02:10:56.907 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T02:10:57.033 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T02:10:57.174 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T02:10:57.469 INFO:teuthology.orchestra.run.smithi191.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T02:10:57.546 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:57.554 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T02:10:58.002 INFO:teuthology.orchestra.run.smithi170.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T02:10:58.067 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:10:58.068 INFO:teuthology.orchestra.run.smithi191.stdout:Adding system user cephadm....done 2024-09-21T02:10:58.224 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:10:58.501 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:10:58.584 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user cephadm....done 2024-09-21T02:10:58.732 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T02:10:58.752 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:10:58.853 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:10:59.016 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T02:10:59.104 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T02:10:59.276 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T02:10:59.377 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T02:10:59.547 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:10:59.620 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T02:10:59.673 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T02:10:59.872 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T02:10:59.998 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T02:10:59.999 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:11:00.299 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T02:11:00.368 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T02:11:00.619 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-21T02:11:00.660 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T02:11:00.870 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T02:11:00.911 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-21T02:11:01.147 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:11:01.162 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T02:11:01.430 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-21T02:11:01.431 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:11:01.674 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:11:01.690 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T02:11:01.950 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T02:11:02.009 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:11:02.210 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T02:11:02.268 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:02.511 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:11:02.544 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-argparse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:02.637 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-21T02:11:02.763 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:11:02.795 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T02:11:02.888 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-21T02:11:02.921 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-21T02:11:03.047 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-21T02:11:03.148 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:11:03.181 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-21T02:11:03.274 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:11:03.441 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-21T02:11:03.558 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:11:03.584 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T02:11:03.843 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-21T02:11:04.003 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T02:11:04.040 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T02:11:04.103 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:11:04.189 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-21T02:11:04.215 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-21T02:11:04.229 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T02:11:04.271 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T02:11:04.372 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T02:11:04.588 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:11:04.714 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T02:11:04.724 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T02:11:04.850 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T02:11:04.990 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:11:05.134 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T02:11:05.258 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:11:05.377 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T02:11:05.601 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T02:11:05.678 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up zip (3.0-12build2) ... 2024-09-21T02:11:05.719 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:11:05.796 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T02:11:06.395 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:11:06.431 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T02:11:06.655 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:11:06.699 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T02:11:06.781 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:11:06.800 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T02:11:06.901 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:11:06.906 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T02:11:07.169 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T02:11:07.207 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T02:11:07.545 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T02:11:07.617 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T02:11:07.796 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:11:07.868 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T02:11:08.165 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:11:08.245 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T02:11:08.449 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-21T02:11:08.538 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-21T02:11:08.742 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:11:08.847 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T02:11:09.102 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:11:09.190 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T02:11:09.353 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:11:09.441 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T02:11:09.479 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:09.567 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:09.780 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:11:09.860 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T02:11:10.132 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:11:10.178 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T02:11:11.341 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:11:11.345 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-21T02:11:11.597 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:11:11.618 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:11:11.740 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T02:11:11.752 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T02:11:12.016 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:11:12.028 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T02:11:12.154 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:11:12.280 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:11:12.377 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T02:11:12.543 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T02:11:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:11:12.807 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T02:11:12.828 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T02:11:13.058 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:11:13.079 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T02:11:13.186 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:11:13.330 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:11:13.465 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T02:11:13.496 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:11:13.755 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T02:11:13.791 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T02:11:14.014 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T02:11:14.051 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T02:11:14.266 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:11:14.293 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T02:11:14.525 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T02:11:14.528 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T02:11:14.779 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T02:11:14.801 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:11:14.919 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T02:11:15.047 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T02:11:15.191 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T02:11:15.211 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:11:15.338 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:11:15.483 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T02:11:15.584 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T02:11:15.639 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:11:15.860 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T02:11:15.940 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:11:16.152 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T02:11:16.182 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:11:16.308 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:11:16.404 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-21T02:11:16.434 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:11:16.524 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T02:11:16.546 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-21T02:11:16.672 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T02:11:16.761 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:11:16.770 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T02:11:16.887 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:11:17.008 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T02:11:17.117 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T02:11:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:11:17.502 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T02:11:17.581 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T02:11:17.707 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T02:11:17.753 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-21T02:11:17.896 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T02:11:17.974 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:11:18.100 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:11:18.155 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T02:11:18.289 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T02:11:18.443 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:11:18.599 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T02:11:18.736 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:11:18.892 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T02:11:19.097 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T02:11:19.223 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T02:11:19.237 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T02:11:19.363 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T02:11:19.491 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T02:11:19.639 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T02:11:19.751 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:11:19.924 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T02:11:20.175 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:11:20.206 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T02:11:20.519 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T02:11:20.587 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T02:11:20.804 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:11:20.905 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T02:11:21.131 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:11:21.241 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T02:11:21.257 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:21.350 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librados2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:21.358 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:21.451 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libsqlite3-mod-ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:21.460 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:11:21.569 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-21T02:11:21.788 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:21.905 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libcephfs2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:21.913 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.039 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libradosstriper1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.040 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T02:11:22.140 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T02:11:22.316 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.408 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librbd1 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.434 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.518 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-modules-core (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.535 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.635 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:22.751 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:11:22.751 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:11:22.849 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:11:22.849 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T02:11:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libcephfs-dev (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.396 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.504 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rados (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.522 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.647 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librgw2 (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.648 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.774 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:11:23.782 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephfs-top (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:23.908 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T02:11:25.011 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.104 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rbd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.137 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.230 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up rbd-fuse (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.263 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.356 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.389 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.482 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cephfs (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.515 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.608 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephfs-shell (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.641 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:25.734 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-common (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:26.126 INFO:teuthology.orchestra.run.smithi170.stdout:Adding group ceph....done 2024-09-21T02:11:26.259 INFO:teuthology.orchestra.run.smithi191.stdout:Adding group ceph....done 2024-09-21T02:11:26.528 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user ceph....done 2024-09-21T02:11:26.619 INFO:teuthology.orchestra.run.smithi191.stdout:Adding system user ceph....done 2024-09-21T02:11:26.696 INFO:teuthology.orchestra.run.smithi170.stdout:Setting system user ceph properties....done 2024-09-21T02:11:26.715 INFO:teuthology.orchestra.run.smithi170.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T02:11:26.788 INFO:teuthology.orchestra.run.smithi191.stdout:Setting system user ceph properties....done 2024-09-21T02:11:26.802 INFO:teuthology.orchestra.run.smithi191.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T02:11:26.850 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T02:11:26.936 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T02:11:27.180 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T02:11:27.249 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T02:11:27.660 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:27.734 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephfs-mirror (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:27.858 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:11:27.858 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:11:27.942 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:11:27.942 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:11:28.328 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:28.385 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-test (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:28.454 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:28.511 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up radosgw (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:28.925 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:11:28.925 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:11:28.968 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:11:28.968 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:11:29.380 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:29.438 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-base (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:29.735 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T02:11:29.781 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T02:11:30.199 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:30.256 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mds (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:30.415 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:11:30.415 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:11:30.478 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:11:30.478 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T02:11:30.884 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:30.950 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:31.094 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:11:31.094 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:11:31.163 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:11:31.163 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T02:11:31.560 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:31.642 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-osd (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:31.837 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:11:31.837 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:11:31.900 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:11:31.900 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T02:11:32.320 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:32.394 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-k8sevents (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:32.447 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:32.520 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:32.598 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:32.663 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mon (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:32.809 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:11:32.809 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:11:32.857 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:11:32.857 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T02:11:33.342 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.389 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-cephadm (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.493 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.540 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.611 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.683 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-dashboard (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.762 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:33.833 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-volume (19.3.0-5120-g18432218-1jammy) ... 2024-09-21T02:11:34.114 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:11:34.178 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:11:34.224 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:11:34.287 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:11:34.376 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:11:34.439 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:11:35.452 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:11:35.607 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:11:36.117 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:11:36.120 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-21T02:11:36.175 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T02:11:36.220 INFO:teuthology.orchestra.run.smithi191.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:11:36.223 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-21T02:11:36.282 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-21T02:11:36.386 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T02:11:36.387 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T02:11:36.487 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-21T02:11:36.487 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-21T02:11:36.598 INFO:teuthology.orchestra.run.smithi170.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:11:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:11:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:11:36.600 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:11:36.600 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T02:11:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-21T02:11:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-21T02:11:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-21T02:11:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-21T02:11:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:11:36.603 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T02:11:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-21T02:11:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-21T02:11:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-21T02:11:36.649 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T02:11:36.650 INFO:teuthology.orchestra.run.smithi170.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-21T02:11:36.650 INFO:teuthology.orchestra.run.smithi170.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-21T02:11:36.650 INFO:teuthology.orchestra.run.smithi170.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-21T02:11:36.650 INFO:teuthology.orchestra.run.smithi170.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-21T02:11:36.650 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-21T02:11:36.651 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-21T02:11:36.651 INFO:teuthology.orchestra.run.smithi170.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:11:36.652 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2024-09-21T02:11:36.653 INFO:teuthology.orchestra.run.smithi170.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-21T02:11:36.672 INFO:teuthology.orchestra.run.smithi191.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-21T02:11:36.672 INFO:teuthology.orchestra.run.smithi191.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-21T02:11:36.673 INFO:teuthology.orchestra.run.smithi191.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-21T02:11:36.673 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T02:11:36.673 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T02:11:36.673 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T02:11:36.673 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T02:11:36.673 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T02:11:36.674 INFO:teuthology.orchestra.run.smithi191.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T02:11:36.674 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T02:11:36.675 INFO:teuthology.orchestra.run.smithi191.stdout:The following additional packages will be installed: 2024-09-21T02:11:36.675 INFO:teuthology.orchestra.run.smithi191.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-21T02:11:36.675 INFO:teuthology.orchestra.run.smithi191.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-21T02:11:36.675 INFO:teuthology.orchestra.run.smithi191.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-21T02:11:36.675 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-21T02:11:36.676 INFO:teuthology.orchestra.run.smithi191.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:11:36.676 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2024-09-21T02:11:36.676 INFO:teuthology.orchestra.run.smithi191.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-21T02:11:36.676 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-21T02:11:36.676 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-21T02:11:36.717 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2024-09-21T02:11:36.717 INFO:teuthology.orchestra.run.smithi191.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-21T02:11:36.718 INFO:teuthology.orchestra.run.smithi191.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-21T02:11:36.719 INFO:teuthology.orchestra.run.smithi191.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-21T02:11:36.719 INFO:teuthology.orchestra.run.smithi191.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-21T02:11:36.719 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-21T02:11:36.719 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-21T02:11:36.719 INFO:teuthology.orchestra.run.smithi191.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-21T02:11:36.720 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages will be upgraded: 2024-09-21T02:11:36.720 INFO:teuthology.orchestra.run.smithi191.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-21T02:11:36.724 INFO:teuthology.orchestra.run.smithi170.stdout:4 upgraded, 39 newly installed, 0 to remove and 328 not upgraded. 2024-09-21T02:11:36.724 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 55.8 MB of archives. 2024-09-21T02:11:36.724 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-21T02:11:36.724 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-21T02:11:36.774 INFO:teuthology.orchestra.run.smithi191.stdout:4 upgraded, 39 newly installed, 0 to remove and 328 not upgraded. 2024-09-21T02:11:36.774 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 55.8 MB of archives. 2024-09-21T02:11:36.774 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-21T02:11:36.775 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-21T02:11:36.856 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-21T02:11:36.864 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-21T02:11:36.902 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-21T02:11:36.903 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-21T02:11:36.904 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-21T02:11:36.908 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-21T02:11:36.914 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-21T02:11:36.915 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-21T02:11:36.916 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-21T02:11:36.917 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-21T02:11:36.919 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-21T02:11:36.919 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-21T02:11:36.921 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-21T02:11:36.922 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-21T02:11:36.937 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-21T02:11:36.938 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-21T02:11:36.939 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-21T02:11:36.957 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-21T02:11:36.958 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-21T02:11:36.958 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-21T02:11:36.969 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-21T02:11:36.969 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-21T02:11:36.972 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-21T02:11:36.973 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-21T02:11:36.973 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-21T02:11:36.975 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-21T02:11:36.976 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-21T02:11:36.992 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-21T02:11:36.993 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-21T02:11:36.994 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-21T02:11:37.255 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-21T02:11:37.257 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-21T02:11:37.257 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.4 [262 kB] 2024-09-21T02:11:37.259 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-21T02:11:37.260 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-21T02:11:37.264 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-21T02:11:37.265 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-21T02:11:37.272 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-21T02:11:37.272 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-21T02:11:37.293 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-21T02:11:37.293 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.4 [1,094 kB] 2024-09-21T02:11:37.303 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-21T02:11:37.308 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-21T02:11:37.309 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-21T02:11:37.309 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.4 [262 kB] 2024-09-21T02:11:37.312 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-21T02:11:37.312 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-21T02:11:37.315 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-21T02:11:37.317 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-21T02:11:37.322 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-21T02:11:37.323 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-21T02:11:37.327 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-21T02:11:37.327 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-21T02:11:37.327 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-21T02:11:37.328 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-21T02:11:37.336 INFO:teuthology.orchestra.run.smithi191.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-21T02:11:37.338 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-21T02:11:37.340 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-21T02:11:37.354 INFO:teuthology.orchestra.run.smithi191.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-21T02:11:37.354 INFO:teuthology.orchestra.run.smithi191.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.4 [1,094 kB] 2024-09-21T02:11:37.363 INFO:teuthology.orchestra.run.smithi191.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-21T02:11:37.379 INFO:teuthology.orchestra.run.smithi191.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-21T02:11:37.381 INFO:teuthology.orchestra.run.smithi191.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-21T02:11:37.382 INFO:teuthology.orchestra.run.smithi191.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-21T02:11:37.382 INFO:teuthology.orchestra.run.smithi191.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-21T02:11:37.383 INFO:teuthology.orchestra.run.smithi191.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-21T02:11:37.392 INFO:teuthology.orchestra.run.smithi191.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-21T02:11:37.393 INFO:teuthology.orchestra.run.smithi191.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-21T02:11:37.543 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-21T02:11:37.544 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-21T02:11:37.544 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-21T02:11:37.544 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-21T02:11:37.544 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-21T02:11:37.544 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-21T02:11:37.556 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-21T02:11:37.556 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-21T02:11:37.574 INFO:teuthology.orchestra.run.smithi191.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-21T02:11:37.574 INFO:teuthology.orchestra.run.smithi191.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-21T02:11:37.574 INFO:teuthology.orchestra.run.smithi191.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-21T02:11:37.574 INFO:teuthology.orchestra.run.smithi191.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-21T02:11:37.574 INFO:teuthology.orchestra.run.smithi191.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-21T02:11:37.575 INFO:teuthology.orchestra.run.smithi191.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-21T02:11:37.585 INFO:teuthology.orchestra.run.smithi191.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-21T02:11:37.585 INFO:teuthology.orchestra.run.smithi191.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-21T02:11:37.969 INFO:teuthology.orchestra.run.smithi170.stdout:Preconfiguring packages ... 2024-09-21T02:11:37.993 INFO:teuthology.orchestra.run.smithi191.stdout:Preconfiguring packages ... 2024-09-21T02:11:38.137 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 55.8 MB in 1s (62.3 MB/s) 2024-09-21T02:11:38.177 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 55.8 MB in 1s (65.0 MB/s) 2024-09-21T02:11:38.458 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:11:38.464 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:11:38.494 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:11:38.500 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:11:38.705 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:11:38.800 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:11:39.391 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-21T02:11:39.417 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-21T02:11:40.319 INFO:teuthology.orchestra.run.smithi170.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-21T02:11:40.372 INFO:teuthology.orchestra.run.smithi191.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-21T02:11:40.591 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package flex. 2024-09-21T02:11:40.685 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:11:40.691 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-21T02:11:40.739 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-21T02:11:40.768 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package flex. 2024-09-21T02:11:40.872 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176357 files and directories currently installed.) 2024-09-21T02:11:40.877 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-21T02:11:40.917 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-21T02:11:41.206 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:11:41.362 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-21T02:11:41.425 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:11:41.597 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-21T02:11:41.763 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:11:42.000 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:11:42.318 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:11:42.323 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:11:42.453 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:11:42.528 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:11:42.533 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:11:42.663 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:11:42.845 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:11:42.984 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:11:43.072 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-21T02:11:43.211 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-21T02:11:43.522 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:11:43.895 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package bison. 2024-09-21T02:11:43.897 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:11:43.998 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:11:44.004 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-21T02:11:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:11:44.271 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package bison. 2024-09-21T02:11:44.376 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176430 files and directories currently installed.) 2024-09-21T02:11:44.382 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-21T02:11:44.440 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:11:44.489 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-21T02:11:44.518 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-21T02:11:44.556 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:11:44.840 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-21T02:11:44.868 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-21T02:11:44.884 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package dump. 2024-09-21T02:11:44.907 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:11:44.912 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-21T02:11:44.950 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking dump (0.4b47-3) ... 2024-09-21T02:11:45.363 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package indent. 2024-09-21T02:11:45.379 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:45.410 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package dump. 2024-09-21T02:11:45.437 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:11:45.438 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-21T02:11:45.477 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking dump (0.4b47-3) ... 2024-09-21T02:11:45.756 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package indent. 2024-09-21T02:11:45.772 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:45.805 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:11:45.841 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-21T02:11:45.858 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-21T02:11:45.899 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:11:46.033 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-21T02:11:46.062 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-21T02:11:46.100 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:11:46.229 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-21T02:11:46.257 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-21T02:11:46.295 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:11:46.429 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-21T02:11:46.456 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-21T02:11:46.495 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:11:46.539 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-21T02:11:46.567 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-21T02:11:46.597 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:11:46.748 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-21T02:11:46.776 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-21T02:11:46.814 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:11:46.901 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-21T02:11:46.929 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-21T02:11:46.967 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:11:47.135 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-21T02:11:47.163 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-21T02:11:47.202 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:11:47.220 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-21T02:11:47.248 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-21T02:11:47.287 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:11:47.479 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-21T02:11:47.507 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-21T02:11:47.539 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libjson-perl. 2024-09-21T02:11:47.545 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:11:47.567 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-21T02:11:47.606 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-21T02:11:47.798 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjson-perl. 2024-09-21T02:11:47.814 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-21T02:11:47.856 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-21T02:11:47.875 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-21T02:11:47.892 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-21T02:11:47.925 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:11:48.143 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-21T02:11:48.171 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-21T02:11:48.201 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:11:49.545 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-21T02:11:49.573 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:11:49.611 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:11:49.873 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-21T02:11:49.892 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:11:49.896 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-21T02:11:49.924 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:11:49.930 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:11:49.963 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:11:50.225 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-21T02:11:50.232 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-21T02:11:50.248 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-21T02:11:50.254 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:11:50.283 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:11:50.290 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:11:50.602 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-21T02:11:50.620 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-21T02:11:50.630 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:11:50.648 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-21T02:11:50.660 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:11:50.910 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:11:51.013 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-21T02:11:51.041 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-21T02:11:51.079 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:11:51.433 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-21T02:11:51.457 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-21T02:11:51.461 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:11:51.485 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:11:51.499 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:11:51.523 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:11:51.835 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-21T02:11:51.844 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-21T02:11:51.862 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:51.872 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:51.892 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:11:51.910 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:11:52.212 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-21T02:11:52.230 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-21T02:11:52.241 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:52.258 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:52.279 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:11:52.297 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:11:52.583 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-21T02:11:52.599 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:52.607 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-21T02:11:52.623 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:52.641 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:11:52.665 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:11:52.894 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-21T02:11:52.909 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-21T02:11:52.922 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:11:52.925 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-21T02:11:52.960 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:11:52.967 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:11:53.446 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-21T02:11:53.455 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-21T02:11:53.474 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-21T02:11:53.484 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-21T02:11:53.504 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:11:53.514 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:11:53.866 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package modemmanager. 2024-09-21T02:11:53.885 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package modemmanager. 2024-09-21T02:11:53.892 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:11:53.914 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.4_amd64.deb ... 2024-09-21T02:11:53.932 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:11:53.951 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:11:54.539 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package network-manager. 2024-09-21T02:11:54.567 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:11:54.614 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:11:54.642 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package network-manager. 2024-09-21T02:11:54.670 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-21T02:11:54.717 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:11:55.229 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ppp. 2024-09-21T02:11:55.257 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-21T02:11:55.341 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ppp. 2024-09-21T02:11:55.369 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-21T02:11:55.540 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:11:55.718 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:11:55.875 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package pptp-linux. 2024-09-21T02:11:55.904 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-21T02:11:55.942 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-21T02:11:56.071 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package pptp-linux. 2024-09-21T02:11:56.099 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-21T02:11:56.145 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-21T02:11:56.203 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-21T02:11:56.232 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-21T02:11:56.270 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-21T02:11:56.448 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-21T02:11:56.476 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-21T02:11:56.523 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-21T02:11:56.774 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-21T02:11:56.802 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-21T02:11:56.840 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql-client-common (238) ... 2024-09-21T02:11:56.843 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-21T02:11:56.871 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-21T02:11:56.910 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql-client-common (238) ... 2024-09-21T02:11:57.144 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-21T02:11:57.173 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:57.210 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:11:57.247 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-21T02:11:57.275 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:57.322 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:11:57.824 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql-common. 2024-09-21T02:11:57.853 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-21T02:11:57.908 INFO:teuthology.orchestra.run.smithi191.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-21T02:11:57.938 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql-common (238) ... 2024-09-21T02:11:57.952 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql-common. 2024-09-21T02:11:57.980 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-21T02:11:58.036 INFO:teuthology.orchestra.run.smithi170.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-21T02:11:58.066 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql-common (238) ... 2024-09-21T02:11:58.389 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql-14. 2024-09-21T02:11:58.418 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:58.459 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:11:58.534 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql-14. 2024-09-21T02:11:58.563 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-21T02:11:58.604 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:11:59.854 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql. 2024-09-21T02:11:59.871 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-21T02:11:59.911 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql (14+238) ... 2024-09-21T02:11:59.996 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql. 2024-09-21T02:12:00.013 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-21T02:12:00.054 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql (14+238) ... 2024-09-21T02:12:00.157 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql-client. 2024-09-21T02:12:00.186 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-21T02:12:00.223 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql-client (14+238) ... 2024-09-21T02:12:00.306 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql-client. 2024-09-21T02:12:00.323 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-21T02:12:00.365 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql-client (14+238) ... 2024-09-21T02:12:00.451 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-21T02:12:00.468 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-21T02:12:00.501 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-21T02:12:00.608 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-21T02:12:00.625 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-21T02:12:00.667 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-21T02:12:00.838 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package tcl8.6. 2024-09-21T02:12:00.866 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-21T02:12:00.904 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:12:01.036 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package tcl8.6. 2024-09-21T02:12:01.065 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-21T02:12:01.103 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:12:01.199 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package tcl. 2024-09-21T02:12:01.216 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-21T02:12:01.262 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-21T02:12:01.364 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package tcl. 2024-09-21T02:12:01.380 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-21T02:12:01.426 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-21T02:12:01.603 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package wpasupplicant. 2024-09-21T02:12:01.632 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-21T02:12:01.670 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:12:01.775 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package wpasupplicant. 2024-09-21T02:12:01.804 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-21T02:12:01.841 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:12:02.034 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-21T02:12:02.051 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-21T02:12:02.092 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-21T02:12:02.179 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-21T02:12:02.208 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-21T02:12:02.246 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-21T02:12:02.455 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-21T02:12:02.484 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-21T02:12:02.530 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:12:02.574 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-21T02:12:02.602 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-21T02:12:02.641 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:12:02.827 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql-client-common (238) ... 2024-09-21T02:12:02.971 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql-client-common (238) ... 2024-09-21T02:12:03.021 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-21T02:12:03.148 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:12:03.180 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-21T02:12:03.299 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:12:03.306 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-21T02:12:03.425 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:12:03.448 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:12:03.560 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:12:03.566 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-21T02:12:03.667 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-21T02:12:03.686 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:12:03.793 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:12:04.145 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-21T02:12:04.211 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:12:04.329 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-21T02:12:04.408 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-21T02:12:04.579 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:12:04.579 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:12:04.764 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:12:04.764 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-21T02:12:05.065 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:12:05.174 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:12:05.269 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-21T02:12:05.301 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:12:05.395 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-21T02:12:05.427 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-21T02:12:05.521 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-21T02:12:05.553 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:12:05.647 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-21T02:12:05.680 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:12:05.773 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-21T02:12:05.815 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:12:05.899 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-21T02:12:05.932 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:12:06.000 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-21T02:12:06.101 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-21T02:12:06.764 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:12:06.890 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-21T02:12:06.922 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up dump (0.4b47-3) ... 2024-09-21T02:12:07.109 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-21T02:12:07.167 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:12:07.233 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up dump (0.4b47-3) ... 2024-09-21T02:12:07.268 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:12:07.319 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-21T02:12:07.385 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:12:07.394 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-21T02:12:07.512 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-21T02:12:07.520 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-21T02:12:07.638 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-21T02:12:07.646 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:12:07.737 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-21T02:12:07.765 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-21T02:12:07.797 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:12:07.883 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-21T02:12:07.923 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:12:07.965 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-21T02:12:08.026 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-21T02:12:08.091 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:12:08.152 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-21T02:12:08.192 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:12:08.293 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-21T02:12:08.329 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-21T02:12:08.461 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql-client (14+238) ... 2024-09-21T02:12:08.464 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:12:08.587 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:12:08.590 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-21T02:12:08.759 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql-client (14+238) ... 2024-09-21T02:12:08.910 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout:which means they are currently configured by ifupdown: 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout:- enp3s0f1 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout:remove their configuration from /etc/network/interfaces. 2024-09-21T02:12:09.065 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:09.181 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout:which means they are currently configured by ifupdown: 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout:- enp3s0f1 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout:remove their configuration from /etc/network/interfaces. 2024-09-21T02:12:09.398 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:09.498 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-21T02:12:09.520 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-21T02:12:09.823 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-21T02:12:09.841 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-21T02:12:10.159 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-21T02:12:10.331 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:12:10.508 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:12:10.626 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-21T02:12:10.672 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-21T02:12:10.735 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql-common (238) ... 2024-09-21T02:12:10.824 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-21T02:12:10.934 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-21T02:12:11.043 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql-common (238) ... 2024-09-21T02:12:11.585 INFO:teuthology.orchestra.run.smithi170.stdout:Adding user postgres to group ssl-cert 2024-09-21T02:12:11.809 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:11.809 INFO:teuthology.orchestra.run.smithi170.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-21T02:12:11.929 INFO:teuthology.orchestra.run.smithi191.stdout:Adding user postgres to group ssl-cert 2024-09-21T02:12:11.992 INFO:teuthology.orchestra.run.smithi170.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-21T02:12:11.993 INFO:teuthology.orchestra.run.smithi170.stdout:Removing obsolete dictionary files: 2024-09-21T02:12:12.174 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:12.174 INFO:teuthology.orchestra.run.smithi191.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-21T02:12:12.372 INFO:teuthology.orchestra.run.smithi191.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-21T02:12:12.372 INFO:teuthology.orchestra.run.smithi191.stdout:Removing obsolete dictionary files: 2024-09-21T02:12:12.554 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-21T02:12:12.945 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:12:12.961 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-21T02:12:13.265 INFO:teuthology.orchestra.run.smithi170.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-21T02:12:13.302 INFO:teuthology.orchestra.run.smithi170.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-21T02:12:13.332 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:This user must also own the server process. 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:The database cluster will be initialized with locale "en_US". 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:The default text search configuration will be set to "english". 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:Data page checksums are disabled. 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-21T02:12:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:creating subdirectories ... ok 2024-09-21T02:12:13.343 INFO:teuthology.orchestra.run.smithi170.stdout:selecting dynamic shared memory implementation ... posix 2024-09-21T02:12:13.359 INFO:teuthology.orchestra.run.smithi170.stdout:selecting default max_connections ... 100 2024-09-21T02:12:13.380 INFO:teuthology.orchestra.run.smithi170.stdout:selecting default shared_buffers ... 128MB 2024-09-21T02:12:13.390 INFO:teuthology.orchestra.run.smithi170.stdout:selecting default time zone ... Etc/UTC 2024-09-21T02:12:13.390 INFO:teuthology.orchestra.run.smithi170.stdout:creating configuration files ... ok 2024-09-21T02:12:13.507 INFO:teuthology.orchestra.run.smithi170.stdout:running bootstrap script ... ok 2024-09-21T02:12:13.653 INFO:teuthology.orchestra.run.smithi191.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-21T02:12:13.669 INFO:teuthology.orchestra.run.smithi191.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:This user must also own the server process. 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:The database cluster will be initialized with locale "en_US". 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:The default text search configuration will be set to "english". 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:Data page checksums are disabled. 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:creating subdirectories ... ok 2024-09-21T02:12:13.719 INFO:teuthology.orchestra.run.smithi191.stdout:selecting dynamic shared memory implementation ... posix 2024-09-21T02:12:13.731 INFO:teuthology.orchestra.run.smithi191.stdout:selecting default max_connections ... 100 2024-09-21T02:12:13.750 INFO:teuthology.orchestra.run.smithi191.stdout:selecting default shared_buffers ... 128MB 2024-09-21T02:12:13.757 INFO:teuthology.orchestra.run.smithi191.stdout:selecting default time zone ... Etc/UTC 2024-09-21T02:12:13.758 INFO:teuthology.orchestra.run.smithi191.stdout:creating configuration files ... ok 2024-09-21T02:12:13.873 INFO:teuthology.orchestra.run.smithi191.stdout:running bootstrap script ... ok 2024-09-21T02:12:14.170 INFO:teuthology.orchestra.run.smithi170.stdout:performing post-bootstrap initialization ... ok 2024-09-21T02:12:14.561 INFO:teuthology.orchestra.run.smithi191.stdout:performing post-bootstrap initialization ... ok 2024-09-21T02:12:15.269 INFO:teuthology.orchestra.run.smithi191.stdout:syncing data to disk ... ok 2024-09-21T02:12:15.315 INFO:teuthology.orchestra.run.smithi170.stdout:syncing data to disk ... ok 2024-09-21T02:12:15.661 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-21T02:12:15.697 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-21T02:12:18.432 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:12:18.452 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:12:18.558 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-21T02:12:18.594 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-21T02:12:18.693 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up postgresql (14+238) ... 2024-09-21T02:12:18.712 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up postgresql (14+238) ... 2024-09-21T02:12:18.794 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:12:18.813 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-21T02:12:18.920 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:12:18.930 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.4) ... 2024-09-21T02:12:19.177 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:12:19.177 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:12:19.185 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:12:19.185 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-21T02:12:19.724 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:12:19.724 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-21T02:12:19.800 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-21T02:12:19.808 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-21T02:12:20.128 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:12:20.136 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T02:12:20.289 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:12:20.296 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T02:12:21.637 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-21T02:12:21.658 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-21T02:12:21.745 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:12:21.749 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-21T02:12:22.204 INFO:teuthology.orchestra.run.smithi191.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:12:22.208 DEBUG:teuthology.parallel:result is None 2024-09-21T02:12:22.266 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T02:12:22.270 DEBUG:teuthology.parallel:result is None 2024-09-21T02:12:22.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:12:22.413 DEBUG:teuthology.orchestra.run.smithi170:> dpkg-query -W -f '${Version}' ceph 2024-09-21T02:12:22.431 INFO:teuthology.orchestra.run.smithi170.stdout:19.3.0-5120-g18432218-1jammy 2024-09-21T02:12:22.432 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:12:22.432 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120-g18432218-1jammy is installed. 2024-09-21T02:12:22.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:12:22.565 DEBUG:teuthology.orchestra.run.smithi191:> dpkg-query -W -f '${Version}' ceph 2024-09-21T02:12:22.584 INFO:teuthology.orchestra.run.smithi191.stdout:19.3.0-5120-g18432218-1jammy 2024-09-21T02:12:22.584 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120-g18432218-1jammy 2024-09-21T02:12:22.585 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120-g18432218-1jammy is installed. 2024-09-21T02:12:22.586 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T02:12:22.587 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:22.587 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:12:22.601 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:22.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:12:22.640 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T02:12:22.640 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:22.640 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:12:22.656 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:12:22.710 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:22.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:12:22.724 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:12:22.778 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T02:12:22.779 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:22.779 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:12:22.792 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:12:22.846 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:22.847 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:12:22.859 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:12:22.914 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T02:12:22.914 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:22.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:12:22.928 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:12:22.983 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:22.983 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:12:22.996 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:12:23.050 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T02:12:23.124 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T02:12:23.124 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2024-09-21T02:12:23.126 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2024-09-21T02:12:23.138 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T02:12:23.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T02:12:23.181 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T02:12:23.194 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T02:12:23.195 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T02:12:23.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T02:12:23.254 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T02:12:23.254 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T02:12:23.254 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T02:12:23.255 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:23.255 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T02:12:23.292 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:23.366 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:23.367 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T02:12:23.372 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:23.451 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T02:12:23.451 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T02:12:23.452 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913409', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJlILFpLgG9S3GWq0Ng/yZ+JmIXVebw+08zEikgf3Xc4S9cqFlQSoiNsKIDweVwBpI4G/bx/2SdHBuBYzGguHsY=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaV1+dBiNRWxho5lhJmqrQEXx5I7IHF9ub/o3sl8ooRLE5iMay/PfRAYAG7VwqyHQTJ/UVLQHgRmvkXAL7Jd3o='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T02:12:23.452 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T02:12:23.455 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T02:12:23.492 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T02:12:23.501 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T02:12:23.548 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:23.548 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:12:23.560 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:12:23.560 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-09-21T02:12:23.608 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:12:23.608 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:23.608 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-09-21T02:12:23.608 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:23.608 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 02:06:06.588902442 +0000 2024-09-21T02:12:23.609 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 02:06:06.344904621 +0000 2024-09-21T02:12:23.609 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 02:06:06.344904621 +0000 2024-09-21T02:12:23.609 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T02:12:23.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:12:23.662 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:12:23.662 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:12:23.662 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000325514 s, 1.6 MB/s 2024-09-21T02:12:23.663 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:12:23.713 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-09-21T02:12:23.759 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 02:06:06.824900334 +0000 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 02:06:06.820900370 +0000 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 02:06:06.820900370 +0000 2024-09-21T02:12:23.760 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T02:12:23.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:12:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:12:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:12:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000320761 s, 1.6 MB/s 2024-09-21T02:12:23.815 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:12:23.865 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-09-21T02:12:23.911 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 02:06:07.540893939 +0000 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 02:06:07.296896118 +0000 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 02:06:07.296896118 +0000 2024-09-21T02:12:23.912 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T02:12:23.912 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:12:23.965 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:12:23.966 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:12:23.966 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000234022 s, 2.2 MB/s 2024-09-21T02:12:23.967 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:12:24.017 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 02:06:08.032889545 +0000 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 02:06:07.780891796 +0000 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 02:06:07.780891796 +0000 2024-09-21T02:12:24.064 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T02:12:24.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:12:24.118 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:12:24.118 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:12:24.118 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000325534 s, 1.6 MB/s 2024-09-21T02:12:24.119 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:12:24.169 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:24.169 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:24.169 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:12:24.175 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:12:24.175 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-09-21T02:12:24.224 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:12:24.224 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:24.224 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-09-21T02:12:24.224 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:24.225 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-21 02:06:03.317911458 +0000 2024-09-21T02:12:24.225 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-21 02:06:03.061913746 +0000 2024-09-21T02:12:24.225 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-21 02:06:03.061913746 +0000 2024-09-21T02:12:24.225 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-21T02:12:24.225 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:12:24.278 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-21T02:12:24.278 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-21T02:12:24.278 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000458347 s, 1.1 MB/s 2024-09-21T02:12:24.280 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:12:24.329 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-09-21T02:12:24.375 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:12:24.375 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:24.376 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-09-21T02:12:24.376 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:24.376 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-21 02:06:03.821906957 +0000 2024-09-21T02:12:24.376 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-21 02:06:03.561909279 +0000 2024-09-21T02:12:24.376 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-21 02:06:03.561909279 +0000 2024-09-21T02:12:24.376 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-21T02:12:24.376 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:12:24.429 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-21T02:12:24.429 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-21T02:12:24.429 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000333215 s, 1.5 MB/s 2024-09-21T02:12:24.430 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:12:24.477 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-21 02:06:04.357902169 +0000 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-21 02:06:04.085904599 +0000 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-21 02:06:04.085904599 +0000 2024-09-21T02:12:24.524 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-21T02:12:24.525 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:12:24.578 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-21T02:12:24.578 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-21T02:12:24.578 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000320162 s, 1.6 MB/s 2024-09-21T02:12:24.579 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:12:24.628 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-21 02:06:04.849897774 +0000 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-21 02:06:04.573900239 +0000 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-21 02:06:04.573900239 +0000 2024-09-21T02:12:24.676 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-21T02:12:24.677 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:12:24.730 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-21T02:12:24.730 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-21T02:12:24.731 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00038618 s, 1.3 MB/s 2024-09-21T02:12:24.732 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:12:24.781 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:24.781 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T02:12:24.781 INFO:tasks.ceph:Generating config... 2024-09-21T02:12:24.782 INFO:tasks.ceph:[client] pid file = 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T02:12:24.783 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T02:12:24.783 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T02:12:24.783 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T02:12:24.783 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T02:12:24.783 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T02:12:24.783 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T02:12:24.784 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T02:12:24.784 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T02:12:24.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T02:12:24.849 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T02:12:24.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T02:12:24.932 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:12:24.943 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.170'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2024-09-21T02:12:24.944 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.170,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T02:12:25.014 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:25.014 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T02:12:25.019 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.170 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:25.135 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:25.135 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: generated fsid 75527983-1c6c-4a94-aa07-cb5c10ea5972 2024-09-21T02:12:25.135 INFO:teuthology.orchestra.run.smithi170.stdout:setting min_mon_release = quincy 2024-09-21T02:12:25.135 INFO:teuthology.orchestra.run.smithi170.stdout:epoch 0 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:fsid 75527983-1c6c-4a94-aa07-cb5c10ea5972 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:last_changed 2024-09-21T02:12:25.138857+0000 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:created 2024-09-21T02:12:25.138857+0000 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:min_mon_release 17 (quincy) 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:election_strategy: 1 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:0: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.a 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2024-09-21T02:12:25.136 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T02:12:25.140 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T02:12:25.145 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 75527983-1c6c-4a94-aa07-cb5c10ea5972... 2024-09-21T02:12:25.146 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T02:12:25.189 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2024-09-21T02:12:25.216 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2024-09-21T02:12:25.217 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2024-09-21T02:12:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.170,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 75527983-1c6c-4a94-aa07-cb5c10ea5972 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2024-09-21T02:12:25.219 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug trim objects = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.220 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore allocator = bitmap 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2024-09-21T02:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2024-09-21T02:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2024-09-21T02:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: debug finisher = 20 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2024-09-21T02:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.225 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: mon down mkfs grace = 300 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:12:25.226 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = "" 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2024-09-21T02:12:25.227 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2024-09-21T02:12:25.228 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2024-09-21T02:12:25.228 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2024-09-21T02:12:25.228 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2024-09-21T02:12:25.228 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.229 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.230 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2024-09-21T02:12:25.231 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2024-09-21T02:12:25.232 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.170,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 75527983-1c6c-4a94-aa07-cb5c10ea5972 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2024-09-21T02:12:25.233 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.234 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug trim objects = true 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2024-09-21T02:12:25.235 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2024-09-21T02:12:25.236 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:12:25.237 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: debug finisher = 20 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2024-09-21T02:12:25.238 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: mon down mkfs grace = 300 2024-09-21T02:12:25.239 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = "" 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-09-21T02:12:25.240 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2024-09-21T02:12:25.242 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T02:12:25.242 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T02:12:25.363 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T02:12:25.363 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:25.364 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T02:12:25.369 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:25.369 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T02:12:25.416 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:12:25.416 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:25.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T02:12:25.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:12:25.480 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:25.480 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:25.528 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:12:25.528 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:25.528 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T02:12:25.528 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T02:12:25.552 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:25.552 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:25.599 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T02:12:25.600 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T02:12:25.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T02:12:25.673 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T02:12:25.680 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T02:12:25.754 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T02:12:25.779 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T02:12:25.780 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T02:12:25.855 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T02:12:25.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T02:12:25.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T02:12:25.981 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T02:12:25.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T02:12:25.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T02:12:26.072 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T02:12:26.079 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T02:12:26.092 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T02:12:26.203 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T02:12:26.210 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T02:12:26.220 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T02:12:26.220 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T02:12:26.287 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T02:12:26.302 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T02:12:26.362 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T02:12:26.375 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T02:12:26.440 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T02:12:26.455 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T02:12:26.509 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T02:12:26.523 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T02:12:26.523 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T02:12:26.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T02:12:26.537 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:26.537 INFO:tasks.ceph:role: osd.0 2024-09-21T02:12:26.537 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:12:26.537 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T02:12:26.591 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:26.592 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:26.618 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T02:12:26.619 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T02:12:26.620 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T02:12:26.678 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T02:12:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:26.729 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:26.729 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T02:12:26.783 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:26.783 INFO:tasks.ceph:role: osd.1 2024-09-21T02:12:26.783 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:12:26.783 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:26.840 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:26.866 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T02:12:26.867 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T02:12:26.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T02:12:26.931 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T02:12:26.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:26.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T02:12:27.036 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:27.036 INFO:tasks.ceph:role: osd.2 2024-09-21T02:12:27.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:12:27.037 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T02:12:27.091 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:27.091 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:27.091 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:27.091 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:27.091 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:27.091 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:27.092 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:27.092 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:27.092 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:27.092 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:27.118 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T02:12:27.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T02:12:27.119 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T02:12:27.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T02:12:27.235 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:27.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:27.237 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T02:12:27.289 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:27.289 INFO:tasks.ceph:role: osd.3 2024-09-21T02:12:27.289 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:12:27.289 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:27.343 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:27.369 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T02:12:27.370 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T02:12:27.370 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T02:12:27.426 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T02:12:27.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:27.480 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:27.480 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:27.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:27.621+0000 7fcff98d97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T02:12:27.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:27.621+0000 7fcff98d97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T02:12:27.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:27.621+0000 7fcff98d97c0 -1 bdev(0x55cccc3a4a00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:27.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:27.621+0000 7fcff98d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T02:12:30.769 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T02:12:30.782 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:30.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:30.893+0000 7f9c1712f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T02:12:30.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:30.893+0000 7f9c1712f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T02:12:30.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:30.893+0000 7f9c1712f7c0 -1 bdev(0x55ebab410a00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:30.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:30.893+0000 7f9c1712f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T02:12:34.286 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T02:12:34.301 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:34.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:34.413+0000 7f4a49f3a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T02:12:34.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:34.413+0000 7f4a49f3a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T02:12:34.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:34.413+0000 7f4a49f3a7c0 -1 bdev(0x557a36388a00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:34.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:34.413+0000 7f4a49f3a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T02:12:37.563 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T02:12:37.576 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:37.689+0000 7fea1d40f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T02:12:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:37.689+0000 7fea1d40f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T02:12:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:37.689+0000 7fea1d40f7c0 -1 bdev(0x564185990a00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:12:37.693+0000 7fea1d40f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T02:12:40.839 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T02:12:40.853 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T02:12:40.868 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:40.868 INFO:tasks.ceph:role: osd.4 2024-09-21T02:12:40.868 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:12:40.868 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:40.931 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:41.383 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-21T02:12:41.386 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-21T02:12:41.386 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T02:12:41.491 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T02:12:41.502 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:41.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:41.503 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T02:12:41.559 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:41.559 INFO:tasks.ceph:role: osd.5 2024-09-21T02:12:41.559 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:12:41.559 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T02:12:41.623 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:41.624 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:42.097 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-21T02:12:42.100 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-21T02:12:42.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T02:12:42.263 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T02:12:42.273 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:42.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:42.274 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T02:12:42.327 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:42.327 INFO:tasks.ceph:role: osd.6 2024-09-21T02:12:42.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:12:42.327 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T02:12:42.391 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:42.391 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:42.391 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:42.392 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:42.863 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-21T02:12:42.865 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-21T02:12:42.866 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T02:12:43.008 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T02:12:43.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:43.019 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:43.019 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T02:12:43.074 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T02:12:43.074 INFO:tasks.ceph:role: osd.7 2024-09-21T02:12:43.074 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:12:43.074 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T02:12:43.135 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T02:12:43.135 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T02:12:43.135 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T02:12:43.135 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T02:12:43.136 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T02:12:43.136 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-21T02:12:43.136 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T02:12:43.136 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T02:12:43.136 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T02:12:43.136 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T02:12:43.566 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-21T02:12:43.569 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-21T02:12:43.569 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T02:12:43.664 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T02:12:43.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:12:43.674 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T02:12:43.674 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:43.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:43.815+0000 7f2c266c87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T02:12:43.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:43.819+0000 7f2c266c87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T02:12:43.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:43.819+0000 7f2c266c87c0 -1 bdev(0x55b7c8f88a00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:43.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:43.819+0000 7f2c266c87c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T02:12:46.983 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T02:12:46.997 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:47.107 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:47.107+0000 7f4e8e2867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T02:12:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:47.107+0000 7f4e8e2867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T02:12:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:47.107+0000 7f4e8e2867c0 -1 bdev(0x55dddc8f6a00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:47.108 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:47.107+0000 7f4e8e2867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T02:12:50.254 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T02:12:50.267 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:50.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:50.379+0000 7fd519e677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T02:12:50.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:50.379+0000 7fd519e677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T02:12:50.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:50.379+0000 7fd519e677c0 -1 bdev(0x558a5bf88a00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:50.382 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:50.379+0000 7fd519e677c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T02:12:53.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:53.456 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:12:53.561 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T02:12:53.574 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:53.683 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:53.683+0000 7f42e2b7e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T02:12:53.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:53.683+0000 7f42e2b7e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T02:12:53.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:53.683+0000 7f42e2b7e7c0 -1 bdev(0x5558ea5bea00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T02:12:53.684 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:12:53.683+0000 7f42e2b7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T02:12:56.820 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T02:12:56.834 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T02:12:56.834 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:56.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T02:12:56.850 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:56.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T02:12:56.902 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:56.902 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T02:12:56.959 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:56.959 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T02:12:57.014 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:57.014 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T02:12:57.070 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:57.070 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T02:12:57.122 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:57.122 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T02:12:57.174 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T02:12:57.187 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.187 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T02:12:57.243 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.243 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T02:12:57.298 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.299 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T02:12:57.354 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T02:12:57.410 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.410 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T02:12:57.462 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T02:12:57.519 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:57.519 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T02:12:57.526 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:12:57.526 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T02:12:57.575 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.575 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T02:12:57.581 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:12:57.581 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T02:12:57.627 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T02:12:57.627 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T02:12:57.630 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T02:12:57.673 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.y] 2024-09-21T02:12:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg== 2024-09-21T02:12:57.673 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2024-09-21T02:12:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.3] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA== 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2024-09-21T02:12:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.4] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.5] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.6] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.7] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[client.2] 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ== 2024-09-21T02:12:57.675 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2024-09-21T02:12:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA== 2024-09-21T02:12:57.676 INFO:teuthology.orchestra.run.smithi170.stdout:[client.3] 2024-09-21T02:12:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA== 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg== 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg== 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A== 2024-09-21T02:12:57.681 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g== 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A== 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA== 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg== 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA== 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA== 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2024-09-21T02:12:57.682 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA== 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout:[client.3] 2024-09-21T02:12:57.683 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA== 2024-09-21T02:12:57.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:12:57.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:12:57.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:57.807 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:57.881 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:57.883 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:57.960 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:57.962 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.113 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.115 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.183 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.253 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:12:58.255 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T02:12:58.322 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:58.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:58.396 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:58.398 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T02:12:58.469 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.539 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.542 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.612 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.614 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.683 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T02:12:58.751 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.754 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.825 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.827 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.899 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.901 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.971 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:58.973 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T02:12:59.040 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T02:12:59.040 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T02:12:59.049 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:12:59.159 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T02:12:59.169 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T02:12:59.180 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:12:59.301 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T02:12:59.311 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T02:12:59.365 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T02:12:59.489 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T02:12:59.499 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:59.502 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T02:12:59.546 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T02:12:59.546 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T02:12:59.547 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T02:12:59.549 INFO:tasks.ceph.mon.a:Started 2024-09-21T02:12:59.549 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T02:12:59.549 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T02:12:59.588 INFO:tasks.ceph.mon.b:Started 2024-09-21T02:12:59.588 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T02:12:59.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T02:12:59.591 INFO:tasks.ceph.mon.c:Started 2024-09-21T02:12:59.591 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T02:12:59.591 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T02:12:59.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T02:12:59.594 INFO:tasks.ceph.mgr.y:Started 2024-09-21T02:12:59.594 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T02:12:59.594 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T02:12:59.597 INFO:tasks.ceph.mgr.x:Started 2024-09-21T02:12:59.597 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T02:12:59.597 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T02:12:59.599 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T02:12:59.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T02:12:59.643 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T02:12:59.644 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T02:12:59.691 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T02:12:59.691 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T02:12:59.739 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T02:12:59.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T02:12:59.783 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T02:12:59.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T02:12:59.831 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T02:12:59.831 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T02:12:59.875 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T02:12:59.875 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T02:12:59.919 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T02:12:59.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T02:12:59.967 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T02:12:59.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T02:13:00.015 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T02:13:00.015 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T02:13:00.059 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T02:13:00.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T02:13:00.103 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T02:13:00.104 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T02:13:00.147 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T02:13:00.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T02:13:00.196 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T02:13:00.197 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T02:13:00.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 -- 172.21.15.170:0/264786013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f188c005360 msgr2=0x7f18a00c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 --2- 172.21.15.170:0/264786013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f188c005360 0x7f18a00c8350 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f189c003f30 tx=0x7f189c00f7e0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.201 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T02:13:00.202 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 -- 172.21.15.170:0/264786013 shutdown_connections 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 --2- 172.21.15.170:0/264786013 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18a01f79c0 0x7f18a0070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 --2- 172.21.15.170:0/264786013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18a00710b0 0x7f18a0160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 --2- 172.21.15.170:0/264786013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f188c005360 0x7f18a00c8350 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 -- 172.21.15.170:0/264786013 >> 172.21.15.170:0/264786013 conn(0x7f18a009a9c0 msgr2=0x7f18a00716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 -- 172.21.15.170:0/264786013 shutdown_connections 2024-09-21T02:13:00.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.201+0000 7f18a7f35640 1 -- 172.21.15.170:0/264786013 wait complete. 2024-09-21T02:13:00.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.205+0000 7fa6108c4640 1 -- 172.21.15.170:0/1204217596 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fa60c06ce00 con 0x7fa60c0716e0 2024-09-21T02:13:00.205 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T02:13:00.205 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T02:13:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.209+0000 7f029ffff640 1 -- 172.21.15.170:0/836262724 <== mon.0 v2:172.21.15.170:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7f02a8011a80 con 0x7f02a0004810 2024-09-21T02:13:00.210 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T02:13:00.210 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T02:13:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7fb5997fa640 1 -- 172.21.15.170:0/363496 <== mon.2 v2:172.21.15.191:3301/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7fb5880c2450 con 0x7fb59c15fd30 2024-09-21T02:13:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7fa5faffd640 1 -- 172.21.15.170:0/1204217596 <== mon.2 v2:172.21.15.191:3301/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7fa600010b50 con 0x7fa60c0716e0 2024-09-21T02:13:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7fd02a7fc640 1 -- 172.21.15.170:0/1616890812 <== mon.2 v2:172.21.15.191:3301/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7fd030012ce0 con 0x7fd020004d50 2024-09-21T02:13:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4120982309 <== mon.2 v2:172.21.15.191:3301/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7fe6d0003ad0 con 0x7fe6e0157140 2024-09-21T02:13:00.214 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T02:13:00.214 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T02:13:00.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7fa5faffd640 1 -- 172.21.15.170:0/1204217596 <== mon.2 v2:172.21.15.191:3301/0 9 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v3)=0 v3) ==== 114+0+0 (secure 0 0 0) 0x7fa600025480 con 0x7fa60c0716e0 2024-09-21T02:13:00.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7f70f8ff9640 1 -- 172.21.15.170:0/3778267542 <== mon.1 v2:172.21.15.191:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7f70ec010890 con 0x7f70fc070f90 2024-09-21T02:13:00.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7f09a4ff9640 1 -- 172.21.15.170:0/2529825059 <== mon.1 v2:172.21.15.191:3300/0 8 ==== config(2 keys) ==== 48+0+0 (secure 0 0 0) 0x7f099c025430 con 0x7f09a80b4270 2024-09-21T02:13:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7f79bce95640 1 -- 172.21.15.170:0/3091133169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b80cf750 con 0x7f79b8162a60 2024-09-21T02:13:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.213+0000 7f79bce95640 1 -- 172.21.15.170:0/3091133169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b8147a40 con 0x7f79b8162a60 2024-09-21T02:13:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.217+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 5 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f79a800dca0 con 0x7f79b8162a60 2024-09-21T02:13:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.217+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 6 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f79a8013950 con 0x7f79b8162a60 2024-09-21T02:13:00.217 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T02:13:00.217 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T02:13:00.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7f79bce95640 1 -- 172.21.15.170:0/3091133169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b815a1a0 con 0x7f79b8162a60 2024-09-21T02:13:00.228 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T02:13:00.228 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 -- 172.21.15.170:0/1204217596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa60c0716e0 msgr2=0x7fa60c1590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 --2- 172.21.15.170:0/1204217596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa60c0716e0 0x7fa60c1590f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c5b0 tx=0x7fa60000ca70 comp rx=0 tx=0).stop 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 -- 172.21.15.170:0/1204217596 shutdown_connections 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 --2- 172.21.15.170:0/1204217596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa60c0716e0 0x7fa60c1590f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 --2- 172.21.15.170:0/1204217596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa60c159630 0x7fa60c0cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 --2- 172.21.15.170:0/1204217596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f40051b0 0x7fa60c0c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 -- 172.21.15.170:0/1204217596 >> 172.21.15.170:0/1204217596 conn(0x7fa60c09a9c0 msgr2=0x7fa60c0b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 -- 172.21.15.170:0/1204217596 shutdown_connections 2024-09-21T02:13:00.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.229+0000 7fa6108c4640 1 -- 172.21.15.170:0/1204217596 wait complete. 2024-09-21T02:13:00.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.233+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f79a8013ba0 con 0x7f79b8162a60 2024-09-21T02:13:00.232 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T02:13:00.232 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T02:13:00.275 DEBUG:tasks.ceph:set 22 configs 2024-09-21T02:13:00.275 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.277+0000 7fb5a2f45640 1 -- 172.21.15.170:0/363496 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fb574005110 con 0x7fb59c15fd30 2024-09-21T02:13:00.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.281+0000 7f029ffff640 1 -- 172.21.15.170:0/836262724 <== mon.0 v2:172.21.15.170:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f02a8011c70 con 0x7f02a0004810 2024-09-21T02:13:00.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.281+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 8 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f79a8013d80 con 0x7f79b8162a60 2024-09-21T02:13:00.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.285+0000 7fb5997fa640 1 -- 172.21.15.170:0/363496 <== mon.2 v2:172.21.15.191:3301/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fb588004fc0 con 0x7fb59c15fd30 2024-09-21T02:13:00.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.285+0000 7fb5997fa640 1 -- 172.21.15.170:0/363496 <== mon.2 v2:172.21.15.191:3301/0 10 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v4)=0 v4) ==== 123+0+0 (secure 0 0 0) 0x7fb588016eb0 con 0x7fb59c15fd30 2024-09-21T02:13:00.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.285+0000 7fd02a7fc640 1 -- 172.21.15.170:0/1616890812 <== mon.2 v2:172.21.15.191:3301/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fd0300113e0 con 0x7fd020004d50 2024-09-21T02:13:00.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.285+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4120982309 <== mon.2 v2:172.21.15.191:3301/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fe6d0003d20 con 0x7fe6e0157140 2024-09-21T02:13:00.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.285+0000 7f09a4ff9640 1 -- 172.21.15.170:0/2529825059 <== mon.1 v2:172.21.15.191:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f099c025630 con 0x7f09a80b4270 2024-09-21T02:13:00.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.289+0000 7f70f8ff9640 1 -- 172.21.15.170:0/3778267542 <== mon.1 v2:172.21.15.191:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f70ec010a80 con 0x7f70fc070f90 2024-09-21T02:13:00.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.301+0000 7fd01ffff640 1 -- 172.21.15.170:0/1616890812 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fd00c005110 con 0x7fd020004d50 2024-09-21T02:13:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.305+0000 7fb5a2f45640 1 -- 172.21.15.170:0/363496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb59c15fd30 msgr2=0x7fb59c0b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.305+0000 7fb5a2f45640 1 --2- 172.21.15.170:0/363496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb59c15fd30 0x7fb59c0b1460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb5880b0040 tx=0x7fb58c005840 comp rx=0 tx=0).stop 2024-09-21T02:13:00.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.305+0000 7f029ffff640 1 -- 172.21.15.170:0/836262724 <== mon.0 v2:172.21.15.170:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f02a801a380 con 0x7f02a0004810 2024-09-21T02:13:00.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.305+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 9 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f79a8006720 con 0x7f79b8162a60 2024-09-21T02:13:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fd02a7fc640 1 -- 172.21.15.170:0/1616890812 <== mon.2 v2:172.21.15.191:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd030011610 con 0x7fd020004d50 2024-09-21T02:13:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fd02a7fc640 1 -- 172.21.15.170:0/1616890812 <== mon.2 v2:172.21.15.191:3301/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7fd0300118c0 con 0x7fd020004d50 2024-09-21T02:13:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4120982309 <== mon.2 v2:172.21.15.191:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fe6d0011440 con 0x7fe6e0157140 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 -- 172.21.15.170:0/363496 shutdown_connections 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 --2- 172.21.15.170:0/363496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb59c15fd30 0x7fb59c0b1460 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 --2- 172.21.15.170:0/363496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb59c1f4c60 0x7fb59c0b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 --2- 172.21.15.170:0/363496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb58c004810 0x7fb59c0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 -- 172.21.15.170:0/363496 >> 172.21.15.170:0/363496 conn(0x7fb59c09a9c0 msgr2=0x7fb59c0d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 -- 172.21.15.170:0/363496 shutdown_connections 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7fb5a2f45640 1 -- 172.21.15.170:0/363496 wait complete. 2024-09-21T02:13:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7f70f8ff9640 1 -- 172.21.15.170:0/3778267542 <== mon.1 v2:172.21.15.191:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f70ec00f3e0 con 0x7f70fc070f90 2024-09-21T02:13:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.309+0000 7f09a4ff9640 1 -- 172.21.15.170:0/2529825059 <== mon.1 v2:172.21.15.191:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f099c025940 con 0x7f09a80b4270 2024-09-21T02:13:00.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.321+0000 7fd01ffff640 1 -- 172.21.15.170:0/1616890812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd020004d50 msgr2=0x7fd0340ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.321+0000 7fd01ffff640 1 --2- 172.21.15.170:0/1616890812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd020004d50 0x7fd0340ca990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd030000f70 tx=0x7fd03000c630 comp rx=0 tx=0).stop 2024-09-21T02:13:00.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.321+0000 7fd01ffff640 1 -- 172.21.15.170:0/1616890812 shutdown_connections 2024-09-21T02:13:00.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.321+0000 7fd01ffff640 1 --2- 172.21.15.170:0/1616890812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd034206610 0x7fd0340b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.321+0000 7fd01ffff640 1 --2- 172.21.15.170:0/1616890812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0340b75c0 0x7fd0342060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.321+0000 7fd01ffff640 1 --2- 172.21.15.170:0/1616890812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd020004d50 0x7fd0340ca990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.325+0000 7fd01ffff640 1 -- 172.21.15.170:0/1616890812 >> 172.21.15.170:0/1616890812 conn(0x7fd0340d2a30 msgr2=0x7fd034160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.325+0000 7fd01ffff640 1 -- 172.21.15.170:0/1616890812 shutdown_connections 2024-09-21T02:13:00.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.325+0000 7fd01ffff640 1 -- 172.21.15.170:0/1616890812 wait complete. 2024-09-21T02:13:00.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.329+0000 7fe6e52d9640 1 -- 172.21.15.170:0/4120982309 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fe6b80050f0 con 0x7fe6e0157140 2024-09-21T02:13:00.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.337+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4120982309 <== mon.2 v2:172.21.15.191:3301/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v6)=0 v6) ==== 108+0+0 (secure 0 0 0) 0x7fe6d0011740 con 0x7fe6e0157140 2024-09-21T02:13:00.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.349+0000 7f02bdc66640 1 -- 172.21.15.170:0/836262724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f0290005110 con 0x7f02a0004810 2024-09-21T02:13:00.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.353+0000 7f029ffff640 1 -- 172.21.15.170:0/836262724 <== mon.0 v2:172.21.15.170:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v7)=0 v7) ==== 123+0+0 (secure 0 0 0) 0x7f02b81ffe70 con 0x7f02a0004810 2024-09-21T02:13:00.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.353+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79a8006a20 con 0x7f79b8162a60 2024-09-21T02:13:00.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.353+0000 7f70f8ff9640 1 -- 172.21.15.170:0/3778267542 <== mon.1 v2:172.21.15.191:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70ec00f760 con 0x7f70fc070f90 2024-09-21T02:13:00.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.353+0000 7f09a4ff9640 1 -- 172.21.15.170:0/2529825059 <== mon.1 v2:172.21.15.191:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f099c025da0 con 0x7f09a80b4270 2024-09-21T02:13:00.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.353+0000 7f029ffff640 1 -- 172.21.15.170:0/836262724 <== mon.0 v2:172.21.15.170:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02a8025a50 con 0x7f02a0004810 2024-09-21T02:13:00.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.357+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4120982309 <== mon.2 v2:172.21.15.191:3301/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6d0011a20 con 0x7fe6e0157140 2024-09-21T02:13:00.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 -- 172.21.15.170:0/4120982309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6e0157140 msgr2=0x7fe6e0154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 --2- 172.21.15.170:0/4120982309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6e0157140 0x7fe6e0154010 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005fd0 tx=0x7fe6d0005410 comp rx=0 tx=0).stop 2024-09-21T02:13:00.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 -- 172.21.15.170:0/4120982309 shutdown_connections 2024-09-21T02:13:00.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 --2- 172.21.15.170:0/4120982309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e015e870 0x7fe6e0155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 --2- 172.21.15.170:0/4120982309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6e0156580 0x7fe6e015e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 --2- 172.21.15.170:0/4120982309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6e0157140 0x7fe6e0154010 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 -- 172.21.15.170:0/4120982309 >> 172.21.15.170:0/4120982309 conn(0x7fe6e009a9c0 msgr2=0x7fe6e00b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 -- 172.21.15.170:0/4120982309 shutdown_connections 2024-09-21T02:13:00.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7fe6e52d9640 1 -- 172.21.15.170:0/4120982309 wait complete. 2024-09-21T02:13:00.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.361+0000 7f02bdc66640 1 -- 172.21.15.170:0/836262724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02a0004810 msgr2=0x7f02b815f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.365+0000 7f02bdc66640 1 --2- 172.21.15.170:0/836262724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02a0004810 0x7f02b815f160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f02a80109a0 tx=0x7f02a8010e60 comp rx=0 tx=0).stop 2024-09-21T02:13:00.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 -- 172.21.15.170:0/836262724 shutdown_connections 2024-09-21T02:13:00.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 --2- 172.21.15.170:0/836262724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02b81ffa60 0x7f02b80b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 --2- 172.21.15.170:0/836262724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02b815f8e0 0x7f02b81ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 --2- 172.21.15.170:0/836262724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02a0004810 0x7f02b815f160 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 -- 172.21.15.170:0/836262724 >> 172.21.15.170:0/836262724 conn(0x7f02b80cd5d0 msgr2=0x7f02b80b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 -- 172.21.15.170:0/836262724 shutdown_connections 2024-09-21T02:13:00.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.377+0000 7f02bdc66640 1 -- 172.21.15.170:0/836262724 wait complete. 2024-09-21T02:13:00.391 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.391 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.391 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 -- 172.21.15.170:0/659305133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 msgr2=0x7f852c153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 --2- 172.21.15.170:0/659305133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 0x7f852c153f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c02d5d0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 -- 172.21.15.170:0/659305133 shutdown_connections 2024-09-21T02:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 --2- 172.21.15.170:0/659305133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 0x7f852c153f40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 -- 172.21.15.170:0/659305133 >> 172.21.15.170:0/659305133 conn(0x7f852c0d00a0 msgr2=0x7f852c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 -- 172.21.15.170:0/659305133 shutdown_connections 2024-09-21T02:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.397+0000 7f853454b640 1 -- 172.21.15.170:0/659305133 wait complete. 2024-09-21T02:13:00.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 Processor -- start 2024-09-21T02:13:00.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 -- start start 2024-09-21T02:13:00.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 0x7f852c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f852c150c60 0x7f852c146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f85322e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 0x7f852c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f85322e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 0x7f852c0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52904/0 (socket says 172.21.15.170:52904) 2024-09-21T02:13:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f85322e7640 1 -- 172.21.15.170:0/1481117148 learned_addr learned my addr 172.21.15.170:0/1481117148 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 0x7f852c1f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f852c155190 con 0x7f852c070770 2024-09-21T02:13:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f852c155010 con 0x7f852c150c60 2024-09-21T02:13:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f852c155310 con 0x7f852c151780 2024-09-21T02:13:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.401+0000 7f8531ae6640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f852c150c60 0x7f852c146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.405+0000 7f85322e7640 1 -- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 msgr2=0x7f852c1f7710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:13:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.405+0000 7f85322e7640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 0x7f852c1f7710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.405+0000 7f85322e7640 1 -- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f852c150c60 msgr2=0x7f852c146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.405+0000 7f85322e7640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f852c150c60 0x7f852c146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.405+0000 7f85322e7640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c147380 con 0x7f852c070770 2024-09-21T02:13:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.405+0000 7f85322e7640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 0x7f852c0cf270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f852800c700 tx=0x7f852800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.413+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528017070 con 0x7f852c070770 2024-09-21T02:13:00.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.413+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1f8540 con 0x7f852c070770 2024-09-21T02:13:00.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.413+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c146f60 con 0x7f852c070770 2024-09-21T02:13:00.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.413+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8528005360 con 0x7f852c070770 2024-09-21T02:13:00.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.421+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852800a8a0 con 0x7f852c070770 2024-09-21T02:13:00.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.421+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f852801a070 con 0x7f852c070770 2024-09-21T02:13:00.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.421+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f852801c440 con 0x7f852c070770 2024-09-21T02:13:00.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.421+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c1557c0 con 0x7f852c070770 2024-09-21T02:13:00.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.425+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8528003600 con 0x7f852c070770 2024-09-21T02:13:00.457 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.457 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.461 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.477+0000 7f710230e640 1 -- 172.21.15.170:0/3778267542 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f70dc005300 con 0x7f70fc070f90 2024-09-21T02:13:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.477+0000 7f09ae70e640 1 -- 172.21.15.170:0/2529825059 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f0988005300 con 0x7f09a80b4270 2024-09-21T02:13:00.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.489+0000 7f09a4ff9640 1 -- 172.21.15.170:0/2529825059 <== mon.1 v2:172.21.15.191:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v8)=0 v8) ==== 106+0+0 (secure 0 0 0) 0x7f099c01a3d0 con 0x7f09a80b4270 2024-09-21T02:13:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.497+0000 7f70f8ff9640 1 -- 172.21.15.170:0/3778267542 <== mon.1 v2:172.21.15.191:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v9)=0 v9) ==== 117+0+0 (secure 0 0 0) 0x7f70ec00f910 con 0x7f70fc070f90 2024-09-21T02:13:00.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 -- 172.21.15.170:0/3778267542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70fc070f90 msgr2=0x7f70fc156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f79bce95640 1 -- 172.21.15.170:0/3091133169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f7990005300 con 0x7f79b8162a60 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 --2- 172.21.15.170:0/3778267542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70fc070f90 0x7f70fc156f90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0011d0 tx=0x7f70ec001690 comp rx=0 tx=0).stop 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 -- 172.21.15.170:0/3778267542 shutdown_connections 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 --2- 172.21.15.170:0/3778267542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70fc158da0 0x7f70fc15d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 --2- 172.21.15.170:0/3778267542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70fc15e580 0x7f70fc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 --2- 172.21.15.170:0/3778267542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70fc070f90 0x7f70fc156f90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.501+0000 7f710230e640 1 -- 172.21.15.170:0/3778267542 >> 172.21.15.170:0/3778267542 conn(0x7f70fc0d0910 msgr2=0x7f70fc0cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f710230e640 1 -- 172.21.15.170:0/3778267542 shutdown_connections 2024-09-21T02:13:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f710230e640 1 -- 172.21.15.170:0/3778267542 wait complete. 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 -- 172.21.15.170:0/2529825059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09a80b4270 msgr2=0x7f09a8145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 --2- 172.21.15.170:0/2529825059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09a80b4270 0x7f09a8145b10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f099c00a210 tx=0x7f099c00a6d0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 -- 172.21.15.170:0/2529825059 shutdown_connections 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 --2- 172.21.15.170:0/2529825059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09a8157be0 0x7f09a8152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 --2- 172.21.15.170:0/2529825059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09a815add0 0x7f09a8152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 --2- 172.21.15.170:0/2529825059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09a80b4270 0x7f09a8145b10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 -- 172.21.15.170:0/2529825059 >> 172.21.15.170:0/2529825059 conn(0x7f09a80d00a0 msgr2=0x7f09a8156090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 -- 172.21.15.170:0/2529825059 shutdown_connections 2024-09-21T02:13:00.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f09ae70e640 1 -- 172.21.15.170:0/2529825059 wait complete. 2024-09-21T02:13:00.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.505+0000 7f79a6ffd640 1 -- 172.21.15.170:0/3091133169 <== mon.0 v2:172.21.15.170:3300/0 11 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f79a8006e00 con 0x7f79b8162a60 2024-09-21T02:13:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.529+0000 7f79a4ff9640 1 -- 172.21.15.170:0/3091133169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b8162a60 msgr2=0x7f79b8146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.529+0000 7f79a4ff9640 1 --2- 172.21.15.170:0/3091133169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b8162a60 0x7f79b8146a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f79a800f8b0 tx=0x7f79a800fd70 comp rx=0 tx=0).stop 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 -- 172.21.15.170:0/3091133169 shutdown_connections 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 --2- 172.21.15.170:0/3091133169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79b8147230 0x7f79b80cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 --2- 172.21.15.170:0/3091133169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b8162a60 0x7f79b8146a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 --2- 172.21.15.170:0/3091133169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79b8071460 0x7f79b815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 -- 172.21.15.170:0/3091133169 >> 172.21.15.170:0/3091133169 conn(0x7f79b80d00a0 msgr2=0x7f79b80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 -- 172.21.15.170:0/3091133169 shutdown_connections 2024-09-21T02:13:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f79a4ff9640 1 -- 172.21.15.170:0/3091133169 wait complete. 2024-09-21T02:13:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f1bca00f640 1 -- 172.21.15.170:0/495978591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4070770 msgr2=0x7f1bc4070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f1bca00f640 1 --2- 172.21.15.170:0/495978591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4070770 0x7f1bc4070f20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40cc170 tx=0x7f1bb401c6b0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f1bca00f640 1 -- 172.21.15.170:0/495978591 shutdown_connections 2024-09-21T02:13:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f1bca00f640 1 --2- 172.21.15.170:0/495978591 >> v2:172.21.15.191:3300/0 conn(0x7f1bc4071460 0x7f1bc4162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f1bca00f640 1 --2- 172.21.15.170:0/495978591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4070770 0x7f1bc4070f20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.541+0000 7f1bca00f640 1 -- 172.21.15.170:0/495978591 >> 172.21.15.170:0/495978591 conn(0x7f1bc40d00a0 msgr2=0x7f1bc40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.557+0000 7f1bca00f640 1 -- 172.21.15.170:0/495978591 shutdown_connections 2024-09-21T02:13:00.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.557+0000 7f1bca00f640 1 -- 172.21.15.170:0/495978591 wait complete. 2024-09-21T02:13:00.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.557+0000 7f1bca00f640 1 Processor -- start 2024-09-21T02:13:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.557+0000 7f1bca00f640 1 -- start start 2024-09-21T02:13:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bca00f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc4070770 0x7f1bc4158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bca00f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4071460 0x7f1bc42061c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bca00f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4162890 0x7f1bc4205450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bca00f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc4163030 con 0x7f1bc4071460 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc4070770 0x7f1bc4158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bca00f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc4162eb0 con 0x7f1bc4070770 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bca00f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc41631b0 con 0x7f1bc4162890 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc37fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc4070770 0x7f1bc4158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:59836/0 (socket says 172.21.15.170:59836) 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc37fe640 1 -- 172.21.15.170:0/3313960193 learned_addr learned my addr 172.21.15.170:0/3313960193 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4071460 0x7f1bc42061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 -- 172.21.15.170:0/3313960193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4162890 msgr2=0x7f1bc4205450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4162890 0x7f1bc4205450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 -- 172.21.15.170:0/3313960193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc4070770 msgr2=0x7f1bc4158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc4070770 0x7f1bc4158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 -- 172.21.15.170:0/3313960193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc415b930 con 0x7f1bc4071460 2024-09-21T02:13:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.561+0000 7f1bc2ffd640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4071460 0x7f1bc42061c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800b440 tx=0x7f1bb800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.573+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8015020 con 0x7f1bc4071460 2024-09-21T02:13:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.573+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc4151960 con 0x7f1bc4071460 2024-09-21T02:13:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.573+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc415bbf0 con 0x7f1bc4071460 2024-09-21T02:13:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.573+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bb8011920 con 0x7f1bc4071460 2024-09-21T02:13:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.573+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8011d70 con 0x7f1bc4071460 2024-09-21T02:13:00.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.573+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba40052a0 con 0x7f1bc4071460 2024-09-21T02:13:00.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.585+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1bb8018020 con 0x7f1bc4071460 2024-09-21T02:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.585+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1bb8019860 con 0x7f1bc4071460 2024-09-21T02:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.585+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1bb801f020 con 0x7f1bc4071460 2024-09-21T02:13:00.593 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.593 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.599 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.645+0000 7fde92556640 1 -- 172.21.15.170:0/1237368704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c070770 msgr2=0x7fde8c070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.645+0000 7fde92556640 1 --2- 172.21.15.170:0/1237368704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c070770 0x7fde8c070f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde84009c80 tx=0x7fde8401cda0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.645+0000 7fde92556640 1 -- 172.21.15.170:0/1237368704 shutdown_connections 2024-09-21T02:13:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.645+0000 7fde92556640 1 --2- 172.21.15.170:0/1237368704 >> v2:172.21.15.170:3300/0 conn(0x7fde8c162a60 0x7fde8c0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.645+0000 7fde92556640 1 --2- 172.21.15.170:0/1237368704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c070770 0x7fde8c070f20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.645+0000 7fde92556640 1 -- 172.21.15.170:0/1237368704 >> 172.21.15.170:0/1237368704 conn(0x7fde8c0d00a0 msgr2=0x7fde8c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.653+0000 7fde92556640 1 -- 172.21.15.170:0/1237368704 shutdown_connections 2024-09-21T02:13:00.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.653+0000 7fde92556640 1 -- 172.21.15.170:0/1237368704 wait complete. 2024-09-21T02:13:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.653+0000 7fde92556640 1 Processor -- start 2024-09-21T02:13:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.653+0000 7fde92556640 1 -- start start 2024-09-21T02:13:00.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde92556640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 0x7fde8c14a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde92556640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c162a60 0x7fde8c14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde92556640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde8c16c0f0 0x7fde8c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde92556640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c154410 con 0x7fde8c16c0f0 2024-09-21T02:13:00.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde92556640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c154290 con 0x7fde8c071460 2024-09-21T02:13:00.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde92556640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c154590 con 0x7fde8c162a60 2024-09-21T02:13:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8b7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c162a60 0x7fde8c14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde90af3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde8c16c0f0 0x7fde8c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 0x7fde8c14a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 0x7fde8c14a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:59850/0 (socket says 172.21.15.170:59850) 2024-09-21T02:13:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde90af3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde8c16c0f0 0x7fde8c14f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52934/0 (socket says 172.21.15.170:52934) 2024-09-21T02:13:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 -- 172.21.15.170:0/634476210 learned_addr learned my addr 172.21.15.170:0/634476210 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 -- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c162a60 msgr2=0x7fde8c14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c162a60 0x7fde8c14eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 -- 172.21.15.170:0/634476210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde8c16c0f0 msgr2=0x7fde8c14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde8c16c0f0 0x7fde8c14f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 -- 172.21.15.170:0/634476210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c0750b0 con 0x7fde8c071460 2024-09-21T02:13:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.657+0000 7fde8bfff640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 0x7fde8c14a680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fde84009bd0 tx=0x7fde840a5ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840b2040 con 0x7fde8c071460 2024-09-21T02:13:00.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c2009f0 con 0x7fde8c071460 2024-09-21T02:13:00.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c200fb0 con 0x7fde8c071460 2024-09-21T02:13:00.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde84003c70 con 0x7fde8c071460 2024-09-21T02:13:00.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84004270 con 0x7fde8c071460 2024-09-21T02:13:00.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde840050c0 con 0x7fde8c071460 2024-09-21T02:13:00.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.665+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde840b8780 con 0x7fde8c071460 2024-09-21T02:13:00.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 -- 172.21.15.170:0/2442110055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 msgr2=0x7f2ca80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2442110055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 0x7f2ca80cc990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009840 tx=0x7f2ca401c610 comp rx=0 tx=0).stop 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 -- 172.21.15.170:0/2442110055 shutdown_connections 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2442110055 >> v2:172.21.15.191:3300/0 conn(0x7f2ca80cced0 0x7f2ca815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2442110055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 0x7f2ca80cc990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 -- 172.21.15.170:0/2442110055 >> 172.21.15.170:0/2442110055 conn(0x7f2ca806d650 msgr2=0x7f2ca806ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 -- 172.21.15.170:0/2442110055 shutdown_connections 2024-09-21T02:13:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.669+0000 7f2caeb32640 1 -- 172.21.15.170:0/2442110055 wait complete. 2024-09-21T02:13:00.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 Processor -- start 2024-09-21T02:13:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 -- start start 2024-09-21T02:13:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 0x7f2ca8200220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 0x7f2ca8074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ca815c170 0x7f2ca815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8170070 con 0x7f2ca80cced0 2024-09-21T02:13:00.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca816fef0 con 0x7f2ca815c170 2024-09-21T02:13:00.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2caeb32640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca81701f0 con 0x7f2ca80cc210 2024-09-21T02:13:00.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2cad32f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 0x7f2ca8074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2cad32f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 0x7f2ca8074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52950/0 (socket says 172.21.15.170:52950) 2024-09-21T02:13:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2cad32f640 1 -- 172.21.15.170:0/2932455087 learned_addr learned my addr 172.21.15.170:0/2932455087 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2cadb30640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 0x7f2ca8200220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.673+0000 7f2cae331640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ca815c170 0x7f2ca815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6c0052a0 con 0x7fde8c071460 2024-09-21T02:13:00.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7f2cad32f640 1 -- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 msgr2=0x7f2ca8200220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7f2cad32f640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 0x7f2ca8200220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7f2cad32f640 1 -- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ca815c170 msgr2=0x7f2ca815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7f2cad32f640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ca815c170 0x7f2ca815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7f2cad32f640 1 -- 172.21.15.170:0/2932455087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca8200c00 con 0x7f2ca80cced0 2024-09-21T02:13:00.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.677+0000 7f2cad32f640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 0x7f2ca8074ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800e7c0 tx=0x7f2c9800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.681+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fde84016990 con 0x7fde8c071460 2024-09-21T02:13:00.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.685+0000 7f6ea262b640 1 -- 172.21.15.170:0/2060195078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 msgr2=0x7f6e9c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.685+0000 7f6ea262b640 1 --2- 172.21.15.170:0/2060195078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 0x7f6e9c070180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009bc0 tx=0x7f6e9401ca50 comp rx=0 tx=0).stop 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800ccc0 con 0x7f2ca80cced0 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c9800ee70 con 0x7f2ca80cced0 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca8200ee0 con 0x7f2ca80cced0 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98010690 con 0x7f2ca80cced0 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca8074ff0 con 0x7f2ca80cced0 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f6ea262b640 1 -- 172.21.15.170:0/2060195078 shutdown_connections 2024-09-21T02:13:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f6ea262b640 1 --2- 172.21.15.170:0/2060195078 >> v2:172.21.15.191:3300/0 conn(0x7f6e9c0b41a0 0x7f6e9c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f6ea262b640 1 --2- 172.21.15.170:0/2060195078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 0x7f6e9c070180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f6ea262b640 1 -- 172.21.15.170:0/2060195078 >> 172.21.15.170:0/2060195078 conn(0x7f6e9c0d00a0 msgr2=0x7f6e9c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c98010830 con 0x7f2ca80cced0 2024-09-21T02:13:00.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c98018c70 con 0x7f2ca80cced0 2024-09-21T02:13:00.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f6ea262b640 1 -- 172.21.15.170:0/2060195078 shutdown_connections 2024-09-21T02:13:00.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.689+0000 7f6ea262b640 1 -- 172.21.15.170:0/2060195078 wait complete. 2024-09-21T02:13:00.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 Processor -- start 2024-09-21T02:13:00.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 -- start start 2024-09-21T02:13:00.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f852c06de10 con 0x7f852c070770 2024-09-21T02:13:00.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca8201120 con 0x7f2ca80cced0 2024-09-21T02:13:00.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e9c0b41a0 0x7f6e9c13b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 0x7f6e9c1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 0x7f6e9c0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c0cee60 con 0x7f6e9c155280 2024-09-21T02:13:00.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c0cece0 con 0x7f6e9c0b41a0 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea262b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9c0cefe0 con 0x7f6e9c1558e0 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 0x7f6e9c0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 0x7f6e9c0b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52964/0 (socket says 172.21.15.170:52964) 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 -- 172.21.15.170:0/923408616 learned_addr learned my addr 172.21.15.170:0/923408616 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6e9bfff640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e9c0b41a0 0x7f6e9c13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 -- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 msgr2=0x7f6e9c1f5d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 0x7f6e9c1f5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 -- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e9c0b41a0 msgr2=0x7f6e9c13b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e9c0b41a0 0x7f6e9c13b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 -- 172.21.15.170:0/923408616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c0d09a0 con 0x7f6e9c155280 2024-09-21T02:13:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.693+0000 7f6ea0bc8640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 0x7f6e9c0b0c50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000b410 tx=0x7f6e9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.697+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2ca8201120 con 0x7f2ca80cced0 2024-09-21T02:13:00.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.701+0000 7f8522ffd640 1 -- 172.21.15.170:0/1481117148 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f85280037e0 con 0x7f852c070770 2024-09-21T02:13:00.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.705+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90015020 con 0x7f6e9c155280 2024-09-21T02:13:00.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.705+0000 7f6ea262b640 1 -- 172.21.15.170:0/923408616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c1f6710 con 0x7f6e9c155280 2024-09-21T02:13:00.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.705+0000 7f6ea262b640 1 -- 172.21.15.170:0/923408616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c14fde0 con 0x7f6e9c155280 2024-09-21T02:13:00.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.705+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e90011920 con 0x7f6e9c155280 2024-09-21T02:13:00.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.705+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90011d70 con 0x7f6e9c155280 2024-09-21T02:13:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.709+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e90018020 con 0x7f6e9c155280 2024-09-21T02:13:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.709+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e900198b0 con 0x7f6e9c155280 2024-09-21T02:13:00.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.713+0000 7f6ea262b640 1 -- 172.21.15.170:0/923408616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e780052a0 con 0x7f6e9c155280 2024-09-21T02:13:00.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 msgr2=0x7f852c0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 0x7f852c0cf270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f852800c700 tx=0x7f852800cbc0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6e9001f020 con 0x7f6e9c155280 2024-09-21T02:13:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 shutdown_connections 2024-09-21T02:13:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f852c151780 0x7f852c1f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f852c150c60 0x7f852c146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 --2- 172.21.15.170:0/1481117148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f852c070770 0x7f852c0cf270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.717+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 >> 172.21.15.170:0/1481117148 conn(0x7f852c0d00a0 msgr2=0x7f852c151070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7fe229cfb640 1 -- 172.21.15.170:0/1978998492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224154ce0 msgr2=0x7fe2240b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7fe229cfb640 1 --2- 172.21.15.170:0/1978998492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224154ce0 0x7fe2240b3650 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe214009900 tx=0x7fe21401c840 comp rx=0 tx=0).stop 2024-09-21T02:13:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 shutdown_connections 2024-09-21T02:13:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f853454b640 1 -- 172.21.15.170:0/1481117148 wait complete. 2024-09-21T02:13:00.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f0a3e076640 1 -- 172.21.15.170:0/3749125883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a20005160 msgr2=0x7f0a20009b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f0a3e076640 1 --2- 172.21.15.170:0/3749125883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a20005160 0x7f0a20009b60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0a280017d0 tx=0x7f0a2801c8b0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f0a37fff640 1 -- 172.21.15.170:0/3749125883 reap_dead start 2024-09-21T02:13:00.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f0a3e076640 1 -- 172.21.15.170:0/3749125883 shutdown_connections 2024-09-21T02:13:00.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f0a3e076640 1 -- 172.21.15.170:0/3749125883 >> 172.21.15.170:0/3749125883 conn(0x7f0a380d0170 msgr2=0x7f0a380d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.729+0000 7f0a3e076640 1 -- 172.21.15.170:0/3749125883 shutdown_connections 2024-09-21T02:13:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.733+0000 7f0a3e076640 1 -- 172.21.15.170:0/3749125883 wait complete. 2024-09-21T02:13:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.733+0000 7f0a3e076640 1 Processor -- start 2024-09-21T02:13:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.733+0000 7f0a3e076640 1 -- start start 2024-09-21T02:13:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a3e076640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380b1f00 0x7f0a380b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380b1f00 0x7f0a380b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380b1f00 0x7f0a380b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52978/0 (socket says 172.21.15.170:52978) 2024-09-21T02:13:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a3e076640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a380b4d70 0x7f0a38162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a377fe640 1 -- 172.21.15.170:0/1040920100 learned_addr learned my addr 172.21.15.170:0/1040920100 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a3e076640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38162e80 0x7f0a381f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a36ffd640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a380b4d70 0x7f0a38162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a3e076640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a380cbd70 con 0x7f0a380b1f00 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a3e076640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0a380cbbf0 con 0x7f0a38162e80 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a3e076640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0a38074350 con 0x7f0a380b4d70 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a37fff640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38162e80 0x7f0a381f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f1ba4005860 con 0x7f1bc4071460 2024-09-21T02:13:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a36ffd640 1 -- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38162e80 msgr2=0x7f0a381f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a36ffd640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38162e80 0x7f0a381f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a36ffd640 1 -- 172.21.15.170:0/1040920100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380b1f00 msgr2=0x7f0a380b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a36ffd640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380b1f00 0x7f0a380b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f0a36ffd640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a380b5180 con 0x7f0a380b4d70 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f0a36ffd640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a380b4d70 0x7f0a38162940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a380af970 tx=0x7f0a3000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 -- 172.21.15.170:0/1978998492 shutdown_connections 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 --2- 172.21.15.170:0/1978998492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224154ce0 0x7fe2240b3650 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 --2- 172.21.15.170:0/1978998492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe224154390 0x7fe2241547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 -- 172.21.15.170:0/1978998492 >> 172.21.15.170:0/1978998492 conn(0x7fe2240d0080 msgr2=0x7fe2240d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 -- 172.21.15.170:0/1978998492 shutdown_connections 2024-09-21T02:13:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 -- 172.21.15.170:0/3277969929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b480cd0a0 msgr2=0x7f4b4815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/3277969929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b480cd0a0 0x7f4b4815bc30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009840 tx=0x7f4b3801c680 comp rx=0 tx=0).stop 2024-09-21T02:13:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.737+0000 7f6c2e270640 1 -- 172.21.15.170:0/59032301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c10003c20 msgr2=0x7f6c10004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f6c2e270640 1 --2- 172.21.15.170:0/59032301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c10003c20 0x7f6c10004070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c18006ee0 tx=0x7f6c1801ce60 comp rx=0 tx=0).stop 2024-09-21T02:13:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f6c2c80d640 1 -- 172.21.15.170:0/59032301 reap_dead start 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 -- 172.21.15.170:0/3277969929 shutdown_connections 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/3277969929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b4815c170 0x7f4b480cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/3277969929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b480cd0a0 0x7f4b4815bc30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 -- 172.21.15.170:0/3277969929 >> 172.21.15.170:0/3277969929 conn(0x7f4b4806d650 msgr2=0x7f4b4806ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 -- 172.21.15.170:0/3277969929 shutdown_connections 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 -- 172.21.15.170:0/3277969929 wait complete. 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 Processor -- start 2024-09-21T02:13:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30003d90 con 0x7f0a380b4d70 2024-09-21T02:13:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a30004b30 con 0x7f0a380b4d70 2024-09-21T02:13:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004050 con 0x7f0a380b4d70 2024-09-21T02:13:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f6c2e270640 1 -- 172.21.15.170:0/59032301 shutdown_connections 2024-09-21T02:13:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f6c2e270640 1 -- 172.21.15.170:0/59032301 >> 172.21.15.170:0/59032301 conn(0x7f6c280cdc30 msgr2=0x7f6c280ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 -- 172.21.15.170:0/1978998492 wait complete. 2024-09-21T02:13:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 Processor -- start 2024-09-21T02:13:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7f4b4d38e640 1 -- start start 2024-09-21T02:13:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 0x7f4b480b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b480cd0a0 0x7f4b482059d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b4815c170 0x7f4b48161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f0a3e076640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a381613c0 con 0x7f0a380b4d70 2024-09-21T02:13:00.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 0x7f4b480b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 0x7f4b480b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52986/0 (socket says 172.21.15.170:52986) 2024-09-21T02:13:00.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f6c2e270640 1 -- 172.21.15.170:0/59032301 shutdown_connections 2024-09-21T02:13:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 -- 172.21.15.170:0/2272978167 learned_addr learned my addr 172.21.15.170:0/2272978167 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f0a3e076640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a380b2310 con 0x7f0a380b4d70 2024-09-21T02:13:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f6c2e270640 1 -- 172.21.15.170:0/59032301 wait complete. 2024-09-21T02:13:00.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48170070 con 0x7f4b480cc210 2024-09-21T02:13:00.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4816fef0 con 0x7f4b4815c170 2024-09-21T02:13:00.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4b481701f0 con 0x7f4b480cd0a0 2024-09-21T02:13:00.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4cb8d640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b4815c170 0x7f4b48161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b477fe640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b480cd0a0 0x7f4b482059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.741+0000 7fe229cfb640 1 -- start start 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe229cfb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 0x7fe2241611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 -- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b480cd0a0 msgr2=0x7f4b482059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b480cd0a0 0x7f4b482059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 -- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b4815c170 msgr2=0x7f4b48161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b4815c170 0x7f4b48161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b480b1bf0 con 0x7f4b480cc210 2024-09-21T02:13:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe229cfb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe224154390 0x7fe22414e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe229cfb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe224154ce0 0x7fe22414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b47fff640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 0x7f4b480b16b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000cad0 tx=0x7f4b4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe229cfb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe2240b4730 con 0x7fe224071310 2024-09-21T02:13:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40017070 con 0x7f4b480cc210 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 0x7fe2241611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 0x7fe2241611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52990/0 (socket says 172.21.15.170:52990) 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 -- 172.21.15.170:0/4041365870 learned_addr learned my addr 172.21.15.170:0/4041365870 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe223fff640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe224154390 0x7fe22414e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe2294fa640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe224154ce0 0x7fe22414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe229cfb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2240b45b0 con 0x7fe224154ce0 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe224157670 con 0x7fe224154390 2024-09-21T02:13:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b4000a700 con 0x7f4b480cc210 2024-09-21T02:13:00.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000aaf0 con 0x7f4b480cc210 2024-09-21T02:13:00.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 -- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe224154390 msgr2=0x7fe22414e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe224154390 0x7fe22414e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 -- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe224154ce0 msgr2=0x7fe22414eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe224154ce0 0x7fe22414eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 -- 172.21.15.170:0/4041365870 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe224150a40 con 0x7fe224071310 2024-09-21T02:13:00.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7fe228cf9640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 0x7fe2241611a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe21800b3e0 tx=0x7fe21800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b482051f0 con 0x7f4b480cc210 2024-09-21T02:13:00.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4814f7a0 con 0x7f4b480cc210 2024-09-21T02:13:00.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b4001a070 con 0x7f4b480cc210 2024-09-21T02:13:00.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b4001c810 con 0x7f4b480cc210 2024-09-21T02:13:00.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a100052a0 con 0x7f0a380b4d70 2024-09-21T02:13:00.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a30004580 con 0x7f0a380b4d70 2024-09-21T02:13:00.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.745+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a30022780 con 0x7f0a380b4d70 2024-09-21T02:13:00.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f6c2e270640 1 Processor -- start 2024-09-21T02:13:00.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1babfff640 1 -- 172.21.15.170:0/3313960193 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f1bb80193b0 con 0x7f1bc4071460 2024-09-21T02:13:00.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f6c2e270640 1 -- start start 2024-09-21T02:13:00.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4071460 msgr2=0x7f1bc42061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4071460 0x7f1bc42061c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800b440 tx=0x7f1bb800b840 comp rx=0 tx=0).stop 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 shutdown_connections 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bc4162890 0x7f1bc4205450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc4071460 0x7f1bc42061c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 --2- 172.21.15.170:0/3313960193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bc4070770 0x7f1bc4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 >> 172.21.15.170:0/3313960193 conn(0x7f1bc40d00a0 msgr2=0x7f1bc415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 shutdown_connections 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f1bca00f640 1 -- 172.21.15.170:0/3313960193 wait complete. 2024-09-21T02:13:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218015020 con 0x7fe224071310 2024-09-21T02:13:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0a380b5180 con 0x7f0a380b4d70 2024-09-21T02:13:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe224150d20 con 0x7fe224071310 2024-09-21T02:13:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.749+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2241512e0 con 0x7fe224071310 2024-09-21T02:13:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.753+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe218011940 con 0x7fe224071310 2024-09-21T02:13:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.753+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218011d90 con 0x7fe224071310 2024-09-21T02:13:00.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.753+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b480cdcb0 con 0x7f4b480cc210 2024-09-21T02:13:00.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.753+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe218018020 con 0x7fe224071310 2024-09-21T02:13:00.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.753+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2180199d0 con 0x7fe224071310 2024-09-21T02:13:00.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.753+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2040052a0 con 0x7fe224071310 2024-09-21T02:13:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c10003c20 0x7f6c281f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c281ec3f0 0x7f6c281f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c27fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c10003c20 0x7f6c281f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c27fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c10003c20 0x7f6c281f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:47572/0 (socket says 172.21.15.170:47572) 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c27fff640 1 -- 172.21.15.170:0/656718835 learned_addr learned my addr 172.21.15.170:0/656718835 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c281ec3f0 0x7f6c281f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c281f8100 0x7f6c2815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280692d0 con 0x7f6c281f8100 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28069150 con 0x7f6c281ec3f0 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6c28069450 con 0x7f6c10003c20 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 -- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c10003c20 msgr2=0x7f6c281f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c10003c20 0x7f6c281f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 -- 172.21.15.170:0/656718835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c281f8100 msgr2=0x7f6c2815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c281f8100 0x7f6c2815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2815c220 con 0x7f6c281ec3f0 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c277fe640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c281ec3f0 0x7f6c281f7aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c003d90 tx=0x7f6c1c001810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011640 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c1c016020 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011a80 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2815c500 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.757+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c281ec8a0 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.761+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c1c01b410 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.761+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6c1c01ab10 con 0x7f6c281ec3f0 2024-09-21T02:13:00.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.761+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c28074ff0 con 0x7f6c281ec3f0 2024-09-21T02:13:00.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.765+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4b4000ac90 con 0x7f4b480cc210 2024-09-21T02:13:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.765+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6c1c020020 con 0x7f6c281ec3f0 2024-09-21T02:13:00.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.769+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe21801f020 con 0x7fe224071310 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 -- 172.21.15.170:0/2123850430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328154390 msgr2=0x7f33281547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 --2- 172.21.15.170:0/2123850430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328154390 0x7f33281547a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f331c009840 tx=0x7f331c01c760 comp rx=0 tx=0).stop 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 -- 172.21.15.170:0/2123850430 shutdown_connections 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 --2- 172.21.15.170:0/2123850430 >> v2:172.21.15.170:3300/0 conn(0x7f3328154ce0 0x7f33280b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 --2- 172.21.15.170:0/2123850430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328154390 0x7f33281547a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 -- 172.21.15.170:0/2123850430 >> 172.21.15.170:0/2123850430 conn(0x7f33280d0080 msgr2=0x7f33280d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 -- 172.21.15.170:0/2123850430 shutdown_connections 2024-09-21T02:13:00.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.789+0000 7f332d374640 1 -- 172.21.15.170:0/2123850430 wait complete. 2024-09-21T02:13:00.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 Processor -- start 2024-09-21T02:13:00.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 -- start start 2024-09-21T02:13:00.792 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328071310 0x7f33282073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3328154ce0 0x7f33280c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3328207910 0x7f33280c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f3327fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328071310 0x7f33282073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f3327fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328071310 0x7f33282073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:47590/0 (socket says 172.21.15.170:47590) 2024-09-21T02:13:00.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33280b4730 con 0x7f3328207910 2024-09-21T02:13:00.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f3327fff640 1 -- 172.21.15.170:0/306126641 learned_addr learned my addr 172.21.15.170:0/306126641 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33280b45b0 con 0x7f3328154ce0 2024-09-21T02:13:00.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3328157670 con 0x7f3328071310 2024-09-21T02:13:00.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f332cb73640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3328207910 0x7f33280c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.793+0000 7f33277fe640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3328154ce0 0x7f33280c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.797+0000 7f332cb73640 1 -- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328071310 msgr2=0x7f33282073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.797+0000 7f332cb73640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328071310 0x7f33282073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.797+0000 7f332cb73640 1 -- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3328154ce0 msgr2=0x7f33280c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.797+0000 7f332cb73640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3328154ce0 0x7f33280c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.797+0000 7f332cb73640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3328153fe0 con 0x7f3328207910 2024-09-21T02:13:00.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.797+0000 7f332cb73640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3328207910 0x7f33280c7e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f331800b410 tx=0x7f331800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318015020 con 0x7f3328207910 2024-09-21T02:13:00.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3318011920 con 0x7f3328207910 2024-09-21T02:13:00.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318011d70 con 0x7f3328207910 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/1086766386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cac00 msgr2=0x7f9b34162700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/1086766386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cac00 0x7f9b34162700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009900 tx=0x7f9b3001c870 comp rx=0 tx=0).stop 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/1086766386 shutdown_connections 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/1086766386 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cb010 0x7f9b34162c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/1086766386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cac00 0x7f9b34162700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/1086766386 >> 172.21.15.170:0/1086766386 conn(0x7f9b340d0170 msgr2=0x7f9b340d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/1086766386 shutdown_connections 2024-09-21T02:13:00.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/1086766386 wait complete. 2024-09-21T02:13:00.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33282080c0 con 0x7f3328207910 2024-09-21T02:13:00.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3328208680 con 0x7f3328207910 2024-09-21T02:13:00.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3318018020 con 0x7f3328207910 2024-09-21T02:13:00.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.801+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3318019890 con 0x7f3328207910 2024-09-21T02:13:00.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 Processor -- start 2024-09-21T02:13:00.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 -- start start 2024-09-21T02:13:00.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7fecb09c9640 1 -- 172.21.15.170:0/1974459195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15fa20 msgr2=0x7fecac15fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1974459195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15fa20 0x7fecac15fe30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fecac15f440 tx=0x7fec9c01c770 comp rx=0 tx=0).stop 2024-09-21T02:13:00.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 -- 172.21.15.170:0/924703077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c159020 msgr2=0x7f286c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 --2- 172.21.15.170:0/924703077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c159020 0x7f286c0cdbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f286c0d1070 tx=0x7f285c01c6b0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7fecb09c9640 1 -- 172.21.15.170:0/1974459195 shutdown_connections 2024-09-21T02:13:00.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1974459195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15fa20 0x7fecac15fe30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1974459195 >> v2:172.21.15.170:3300/0 conn(0x7fecac15b8c0 0x7fecac15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7fecb09c9640 1 -- 172.21.15.170:0/1974459195 >> 172.21.15.170:0/1974459195 conn(0x7fecac09a9c0 msgr2=0x7fecac0b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 -- 172.21.15.170:0/924703077 shutdown_connections 2024-09-21T02:13:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 --2- 172.21.15.170:0/924703077 >> v2:172.21.15.191:3300/0 conn(0x7f286c15c2a0 0x7f286c0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 --2- 172.21.15.170:0/924703077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c159020 0x7f286c0cdbb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 -- 172.21.15.170:0/924703077 >> 172.21.15.170:0/924703077 conn(0x7f286c06d650 msgr2=0x7f286c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 0x7f9b34159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33080052a0 con 0x7f3328207910 2024-09-21T02:13:00.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 -- 172.21.15.170:0/924703077 shutdown_connections 2024-09-21T02:13:00.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cb010 0x7f9b340af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f2871208640 1 -- 172.21.15.170:0/924703077 wait complete. 2024-09-21T02:13:00.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b3415edf0 0x7f9b341521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9b34170830 con 0x7f9b340cb010 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341706b0 con 0x7f9b3415edf0 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b3b4eb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9b341709b0 con 0x7f9b340cac00 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b39287640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 0x7f9b34159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b39287640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 0x7f9b34159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:47602/0 (socket says 172.21.15.170:47602) 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b39287640 1 -- 172.21.15.170:0/3721699133 learned_addr learned my addr 172.21.15.170:0/3721699133 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.805+0000 7f9b39a88640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b3415edf0 0x7f9b341521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f9b39287640 1 -- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b3415edf0 msgr2=0x7f9b341521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f9b39287640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b3415edf0 0x7f9b341521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f9b39287640 1 -- 172.21.15.170:0/3721699133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cb010 msgr2=0x7f9b340af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f9b39287640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cb010 0x7f9b340af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f9b39287640 1 -- 172.21.15.170:0/3721699133 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b34200b00 con 0x7f9b340cac00 2024-09-21T02:13:00.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f9b39287640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 0x7f9b34159100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9b3001d110 tx=0x7f9b30005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f2871208640 1 Processor -- start 2024-09-21T02:13:00.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f2871208640 1 -- start start 2024-09-21T02:13:00.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.809+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f331801f020 con 0x7f3328207910 2024-09-21T02:13:00.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2871208640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 0x7f286c1f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f286bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 0x7f286c1f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f286bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 0x7f286c1f95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53066/0 (socket says 172.21.15.170:53066) 2024-09-21T02:13:00.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f286bfff640 1 -- 172.21.15.170:0/4075261879 learned_addr learned my addr 172.21.15.170:0/4075261879 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2871208640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f286c15c6b0 0x7f286c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2871208640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f286c153d40 0x7f286c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0d0470 con 0x7f286c15c2a0 2024-09-21T02:13:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0caf50 con 0x7f286c15c6b0 2024-09-21T02:13:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2870a07640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f286c153d40 0x7f286c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f286c0d05f0 con 0x7f286c153d40 2024-09-21T02:13:00.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f286b7fe640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f286c15c6b0 0x7f286c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2870a07640 1 -- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f286c15c6b0 msgr2=0x7f286c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2870a07640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f286c15c6b0 0x7f286c0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2870a07640 1 -- 172.21.15.170:0/4075261879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 msgr2=0x7f286c1f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2870a07640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 0x7f286c1f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f2870a07640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c154690 con 0x7f286c153d40 2024-09-21T02:13:00.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.813+0000 7f286bfff640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 0x7f286c1f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:00.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2870a07640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f286c153d40 0x7f286c154150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f286400fcb0 tx=0x7f286400d430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fde6c005590 con 0x7fde8c071460 2024-09-21T02:13:00.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286401a070 con 0x7f286c153d40 2024-09-21T02:13:00.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2864006aa0 con 0x7f286c153d40 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864008070 con 0x7f286c153d40 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c0c9c40 con 0x7f286c153d40 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c0c1ec0 con 0x7f286c153d40 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f286b7fe640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f286c15c6b0 0x7f286c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b300bc070 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b34200de0 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b342013a0 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b300b1040 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b300b0070 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b300bf070 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b140052a0 con 0x7f9b340cac00 2024-09-21T02:13:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b30004e20 con 0x7f9b340cac00 2024-09-21T02:13:00.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7fecb09c9640 1 -- 172.21.15.170:0/1974459195 shutdown_connections 2024-09-21T02:13:00.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c0052a0 con 0x7f286c153d40 2024-09-21T02:13:00.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f286401d070 con 0x7f286c153d40 2024-09-21T02:13:00.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f286401fb30 con 0x7f286c153d40 2024-09-21T02:13:00.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.817+0000 7fecb09c9640 1 -- 172.21.15.170:0/1974459195 wait complete. 2024-09-21T02:13:00.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 Processor -- start 2024-09-21T02:13:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9b300c6020 con 0x7f9b340cac00 2024-09-21T02:13:00.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 -- start start 2024-09-21T02:13:00.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecac0d1250 0x7fecac205ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 0x7fecac0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15fa20 0x7fecac0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecaa59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 0x7fecac0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecaa59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 0x7fecac0b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:59982/0 (socket says 172.21.15.170:59982) 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecaa59c640 1 -- 172.21.15.170:0/1476846617 learned_addr learned my addr 172.21.15.170:0/1476846617 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecab59e640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15fa20 0x7fecac0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fecac169090 con 0x7fecac0d1250 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fecac06e860 con 0x7fecac15b8c0 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.821+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fecac169210 con 0x7fecac15fa20 2024-09-21T02:13:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaad9d640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecac0d1250 0x7fecac205ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaa59c640 1 -- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15fa20 msgr2=0x7fecac0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaa59c640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15fa20 0x7fecac0b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaa59c640 1 -- 172.21.15.170:0/1476846617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecac0d1250 msgr2=0x7fecac205ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaa59c640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecac0d1250 0x7fecac205ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaa59c640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac0c9810 con 0x7fecac15b8c0 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7fecaa59c640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 0x7fecac0b02e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009790 tx=0x7fec9c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.825+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2864024020 con 0x7f286c153d40 2024-09-21T02:13:00.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.829+0000 7fde88ff9640 1 -- 172.21.15.170:0/634476210 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7fde840055e0 con 0x7fde8c071460 2024-09-21T02:13:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.829+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0bb070 con 0x7fecac15b8c0 2024-09-21T02:13:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.829+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec9c005030 con 0x7fecac15b8c0 2024-09-21T02:13:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.829+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0b2040 con 0x7fecac15b8c0 2024-09-21T02:13:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.829+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac0c9af0 con 0x7fecac15b8c0 2024-09-21T02:13:00.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.829+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac15d400 con 0x7fecac15b8c0 2024-09-21T02:13:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.837+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec9c005480 con 0x7fecac15b8c0 2024-09-21T02:13:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.837+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec9c003600 con 0x7fecac15b8c0 2024-09-21T02:13:00.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.841+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecac206720 con 0x7fecac15b8c0 2024-09-21T02:13:00.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.845+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fec9c0c4020 con 0x7fecac15b8c0 2024-09-21T02:13:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 msgr2=0x7fde8c14a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 0x7fde8c14a680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fde84009bd0 tx=0x7fde840a5ad0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 shutdown_connections 2024-09-21T02:13:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde8c162a60 0x7fde8c14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde8c16c0f0 0x7fde8c14f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 --2- 172.21.15.170:0/634476210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde8c071460 0x7fde8c14a680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 >> 172.21.15.170:0/634476210 conn(0x7fde8c0d00a0 msgr2=0x7fde8c15f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 shutdown_connections 2024-09-21T02:13:00.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.849+0000 7fde92556640 1 -- 172.21.15.170:0/634476210 wait complete. 2024-09-21T02:13:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.865+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f2ca80cc990 con 0x7f2ca80cced0 2024-09-21T02:13:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.869+0000 7f2c9e7fc640 1 -- 172.21.15.170:0/2932455087 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7f2c98010cb0 con 0x7f2ca80cced0 2024-09-21T02:13:00.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.877+0000 7f6ea262b640 1 -- 172.21.15.170:0/923408616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f6e78005590 con 0x7f6e9c155280 2024-09-21T02:13:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.881+0000 7f6e98ff9640 1 -- 172.21.15.170:0/923408616 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f6e900193b0 con 0x7f6e9c155280 2024-09-21T02:13:00.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.881+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 msgr2=0x7f2ca8074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.881+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 0x7f2ca8074ab0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800e7c0 tx=0x7f2c9800ec80 comp rx=0 tx=0).stop 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 -- 172.21.15.170:0/923408616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 msgr2=0x7f6e9c0b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 0x7f6e9c0b0c50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000b410 tx=0x7f6e9000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 -- 172.21.15.170:0/923408616 shutdown_connections 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e9c1558e0 0x7f6e9c1f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e9c155280 0x7f6e9c0b0c50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 --2- 172.21.15.170:0/923408616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e9c0b41a0 0x7f6e9c13b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 -- 172.21.15.170:0/923408616 >> 172.21.15.170:0/923408616 conn(0x7f6e9c0d00a0 msgr2=0x7f6e9c15e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 shutdown_connections 2024-09-21T02:13:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ca815c170 0x7f2ca815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca80cced0 0x7f2ca8074ab0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 --2- 172.21.15.170:0/2932455087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca80cc210 0x7f2ca8200220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 >> 172.21.15.170:0/2932455087 conn(0x7f2ca806d650 msgr2=0x7f2ca816c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 -- 172.21.15.170:0/923408616 shutdown_connections 2024-09-21T02:13:00.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f6e8a7fc640 1 -- 172.21.15.170:0/923408616 wait complete. 2024-09-21T02:13:00.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 shutdown_connections 2024-09-21T02:13:00.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.885+0000 7f2caeb32640 1 -- 172.21.15.170:0/2932455087 wait complete. 2024-09-21T02:13:00.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.909+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f4b480d0af0 con 0x7f4b480cc210 2024-09-21T02:13:00.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.913+0000 7f4b44ff9640 1 -- 172.21.15.170:0/2272978167 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) ==== 120+0+0 (secure 0 0 0) 0x7f4b4001ca10 con 0x7f4b480cc210 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.913+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 msgr2=0x7f4b480b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.913+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 0x7f4b480b16b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000cad0 tx=0x7f4b4000cf90 comp rx=0 tx=0).stop 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.913+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 shutdown_connections 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.913+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b480cd0a0 0x7f4b482059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.913+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b4815c170 0x7f4b48161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.917+0000 7f4b4d38e640 1 --2- 172.21.15.170:0/2272978167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b480cc210 0x7f4b480b16b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.917+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 >> 172.21.15.170:0/2272978167 conn(0x7f4b4806d650 msgr2=0x7f4b4815e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.917+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 shutdown_connections 2024-09-21T02:13:00.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.917+0000 7f4b4d38e640 1 -- 172.21.15.170:0/2272978167 wait complete. 2024-09-21T02:13:00.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.917+0000 7f6c2e270640 1 -- 172.21.15.170:0/656718835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f6c28069b00 con 0x7f6c281ec3f0 2024-09-21T02:13:00.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c24ff9640 1 -- 172.21.15.170:0/656718835 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v17)=0 v17) ==== 108+0+0 (secure 0 0 0) 0x7f6c1c01ad10 con 0x7f6c281ec3f0 2024-09-21T02:13:00.924 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.924 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.925 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f0a10005590 con 0x7f0a380b4d70 2024-09-21T02:13:00.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 -- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c281ec3f0 msgr2=0x7f6c281f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c281ec3f0 0x7f6c281f7aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c003d90 tx=0x7f6c1c001810 comp rx=0 tx=0).stop 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 -- 172.21.15.170:0/656718835 shutdown_connections 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c281f8100 0x7f6c2815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c281ec3f0 0x7f6c281f7aa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 --2- 172.21.15.170:0/656718835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c10003c20 0x7f6c281f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 -- 172.21.15.170:0/656718835 >> 172.21.15.170:0/656718835 conn(0x7f6c280cdc30 msgr2=0x7f6c280cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.925+0000 7f6c0e7fc640 1 -- 172.21.15.170:0/656718835 shutdown_connections 2024-09-21T02:13:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.929+0000 7f6c0e7fc640 1 -- 172.21.15.170:0/656718835 wait complete. 2024-09-21T02:13:00.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.937+0000 7f0a1ffff640 1 -- 172.21.15.170:0/1040920100 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v18)=0 v18) ==== 111+0+0 (secure 0 0 0) 0x7f0a300059d0 con 0x7f0a380b4d70 2024-09-21T02:13:00.946 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.946 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi170') 2024-09-21T02:13:00.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.945+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fe204005590 con 0x7fe224071310 2024-09-21T02:13:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.945+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a380b4d70 msgr2=0x7f0a38162940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.945+0000 7f0a1dffb640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a380b4d70 0x7f0a38162940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a380af970 tx=0x7f0a3000f790 comp rx=0 tx=0).stop 2024-09-21T02:13:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 shutdown_connections 2024-09-21T02:13:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a380b4d70 0x7f0a38162940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38162e80 0x7f0a381f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 --2- 172.21.15.170:0/1040920100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380b1f00 0x7f0a380b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 >> 172.21.15.170:0/1040920100 conn(0x7f0a380d0170 msgr2=0x7f0a380ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 shutdown_connections 2024-09-21T02:13:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.949+0000 7f0a1dffb640 1 -- 172.21.15.170:0/1040920100 wait complete. 2024-09-21T02:13:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.953+0000 7fe2217fa640 1 -- 172.21.15.170:0/4041365870 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v19)=0 v19) ==== 125+0+0 (secure 0 0 0) 0x7fe2180193b0 con 0x7fe224071310 2024-09-21T02:13:00.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 msgr2=0x7fe2241611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 0x7fe2241611a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe21800b3e0 tx=0x7fe21800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 shutdown_connections 2024-09-21T02:13:00.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe224154390 0x7fe22414e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe224154ce0 0x7fe22414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 --2- 172.21.15.170:0/4041365870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe224071310 0x7fe2241611a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 >> 172.21.15.170:0/4041365870 conn(0x7fe2240d0080 msgr2=0x7fe2240cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 shutdown_connections 2024-09-21T02:13:00.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7fe229cfb640 1 -- 172.21.15.170:0/4041365870 wait complete. 2024-09-21T02:13:00.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.957+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f9b14005590 con 0x7f9b340cac00 2024-09-21T02:13:00.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.965+0000 7f9b29ffb640 1 -- 172.21.15.170:0/3721699133 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f9b300164f0 con 0x7f9b340cac00 2024-09-21T02:13:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 msgr2=0x7f9b34159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 0x7f9b34159100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9b3001d110 tx=0x7f9b30005760 comp rx=0 tx=0).stop 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 shutdown_connections 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b3415edf0 0x7f9b341521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b340cb010 0x7f9b340af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 --2- 172.21.15.170:0/3721699133 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b340cac00 0x7f9b34159100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 >> 172.21.15.170:0/3721699133 conn(0x7f9b340d0170 msgr2=0x7f9b3415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 shutdown_connections 2024-09-21T02:13:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.969+0000 7f9b3b4eb640 1 -- 172.21.15.170:0/3721699133 wait complete. 2024-09-21T02:13:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.973+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fecac15f4e0 con 0x7fecac15b8c0 2024-09-21T02:13:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.977+0000 7fec937fe640 1 -- 172.21.15.170:0/1476846617 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v21)=0 v21) ==== 106+0+0 (secure 0 0 0) 0x7fec9c0163f0 con 0x7fecac15b8c0 2024-09-21T02:13:00.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 msgr2=0x7fecac0b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 0x7fecac0b02e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009790 tx=0x7fec9c004040 comp rx=0 tx=0).stop 2024-09-21T02:13:00.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 shutdown_connections 2024-09-21T02:13:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecac15fa20 0x7fecac0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecac15b8c0 0x7fecac0b02e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 --2- 172.21.15.170:0/1476846617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecac0d1250 0x7fecac205ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 >> 172.21.15.170:0/1476846617 conn(0x7fecac09a9c0 msgr2=0x7fecac0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 shutdown_connections 2024-09-21T02:13:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.981+0000 7fecb09c9640 1 -- 172.21.15.170:0/1476846617 wait complete. 2024-09-21T02:13:00.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.993+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f3308005590 con 0x7f3328207910 2024-09-21T02:13:00.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.993+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f284c005860 con 0x7f286c153d40 2024-09-21T02:13:00.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f3324ff9640 1 -- 172.21.15.170:0/306126641 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f33180193b0 con 0x7f3328207910 2024-09-21T02:13:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3328207910 msgr2=0x7f33280c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3328207910 0x7f33280c7e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f331800b410 tx=0x7f331800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 shutdown_connections 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3328207910 0x7f33280c7e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3328154ce0 0x7f33280c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 --2- 172.21.15.170:0/306126641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3328071310 0x7f33282073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 >> 172.21.15.170:0/306126641 conn(0x7f33280d0080 msgr2=0x7f3328161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 shutdown_connections 2024-09-21T02:13:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:00.997+0000 7f332d374640 1 -- 172.21.15.170:0/306126641 wait complete. 2024-09-21T02:13:01.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2868ff9640 1 -- 172.21.15.170:0/4075261879 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f286c154690 con 0x7f286c153d40 2024-09-21T02:13:01.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f286c153d40 msgr2=0x7f286c154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f286c153d40 0x7f286c154150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f286400fcb0 tx=0x7f286400d430 comp rx=0 tx=0).stop 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 shutdown_connections 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f286c153d40 0x7f286c154150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f286c15c6b0 0x7f286c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 --2- 172.21.15.170:0/4075261879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c15c2a0 0x7f286c1f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 >> 172.21.15.170:0/4075261879 conn(0x7f286c06d650 msgr2=0x7f286c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 shutdown_connections 2024-09-21T02:13:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.005+0000 7f2871208640 1 -- 172.21.15.170:0/4075261879 wait complete. 2024-09-21T02:13:01.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi170') 2024-09-21T02:13:01.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi170') 2024-09-21T02:13:01.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi170') 2024-09-21T02:13:01.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi170') 2024-09-21T02:13:01.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi170') 2024-09-21T02:13:01.025 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi170') 2024-09-21T02:13:01.025 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config dump 2024-09-21T02:13:01.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/1505552805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b0003a10 msgr2=0x7fc7b0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c8ad8640 1 --2- 172.21.15.170:0/1505552805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b0003a10 0x7fc7b0003e80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009b30 tx=0x7fc7b401d270 comp rx=0 tx=0).stop 2024-09-21T02:13:01.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c2d9d640 1 -- 172.21.15.170:0/1505552805 reap_dead start 2024-09-21T02:13:01.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/1505552805 shutdown_connections 2024-09-21T02:13:01.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/1505552805 >> 172.21.15.170:0/1505552805 conn(0x7fc7c409a9c0 msgr2=0x7fc7c40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/1505552805 shutdown_connections 2024-09-21T02:13:01.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.201+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/1505552805 wait complete. 2024-09-21T02:13:01.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 Processor -- start 2024-09-21T02:13:01.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- start start 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7b0003a10 0x7fc7c415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 0x7fc7c4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7c4076120 0x7fc7c4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4166db0 con 0x7fc7c41588f0 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c40ccad0 con 0x7fc7b0003a10 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c4166f30 con 0x7fc7c4076120 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 0x7fc7c4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 0x7fc7c4158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53082/0 (socket says 172.21.15.170:53082) 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 -- 172.21.15.170:0/2178458479 learned_addr learned my addr 172.21.15.170:0/2178458479 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c2d9d640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7c4076120 0x7fc7c4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c259c640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7b0003a10 0x7fc7c415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 -- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7c4076120 msgr2=0x7fc7c4076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7c4076120 0x7fc7c4076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 -- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7b0003a10 msgr2=0x7fc7c415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7b0003a10 0x7fc7c415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 -- 172.21.15.170:0/2178458479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c4076a70 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c1d9b640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 0x7fc7c4158100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00d950 tx=0x7fc7ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac016070 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7ac009d20 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c40c9190 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0106c0 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c415c720 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7ac019070 con 0x7fc7c41588f0 2024-09-21T02:13:01.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc7ac01bcd0 con 0x7fc7c41588f0 2024-09-21T02:13:01.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.205+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c415ff00 con 0x7fc7c41588f0 2024-09-21T02:13:01.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.209+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc7c4076a70 con 0x7fc7c41588f0 2024-09-21T02:13:01.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc7c40ce9d0 con 0x7fc7c41588f0 2024-09-21T02:13:01.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7baffd640 1 -- 172.21.15.170:0/2178458479 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7fc7ac01b450 con 0x7fc7c41588f0 2024-09-21T02:13:01.265 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:13:01.265 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:13:01.265 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:13:01.265 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:13:01.265 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:13:01.266 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:13:01.267 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:13:01.267 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:13:01.267 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 msgr2=0x7fc7c4158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 0x7fc7c4158100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00d950 tx=0x7fc7ac00de10 comp rx=0 tx=0).stop 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 shutdown_connections 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7c4076120 0x7fc7c4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c41588f0 0x7fc7c4158100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 --2- 172.21.15.170:0/2178458479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7b0003a10 0x7fc7c415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 >> 172.21.15.170:0/2178458479 conn(0x7fc7c409a9c0 msgr2=0x7fc7c4166630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 shutdown_connections 2024-09-21T02:13:01.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.265+0000 7fc7c8ad8640 1 -- 172.21.15.170:0/2178458479 wait complete. 2024-09-21T02:13:01.284 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T02:13:01.285 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 -- 172.21.15.170:0/651856530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 msgr2=0x7f48b415f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/651856530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415f880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009c80 tx=0x7f48ac01cd10 comp rx=0 tx=0).stop 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 -- 172.21.15.170:0/651856530 shutdown_connections 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/651856530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415f880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/651856530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48b40cc8a0 0x7f48b40cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 -- 172.21.15.170:0/651856530 >> 172.21.15.170:0/651856530 conn(0x7f48b409a9c0 msgr2=0x7f48b40d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 -- 172.21.15.170:0/651856530 shutdown_connections 2024-09-21T02:13:01.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 -- 172.21.15.170:0/651856530 wait complete. 2024-09-21T02:13:01.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 Processor -- start 2024-09-21T02:13:01.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.429+0000 7f48bc4ad640 1 -- start start 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48b40b04e0 0x7f48b40b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 0x7f48b41fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4169350 con 0x7f48b40cc8a0 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f48b41691d0 con 0x7f48b415f470 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f48b41694d0 con 0x7f48b40b04e0 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 0x7f48b41fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 0x7f48b41fd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53090/0 (socket says 172.21.15.170:53090) 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 -- 172.21.15.170:0/2457613730 learned_addr learned my addr 172.21.15.170:0/2457613730 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48ba249640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48b40b04e0 0x7f48b40b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48baa4a640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 -- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48b40b04e0 msgr2=0x7f48b40b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48b40b04e0 0x7f48b40b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 -- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 msgr2=0x7f48b415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 -- 172.21.15.170:0/2457613730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b41fd600 con 0x7f48b40cc8a0 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48baa4a640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48b9a48640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 0x7f48b41fd0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48a400e7a0 tx=0x7f48a400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a400ccc0 con 0x7f48b40cc8a0 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48a4010430 con 0x7f48b40cc8a0 2024-09-21T02:13:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4010860 con 0x7f48b40cc8a0 2024-09-21T02:13:01.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b40b40d0 con 0x7f48b40cc8a0 2024-09-21T02:13:01.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b4155260 con 0x7f48b40cc8a0 2024-09-21T02:13:01.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f48a4010a00 con 0x7f48b40cc8a0 2024-09-21T02:13:01.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f48a4018d60 con 0x7f48b40cc8a0 2024-09-21T02:13:01.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.433+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b40cccb0 con 0x7f48b40cc8a0 2024-09-21T02:13:01.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.437+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f48b41fd600 con 0x7f48b40cc8a0 2024-09-21T02:13:01.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.497+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f48b40d19b0 con 0x7f48b40cc8a0 2024-09-21T02:13:01.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.497+0000 7f48aaffd640 1 -- 172.21.15.170:0/2457613730 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f48a4018420 con 0x7f48b40cc8a0 2024-09-21T02:13:01.498 INFO:teuthology.orchestra.run.smithi170.stderr:adjusted tunables profile to default 2024-09-21T02:13:01.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 msgr2=0x7f48b41fd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 0x7f48b41fd0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48a400e7a0 tx=0x7f48a400ec60 comp rx=0 tx=0).stop 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 shutdown_connections 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b415f470 0x7f48b415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48b40cc8a0 0x7f48b41fd0c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 --2- 172.21.15.170:0/2457613730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48b40b04e0 0x7f48b40b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 >> 172.21.15.170:0/2457613730 conn(0x7f48b409a9c0 msgr2=0x7f48b40b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 shutdown_connections 2024-09-21T02:13:01.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:01.501+0000 7f48bc4ad640 1 -- 172.21.15.170:0/2457613730 wait complete. 2024-09-21T02:13:01.518 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T02:13:01.518 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T02:13:01.518 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:01.518 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T02:13:01.529 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:01.530 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T02:13:01.581 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:01.582 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T02:13:01.634 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:01.634 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T02:13:01.686 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:13:01.686 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T02:13:01.699 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:13:01.699 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T02:13:01.753 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:13:01.753 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T02:13:01.806 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:13:01.806 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T02:13:01.858 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 46247445-2c36-4916-9356-d55f137755bd 0 2024-09-21T02:13:02.048 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 -- 172.21.15.191:0/997834730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c0d0420 msgr2=0x7f6c2c0d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/997834730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c0d0420 0x7f6c2c0d0830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c2001c8b0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 -- 172.21.15.191:0/997834730 shutdown_connections 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/997834730 >> v2:172.21.15.170:3300/0 conn(0x7f6c2c0b6110 0x7f6c2c0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/997834730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c0d0420 0x7f6c2c0d0830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 -- 172.21.15.191:0/997834730 >> 172.21.15.191:0/997834730 conn(0x7f6c2c09a9c0 msgr2=0x7f6c2c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 -- 172.21.15.191:0/997834730 shutdown_connections 2024-09-21T02:13:02.049 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.047+0000 7f6c33f8d640 1 -- 172.21.15.191:0/997834730 wait complete. 2024-09-21T02:13:02.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 Processor -- start 2024-09-21T02:13:02.051 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 -- start start 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c2c0b6110 0x7f6c2c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 0x7f6c2c1fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 0x7f6c2c1fca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c15d9d0 con 0x7f6c2c0b6110 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c15d850 con 0x7f6c2c0d0420 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c15db50 con 0x7f6c2c15a400 2024-09-21T02:13:02.055 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 0x7f6c2c1fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c31528640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 0x7f6c2c1fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 0x7f6c2c1fca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:40672/0 (socket says 172.21.15.191:40672) 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c31528640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 0x7f6c2c1fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59126/0 (socket says 172.21.15.191:59126) 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 -- 172.21.15.191:0/1843821065 learned_addr learned my addr 172.21.15.191:0/1843821065 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c31d29640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c2c0b6110 0x7f6c2c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 -- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 msgr2=0x7f6c2c1fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 0x7f6c2c1fc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 -- 172.21.15.191:0/1843821065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c2c0b6110 msgr2=0x7f6c2c159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c2c0b6110 0x7f6c2c159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 -- 172.21.15.191:0/1843821065 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c0b5540 con 0x7f6c2c15a400 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c31528640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 0x7f6c2c1fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c3252a640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 0x7f6c2c1fca50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e910 tx=0x7f6c2400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24004d30 con 0x7f6c2c15a400 2024-09-21T02:13:02.056 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c2400cac0 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c15e200 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24005020 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.051+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c15e7c0 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.055+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c240051c0 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.055+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f6c24011b80 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.055+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c0b6520 con 0x7f6c2c15a400 2024-09-21T02:13:02.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.055+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6c24016030 con 0x7f6c2c15a400 2024-09-21T02:13:02.128 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.127+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "46247445-2c36-4916-9356-d55f137755bd", "id": 0} v 0) -- 0x7f6c2c0ceb10 con 0x7f6c2c15a400 2024-09-21T02:13:02.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c1e7fc640 1 -- 172.21.15.191:0/1843821065 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "46247445-2c36-4916-9356-d55f137755bd", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f6c24011470 con 0x7f6c2c15a400 2024-09-21T02:13:02.135 INFO:teuthology.orchestra.run.smithi191.stdout:0 2024-09-21T02:13:02.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 msgr2=0x7f6c2c1fca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 0x7f6c2c1fca50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e910 tx=0x7f6c2400edd0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 shutdown_connections 2024-09-21T02:13:02.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c2c15a400 0x7f6c2c1fca50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c2c0d0420 0x7f6c2c1fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.137 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 --2- 172.21.15.191:0/1843821065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c2c0b6110 0x7f6c2c159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 >> 172.21.15.191:0/1843821065 conn(0x7f6c2c09a9c0 msgr2=0x7f6c2c0b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 shutdown_connections 2024-09-21T02:13:02.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.135+0000 7f6c33f8d640 1 -- 172.21.15.191:0/1843821065 wait complete. 2024-09-21T02:13:02.154 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new dfc90b85-149d-4f5c-8d74-883fcc23d5cf 1 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 -- 172.21.15.191:0/1134512674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 msgr2=0x7f434400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 --2- 172.21.15.191:0/1134512674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f434400b9d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4354009eb0 tx=0x7f435401d5f0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 -- 172.21.15.191:0/1134512674 shutdown_connections 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 --2- 172.21.15.191:0/1134512674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f434400b9d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 -- 172.21.15.191:0/1134512674 >> 172.21.15.191:0/1134512674 conn(0x7f435c09a9c0 msgr2=0x7f435c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 -- 172.21.15.191:0/1134512674 shutdown_connections 2024-09-21T02:13:02.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.303+0000 7f4361741640 1 -- 172.21.15.191:0/1134512674 wait complete. 2024-09-21T02:13:02.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 Processor -- start 2024-09-21T02:13:02.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- start start 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f435c0c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f435c0c4dd0 0x7f435c15f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f435c1564d0 0x7f435c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f435c0b10b0 con 0x7f435c0c4dd0 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f435c0b0f30 con 0x7f4344005360 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f435c0b1230 con 0x7f435c1564d0 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f435c0c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f435c0c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59140/0 (socket says 172.21.15.191:59140) 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 -- 172.21.15.191:0/3602042238 learned_addr learned my addr 172.21.15.191:0/3602042238 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435b7fe640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f435c1564d0 0x7f435c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435a7fc640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f435c0c4dd0 0x7f435c15f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 -- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f435c1564d0 msgr2=0x7f435c06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f435c1564d0 0x7f435c06ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 -- 172.21.15.191:0/3602042238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f435c0c4dd0 msgr2=0x7f435c15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f435c0c4dd0 0x7f435c15f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 -- 172.21.15.191:0/3602042238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c15de00 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f435affd640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f435c0c43a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f434c006480 tx=0x7f434c0039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c015070 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f434c011900 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c15e0e0 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c011d30 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c0b43b0 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f434c018070 con 0x7f4344005360 2024-09-21T02:13:02.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f434c023750 con 0x7f4344005360 2024-09-21T02:13:02.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.307+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435c0cddf0 con 0x7f4344005360 2024-09-21T02:13:02.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.311+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f434c0239f0 con 0x7f4344005360 2024-09-21T02:13:02.379 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.375+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dfc90b85-149d-4f5c-8d74-883fcc23d5cf", "id": 1} v 0) -- 0x7f435c0cb6d0 con 0x7f4344005360 2024-09-21T02:13:02.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f43437fe640 1 -- 172.21.15.191:0/3602042238 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dfc90b85-149d-4f5c-8d74-883fcc23d5cf", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f434c023d00 con 0x7f4344005360 2024-09-21T02:13:02.383 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:13:02.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 msgr2=0x7f435c0c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f435c0c43a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f434c006480 tx=0x7f434c0039d0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 shutdown_connections 2024-09-21T02:13:02.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f435c1564d0 0x7f435c06ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.384 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f435c0c4dd0 0x7f435c15f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 --2- 172.21.15.191:0/3602042238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4344005360 0x7f435c0c43a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 >> 172.21.15.191:0/3602042238 conn(0x7f435c09a9c0 msgr2=0x7f435c0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 shutdown_connections 2024-09-21T02:13:02.385 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.383+0000 7f4361741640 1 -- 172.21.15.191:0/3602042238 wait complete. 2024-09-21T02:13:02.401 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 16bebfcd-e061-4b85-b764-34d910585a69 2 2024-09-21T02:13:02.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 -- 172.21.15.191:0/358592538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09200cc8a0 msgr2=0x7f09200cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 --2- 172.21.15.191:0/358592538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09200cc8a0 0x7f09200cccb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0910009900 tx=0x7f091001c850 comp rx=0 tx=0).stop 2024-09-21T02:13:02.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 -- 172.21.15.191:0/358592538 shutdown_connections 2024-09-21T02:13:02.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 --2- 172.21.15.191:0/358592538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09200cc8a0 0x7f09200cccb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 --2- 172.21.15.191:0/358592538 >> v2:172.21.15.191:3300/0 conn(0x7f09200b04e0 0x7f09200b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 -- 172.21.15.191:0/358592538 >> 172.21.15.191:0/358592538 conn(0x7f092009a9c0 msgr2=0x7f09200d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 -- 172.21.15.191:0/358592538 shutdown_connections 2024-09-21T02:13:02.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 -- 172.21.15.191:0/358592538 wait complete. 2024-09-21T02:13:02.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 Processor -- start 2024-09-21T02:13:02.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.539+0000 7f0926ab3640 1 -- start start 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09200b04e0 0x7f0920150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09200cc8a0 0x7f09200ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 0x7f09200caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0920169350 con 0x7f09200cc8a0 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09201691d0 con 0x7f09200b04e0 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f09201694d0 con 0x7f092015f470 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 0x7f09200caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f092484f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09200b04e0 0x7f0920150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 0x7f09200caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:40704/0 (socket says 172.21.15.191:40704) 2024-09-21T02:13:02.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 -- 172.21.15.191:0/350130301 learned_addr learned my addr 172.21.15.191:0/350130301 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091ffff640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09200cc8a0 0x7f09200ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 -- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09200b04e0 msgr2=0x7f0920150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09200b04e0 0x7f0920150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 -- 172.21.15.191:0/350130301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09200cc8a0 msgr2=0x7f09200ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09200cc8a0 0x7f09200ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 -- 172.21.15.191:0/350130301 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09200cb720 con 0x7f092015f470 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091ffff640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09200cc8a0 0x7f09200ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0925050640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 0x7f09200caaa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f091800b9e0 tx=0x7f091800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918003b80 con 0x7f092015f470 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0918003e10 con 0x7f092015f470 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0920150aa0 con 0x7f092015f470 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091800cb80 con 0x7f092015f470 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0920159b70 con 0x7f092015f470 2024-09-21T02:13:02.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0918006d10 con 0x7f092015f470 2024-09-21T02:13:02.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0918014a40 con 0x7f092015f470 2024-09-21T02:13:02.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.543+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09200d19b0 con 0x7f092015f470 2024-09-21T02:13:02.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.547+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0918014d60 con 0x7f092015f470 2024-09-21T02:13:02.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.663+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "16bebfcd-e061-4b85-b764-34d910585a69", "id": 2} v 0) -- 0x7f09200afc00 con 0x7f092015f470 2024-09-21T02:13:02.667 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f091d7fa640 1 -- 172.21.15.191:0/350130301 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "16bebfcd-e061-4b85-b764-34d910585a69", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0918014620 con 0x7f092015f470 2024-09-21T02:13:02.668 INFO:teuthology.orchestra.run.smithi191.stdout:2 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 msgr2=0x7f09200caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 0x7f09200caaa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f091800b9e0 tx=0x7f091800bea0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 shutdown_connections 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f092015f470 0x7f09200caaa0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09200cc8a0 0x7f09200ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 --2- 172.21.15.191:0/350130301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09200b04e0 0x7f0920150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 >> 172.21.15.191:0/350130301 conn(0x7f092009a9c0 msgr2=0x7f09200b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 shutdown_connections 2024-09-21T02:13:02.669 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.667+0000 7f0926ab3640 1 -- 172.21.15.191:0/350130301 wait complete. 2024-09-21T02:13:02.685 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new fb2695db-07dd-4a82-a7a5-9bf70a4df160 3 2024-09-21T02:13:02.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- 172.21.15.191:0/2546418599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26201606e0 msgr2=0x7f2620160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 --2- 172.21.15.191:0/2546418599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26201606e0 0x7f2620160af0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f26200ce420 tx=0x7f261001cd10 comp rx=0 tx=0).stop 2024-09-21T02:13:02.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- 172.21.15.191:0/2546418599 shutdown_connections 2024-09-21T02:13:02.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 --2- 172.21.15.191:0/2546418599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26201606e0 0x7f2620160af0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- 172.21.15.191:0/2546418599 >> 172.21.15.191:0/2546418599 conn(0x7f26200d2a30 msgr2=0x7f26200cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- 172.21.15.191:0/2546418599 shutdown_connections 2024-09-21T02:13:02.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- 172.21.15.191:0/2546418599 wait complete. 2024-09-21T02:13:02.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 Processor -- start 2024-09-21T02:13:02.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- start start 2024-09-21T02:13:02.823 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26200d0700 0x7f2620151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26200d0b10 0x7f2620154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 0x7f2620154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f262016aed0 con 0x7f26200d0700 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2620071a70 con 0x7f26200d0b10 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f2625928640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f262016b050 con 0x7f26201606e0 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26200d0b10 0x7f2620154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 0x7f2620154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 0x7f2620154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:40714/0 (socket says 172.21.15.191:40714) 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26200d0b10 0x7f2620154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59180/0 (socket says 172.21.15.191:59180) 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 -- 172.21.15.191:0/3498597846 learned_addr learned my addr 172.21.15.191:0/3498597846 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261effd640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26200d0700 0x7f2620151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 -- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26200d0b10 msgr2=0x7f2620154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26200d0b10 0x7f2620154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 -- 172.21.15.191:0/3498597846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26200d0700 msgr2=0x7f2620151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26200d0700 0x7f2620151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.819+0000 7f261f7fe640 1 -- 172.21.15.191:0/3498597846 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2620140980 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f261f7fe640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 0x7f2620154c70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f261800b410 tx=0x7f261800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618015020 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2618011920 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2620140c20 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618011d70 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201523b0 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2618018020 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f2618019c50 con 0x7f26201606e0 2024-09-21T02:13:02.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2620160af0 con 0x7f26201606e0 2024-09-21T02:13:02.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.823+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f261801f020 con 0x7f26201606e0 2024-09-21T02:13:02.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.895+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb2695db-07dd-4a82-a7a5-9bf70a4df160", "id": 3} v 0) -- 0x7f2620071290 con 0x7f26201606e0 2024-09-21T02:13:02.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2617fff640 1 -- 172.21.15.191:0/3498597846 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb2695db-07dd-4a82-a7a5-9bf70a4df160", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f26180193b0 con 0x7f26201606e0 2024-09-21T02:13:02.900 INFO:teuthology.orchestra.run.smithi191.stdout:3 2024-09-21T02:13:02.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 msgr2=0x7f2620154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:02.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 0x7f2620154c70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f261800b410 tx=0x7f261800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.901 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 shutdown_connections 2024-09-21T02:13:02.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26201606e0 0x7f2620154c70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26200d0b10 0x7f2620154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 --2- 172.21.15.191:0/3498597846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26200d0700 0x7f2620151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:02.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 >> 172.21.15.191:0/3498597846 conn(0x7f26200d2a30 msgr2=0x7f262015bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:02.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 shutdown_connections 2024-09-21T02:13:02.902 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:02.899+0000 7f2625928640 1 -- 172.21.15.191:0/3498597846 wait complete. 2024-09-21T02:13:02.909 INFO:tasks.ceph.mgr.x.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T02:13:02.909 INFO:tasks.ceph.mgr.x.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T02:13:02.909 INFO:tasks.ceph.mgr.x.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-21T02:13:02.918 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 009a9026-b1d1-4160-b28e-fc3ccb6e12b2 4 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- 172.21.15.191:0/889252881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14540cc8a0 msgr2=0x7f14540cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- 172.21.15.191:0/889252881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14540cc8a0 0x7f14540cccb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1454072930 tx=0x7f144801c770 comp rx=0 tx=0).stop 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- 172.21.15.191:0/889252881 shutdown_connections 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- 172.21.15.191:0/889252881 >> v2:172.21.15.170:3300/0 conn(0x7f145415f470 0x7f145415f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- 172.21.15.191:0/889252881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14540cc8a0 0x7f14540cccb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- 172.21.15.191:0/889252881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14540b04e0 0x7f14540b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- 172.21.15.191:0/889252881 >> 172.21.15.191:0/889252881 conn(0x7f145409a9c0 msgr2=0x7f14540d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- 172.21.15.191:0/889252881 shutdown_connections 2024-09-21T02:13:03.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- 172.21.15.191:0/889252881 wait complete. 2024-09-21T02:13:03.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 Processor -- start 2024-09-21T02:13:03.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- start start 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14540b04e0 0x7f145415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14540cc8a0 0x7f14540ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 0x7f14540ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14540ce470 con 0x7f14540b04e0 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f14540ce2f0 con 0x7f145415f470 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.059+0000 7f145893a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f14540ce5f0 con 0x7f14540cc8a0 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 0x7f14540ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1451d9b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14540cc8a0 0x7f14540ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 0x7f14540ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59192/0 (socket says 172.21.15.191:59192) 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 -- 172.21.15.191:0/1847734971 learned_addr learned my addr 172.21.15.191:0/1847734971 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f145259c640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14540b04e0 0x7f145415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 -- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14540cc8a0 msgr2=0x7f14540ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14540cc8a0 0x7f14540ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 -- 172.21.15.191:0/1847734971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14540b04e0 msgr2=0x7f145415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14540b04e0 0x7f145415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 -- 172.21.15.191:0/1847734971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145414f2c0 con 0x7f145415f470 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1452d9d640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 0x7f14540ae780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e940 tx=0x7f144c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c004d30 con 0x7f145415f470 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f144c00cb20 con 0x7f145415f470 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145415e860 con 0x7f145415f470 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c005020 con 0x7f145415f470 2024-09-21T02:13:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145415ee20 con 0x7f145415f470 2024-09-21T02:13:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14540cccb0 con 0x7f145415f470 2024-09-21T02:13:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f144c0051c0 con 0x7f145415f470 2024-09-21T02:13:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f144c01a400 con 0x7f145415f470 2024-09-21T02:13:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.063+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f144c016030 con 0x7f145415f470 2024-09-21T02:13:03.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.135+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "009a9026-b1d1-4160-b28e-fc3ccb6e12b2", "id": 4} v 0) -- 0x7f14540d19b0 con 0x7f145415f470 2024-09-21T02:13:03.142 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.139+0000 7f1442ffd640 1 -- 172.21.15.191:0/1847734971 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "009a9026-b1d1-4160-b28e-fc3ccb6e12b2", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f144c011540 con 0x7f145415f470 2024-09-21T02:13:03.142 INFO:teuthology.orchestra.run.smithi191.stdout:4 2024-09-21T02:13:03.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 msgr2=0x7f14540ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 0x7f14540ae780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e940 tx=0x7f144c00ee00 comp rx=0 tx=0).stop 2024-09-21T02:13:03.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 shutdown_connections 2024-09-21T02:13:03.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14540cc8a0 0x7f14540ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f145415f470 0x7f14540ae780 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 --2- 172.21.15.191:0/1847734971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14540b04e0 0x7f145415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 >> 172.21.15.191:0/1847734971 conn(0x7f145409a9c0 msgr2=0x7f14540b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 shutdown_connections 2024-09-21T02:13:03.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.143+0000 7f145893a640 1 -- 172.21.15.191:0/1847734971 wait complete. 2024-09-21T02:13:03.161 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 0a1faa45-e320-423a-b677-6e0e691f3988 5 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 -- 172.21.15.191:0/758546564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb190160af0 msgr2=0x7fb19015d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 --2- 172.21.15.191:0/758546564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb190160af0 0x7fb19015d3e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb184009900 tx=0x7fb18401c850 comp rx=0 tx=0).stop 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 -- 172.21.15.191:0/758546564 shutdown_connections 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 --2- 172.21.15.191:0/758546564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb190160af0 0x7fb19015d3e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 -- 172.21.15.191:0/758546564 >> 172.21.15.191:0/758546564 conn(0x7fb1900d2a30 msgr2=0x7fb1900cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 -- 172.21.15.191:0/758546564 shutdown_connections 2024-09-21T02:13:03.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 -- 172.21.15.191:0/758546564 wait complete. 2024-09-21T02:13:03.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 Processor -- start 2024-09-21T02:13:03.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.299+0000 7fb196685640 1 -- start start 2024-09-21T02:13:03.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb19015d920 0x7fb1900b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 0x7fb19015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb190160af0 0x7fb190150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb19016aed0 con 0x7fb19015d920 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb190071ac0 con 0x7fb1901606e0 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb19016b050 con 0x7fb190160af0 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 0x7fb19015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 0x7fb19015f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59210/0 (socket says 172.21.15.191:59210) 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18f7fe640 1 -- 172.21.15.191:0/1334451821 learned_addr learned my addr 172.21.15.191:0/1334451821 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb190160af0 0x7fb190150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18ffff640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb19015d920 0x7fb1900b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 -- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 msgr2=0x7fb19015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 0x7fb19015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 -- 172.21.15.191:0/1334451821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb19015d920 msgr2=0x7fb1900b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb19015d920 0x7fb1900b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 -- 172.21.15.191:0/1334451821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1900b6300 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18f7fe640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 0x7fb19015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb194c22640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb190160af0 0x7fb190150190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb1900cded0 tx=0x7fb18800cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188003b80 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb188003d20 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1900b65e0 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188005330 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1900b6ba0 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb1880054d0 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fb18801ba70 con 0x7fb190160af0 2024-09-21T02:13:03.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.303+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19015d3e0 con 0x7fb190160af0 2024-09-21T02:13:03.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.307+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb18801b4f0 con 0x7fb190160af0 2024-09-21T02:13:03.368 INFO:tasks.ceph.mgr.y.smithi170.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T02:13:03.369 INFO:tasks.ceph.mgr.y.smithi170.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T02:13:03.369 INFO:tasks.ceph.mgr.y.smithi170.stderr: from numpy import show_config as show_numpy_config 2024-09-21T02:13:03.376 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.375+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0a1faa45-e320-423a-b677-6e0e691f3988", "id": 5} v 0) -- 0x7fb19015ca90 con 0x7fb190160af0 2024-09-21T02:13:03.381 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb18cff9640 1 -- 172.21.15.191:0/1334451821 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0a1faa45-e320-423a-b677-6e0e691f3988", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fb18801b6d0 con 0x7fb190160af0 2024-09-21T02:13:03.381 INFO:teuthology.orchestra.run.smithi191.stdout:5 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb190160af0 msgr2=0x7fb190150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb190160af0 0x7fb190150190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb1900cded0 tx=0x7fb18800cae0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 shutdown_connections 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb190160af0 0x7fb190150190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1901606e0 0x7fb19015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 --2- 172.21.15.191:0/1334451821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb19015d920 0x7fb1900b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 >> 172.21.15.191:0/1334451821 conn(0x7fb1900d2a30 msgr2=0x7fb1900b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 shutdown_connections 2024-09-21T02:13:03.383 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.379+0000 7fb196685640 1 -- 172.21.15.191:0/1334451821 wait complete. 2024-09-21T02:13:03.400 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new beeea962-a5f5-4cad-a37a-f6d0f5c91199 6 2024-09-21T02:13:03.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 -- 172.21.15.191:0/2079071325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f8815a400 msgr2=0x7f9f8806ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 --2- 172.21.15.191:0/2079071325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f8815a400 0x7f9f8806ce00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f880ced30 tx=0x7f9f7801c8a0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 -- 172.21.15.191:0/2079071325 shutdown_connections 2024-09-21T02:13:03.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 --2- 172.21.15.191:0/2079071325 >> v2:172.21.15.170:3300/0 conn(0x7f9f8806d340 0x7f9f880d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 --2- 172.21.15.191:0/2079071325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f8815a400 0x7f9f8806ce00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.540 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 -- 172.21.15.191:0/2079071325 >> 172.21.15.191:0/2079071325 conn(0x7f9f8809a9c0 msgr2=0x7f9f880d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 -- 172.21.15.191:0/2079071325 shutdown_connections 2024-09-21T02:13:03.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 -- 172.21.15.191:0/2079071325 wait complete. 2024-09-21T02:13:03.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 Processor -- start 2024-09-21T02:13:03.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.539+0000 7f9f8d908640 1 -- start start 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 0x7f9f881674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 0x7f9f880c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f8815a400 0x7f9f880c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8815d9d0 con 0x7f9f8815a400 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8815d850 con 0x7f9f880d0960 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8815db50 con 0x7f9f8806d340 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 0x7f9f881674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f867fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 0x7f9f880c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 0x7f9f881674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:40744/0 (socket says 172.21.15.191:40744) 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 -- 172.21.15.191:0/454406836 learned_addr learned my addr 172.21.15.191:0/454406836 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f867fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 0x7f9f880c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59226/0 (socket says 172.21.15.191:59226) 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f877fe640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f8815a400 0x7f9f880c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 -- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 msgr2=0x7f9f880c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 0x7f9f880c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 -- 172.21.15.191:0/454406836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f8815a400 msgr2=0x7f9f880c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f8815a400 0x7f9f880c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 -- 172.21.15.191:0/454406836 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f881fc720 con 0x7f9f8806d340 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f867fc640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 0x7f9f880c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f86ffd640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 0x7f9f881674e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9f78000c10 tx=0x7f9f780a5760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f780bb070 con 0x7f9f8806d340 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f78005030 con 0x7f9f8806d340 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f881fca00 con 0x7f9f8806d340 2024-09-21T02:13:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f780b2040 con 0x7f9f8806d340 2024-09-21T02:13:03.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f880c7380 con 0x7f9f8806d340 2024-09-21T02:13:03.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f780be070 con 0x7f9f8806d340 2024-09-21T02:13:03.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9f78003bf0 con 0x7f9f8806d340 2024-09-21T02:13:03.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.543+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f880c75e0 con 0x7f9f8806d340 2024-09-21T02:13:03.547 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.547+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9f78016490 con 0x7f9f8806d340 2024-09-21T02:13:03.619 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.619+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "beeea962-a5f5-4cad-a37a-f6d0f5c91199", "id": 6} v 0) -- 0x7f9f88165990 con 0x7f9f8806d340 2024-09-21T02:13:03.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.623+0000 7f9f6ffff640 1 -- 172.21.15.191:0/454406836 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "beeea962-a5f5-4cad-a37a-f6d0f5c91199", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9f780046d0 con 0x7f9f8806d340 2024-09-21T02:13:03.626 INFO:teuthology.orchestra.run.smithi191.stdout:6 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 msgr2=0x7f9f881674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 0x7f9f881674e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9f78000c10 tx=0x7f9f780a5760 comp rx=0 tx=0).stop 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 shutdown_connections 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f8815a400 0x7f9f880c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f880d0960 0x7f9f880c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 --2- 172.21.15.191:0/454406836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f8806d340 0x7f9f881674e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 >> 172.21.15.191:0/454406836 conn(0x7f9f8809a9c0 msgr2=0x7f9f880cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 shutdown_connections 2024-09-21T02:13:03.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.627+0000 7f9f8d908640 1 -- 172.21.15.191:0/454406836 wait complete. 2024-09-21T02:13:03.645 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 274c2ada-cfc8-41eb-aa4f-88f17b4adb3e 7 2024-09-21T02:13:03.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 -- 172.21.15.191:0/803305612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a04160af0 msgr2=0x7f9a0415d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 --2- 172.21.15.191:0/803305612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a04160af0 0x7f9a0415d3e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009900 tx=0x7f9a0001c850 comp rx=0 tx=0).stop 2024-09-21T02:13:03.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 -- 172.21.15.191:0/803305612 shutdown_connections 2024-09-21T02:13:03.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 --2- 172.21.15.191:0/803305612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a04160af0 0x7f9a0415d3e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.789 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 -- 172.21.15.191:0/803305612 >> 172.21.15.191:0/803305612 conn(0x7f9a040d2a30 msgr2=0x7f9a040cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 -- 172.21.15.191:0/803305612 shutdown_connections 2024-09-21T02:13:03.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 -- 172.21.15.191:0/803305612 wait complete. 2024-09-21T02:13:03.790 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.787+0000 7f9a0b96d640 1 Processor -- start 2024-09-21T02:13:03.791 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- start start 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 0x7f9a0414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 0x7f9a0415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a04160af0 0x7f9a0415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0416ae40 con 0x7f9a04160af0 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0416acc0 con 0x7f9a0415d920 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0416afc0 con 0x7f9a041606e0 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 0x7f9a0415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.793 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a09709640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 0x7f9a0414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 0x7f9a0415eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:40750/0 (socket says 172.21.15.191:40750) 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a09709640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 0x7f9a0414f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:59238/0 (socket says 172.21.15.191:59238) 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 -- 172.21.15.191:0/424909372 learned_addr learned my addr 172.21.15.191:0/424909372 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a09f0a640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a04160af0 0x7f9a0415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 -- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 msgr2=0x7f9a0414f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 0x7f9a0414f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 -- 172.21.15.191:0/424909372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a04160af0 msgr2=0x7f9a0415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a04160af0 0x7f9a0415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 -- 172.21.15.191:0/424909372 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a041663b0 con 0x7f9a041606e0 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a09709640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 0x7f9a0414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:03.794 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a08f08640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 0x7f9a0415eee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009850 tx=0x7f9a000a5590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:03.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000b2040 con 0x7f9a041606e0 2024-09-21T02:13:03.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a000043b0 con 0x7f9a041606e0 2024-09-21T02:13:03.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00003860 con 0x7f9a041606e0 2024-09-21T02:13:03.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0406fd30 con 0x7f9a041606e0 2024-09-21T02:13:03.795 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a04165bb0 con 0x7f9a041606e0 2024-09-21T02:13:03.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a000ad440 con 0x7f9a041606e0 2024-09-21T02:13:03.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f9a000c0a90 con 0x7f9a041606e0 2024-09-21T02:13:03.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.791+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0415d3e0 con 0x7f9a041606e0 2024-09-21T02:13:03.796 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.795+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9a000164d0 con 0x7f9a041606e0 2024-09-21T02:13:03.863 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.859+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "274c2ada-cfc8-41eb-aa4f-88f17b4adb3e", "id": 7} v 0) -- 0x7f9a0415a000 con 0x7f9a041606e0 2024-09-21T02:13:03.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f99f5ffb640 1 -- 172.21.15.191:0/424909372 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "274c2ada-cfc8-41eb-aa4f-88f17b4adb3e", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9a000c0430 con 0x7f9a041606e0 2024-09-21T02:13:03.869 INFO:teuthology.orchestra.run.smithi191.stdout:7 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 msgr2=0x7f9a0415eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 0x7f9a0415eee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009850 tx=0x7f9a000a5590 comp rx=0 tx=0).stop 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 shutdown_connections 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a041606e0 0x7f9a0415eee0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a04160af0 0x7f9a0415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 --2- 172.21.15.191:0/424909372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0415d920 0x7f9a0414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 >> 172.21.15.191:0/424909372 conn(0x7f9a040d2a30 msgr2=0x7f9a0415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 shutdown_connections 2024-09-21T02:13:03.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:03.867+0000 7f9a0b96d640 1 -- 172.21.15.191:0/424909372 wait complete. 2024-09-21T02:13:03.889 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T02:13:03.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T02:13:03.892 INFO:tasks.ceph.osd.0:Started 2024-09-21T02:13:03.892 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T02:13:03.892 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T02:13:03.894 INFO:tasks.ceph.osd.1:Started 2024-09-21T02:13:03.894 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T02:13:03.894 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T02:13:03.897 INFO:tasks.ceph.osd.2:Started 2024-09-21T02:13:03.897 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T02:13:03.897 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T02:13:03.899 INFO:tasks.ceph.osd.3:Started 2024-09-21T02:13:03.899 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T02:13:03.899 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T02:13:03.901 INFO:tasks.ceph.osd.4:Started 2024-09-21T02:13:03.901 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T02:13:03.902 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T02:13:03.903 INFO:tasks.ceph.osd.5:Started 2024-09-21T02:13:03.903 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T02:13:03.904 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T02:13:03.905 INFO:tasks.ceph.osd.6:Started 2024-09-21T02:13:03.905 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T02:13:03.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T02:13:03.908 INFO:tasks.ceph.osd.7:Started 2024-09-21T02:13:03.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- 172.21.15.170:0/3521964839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780cc1b0 msgr2=0x7f18780cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 --2- 172.21.15.170:0/3521964839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780cc1b0 0x7f18780cc5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18780caf40 tx=0x7f187001d750 comp rx=0 tx=0).stop 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- 172.21.15.170:0/3521964839 shutdown_connections 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 --2- 172.21.15.170:0/3521964839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780cc1b0 0x7f18780cc5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 --2- 172.21.15.170:0/3521964839 >> v2:172.21.15.191:3300/0 conn(0x7f1878071bd0 0x7f187815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- 172.21.15.170:0/3521964839 >> 172.21.15.170:0/3521964839 conn(0x7f18780cd3c0 msgr2=0x7f18780cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- 172.21.15.170:0/3521964839 shutdown_connections 2024-09-21T02:13:04.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- 172.21.15.170:0/3521964839 wait complete. 2024-09-21T02:13:04.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 Processor -- start 2024-09-21T02:13:04.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- start start 2024-09-21T02:13:04.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878071bd0 0x7f1878158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18780cc1b0 0x7f1878150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 0x7f1878150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1878168de0 con 0x7f18780ce510 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1878168c60 con 0x7f1878071bd0 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187d10d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1878168f60 con 0x7f18780cc1b0 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.093+0000 7f187759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 0x7f1878150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 0x7f1878150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53158/0 (socket says 172.21.15.170:53158) 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 -- 172.21.15.170:0/1452527317 learned_addr learned my addr 172.21.15.170:0/1452527317 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f186ffff640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18780cc1b0 0x7f1878150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f1876d9d640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878071bd0 0x7f1878158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 -- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18780cc1b0 msgr2=0x7f1878150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18780cc1b0 0x7f1878150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 -- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878071bd0 msgr2=0x7f1878158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878071bd0 0x7f1878158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 -- 172.21.15.170:0/1452527317 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187814f1a0 con 0x7f18780ce510 2024-09-21T02:13:04.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187759e640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 0x7f1878150850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f187001cd30 tx=0x7f1870003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:04.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700bb070 con 0x7f18780ce510 2024-09-21T02:13:04.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1870004b40 con 0x7f18780ce510 2024-09-21T02:13:04.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700b2040 con 0x7f18780ce510 2024-09-21T02:13:04.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187814fb30 con 0x7f18780ce510 2024-09-21T02:13:04.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187814f440 con 0x7f18780ce510 2024-09-21T02:13:04.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1870004000 con 0x7f18780ce510 2024-09-21T02:13:04.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1870005890 con 0x7f18780ce510 2024-09-21T02:13:04.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.097+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187815b5e0 con 0x7f18780ce510 2024-09-21T02:13:04.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.101+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f18700c5080 con 0x7f18780ce510 2024-09-21T02:13:04.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18781534c0 con 0x7f18780ce510 2024-09-21T02:13:04.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f186f7fe640 1 -- 172.21.15.170:0/1452527317 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1870016970 con 0x7f18780ce510 2024-09-21T02:13:04.161 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:04.161 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":10,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:03.867885+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 msgr2=0x7f1878150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 0x7f1878150850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f187001cd30 tx=0x7f1870003f80 comp rx=0 tx=0).stop 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 shutdown_connections 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18780cc1b0 0x7f1878150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18780ce510 0x7f1878150850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 --2- 172.21.15.170:0/1452527317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878071bd0 0x7f1878158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.161+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 >> 172.21.15.170:0/1452527317 conn(0x7f18780cd3c0 msgr2=0x7f18780cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.165+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 shutdown_connections 2024-09-21T02:13:04.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:04.165+0000 7f187d10d640 1 -- 172.21.15.170:0/1452527317 wait complete. 2024-09-21T02:13:04.181 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T02:13:04.181 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T02:13:04.986 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:05.053 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-09-21T02:13:05.051+0000 7f83696a47c0 -1 Falling back to public interface 2024-09-21T02:13:05.133 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 -- 172.21.15.170:0/19704624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 msgr2=0x7f3b9006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:05.133 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 --2- 172.21.15.170:0/19704624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 0x7f3b9006d210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009900 tx=0x7f3b8401c820 comp rx=0 tx=0).stop 2024-09-21T02:13:05.133 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 -- 172.21.15.170:0/19704624 shutdown_connections 2024-09-21T02:13:05.133 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 --2- 172.21.15.170:0/19704624 >> v2:172.21.15.191:3300/0 conn(0x7f3b901607b0 0x7f3b90160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.133 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 --2- 172.21.15.170:0/19704624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 0x7f3b9006d210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.133 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 -- 172.21.15.170:0/19704624 >> 172.21.15.170:0/19704624 conn(0x7f3b900d0b40 msgr2=0x7f3b900d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:05.134 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 -- 172.21.15.170:0/19704624 shutdown_connections 2024-09-21T02:13:05.134 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.132+0000 7f3b968e4640 1 -- 172.21.15.170:0/19704624 wait complete. 2024-09-21T02:13:05.135 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 Processor -- start 2024-09-21T02:13:05.135 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 -- start start 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 0x7f3b90152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b900b6400 0x7f3b90156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 0x7f3b901fada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90169bb0 con 0x7f3b901607b0 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90169d30 con 0x7f3b900b6400 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3b90169eb0 con 0x7f3b9006ce00 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 0x7f3b901fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:05.138 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 0x7f3b901fada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53166/0 (socket says 172.21.15.170:53166) 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 -- 172.21.15.170:0/2550732712 learned_addr learned my addr 172.21.15.170:0/2550732712 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8ffff640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 0x7f3b90152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8f7fe640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b900b6400 0x7f3b90156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 -- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 msgr2=0x7f3b90152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 0x7f3b90152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 -- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b900b6400 msgr2=0x7f3b90156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b900b6400 0x7f3b90156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 -- 172.21.15.170:0/2550732712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b901fa600 con 0x7f3b901607b0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8f7fe640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b900b6400 0x7f3b90156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b94e81640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 0x7f3b901fada0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e9a0 tx=0x7f3b8000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000ccc0 con 0x7f3b901607b0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b8000ce60 con 0x7f3b901607b0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b901573f0 con 0x7f3b901607b0 2024-09-21T02:13:05.139 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80004280 con 0x7f3b901607b0 2024-09-21T02:13:05.140 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b90158910 con 0x7f3b901607b0 2024-09-21T02:13:05.140 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.136+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 2) ==== 48655+0+0 (secure 0 0 0) 0x7f3b80003600 con 0x7f3b901607b0 2024-09-21T02:13:05.140 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.140+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3b8004f6b0 con 0x7f3b901607b0 2024-09-21T02:13:05.140 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.140+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9006d210 con 0x7f3b901607b0 2024-09-21T02:13:05.143 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.144+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3b901fa600 con 0x7f3b901607b0 2024-09-21T02:13:05.148 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-09-21T02:13:05.148+0000 7fca5511c7c0 -1 Falling back to public interface 2024-09-21T02:13:05.202 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.200+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b90156c70 con 0x7f3b901607b0 2024-09-21T02:13:05.203 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b8cff9640 1 -- 172.21.15.170:0/2550732712 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3b8000fdb0 con 0x7f3b901607b0 2024-09-21T02:13:05.203 INFO:teuthology.misc.health.smithi170.stdout: 2024-09-21T02:13:05.203 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":10,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:03.867885+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:05.204 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 msgr2=0x7f3b901fada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 0x7f3b901fada0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000e9a0 tx=0x7f3b8000ee60 comp rx=0 tx=0).stop 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 shutdown_connections 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b901607b0 0x7f3b901fada0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b900b6400 0x7f3b90156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 --2- 172.21.15.170:0/2550732712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b9006ce00 0x7f3b90152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 >> 172.21.15.170:0/2550732712 conn(0x7f3b900d0b40 msgr2=0x7f3b9015f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 shutdown_connections 2024-09-21T02:13:05.205 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:05.204+0000 7f3b968e4640 1 -- 172.21.15.170:0/2550732712 wait complete. 2024-09-21T02:13:05.221 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T02:13:05.581 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-09-21T02:13:05.580+0000 7f135ede27c0 -1 Falling back to public interface 2024-09-21T02:13:05.793 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-09-21T02:13:05.791+0000 7fa7d3abc7c0 -1 Falling back to public interface 2024-09-21T02:13:05.801 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-09-21T02:13:05.799+0000 7f2417ce37c0 -1 Falling back to public interface 2024-09-21T02:13:05.820 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-09-21T02:13:05.819+0000 7f3008c037c0 -1 Falling back to public interface 2024-09-21T02:13:05.829 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-09-21T02:13:05.828+0000 7f08097897c0 -1 Falling back to public interface 2024-09-21T02:13:05.871 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:13:05.871+0000 7f1a64638640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T02:13:05.871 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:13:05.871+0000 7f1a64638640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T02:13:05.896 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-09-21T02:13:05.896+0000 7f69c974b7c0 -1 Falling back to public interface 2024-09-21T02:13:09.043 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-09-21T02:13:09.043+0000 7f83696a47c0 -1 osd.4 0 log_to_monitors true 2024-09-21T02:13:09.417 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-09-21T02:13:09.416+0000 7fca5511c7c0 -1 osd.0 0 log_to_monitors true 2024-09-21T02:13:09.650 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-09-21T02:13:09.648+0000 7f135ede27c0 -1 osd.2 0 log_to_monitors true 2024-09-21T02:13:09.839 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-09-21T02:13:09.836+0000 7f08097897c0 -1 osd.3 0 log_to_monitors true 2024-09-21T02:13:09.858 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-09-21T02:13:09.855+0000 7f2417ce37c0 -1 osd.5 0 log_to_monitors true 2024-09-21T02:13:10.057 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-09-21T02:13:10.055+0000 7fa7d3abc7c0 -1 osd.7 0 log_to_monitors true 2024-09-21T02:13:10.175 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-09-21T02:13:10.175+0000 7f3008c037c0 -1 osd.6 0 log_to_monitors true 2024-09-21T02:13:10.272 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-09-21T02:13:10.272+0000 7f69c974b7c0 -1 osd.1 0 log_to_monitors true 2024-09-21T02:13:11.148 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-09-21T02:13:11.147+0000 7f8365e5f640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T02:13:11.214 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-09-21T02:13:11.211+0000 7f835d447640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:12.027 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:12.232 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 -- 172.21.15.170:0/518928581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0cd370 msgr2=0x7f41bc06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:12.232 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 --2- 172.21.15.170:0/518928581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0cd370 0x7f41bc06ed80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009900 tx=0x7f41ac01c7c0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.232 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c101e640 1 -- 172.21.15.170:0/518928581 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a5cd0 con 0x7f41bc0cd370 2024-09-21T02:13:12.232 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 -- 172.21.15.170:0/518928581 shutdown_connections 2024-09-21T02:13:12.233 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 --2- 172.21.15.170:0/518928581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41bc06f2c0 0x7f41bc0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.233 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 --2- 172.21.15.170:0/518928581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0cd370 0x7f41bc06ed80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.233 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 -- 172.21.15.170:0/518928581 >> 172.21.15.170:0/518928581 conn(0x7f41bc06d790 msgr2=0x7f41bc0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:12.233 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 -- 172.21.15.170:0/518928581 shutdown_connections 2024-09-21T02:13:12.235 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 -- 172.21.15.170:0/518928581 wait complete. 2024-09-21T02:13:12.235 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 Processor -- start 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.232+0000 7f41c4284640 1 -- start start 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41bc0cf1f0 0x7f41bc0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41bc0a9e20 0x7f41bc0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 0x7f41bc0b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc15d370 con 0x7f41bc1598c0 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 0x7f41bc0b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc15d1f0 con 0x7f41bc0a9e20 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 0x7f41bc0b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45742/0 (socket says 172.21.15.170:45742) 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 -- 172.21.15.170:0/4004390827 learned_addr learned my addr 172.21.15.170:0/4004390827 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc15d4f0 con 0x7f41bc0cf1f0 2024-09-21T02:13:12.238 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c181f640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41bc0a9e20 0x7f41bc0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:12.239 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2020640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41bc0cf1f0 0x7f41bc0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:12.239 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 -- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41bc0cf1f0 msgr2=0x7f41bc0c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41bc0cf1f0 0x7f41bc0c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 -- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41bc0a9e20 msgr2=0x7f41bc0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41bc0a9e20 0x7f41bc0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 -- 172.21.15.170:0/4004390827 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc0ac6f0 con 0x7f41bc1598c0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c2821640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 0x7f41bc0b56a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c6e0 tx=0x7f41b400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4003b80 con 0x7f41bc1598c0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41b4003d40 con 0x7f41bc1598c0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4005330 con 0x7f41bc1598c0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc0ac9d0 con 0x7f41bc1598c0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.236+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc0acf90 con 0x7f41bc1598c0 2024-09-21T02:13:12.240 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.240+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc0aa370 con 0x7f41bc1598c0 2024-09-21T02:13:12.241 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.240+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f41b40054d0 con 0x7f41bc1598c0 2024-09-21T02:13:12.248 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.240+0000 7f41b2ffd640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41a0073170 0x7f41a0075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:12.248 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.240+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f41b4093180 con 0x7f41bc1598c0 2024-09-21T02:13:12.248 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.240+0000 7f41c2020640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41a0073170 0x7f41a0075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:12.248 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.244+0000 7f41c2020640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41a0073170 0x7f41a0075680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0af000 tx=0x7f41ac01b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:12.248 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.248+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f41b405e8a0 con 0x7f41bc1598c0 2024-09-21T02:13:12.491 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-09-21T02:13:12.488+0000 7f135ad88640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T02:13:12.509 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-09-21T02:13:12.507+0000 7f2413c89640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T02:13:12.518 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-09-21T02:13:12.516+0000 7f1352b85640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:12.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.520+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41bc0cecb0 con 0x7f41bc1598c0 2024-09-21T02:13:12.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.520+0000 7f41b2ffd640 1 -- 172.21.15.170:0/4004390827 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7f41b40632d0 con 0x7f41bc1598c0 2024-09-21T02:13:12.521 INFO:teuthology.misc.health.smithi170.stdout: 2024-09-21T02:13:12.521 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":14,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:12.117749+0000","last_up_change":"2024-09-21T02:13:12.117749+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41a0073170 msgr2=0x7f41a0075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41a0073170 0x7f41a0075680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0af000 tx=0x7f41ac01b000 comp rx=0 tx=0).stop 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 msgr2=0x7f41bc0b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 0x7f41bc0b56a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c6e0 tx=0x7f41b400cba0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 shutdown_connections 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41a0073170 0x7f41a0075680 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc1598c0 0x7f41bc0b56a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41bc0a9e20 0x7f41bc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 --2- 172.21.15.170:0/4004390827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41bc0cf1f0 0x7f41bc0c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:12.524 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 >> 172.21.15.170:0/4004390827 conn(0x7f41bc06d790 msgr2=0x7f41bc162370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:12.525 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 shutdown_connections 2024-09-21T02:13:12.525 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:12.524+0000 7f41c4284640 1 -- 172.21.15.170:0/4004390827 wait complete. 2024-09-21T02:13:12.537 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-09-21T02:13:12.535+0000 7fa7cfa62640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T02:13:12.540 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-09-21T02:13:12.539+0000 7f240ba86640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:12.549 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-21T02:13:12.582 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-09-21T02:13:12.579+0000 7fa7c785f640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:12.603 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-09-21T02:13:12.600+0000 7fca510c2640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T02:13:12.641 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-09-21T02:13:12.640+0000 7fca48ebf640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:12.661 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-09-21T02:13:12.660+0000 7f0805f44640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T02:13:12.681 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-09-21T02:13:12.680+0000 7f07fd52c640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:13.274 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-09-21T02:13:13.271+0000 7f30053be640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T02:13:13.296 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-09-21T02:13:13.295+0000 7f2ffc9a6640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:13.317 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-09-21T02:13:13.316+0000 7f69c56f1640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T02:13:13.345 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-09-21T02:13:13.344+0000 7f69bd4ee640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:13:19.356 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0ee65640 1 -- 172.21.15.170:0/2479710845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4006270 msgr2=0x7faaf400cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0ee65640 1 --2- 172.21.15.170:0/2479710845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faaf4006270 0x7faaf400cae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faaf80016f0 tx=0x7faaf801c7b0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0d402640 1 -- 172.21.15.170:0/2479710845 reap_dead start 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0ee65640 1 -- 172.21.15.170:0/2479710845 shutdown_connections 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0ee65640 1 -- 172.21.15.170:0/2479710845 >> 172.21.15.170:0/2479710845 conn(0x7fab080b1c90 msgr2=0x7fab080b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0ee65640 1 -- 172.21.15.170:0/2479710845 shutdown_connections 2024-09-21T02:13:19.515 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.511+0000 7fab0ee65640 1 -- 172.21.15.170:0/2479710845 wait complete. 2024-09-21T02:13:19.516 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 Processor -- start 2024-09-21T02:13:19.516 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- start start 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab081f2920 0x7fab081583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab081f2d30 0x7fab081588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 0x7fab080c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab080cccc0 con 0x7fab080c5e00 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fab080ccb40 con 0x7fab081f2d30 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fab080cce40 con 0x7fab081f2920 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 0x7fab080c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.519 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 0x7fab080c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57010/0 (socket says 172.21.15.170:57010) 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 -- 172.21.15.170:0/1497882406 learned_addr learned my addr 172.21.15.170:0/1497882406 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab07fff640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab081f2d30 0x7fab081588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0cc01640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab081f2920 0x7fab081583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 -- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab081f2920 msgr2=0x7fab081583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab081f2920 0x7fab081583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 -- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab081f2d30 msgr2=0x7fab081588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab081f2d30 0x7fab081588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 -- 172.21.15.170:0/1497882406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab080c6750 con 0x7fab080c5e00 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0d402640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 0x7fab080c6210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faaf801cc90 tx=0x7faaf8006b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8001310 con 0x7fab080c5e00 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf80a5910 con 0x7fab080c5e00 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab080c9080 con 0x7fab080c5e00 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8006930 con 0x7fab080c5e00 2024-09-21T02:13:19.520 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab080c9640 con 0x7fab080c5e00 2024-09-21T02:13:19.521 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.515+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0806ebb0 con 0x7fab080c5e00 2024-09-21T02:13:19.521 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.519+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faaf8003600 con 0x7fab080c5e00 2024-09-21T02:13:19.521 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.519+0000 7fab057fa640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faae8073200 0x7faae8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.521 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.519+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7faaf812de10 con 0x7fab080c5e00 2024-09-21T02:13:19.522 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.519+0000 7fab0cc01640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faae8073200 0x7faae8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.522 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.519+0000 7fab0cc01640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faae8073200 0x7faae8075710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab00006fd0 tx=0x7fab00001980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:19.525 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.523+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fab080c6750 con 0x7fab080c5e00 2024-09-21T02:13:19.695 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.691+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab080008e0 con 0x7fab080c5e00 2024-09-21T02:13:19.696 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.691+0000 7fab057fa640 1 -- 172.21.15.170:0/1497882406 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12944 (secure 0 0 0) 0x7faaf8016510 con 0x7fab080c5e00 2024-09-21T02:13:19.696 INFO:teuthology.misc.health.smithi170.stdout: 2024-09-21T02:13:19.696 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":19,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:17.544381+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faae8073200 msgr2=0x7faae8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faae8073200 0x7faae8075710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab00006fd0 tx=0x7fab00001980 comp rx=0 tx=0).stop 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 msgr2=0x7fab080c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 0x7fab080c6210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faaf801cc90 tx=0x7faaf8006b00 comp rx=0 tx=0).stop 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 shutdown_connections 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faae8073200 0x7faae8075710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab080c5e00 0x7fab080c6210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab081f2d30 0x7fab081588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 --2- 172.21.15.170:0/1497882406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab081f2920 0x7fab081583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 >> 172.21.15.170:0/1497882406 conn(0x7fab080b1c90 msgr2=0x7fab080b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 shutdown_connections 2024-09-21T02:13:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-09-21T02:13:19.695+0000 7fab0ee65640 1 -- 172.21.15.170:0/1497882406 wait complete. 2024-09-21T02:13:19.715 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T02:13:19.715 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T02:13:19.716 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:13:19.716 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:13:19.718 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:13:19.718 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:13:19.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:13:19.719 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:13:19.720 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:13:19.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:13:19.722 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:13:19.722 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:13:19.723 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:13:19.725 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:13:19.725 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T02:13:19.725 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:19.813 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:13:19.813 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:13:19.841 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:13:19.847 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 -- 172.21.15.170:0/1617488548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c9c004810 msgr2=0x7f6c9c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/1617488548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c9c004810 0x7f6c9c004c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009b60 tx=0x7f6ca401d280 comp rx=0 tx=0).stop 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cb9022640 1 -- 172.21.15.170:0/1617488548 reap_dead start 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 -- 172.21.15.170:0/1617488548 shutdown_connections 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 -- 172.21.15.170:0/1617488548 >> 172.21.15.170:0/1617488548 conn(0x7f6cb408bf50 msgr2=0x7f6cb409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 -- 172.21.15.170:0/1617488548 shutdown_connections 2024-09-21T02:13:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 -- 172.21.15.170:0/1617488548 wait complete. 2024-09-21T02:13:19.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 Processor -- start 2024-09-21T02:13:19.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.867+0000 7f6cbaa85640 1 -- start start 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c9c004810 0x7f6cb415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb415e900 0x7f6cb40b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 0x7f6cb40b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4169ee0 con 0x7f6cb415ee50 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb40d33e0 con 0x7f6cb415e900 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb416a060 con 0x7f6c9c004810 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 0x7f6cb40b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 0x7f6cb40b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57080/0 (socket says 172.21.15.170:57080) 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 -- 172.21.15.170:0/4133228579 learned_addr learned my addr 172.21.15.170:0/4133228579 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb8821640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c9c004810 0x7f6cb415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb3fff640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb415e900 0x7f6cb40b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 -- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c9c004810 msgr2=0x7f6cb415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c9c004810 0x7f6cb415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 -- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb415e900 msgr2=0x7f6cb40b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb415e900 0x7f6cb40b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 -- 172.21.15.170:0/4133228579 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb40ce7b0 con 0x7f6cb415ee50 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb9022640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 0x7f6cb40b5d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6cac001340 tx=0x7f6cac006650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac006990 con 0x7f6cb415ee50 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cac018070 con 0x7f6cb415ee50 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00ed60 con 0x7f6cb415ee50 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb4166f50 con 0x7f6cb415ee50 2024-09-21T02:13:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb4167510 con 0x7f6cb415ee50 2024-09-21T02:13:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb40d1260 con 0x7f6cb415ee50 2024-09-21T02:13:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6cac01d460 con 0x7f6cb415ee50 2024-09-21T02:13:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.871+0000 7f6cb17fa640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6ca0073200 0x7f6ca0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.875+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f6cac094f10 con 0x7f6cb415ee50 2024-09-21T02:13:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.875+0000 7f6cb8821640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6ca0073200 0x7f6ca0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.875+0000 7f6cb8821640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6ca0073200 0x7f6ca0075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005ec0 tx=0x7f6ca80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:19.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:19.875+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6cb40cfeb0 con 0x7f6cb415ee50 2024-09-21T02:13:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.071+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cb40ce540 con 0x7f6cb415ee50 2024-09-21T02:13:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.071+0000 7f6cb17fa640 1 -- 172.21.15.170:0/4133228579 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7f6cac0603f0 con 0x7f6cb415ee50 2024-09-21T02:13:20.075 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:20.075 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"btime":"2024-09-21T02:13:19:829813+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:13:20.075 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 2 2024-09-21T02:13:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6ca0073200 msgr2=0x7f6ca0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6ca0073200 0x7f6ca0075710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005ec0 tx=0x7f6ca80091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 msgr2=0x7f6cb40b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 0x7f6cb40b5d10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6cac001340 tx=0x7f6cac006650 comp rx=0 tx=0).stop 2024-09-21T02:13:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 shutdown_connections 2024-09-21T02:13:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6ca0073200 0x7f6ca0075710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb415ee50 0x7f6cb40b5d10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6cb415e900 0x7f6cb40b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 --2- 172.21.15.170:0/4133228579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c9c004810 0x7f6cb415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 >> 172.21.15.170:0/4133228579 conn(0x7f6cb408bf50 msgr2=0x7f6cb40b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 shutdown_connections 2024-09-21T02:13:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:20.075+0000 7f6cbaa85640 1 -- 172.21.15.170:0/4133228579 wait complete. 2024-09-21T02:13:22.096 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 -- 172.21.15.170:0/813028332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500ce4c0 msgr2=0x7f5e500ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 --2- 172.21.15.170:0/813028332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500ce4c0 0x7f5e500ce8d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e500cdc00 tx=0x7f5e4001c6e0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 -- 172.21.15.170:0/813028332 shutdown_connections 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 --2- 172.21.15.170:0/813028332 >> v2:172.21.15.191:3300/0 conn(0x7f5e500cadf0 0x7f5e500cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 --2- 172.21.15.170:0/813028332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500ce4c0 0x7f5e500ce8d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 -- 172.21.15.170:0/813028332 >> 172.21.15.170:0/813028332 conn(0x7f5e500b1c40 msgr2=0x7f5e500b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 -- 172.21.15.170:0/813028332 shutdown_connections 2024-09-21T02:13:22.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.259+0000 7f5e575b0640 1 -- 172.21.15.170:0/813028332 wait complete. 2024-09-21T02:13:22.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 Processor -- start 2024-09-21T02:13:22.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 -- start start 2024-09-21T02:13:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 0x7f5e500c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e500ce4c0 0x7f5e5009cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e5015f990 0x7f5e5009d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50165fe0 con 0x7f5e500cadf0 2024-09-21T02:13:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50074f00 con 0x7f5e5015f990 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5e50166160 con 0x7f5e500ce4c0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 0x7f5e500c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 0x7f5e500c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57092/0 (socket says 172.21.15.170:57092) 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 -- 172.21.15.170:0/2048919625 learned_addr learned my addr 172.21.15.170:0/2048919625 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e54b4b640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e500ce4c0 0x7f5e5009cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e55b4d640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e5015f990 0x7f5e5009d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 -- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e500ce4c0 msgr2=0x7f5e5009cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e500ce4c0 0x7f5e5009cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 -- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e5015f990 msgr2=0x7f5e5009d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e5015f990 0x7f5e5009d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 -- 172.21.15.170:0/2048919625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5015ce30 con 0x7f5e500cadf0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e5534c640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 0x7f5e500c95d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00b410 tx=0x7f5e4c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c015030 con 0x7f5e500cadf0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e4c004020 con 0x7f5e500cadf0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5015d110 con 0x7f5e500cadf0 2024-09-21T02:13:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0044b0 con 0x7f5e500cadf0 2024-09-21T02:13:22.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.263+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5015d6d0 con 0x7f5e500cadf0 2024-09-21T02:13:22.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.267+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e500cb200 con 0x7f5e500cadf0 2024-09-21T02:13:22.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.267+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e4c018070 con 0x7f5e500cadf0 2024-09-21T02:13:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.267+0000 7f5e45ffb640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5e34073200 0x7f5e34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.267+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f5e4c093b00 con 0x7f5e500cadf0 2024-09-21T02:13:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.267+0000 7f5e54b4b640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5e34073200 0x7f5e34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.267+0000 7f5e54b4b640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5e34073200 0x7f5e34075710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5e400af000 tx=0x7f5e4001af20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.271+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5e4c05ee70 con 0x7f5e500cadf0 2024-09-21T02:13:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.479+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e500ce2a0 con 0x7f5e500cadf0 2024-09-21T02:13:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.479+0000 7f5e45ffb640 1 -- 172.21.15.170:0/2048919625 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3915 (secure 0 0 0) 0x7f5e4c05e770 con 0x7f5e500cadf0 2024-09-21T02:13:22.483 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:22.483 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"btime":"2024-09-21T02:13:20:830959+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:13:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 3 2024-09-21T02:13:22.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5e34073200 msgr2=0x7f5e34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5e34073200 0x7f5e34075710 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5e400af000 tx=0x7f5e4001af20 comp rx=0 tx=0).stop 2024-09-21T02:13:22.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 msgr2=0x7f5e500c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 0x7f5e500c95d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00b410 tx=0x7f5e4c00b8d0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 shutdown_connections 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5e34073200 0x7f5e34075710 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e500ce4c0 0x7f5e5009cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e5015f990 0x7f5e5009d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 --2- 172.21.15.170:0/2048919625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e500cadf0 0x7f5e500c95d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 >> 172.21.15.170:0/2048919625 conn(0x7f5e500b1c40 msgr2=0x7f5e500b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 shutdown_connections 2024-09-21T02:13:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.483+0000 7f5e575b0640 1 -- 172.21.15.170:0/2048919625 wait complete. 2024-09-21T02:13:22.503 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:13:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 -- 172.21.15.170:0/4162283694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3015f990 msgr2=0x7f6a30070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 --2- 172.21.15.170:0/4162283694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3015f990 0x7f6a30070030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a1801c8b0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 -- 172.21.15.170:0/4162283694 shutdown_connections 2024-09-21T02:13:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 --2- 172.21.15.170:0/4162283694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a300cadf0 0x7f6a300cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 --2- 172.21.15.170:0/4162283694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3015f990 0x7f6a30070030 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 -- 172.21.15.170:0/4162283694 >> 172.21.15.170:0/4162283694 conn(0x7f6a300b1c40 msgr2=0x7f6a300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 -- 172.21.15.170:0/4162283694 shutdown_connections 2024-09-21T02:13:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 -- 172.21.15.170:0/4162283694 wait complete. 2024-09-21T02:13:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.655+0000 7f6a351ab640 1 Processor -- start 2024-09-21T02:13:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- start start 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a30070570 0x7f6a30157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 0x7f6a301f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3015f990 0x7f6a3015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a30167ff0 con 0x7f6a300cadf0 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6a30168170 con 0x7f6a30070570 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6a301682f0 con 0x7f6a3015f990 2024-09-21T02:13:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 0x7f6a301f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 0x7f6a301f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57106/0 (socket says 172.21.15.170:57106) 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 -- 172.21.15.170:0/2969467027 learned_addr learned my addr 172.21.15.170:0/2969467027 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2ed9d640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a30070570 0x7f6a30157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2f59e640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3015f990 0x7f6a3015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 -- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3015f990 msgr2=0x7f6a3015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3015f990 0x7f6a3015b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 -- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a30070570 msgr2=0x7f6a30157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a30070570 0x7f6a30157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 -- 172.21.15.170:0/2969467027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3009bab0 con 0x7f6a300cadf0 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2f59e640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3015f990 0x7f6a3015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a2e59c640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 0x7f6a301f5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c400 tx=0x7f6a2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20017070 con 0x7f6a300cadf0 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a20003600 con 0x7f6a300cadf0 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3009bd90 con 0x7f6a300cadf0 2024-09-21T02:13:22.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20003a50 con 0x7f6a300cadf0 2024-09-21T02:13:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3009c350 con 0x7f6a300cadf0 2024-09-21T02:13:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.659+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a301f5ae0 con 0x7f6a300cadf0 2024-09-21T02:13:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.663+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6a2001a070 con 0x7f6a300cadf0 2024-09-21T02:13:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.663+0000 7f6a277fe640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6a10073230 0x7f6a10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.663+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f6a20093130 con 0x7f6a300cadf0 2024-09-21T02:13:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.663+0000 7f6a2ed9d640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6a10073230 0x7f6a10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.663+0000 7f6a2ed9d640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6a10073230 0x7f6a10075740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a180af000 tx=0x7f6a1801b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.667+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6a300cb740 con 0x7f6a300cadf0 2024-09-21T02:13:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6a300cb200 con 0x7f6a300cadf0 2024-09-21T02:13:22.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a277fe640 1 -- 172.21.15.170:0/2969467027 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f6a2005e530 con 0x7f6a300cadf0 2024-09-21T02:13:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:22.838 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:13:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:13:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:13:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:13:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:13:22.839 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6a10073230 msgr2=0x7f6a10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6a10073230 0x7f6a10075740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a180af000 tx=0x7f6a1801b0f0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 msgr2=0x7f6a301f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 0x7f6a301f5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c400 tx=0x7f6a2000c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 shutdown_connections 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6a10073230 0x7f6a10075740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3015f990 0x7f6a3015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a300cadf0 0x7f6a301f5000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 --2- 172.21.15.170:0/2969467027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a30070570 0x7f6a30157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 >> 172.21.15.170:0/2969467027 conn(0x7f6a300b1c40 msgr2=0x7f6a300aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 shutdown_connections 2024-09-21T02:13:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:22.835+0000 7f6a351ab640 1 -- 172.21.15.170:0/2969467027 wait complete. 2024-09-21T02:13:22.858 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:13:22.858 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 -- 172.21.15.170:0/1138698143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0006ce00 msgr2=0x7f9a0006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 --2- 172.21.15.170:0/1138698143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0006ce00 0x7f9a0006d210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009900 tx=0x7f99f401c700 comp rx=0 tx=0).stop 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 -- 172.21.15.170:0/1138698143 shutdown_connections 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 --2- 172.21.15.170:0/1138698143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0006ce00 0x7f9a0006d210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 --2- 172.21.15.170:0/1138698143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a00073ff0 0x7f9a00162990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 -- 172.21.15.170:0/1138698143 >> 172.21.15.170:0/1138698143 conn(0x7f9a000cdd00 msgr2=0x7f9a000ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 -- 172.21.15.170:0/1138698143 shutdown_connections 2024-09-21T02:13:23.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.011+0000 7f9a066fe640 1 -- 172.21.15.170:0/1138698143 wait complete. 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 Processor -- start 2024-09-21T02:13:23.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- start start 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0006ce00 0x7f9a000c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 0x7f9a001483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a000b3150 0x7f9a0014ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0015fab0 con 0x7f9a00073ff0 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0015f930 con 0x7f9a000b3150 2024-09-21T02:13:23.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0015fc30 con 0x7f9a0006ce00 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 0x7f9a001483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 0x7f9a001483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57122/0 (socket says 172.21.15.170:57122) 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 -- 172.21.15.170:0/2008248893 learned_addr learned my addr 172.21.15.170:0/2008248893 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99fffff640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0006ce00 0x7f9a000c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a04c9b640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a000b3150 0x7f9a0014ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 -- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0006ce00 msgr2=0x7f9a000c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0006ce00 0x7f9a000c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 -- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a000b3150 msgr2=0x7f9a0014ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a000b3150 0x7f9a0014ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 -- 172.21.15.170:0/2008248893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0014baf0 con 0x7f9a00073ff0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99ff7fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 0x7f9a001483b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99f401cfa0 tx=0x7f99f40a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f40ac070 con 0x7f9a00073ff0 2024-09-21T02:13:23.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99f4004030 con 0x7f9a00073ff0 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a000c6f70 con 0x7f9a00073ff0 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4004460 con 0x7f9a00073ff0 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a000c6c40 con 0x7f9a00073ff0 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.015+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0014b860 con 0x7f9a00073ff0 2024-09-21T02:13:23.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.019+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f99f40af070 con 0x7f9a00073ff0 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.019+0000 7f99fcff9640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99ec073200 0x7f99ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.019+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f99f412dba0 con 0x7f9a00073ff0 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.019+0000 7f99fffff640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99ec073200 0x7f99ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:23.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.019+0000 7f99fffff640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99ec073200 0x7f99ec075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99f000bb00 tx=0x7f99f0005dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:23.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.023+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9a0006d750 con 0x7f9a00073ff0 2024-09-21T02:13:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.187+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9a0006d210 con 0x7f9a00073ff0 2024-09-21T02:13:23.515 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:23.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:23.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.851+0000 7f99fcff9640 1 -- 172.21.15.170:0/2008248893 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f99f40f8fa0 con 0x7f9a00073ff0 2024-09-21T02:13:23.853 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:13:23.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99ec073200 msgr2=0x7f99ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99ec073200 0x7f99ec075710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99f000bb00 tx=0x7f99f0005dd0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 msgr2=0x7f9a001483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 0x7f9a001483b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99f401cfa0 tx=0x7f99f40a6ac0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 shutdown_connections 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99ec073200 0x7f99ec075710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a000b3150 0x7f9a0014ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00073ff0 0x7f9a001483b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 --2- 172.21.15.170:0/2008248893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0006ce00 0x7f9a000c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 >> 172.21.15.170:0/2008248893 conn(0x7f9a000cdd00 msgr2=0x7f9a000d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 shutdown_connections 2024-09-21T02:13:23.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:23.855+0000 7f9a066fe640 1 -- 172.21.15.170:0/2008248893 wait complete. 2024-09-21T02:13:23.885 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:13:24.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 -- 172.21.15.170:0/2291262920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f540cb380 msgr2=0x7f3f540ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 --2- 172.21.15.170:0/2291262920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f540cb380 0x7f3f540ce070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3f540723d0 tx=0x7f3f4401db40 comp rx=0 tx=0).stop 2024-09-21T02:13:24.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 -- 172.21.15.170:0/2291262920 shutdown_connections 2024-09-21T02:13:24.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 --2- 172.21.15.170:0/2291262920 >> v2:172.21.15.191:3300/0 conn(0x7f3f540cb790 0x7f3f540ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 --2- 172.21.15.170:0/2291262920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f540cb380 0x7f3f540ce070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 -- 172.21.15.170:0/2291262920 >> 172.21.15.170:0/2291262920 conn(0x7f3f540d0170 msgr2=0x7f3f540d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 -- 172.21.15.170:0/2291262920 shutdown_connections 2024-09-21T02:13:24.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.039+0000 7f3f58f91640 1 -- 172.21.15.170:0/2291262920 wait complete. 2024-09-21T02:13:24.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 Processor -- start 2024-09-21T02:13:24.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- start start 2024-09-21T02:13:24.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 0x7f3f5409c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f540cb380 0x7f3f541fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f540cb790 0x7f3f540c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f54074650 con 0x7f3f5406f9a0 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3f540744d0 con 0x7f3f540cb790 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3f540747d0 con 0x7f3f540cb380 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 0x7f3f5409c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 0x7f3f5409c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57144/0 (socket says 172.21.15.170:57144) 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 -- 172.21.15.170:0/1215589160 learned_addr learned my addr 172.21.15.170:0/1215589160 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f52d9d640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f540cb790 0x7f3f540c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f4bfff640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f540cb380 0x7f3f541fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 -- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f540cb380 msgr2=0x7f3f541fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f540cb380 0x7f3f541fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 -- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f540cb790 msgr2=0x7f3f540c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f540cb790 0x7f3f540c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 -- 172.21.15.170:0/1215589160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f541f7740 con 0x7f3f5406f9a0 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f52d9d640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f540cb790 0x7f3f540c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f4bfff640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f540cb380 0x7f3f541fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f5259c640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 0x7f3f5409c500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f440b1040 tx=0x7f3f440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f440be070 con 0x7f3f5406f9a0 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f440044e0 con 0x7f3f5406f9a0 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f541f7a20 con 0x7f3f5406f9a0 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f440b3070 con 0x7f3f5406f9a0 2024-09-21T02:13:24.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f541f7fe0 con 0x7f3f5406f9a0 2024-09-21T02:13:24.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.043+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5406fdb0 con 0x7f3f5406f9a0 2024-09-21T02:13:24.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.047+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3f440c1070 con 0x7f3f5406f9a0 2024-09-21T02:13:24.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.047+0000 7f3f4affd640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3f34073200 0x7f3f34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:24.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.047+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f3f4412ed50 con 0x7f3f5406f9a0 2024-09-21T02:13:24.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.047+0000 7f3f4bfff640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3f34073200 0x7f3f34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:24.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.047+0000 7f3f4bfff640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3f34073200 0x7f3f34075710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f54161250 tx=0x7f3f3c005cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:24.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.051+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3f440177c0 con 0x7f3f5406f9a0 2024-09-21T02:13:24.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.227+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3f540b04c0 con 0x7f3f5406f9a0 2024-09-21T02:13:24.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.863+0000 7f3f4affd640 1 -- 172.21.15.170:0/1215589160 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f3f440fa0c0 con 0x7f3f5406f9a0 2024-09-21T02:13:24.867 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3f34073200 msgr2=0x7f3f34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3f34073200 0x7f3f34075710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f54161250 tx=0x7f3f3c005cc0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 msgr2=0x7f3f5409c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 0x7f3f5409c500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3f440b1040 tx=0x7f3f440042f0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 shutdown_connections 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3f34073200 0x7f3f34075710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f540cb380 0x7f3f541fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f540cb790 0x7f3f540c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 --2- 172.21.15.170:0/1215589160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f5406f9a0 0x7f3f5409c500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 >> 172.21.15.170:0/1215589160 conn(0x7f3f540d0170 msgr2=0x7f3f540b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 shutdown_connections 2024-09-21T02:13:24.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:24.867+0000 7f3f58f91640 1 -- 172.21.15.170:0/1215589160 wait complete. 2024-09-21T02:13:24.911 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 -- 172.21.15.170:0/3633600844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0550004810 msgr2=0x7f0550004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 --2- 172.21.15.170:0/3633600844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0550004810 0x7f0550004c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f055c00bb10 tx=0x7f055c01cd80 comp rx=0 tx=0).stop 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 -- 172.21.15.170:0/3633600844 shutdown_connections 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 --2- 172.21.15.170:0/3633600844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0550004810 0x7f0550004c20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 -- 172.21.15.170:0/3633600844 >> 172.21.15.170:0/3633600844 conn(0x7f056809a9c0 msgr2=0x7f05680d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 -- 172.21.15.170:0/3633600844 shutdown_connections 2024-09-21T02:13:25.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 -- 172.21.15.170:0/3633600844 wait complete. 2024-09-21T02:13:25.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 Processor -- start 2024-09-21T02:13:25.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.067+0000 7f056ec91640 1 -- start start 2024-09-21T02:13:25.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0550004810 0x7f056815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 0x7f05681f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05681536a0 0x7f0568153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f056815fc10 con 0x7f05680cb3f0 2024-09-21T02:13:25.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f056815fa90 con 0x7f05681536a0 2024-09-21T02:13:25.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f056815fd90 con 0x7f0550004810 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 0x7f05681f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 0x7f05681f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57158/0 (socket says 172.21.15.170:57158) 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 -- 172.21.15.170:0/2988225225 learned_addr learned my addr 172.21.15.170:0/2988225225 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ca2d640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0550004810 0x7f056815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056d22e640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05681536a0 0x7f0568153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 -- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0550004810 msgr2=0x7f056815e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0550004810 0x7f056815e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 -- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05681536a0 msgr2=0x7f0568153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05681536a0 0x7f0568153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 -- 172.21.15.170:0/2988225225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056815eb60 con 0x7f05680cb3f0 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f0567fff640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 0x7f05681f4a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ba30 tx=0x7f055c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0ab040 con 0x7f05680cb3f0 2024-09-21T02:13:25.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f055c0be070 con 0x7f05680cb3f0 2024-09-21T02:13:25.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0568152600 con 0x7f05680cb3f0 2024-09-21T02:13:25.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c004a60 con 0x7f05680cb3f0 2024-09-21T02:13:25.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0568152bc0 con 0x7f05680cb3f0 2024-09-21T02:13:25.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0568152370 con 0x7f05680cb3f0 2024-09-21T02:13:25.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f055c0b6070 con 0x7f05680cb3f0 2024-09-21T02:13:25.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.071+0000 7f05657fa640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0554073200 0x7f0554075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:25.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.075+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f055c12f7b0 con 0x7f05680cb3f0 2024-09-21T02:13:25.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.075+0000 7f056ca2d640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0554073200 0x7f0554075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:25.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.075+0000 7f056ca2d640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0554073200 0x7f0554075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0558001d00 tx=0x7f0558001c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:25.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.079+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f055c016a90 con 0x7f05680cb3f0 2024-09-21T02:13:25.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.287+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f05680cdae0 con 0x7f05680cb3f0 2024-09-21T02:13:25.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.867+0000 7f05657fa640 1 -- 172.21.15.170:0/2988225225 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f055c0faa30 con 0x7f05680cb3f0 2024-09-21T02:13:25.870 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T02:13:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0554073200 msgr2=0x7f0554075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0554073200 0x7f0554075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0558001d00 tx=0x7f0558001c90 comp rx=0 tx=0).stop 2024-09-21T02:13:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 msgr2=0x7f05681f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 0x7f05681f4a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ba30 tx=0x7f055c005620 comp rx=0 tx=0).stop 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 shutdown_connections 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0554073200 0x7f0554075710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05681536a0 0x7f0568153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05680cb3f0 0x7f05681f4a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 --2- 172.21.15.170:0/2988225225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0550004810 0x7f056815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 >> 172.21.15.170:0/2988225225 conn(0x7f056809a9c0 msgr2=0x7f056806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 shutdown_connections 2024-09-21T02:13:25.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:25.871+0000 7f056ec91640 1 -- 172.21.15.170:0/2988225225 wait complete. 2024-09-21T02:13:25.912 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 -- 172.21.15.170:0/1108418996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f254806d510 msgr2=0x7f25480adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 --2- 172.21.15.170:0/1108418996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f254806d510 0x7f25480adfc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f253c009900 tx=0x7f253c01c850 comp rx=0 tx=0).stop 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 -- 172.21.15.170:0/1108418996 shutdown_connections 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 --2- 172.21.15.170:0/1108418996 >> v2:172.21.15.191:3300/0 conn(0x7f25480ae500 0x7f2548169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 --2- 172.21.15.170:0/1108418996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f254806d510 0x7f25480adfc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 -- 172.21.15.170:0/1108418996 >> 172.21.15.170:0/1108418996 conn(0x7f254809a9c0 msgr2=0x7f25480d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 -- 172.21.15.170:0/1108418996 shutdown_connections 2024-09-21T02:13:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.059+0000 7f254fec1640 1 -- 172.21.15.170:0/1108418996 wait complete. 2024-09-21T02:13:26.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 Processor -- start 2024-09-21T02:13:26.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 -- start start 2024-09-21T02:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 0x7f2548157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2548162e80 0x7f2548153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25480b40a0 0x7f2548151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f254816c5d0 con 0x7f25480ae500 2024-09-21T02:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f254815ee30 con 0x7f2548162e80 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254fec1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f254816c750 con 0x7f25480b40a0 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 0x7f2548157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 0x7f2548157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42068/0 (socket says 172.21.15.170:42068) 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 -- 172.21.15.170:0/2717845468 learned_addr learned my addr 172.21.15.170:0/2717845468 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254e45e640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25480b40a0 0x7f2548151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254d45c640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2548162e80 0x7f2548153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 -- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25480b40a0 msgr2=0x7f2548151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25480b40a0 0x7f2548151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 -- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2548162e80 msgr2=0x7f2548153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2548162e80 0x7f2548153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.063+0000 7f254dc5d640 1 -- 172.21.15.170:0/2717845468 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2548153570 con 0x7f25480ae500 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f254dc5d640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 0x7f2548157540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f254000c730 tx=0x7f254000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540003bd0 con 0x7f25480ae500 2024-09-21T02:13:26.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2540003d90 con 0x7f25480ae500 2024-09-21T02:13:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2548153cb0 con 0x7f25480ae500 2024-09-21T02:13:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000aab0 con 0x7f25480ae500 2024-09-21T02:13:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25480b1090 con 0x7f25480ae500 2024-09-21T02:13:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25480b0ec0 con 0x7f25480ae500 2024-09-21T02:13:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f254000ac50 con 0x7f25480ae500 2024-09-21T02:13:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f253a7fc640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f252c073200 0x7f252c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2540094080 con 0x7f25480ae500 2024-09-21T02:13:26.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f254d45c640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f252c073200 0x7f252c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.067+0000 7f254d45c640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f252c073200 0x7f252c075710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f253c0af000 tx=0x7f253c01b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:26.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.071+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2548153570 con 0x7f25480ae500 2024-09-21T02:13:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25480adfc0 con 0x7f25480ae500 2024-09-21T02:13:26.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f253a7fc640 1 -- 172.21.15.170:0/2717845468 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16730 (secure 0 0 0) 0x7f254005f130 con 0x7f25480ae500 2024-09-21T02:13:26.238 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:26.238 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":22,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:25.293010+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f252c073200 msgr2=0x7f252c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f252c073200 0x7f252c075710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f253c0af000 tx=0x7f253c01b090 comp rx=0 tx=0).stop 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 msgr2=0x7f2548157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 0x7f2548157540 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f254000c730 tx=0x7f254000cbf0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 shutdown_connections 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f252c073200 0x7f252c075710 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25480b40a0 0x7f2548151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2548162e80 0x7f2548153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 --2- 172.21.15.170:0/2717845468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25480ae500 0x7f2548157540 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.235+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 >> 172.21.15.170:0/2717845468 conn(0x7f254809a9c0 msgr2=0x7f25480717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.239+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 shutdown_connections 2024-09-21T02:13:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.239+0000 7f254fec1640 1 -- 172.21.15.170:0/2717845468 wait complete. 2024-09-21T02:13:26.259 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- 172.21.15.170:0/1847349665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f80b25f0 msgr2=0x7f95f80cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/1847349665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f80b25f0 0x7f95f80cf950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009be0 tx=0x7f95f001cbc0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- 172.21.15.170:0/1847349665 shutdown_connections 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/1847349665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f80b25f0 0x7f95f80cf950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- 172.21.15.170:0/1847349665 >> 172.21.15.170:0/1847349665 conn(0x7f95f809a9c0 msgr2=0x7f95f80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- 172.21.15.170:0/1847349665 shutdown_connections 2024-09-21T02:13:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- 172.21.15.170:0/1847349665 wait complete. 2024-09-21T02:13:26.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 Processor -- start 2024-09-21T02:13:26.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- start start 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 0x7f95f814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f815faf0 0x7f95f814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f8160610 0x7f95f806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95f816a160 con 0x7f95f80b25f0 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f95f8169fe0 con 0x7f95f815faf0 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95ff5c7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f95f816a2e0 con 0x7f95f8160610 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95fd363640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 0x7f95f814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95fd363640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 0x7f95f814f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42082/0 (socket says 172.21.15.170:42082) 2024-09-21T02:13:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.411+0000 7f95fd363640 1 -- 172.21.15.170:0/3355219285 learned_addr learned my addr 172.21.15.170:0/3355219285 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fcb62640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f815faf0 0x7f95f814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fdb64640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f8160610 0x7f95f806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fd363640 1 -- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f8160610 msgr2=0x7f95f806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fd363640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f8160610 0x7f95f806ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fd363640 1 -- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f815faf0 msgr2=0x7f95f814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fd363640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f815faf0 0x7f95f814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fd363640 1 -- 172.21.15.170:0/3355219285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f814c710 con 0x7f95f80b25f0 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fcb62640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f815faf0 0x7f95f814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fd363640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 0x7f95f814f900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95f0000f90 tx=0x7f95f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f00b2040 con 0x7f95f80b25f0 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f00a6e50 con 0x7f95f80b25f0 2024-09-21T02:13:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f00042a0 con 0x7f95f80b25f0 2024-09-21T02:13:26.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f81507b0 con 0x7f95f80b25f0 2024-09-21T02:13:26.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f80cb390 con 0x7f95f80b25f0 2024-09-21T02:13:26.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f8150cd0 con 0x7f95f80b25f0 2024-09-21T02:13:26.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f95f00bf030 con 0x7f95f80b25f0 2024-09-21T02:13:26.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95edffb640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95e4073200 0x7f95e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f95f012e470 con 0x7f95f80b25f0 2024-09-21T02:13:26.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.415+0000 7f95fcb62640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95e4073200 0x7f95e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.419+0000 7f95fcb62640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95e4073200 0x7f95e4075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95f4001130 tx=0x7f95f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:26.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.423+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f95f00c6060 con 0x7f95f80b25f0 2024-09-21T02:13:26.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.595+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95f80cf950 con 0x7f95f80b25f0 2024-09-21T02:13:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95edffb640 1 -- 172.21.15.170:0/3355219285 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16730 (secure 0 0 0) 0x7f95f00167b0 con 0x7f95f80b25f0 2024-09-21T02:13:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:26.602 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":22,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:25.293010+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95e4073200 msgr2=0x7f95e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95e4073200 0x7f95e4075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f95f4001130 tx=0x7f95f4009040 comp rx=0 tx=0).stop 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 msgr2=0x7f95f814f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 0x7f95f814f900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95f0000f90 tx=0x7f95f0009250 comp rx=0 tx=0).stop 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 shutdown_connections 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95e4073200 0x7f95e4075710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95f8160610 0x7f95f806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95f815faf0 0x7f95f814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 --2- 172.21.15.170:0/3355219285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f80b25f0 0x7f95f814f900 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 >> 172.21.15.170:0/3355219285 conn(0x7f95f809a9c0 msgr2=0x7f95f815ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:26.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 shutdown_connections 2024-09-21T02:13:26.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.599+0000 7f95ff5c7640 1 -- 172.21.15.170:0/3355219285 wait complete. 2024-09-21T02:13:26.623 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 -- 172.21.15.170:0/2223690019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c0b2fc0 msgr2=0x7fb18c0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 --2- 172.21.15.170:0/2223690019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c0b2fc0 0x7fb18c0ce540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0720f0 tx=0x7fb18401cd60 comp rx=0 tx=0).stop 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 -- 172.21.15.170:0/2223690019 shutdown_connections 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 --2- 172.21.15.170:0/2223690019 >> v2:172.21.15.191:3300/0 conn(0x7fb18c0cea80 0x7fb18c0cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 --2- 172.21.15.170:0/2223690019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c0b2fc0 0x7fb18c0ce540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 -- 172.21.15.170:0/2223690019 >> 172.21.15.170:0/2223690019 conn(0x7fb18c0d0170 msgr2=0x7fb18c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 -- 172.21.15.170:0/2223690019 shutdown_connections 2024-09-21T02:13:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.763+0000 7fb19327c640 1 -- 172.21.15.170:0/2223690019 wait complete. 2024-09-21T02:13:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 Processor -- start 2024-09-21T02:13:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 -- start start 2024-09-21T02:13:26.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 0x7fb18c1fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb18c0b2fc0 0x7fb18c14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb18c0cea80 0x7fb18c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c169f50 con 0x7fb18c06ef40 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c169dd0 con 0x7fb18c0cea80 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb18c16a0d0 con 0x7fb18c0b2fc0 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 0x7fb18c1fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 0x7fb18c1fa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42102/0 (socket says 172.21.15.170:42102) 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 -- 172.21.15.170:0/3773480787 learned_addr learned my addr 172.21.15.170:0/3773480787 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191819640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb18c0cea80 0x7fb18c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb190817640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb18c0b2fc0 0x7fb18c14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 -- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb18c0b2fc0 msgr2=0x7fb18c14b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb18c0b2fc0 0x7fb18c14b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 -- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb18c0cea80 msgr2=0x7fb18c0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb18c0cea80 0x7fb18c0b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 -- 172.21.15.170:0/3773480787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c1fafd0 con 0x7fb18c06ef40 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb190817640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb18c0b2fc0 0x7fb18c14b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb191018640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 0x7fb18c1fa0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb184019970 tx=0x7fb184005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840bc070 con 0x7fb18c06ef40 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1840040f0 con 0x7fb18c06ef40 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840b2040 con 0x7fb18c06ef40 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c1faa10 con 0x7fb18c06ef40 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.767+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c159d30 con 0x7fb18c06ef40 2024-09-21T02:13:26.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.771+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18c070740 con 0x7fb18c06ef40 2024-09-21T02:13:26.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.771+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb184004520 con 0x7fb18c06ef40 2024-09-21T02:13:26.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.771+0000 7fb189ffb640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb178073200 0x7fb178075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:26.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.771+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb18412e080 con 0x7fb18c06ef40 2024-09-21T02:13:26.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.771+0000 7fb190817640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb178073200 0x7fb178075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:26.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.771+0000 7fb190817640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb178073200 0x7fb178075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb18c14c0e0 tx=0x7fb1800091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:26.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:26.795+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb184016950 con 0x7fb18c06ef40 2024-09-21T02:13:27.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.015+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb18c0d07d0 con 0x7fb18c06ef40 2024-09-21T02:13:27.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb189ffb640 1 -- 172.21.15.170:0/3773480787 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fb1840f90b0 con 0x7fb18c06ef40 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb178073200 msgr2=0x7fb178075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb178073200 0x7fb178075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb18c14c0e0 tx=0x7fb1800091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 msgr2=0x7fb18c1fa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 0x7fb18c1fa0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb184019970 tx=0x7fb184005b00 comp rx=0 tx=0).stop 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 shutdown_connections 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb178073200 0x7fb178075710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb18c0b2fc0 0x7fb18c14b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb18c0cea80 0x7fb18c0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 --2- 172.21.15.170:0/3773480787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb18c06ef40 0x7fb18c1fa0c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:27.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 >> 172.21.15.170:0/3773480787 conn(0x7fb18c0d0170 msgr2=0x7fb18c0afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:27.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 shutdown_connections 2024-09-21T02:13:27.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:27.946+0000 7fb19327c640 1 -- 172.21.15.170:0/3773480787 wait complete. 2024-09-21T02:13:27.976 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:13:27.976 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca9733640 1 -- 172.21.15.170:0/997681775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003a40 msgr2=0x7efc90003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca9733640 1 --2- 172.21.15.170:0/997681775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003a40 0x7efc90003eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c470 tx=0x7efc8c01cf60 comp rx=0 tx=0).stop 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca37fe640 1 -- 172.21.15.170:0/997681775 reap_dead start 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca9733640 1 -- 172.21.15.170:0/997681775 shutdown_connections 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca9733640 1 -- 172.21.15.170:0/997681775 >> 172.21.15.170:0/997681775 conn(0x7efca408bf50 msgr2=0x7efca409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca9733640 1 -- 172.21.15.170:0/997681775 shutdown_connections 2024-09-21T02:13:28.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.122+0000 7efca9733640 1 -- 172.21.15.170:0/997681775 wait complete. 2024-09-21T02:13:28.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 Processor -- start 2024-09-21T02:13:28.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 -- start start 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca40ce070 0x7efca4163600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca4163b40 0x7efca41f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 0x7efca40b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efca416b030 con 0x7efca41f8990 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efca416aeb0 con 0x7efca40ce070 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efca416b1b0 con 0x7efca4163b40 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 0x7efca40b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 0x7efca40b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42122/0 (socket says 172.21.15.170:42122) 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 -- 172.21.15.170:0/1622524878 learned_addr learned my addr 172.21.15.170:0/1622524878 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca27fc640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca4163b40 0x7efca41f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca2ffd640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca40ce070 0x7efca4163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 -- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca4163b40 msgr2=0x7efca41f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:28.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca4163b40 0x7efca41f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 -- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca40ce070 msgr2=0x7efca4163600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca40ce070 0x7efca4163600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 -- 172.21.15.170:0/1622524878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca4163f50 con 0x7efca41f8990 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca37fe640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 0x7efca40b8400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efca40ceb50 tx=0x7efc9c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c003d90 con 0x7efca41f8990 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc9c015070 con 0x7efca41f8990 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c006420 con 0x7efca41f8990 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca40ce630 con 0x7efca41f8990 2024-09-21T02:13:28.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.126+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41fc9f0 con 0x7efca41f8990 2024-09-21T02:13:28.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.130+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca40b3a10 con 0x7efca41f8990 2024-09-21T02:13:28.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.130+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc9c0065c0 con 0x7efca41f8990 2024-09-21T02:13:28.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.130+0000 7efc9b7fe640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc84073200 0x7efc84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:28.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.130+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efc9c0941d0 con 0x7efca41f8990 2024-09-21T02:13:28.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.130+0000 7efca2ffd640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc84073200 0x7efc84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:28.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.130+0000 7efca2ffd640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc84073200 0x7efc84075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc8c001060 tx=0x7efc8c019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:28.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.134+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efca4163f50 con 0x7efca41f8990 2024-09-21T02:13:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.330+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7efca406f620 con 0x7efca41f8990 2024-09-21T02:13:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.946+0000 7efc9b7fe640 1 -- 172.21.15.170:0/1622524878 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7efc9c05f310 con 0x7efca41f8990 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc84073200 msgr2=0x7efc84075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc84073200 0x7efc84075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc8c001060 tx=0x7efc8c019da0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 msgr2=0x7efca40b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 0x7efca40b8400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efca40ceb50 tx=0x7efc9c00da90 comp rx=0 tx=0).stop 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 shutdown_connections 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc84073200 0x7efc84075710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca4163b40 0x7efca41f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca41f8990 0x7efca40b8400 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 --2- 172.21.15.170:0/1622524878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca40ce070 0x7efca4163600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 >> 172.21.15.170:0/1622524878 conn(0x7efca408bf50 msgr2=0x7efca406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 shutdown_connections 2024-09-21T02:13:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:28.950+0000 7efca9733640 1 -- 172.21.15.170:0/1622524878 wait complete. 2024-09-21T02:13:28.979 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:13:28.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 -- 172.21.15.170:0/3508978664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 msgr2=0x7f439815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 --2- 172.21.15.170:0/3508978664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 0x7f439815c850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4390009840 tx=0x7f439001c820 comp rx=0 tx=0).stop 2024-09-21T02:13:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 -- 172.21.15.170:0/3508978664 shutdown_connections 2024-09-21T02:13:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 --2- 172.21.15.170:0/3508978664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980d0aa0 0x7f43980d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 --2- 172.21.15.170:0/3508978664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 0x7f439815c850 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 -- 172.21.15.170:0/3508978664 >> 172.21.15.170:0/3508978664 conn(0x7f43980cee30 msgr2=0x7f43980cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:29.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 -- 172.21.15.170:0/3508978664 shutdown_connections 2024-09-21T02:13:29.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.130+0000 7f439cced640 1 -- 172.21.15.170:0/3508978664 wait complete. 2024-09-21T02:13:29.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 Processor -- start 2024-09-21T02:13:29.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 -- start start 2024-09-21T02:13:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980b3e30 0x7f4398159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 0x7f439815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 0x7f439815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f439816a4c0 con 0x7f43980d0aa0 2024-09-21T02:13:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f439816a340 con 0x7f43980b7ab0 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439cced640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f439816a640 con 0x7f43980b3e30 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f4396d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 0x7f439815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f4396d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 0x7f439815d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42124/0 (socket says 172.21.15.170:42124) 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f4396d9d640 1 -- 172.21.15.170:0/2071883536 learned_addr learned my addr 172.21.15.170:0/2071883536 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f4395d9b640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 0x7f439815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.134+0000 7f439659c640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980b3e30 0x7f4398159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f4396d9d640 1 -- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980b3e30 msgr2=0x7f4398159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f4396d9d640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980b3e30 0x7f4398159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f4396d9d640 1 -- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 msgr2=0x7f439815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f4396d9d640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 0x7f439815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f4396d9d640 1 -- 172.21.15.170:0/2071883536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43980c97d0 con 0x7f43980d0aa0 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f439659c640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980b3e30 0x7f4398159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f4396d9d640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 0x7f439815d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f438800e740 tx=0x7f438800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438800cc60 con 0x7f43980d0aa0 2024-09-21T02:13:29.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f438800ce00 con 0x7f43980d0aa0 2024-09-21T02:13:29.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43980c9ab0 con 0x7f43980d0aa0 2024-09-21T02:13:29.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388005280 con 0x7f43980d0aa0 2024-09-21T02:13:29.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43980c66f0 con 0x7f43980d0aa0 2024-09-21T02:13:29.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4398167270 con 0x7f43980d0aa0 2024-09-21T02:13:29.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4388005420 con 0x7f43980d0aa0 2024-09-21T02:13:29.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f438effd640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073230 0x7f4378075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4388094ae0 con 0x7f43980d0aa0 2024-09-21T02:13:29.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.138+0000 7f439659c640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073230 0x7f4378075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.142+0000 7f439659c640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073230 0x7f4378075740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43900af000 tx=0x7f439001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:29.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.142+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f438805fb00 con 0x7f43980d0aa0 2024-09-21T02:13:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.346+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43980c9520 con 0x7f43980d0aa0 2024-09-21T02:13:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f438effd640 1 -- 172.21.15.170:0/2071883536 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f438805f400 con 0x7f43980d0aa0 2024-09-21T02:13:29.353 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:29.353 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:13:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:13:29.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073230 msgr2=0x7f4378075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073230 0x7f4378075740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43900af000 tx=0x7f439001b060 comp rx=0 tx=0).stop 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 msgr2=0x7f439815d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 0x7f439815d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f438800e740 tx=0x7f438800ec00 comp rx=0 tx=0).stop 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 shutdown_connections 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073230 0x7f4378075740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43980d0aa0 0x7f439815d3f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43980b7ab0 0x7f439815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 --2- 172.21.15.170:0/2071883536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43980b3e30 0x7f4398159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 >> 172.21.15.170:0/2071883536 conn(0x7f43980cee30 msgr2=0x7f43980b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 shutdown_connections 2024-09-21T02:13:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.350+0000 7f439cced640 1 -- 172.21.15.170:0/2071883536 wait complete. 2024-09-21T02:13:29.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 -- 172.21.15.170:0/3708546123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb80b3e30 msgr2=0x7fddb80b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 --2- 172.21.15.170:0/3708546123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb80b3e30 0x7fddb80b4240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fddb80ced00 tx=0x7fddb401c650 comp rx=0 tx=0).stop 2024-09-21T02:13:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 -- 172.21.15.170:0/3708546123 shutdown_connections 2024-09-21T02:13:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 --2- 172.21.15.170:0/3708546123 >> v2:172.21.15.191:3300/0 conn(0x7fddb806df10 0x7fddb806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 --2- 172.21.15.170:0/3708546123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb80b3e30 0x7fddb80b4240 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 -- 172.21.15.170:0/3708546123 >> 172.21.15.170:0/3708546123 conn(0x7fddb80cee30 msgr2=0x7fddb80cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 -- 172.21.15.170:0/3708546123 shutdown_connections 2024-09-21T02:13:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 -- 172.21.15.170:0/3708546123 wait complete. 2024-09-21T02:13:29.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 Processor -- start 2024-09-21T02:13:29.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.510+0000 7fddbfc29640 1 -- start start 2024-09-21T02:13:29.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 0x7fddb81615c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb80b3e30 0x7fddb81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80d0100 0x7fddb80b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fddb816be40 con 0x7fddb806df10 2024-09-21T02:13:29.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fddb816bcc0 con 0x7fddb80d0100 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fddb816bfc0 con 0x7fddb80b3e30 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 0x7fddb81615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 0x7fddb81615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42148/0 (socket says 172.21.15.170:42148) 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 -- 172.21.15.170:0/2508301713 learned_addr learned my addr 172.21.15.170:0/2508301713 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd1c4640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb80b3e30 0x7fddb81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 -- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb80b3e30 msgr2=0x7fddb81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbe1c6640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80d0100 0x7fddb80b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb80b3e30 0x7fddb81524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 -- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80d0100 msgr2=0x7fddb80b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80d0100 0x7fddb80b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 -- 172.21.15.170:0/2508301713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb81529f0 con 0x7fddb806df10 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbd9c5640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 0x7fddb81615c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddb401ad00 tx=0x7fddb40a8610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40042c0 con 0x7fddb806df10 2024-09-21T02:13:29.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb4003600 con 0x7fddb806df10 2024-09-21T02:13:29.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb80b6dd0 con 0x7fddb806df10 2024-09-21T02:13:29.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4003a50 con 0x7fddb806df10 2024-09-21T02:13:29.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb8160510 con 0x7fddb806df10 2024-09-21T02:13:29.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb806e320 con 0x7fddb806df10 2024-09-21T02:13:29.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fddb4003bf0 con 0x7fddb806df10 2024-09-21T02:13:29.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.514+0000 7fddae7fc640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdd9c073200 0x7fdd9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.518+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fddb412ecd0 con 0x7fddb806df10 2024-09-21T02:13:29.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.518+0000 7fddbd1c4640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdd9c073200 0x7fdd9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.518+0000 7fddbd1c4640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdd9c073200 0x7fdd9c075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fddb80b33a0 tx=0x7fdda8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:29.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.522+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fddb40c2030 con 0x7fddb806df10 2024-09-21T02:13:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.690+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddb80b39f0 con 0x7fddb806df10 2024-09-21T02:13:29.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.690+0000 7fddae7fc640 1 -- 172.21.15.170:0/2508301713 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7fddb40ac030 con 0x7fddb806df10 2024-09-21T02:13:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:29.696 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdd9c073200 msgr2=0x7fdd9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdd9c073200 0x7fdd9c075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fddb80b33a0 tx=0x7fdda8008040 comp rx=0 tx=0).stop 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 msgr2=0x7fddb81615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 0x7fddb81615c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddb401ad00 tx=0x7fddb40a8610 comp rx=0 tx=0).stop 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 shutdown_connections 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdd9c073200 0x7fdd9c075710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb80b3e30 0x7fddb81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80d0100 0x7fddb80b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 --2- 172.21.15.170:0/2508301713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddb806df10 0x7fddb81615c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 >> 172.21.15.170:0/2508301713 conn(0x7fddb80cee30 msgr2=0x7fddb815d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 shutdown_connections 2024-09-21T02:13:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.694+0000 7fddbfc29640 1 -- 172.21.15.170:0/2508301713 wait complete. 2024-09-21T02:13:29.717 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:13:29.717 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:29.838 INFO:tasks.ceph.mds.c.smithi170.stderr:2024-09-21T02:13:29.834+0000 7f212ad5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7ae841640 1 -- 172.21.15.170:0/3778146213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 msgr2=0x7ff790003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3778146213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff790003ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff798009b60 tx=0x7ff79801d2b0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7acdde640 1 -- 172.21.15.170:0/3778146213 reap_dead start 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7ae841640 1 -- 172.21.15.170:0/3778146213 shutdown_connections 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7ae841640 1 -- 172.21.15.170:0/3778146213 >> 172.21.15.170:0/3778146213 conn(0x7ff7a80b1c40 msgr2=0x7ff7a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7ae841640 1 -- 172.21.15.170:0/3778146213 shutdown_connections 2024-09-21T02:13:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.854+0000 7ff7ae841640 1 -- 172.21.15.170:0/3778146213 wait complete. 2024-09-21T02:13:29.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 Processor -- start 2024-09-21T02:13:29.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 -- start start 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff7a815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7a8161320 0x7ff7a80c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a80c6930 0x7ff7a80c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8167e40 con 0x7ff790003a10 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8070180 con 0x7ff7a80c6930 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a8167fc0 con 0x7ff7a8161320 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff7a815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff7a815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42174/0 (socket says 172.21.15.170:42174) 2024-09-21T02:13:29.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 -- 172.21.15.170:0/3109020535 learned_addr learned my addr 172.21.15.170:0/3109020535 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a77fe640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7a8161320 0x7ff7a80c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7acdde640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a80c6930 0x7ff7a80c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 -- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7a8161320 msgr2=0x7ff7a80c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7a8161320 0x7ff7a80c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 -- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a80c6930 msgr2=0x7ff7a80c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a80c6930 0x7ff7a80c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 -- 172.21.15.170:0/3109020535 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a815f740 con 0x7ff790003a10 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a7fff640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff7a815b3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff798009a80 tx=0x7ff798005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b1040 con 0x7ff790003a10 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7980bc070 con 0x7ff790003a10 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a80b22a0 con 0x7ff790003a10 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980acb40 con 0x7ff790003a10 2024-09-21T02:13:29.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.858+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a80c60a0 con 0x7ff790003a10 2024-09-21T02:13:29.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.862+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a806fa30 con 0x7ff790003a10 2024-09-21T02:13:29.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.862+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7980acce0 con 0x7ff790003a10 2024-09-21T02:13:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.862+0000 7ff7a4ff9640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff794073200 0x7ff794075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.862+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff79812f330 con 0x7ff790003a10 2024-09-21T02:13:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.862+0000 7ff7a77fe640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff794073200 0x7ff794075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.862+0000 7ff7a77fe640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff794073200 0x7ff794075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8072620 tx=0x7ff79c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:29.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:29.866+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff798016fc0 con 0x7ff790003a10 2024-09-21T02:13:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.030+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7a80b2050 con 0x7ff794073200 2024-09-21T02:13:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.054+0000 7ff7a4ff9640 1 -- 172.21.15.170:0/3109020535 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241530 (secure 0 0 0) 0x7ff7a80b2050 con 0x7ff794073200 2024-09-21T02:13:30.055 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:30.057 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff794073200 msgr2=0x7ff794075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff794073200 0x7ff794075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8072620 tx=0x7ff79c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 msgr2=0x7ff7a815b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff7a815b3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff798009a80 tx=0x7ff798005640 comp rx=0 tx=0).stop 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 shutdown_connections 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff794073200 0x7ff794075710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7a8161320 0x7ff7a80c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a80c6930 0x7ff7a80c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 --2- 172.21.15.170:0/3109020535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff790003a10 0x7ff7a815b3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 >> 172.21.15.170:0/3109020535 conn(0x7ff7a80b1c40 msgr2=0x7ff7a815c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 shutdown_connections 2024-09-21T02:13:30.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.058+0000 7ff7ae841640 1 -- 172.21.15.170:0/3109020535 wait complete. 2024-09-21T02:13:30.074 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T02:13:28.863929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244152,"kb_used_data":2692,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730568,"statfs":{"total":773094113280,"available":772844101632,"internally_reserved":0,"allocated":2756608,"data_stored":1672509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.009170"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.863531+0000","last_change":"2024-09-21T02:13:23.863531+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.863531+0000","last_undegraded":"2024-09-21T02:13:23.863531+0000","last_fullsized":"2024-09-21T02:13:23.863531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869051+0000","last_change":"2024-09-21T02:13:23.869051+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869051+0000","last_undegraded":"2024-09-21T02:13:23.869051+0000","last_fullsized":"2024-09-21T02:13:23.869051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867709+0000","last_change":"2024-09-21T02:13:23.867709+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867709+0000","last_undegraded":"2024-09-21T02:13:23.867709+0000","last_fullsized":"2024-09-21T02:13:23.867709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867719+0000","last_change":"2024-09-21T02:13:23.867719+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867719+0000","last_undegraded":"2024-09-21T02:13:23.867719+0000","last_fullsized":"2024-09-21T02:13:23.867719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866060+0000","last_change":"2024-09-21T02:13:23.866060+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866060+0000","last_undegraded":"2024-09-21T02:13:23.866060+0000","last_fullsized":"2024-09-21T02:13:23.866060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861766+0000","last_change":"2024-09-21T02:13:23.861766+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861766+0000","last_undegraded":"2024-09-21T02:13:23.861766+0000","last_fullsized":"2024-09-21T02:13:23.861766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867856+0000","last_change":"2024-09-21T02:13:23.867856+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867856+0000","last_undegraded":"2024-09-21T02:13:23.867856+0000","last_fullsized":"2024-09-21T02:13:23.867856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.864886+0000","last_change":"2024-09-21T02:13:23.864886+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.864886+0000","last_undegraded":"2024-09-21T02:13:23.864886+0000","last_fullsized":"2024-09-21T02:13:23.864886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.868952+0000","last_change":"2024-09-21T02:13:23.868952+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.868952+0000","last_undegraded":"2024-09-21T02:13:23.868952+0000","last_fullsized":"2024-09-21T02:13:23.868952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866765+0000","last_change":"2024-09-21T02:13:23.866765+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866765+0000","last_undegraded":"2024-09-21T02:13:23.866765+0000","last_fullsized":"2024-09-21T02:13:23.866765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861508+0000","last_change":"2024-09-21T02:13:23.861508+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861508+0000","last_undegraded":"2024-09-21T02:13:23.861508+0000","last_fullsized":"2024-09-21T02:13:23.861508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861470+0000","last_change":"2024-09-21T02:13:23.861470+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861470+0000","last_undegraded":"2024-09-21T02:13:23.861470+0000","last_fullsized":"2024-09-21T02:13:23.861470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.862094+0000","last_change":"2024-09-21T02:13:23.862094+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.862094+0000","last_undegraded":"2024-09-21T02:13:23.862094+0000","last_fullsized":"2024-09-21T02:13:23.862094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867804+0000","last_change":"2024-09-21T02:13:23.867804+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867804+0000","last_undegraded":"2024-09-21T02:13:23.867804+0000","last_fullsized":"2024-09-21T02:13:23.867804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:13:23.866625+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:23.866625+0000","last_peered":"2024-09-21T02:13:23.866625+0000","last_clean":"2024-09-21T02:13:23.866625+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:23.866625+0000","last_undegraded":"2024-09-21T02:13:23.866625+0000","last_fullsized":"2024-09-21T02:13:23.866625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866753+0000","last_change":"2024-09-21T02:13:23.866753+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866753+0000","last_undegraded":"2024-09-21T02:13:23.866753+0000","last_fullsized":"2024-09-21T02:13:23.866753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866314+0000","last_change":"2024-09-21T02:13:23.866314+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866314+0000","last_undegraded":"2024-09-21T02:13:23.866314+0000","last_fullsized":"2024-09-21T02:13:23.866314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867831+0000","last_change":"2024-09-21T02:13:23.867831+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867831+0000","last_undegraded":"2024-09-21T02:13:23.867831+0000","last_fullsized":"2024-09-21T02:13:23.867831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866015+0000","last_change":"2024-09-21T02:13:23.866015+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866015+0000","last_undegraded":"2024-09-21T02:13:23.866015+0000","last_fullsized":"2024-09-21T02:13:23.866015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861727+0000","last_change":"2024-09-21T02:13:23.861727+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861727+0000","last_undegraded":"2024-09-21T02:13:23.861727+0000","last_fullsized":"2024-09-21T02:13:23.861727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.865234+0000","last_change":"2024-09-21T02:13:23.865234+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.865234+0000","last_undegraded":"2024-09-21T02:13:23.865234+0000","last_fullsized":"2024-09-21T02:13:23.865234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.865116+0000","last_change":"2024-09-21T02:13:23.865116+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.865116+0000","last_undegraded":"2024-09-21T02:13:23.865116+0000","last_fullsized":"2024-09-21T02:13:23.865116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.862156+0000","last_change":"2024-09-21T02:13:23.862156+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.862156+0000","last_undegraded":"2024-09-21T02:13:23.862156+0000","last_fullsized":"2024-09-21T02:13:23.862156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869393+0000","last_change":"2024-09-21T02:13:23.869393+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869393+0000","last_undegraded":"2024-09-21T02:13:23.869393+0000","last_fullsized":"2024-09-21T02:13:23.869393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869455+0000","last_change":"2024-09-21T02:13:23.869455+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869455+0000","last_undegraded":"2024-09-21T02:13:23.869455+0000","last_fullsized":"2024-09-21T02:13:23.869455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861621+0000","last_change":"2024-09-21T02:13:23.861621+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861621+0000","last_undegraded":"2024-09-21T02:13:23.861621+0000","last_fullsized":"2024-09-21T02:13:23.861621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861576+0000","last_change":"2024-09-21T02:13:23.861576+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861576+0000","last_undegraded":"2024-09-21T02:13:23.861576+0000","last_fullsized":"2024-09-21T02:13:23.861576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.863225+0000","last_change":"2024-09-21T02:13:23.863225+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.863225+0000","last_undegraded":"2024-09-21T02:13:23.863225+0000","last_fullsized":"2024-09-21T02:13:23.863225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":646954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":62403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36120,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335720,"statfs":{"total":96636764160,"available":96599777280,"internally_reserved":0,"allocated":790528,"data_stored":652771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:30.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 -- 172.21.15.170:0/1438268238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a580cea00 msgr2=0x7f3a580cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/1438268238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a580cea00 0x7f3a580cadf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b2b80 tx=0x7f3a5001cd50 comp rx=0 tx=0).stop 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 -- 172.21.15.170:0/1438268238 shutdown_connections 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/1438268238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a580cea00 0x7f3a580cadf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/1438268238 >> v2:172.21.15.191:3300/0 conn(0x7f3a58070570 0x7f3a580ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/1438268238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a5815f990 0x7f3a58070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 -- 172.21.15.170:0/1438268238 >> 172.21.15.170:0/1438268238 conn(0x7f3a580b1c40 msgr2=0x7f3a580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 -- 172.21.15.170:0/1438268238 shutdown_connections 2024-09-21T02:13:30.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.274+0000 7f3a5cee2640 1 -- 172.21.15.170:0/1438268238 wait complete. 2024-09-21T02:13:30.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 Processor -- start 2024-09-21T02:13:30.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- start start 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 0x7f3a5809d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a580cea00 0x7f3a5814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a5815f990 0x7f3a58157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58073ae0 con 0x7f3a58070570 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58073960 con 0x7f3a5815f990 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3a58073c60 con 0x7f3a580cea00 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 0x7f3a5809d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 0x7f3a5809d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42190/0 (socket says 172.21.15.170:42190) 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 -- 172.21.15.170:0/3198152768 learned_addr learned my addr 172.21.15.170:0/3198152768 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a56d9d640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a5815f990 0x7f3a58157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a55d9b640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a580cea00 0x7f3a5814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 -- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a580cea00 msgr2=0x7f3a5814a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a580cea00 0x7f3a5814a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 -- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a5815f990 msgr2=0x7f3a58157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a5815f990 0x7f3a58157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 -- 172.21.15.170:0/3198152768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5815bed0 con 0x7f3a58070570 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5659c640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 0x7f3a5809d200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800d750 tx=0x7f3a4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48017070 con 0x7f3a58070570 2024-09-21T02:13:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a48009cc0 con 0x7f3a58070570 2024-09-21T02:13:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5815c190 con 0x7f3a58070570 2024-09-21T02:13:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48005280 con 0x7f3a58070570 2024-09-21T02:13:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5815c750 con 0x7f3a58070570 2024-09-21T02:13:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.278+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a580767c0 con 0x7f3a58070570 2024-09-21T02:13:30.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.282+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a48005420 con 0x7f3a58070570 2024-09-21T02:13:30.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.282+0000 7f3a46ffd640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a38073200 0x7f3a38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.282+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3a48094340 con 0x7f3a58070570 2024-09-21T02:13:30.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.282+0000 7f3a55d9b640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a38073200 0x7f3a38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.282+0000 7f3a55d9b640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a38073200 0x7f3a38075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c005fd0 tx=0x7f3a4c005d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.286+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3a4800de60 con 0x7f3a58070570 2024-09-21T02:13:30.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.442+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a58073e40 con 0x7f3a38073200 2024-09-21T02:13:30.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.454+0000 7f3a46ffd640 1 -- 172.21.15.170:0/3198152768 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241530 (secure 0 0 0) 0x7f3a58073e40 con 0x7f3a38073200 2024-09-21T02:13:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:30.458 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:30.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a38073200 msgr2=0x7f3a38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a38073200 0x7f3a38075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c005fd0 tx=0x7f3a4c005d80 comp rx=0 tx=0).stop 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 msgr2=0x7f3a5809d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 0x7f3a5809d200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800d750 tx=0x7f3a4800dc10 comp rx=0 tx=0).stop 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 shutdown_connections 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a38073200 0x7f3a38075710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a580cea00 0x7f3a5814a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a5815f990 0x7f3a58157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 --2- 172.21.15.170:0/3198152768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a58070570 0x7f3a5809d200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 >> 172.21.15.170:0/3198152768 conn(0x7f3a580b1c40 msgr2=0x7f3a58159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 shutdown_connections 2024-09-21T02:13:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.458+0000 7f3a5cee2640 1 -- 172.21.15.170:0/3198152768 wait complete. 2024-09-21T02:13:30.475 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T02:13:28.863929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244152,"kb_used_data":2692,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730568,"statfs":{"total":773094113280,"available":772844101632,"internally_reserved":0,"allocated":2756608,"data_stored":1672509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.009170"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.863531+0000","last_change":"2024-09-21T02:13:23.863531+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.863531+0000","last_undegraded":"2024-09-21T02:13:23.863531+0000","last_fullsized":"2024-09-21T02:13:23.863531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869051+0000","last_change":"2024-09-21T02:13:23.869051+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869051+0000","last_undegraded":"2024-09-21T02:13:23.869051+0000","last_fullsized":"2024-09-21T02:13:23.869051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867709+0000","last_change":"2024-09-21T02:13:23.867709+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867709+0000","last_undegraded":"2024-09-21T02:13:23.867709+0000","last_fullsized":"2024-09-21T02:13:23.867709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867719+0000","last_change":"2024-09-21T02:13:23.867719+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867719+0000","last_undegraded":"2024-09-21T02:13:23.867719+0000","last_fullsized":"2024-09-21T02:13:23.867719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866060+0000","last_change":"2024-09-21T02:13:23.866060+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866060+0000","last_undegraded":"2024-09-21T02:13:23.866060+0000","last_fullsized":"2024-09-21T02:13:23.866060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861766+0000","last_change":"2024-09-21T02:13:23.861766+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861766+0000","last_undegraded":"2024-09-21T02:13:23.861766+0000","last_fullsized":"2024-09-21T02:13:23.861766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867856+0000","last_change":"2024-09-21T02:13:23.867856+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867856+0000","last_undegraded":"2024-09-21T02:13:23.867856+0000","last_fullsized":"2024-09-21T02:13:23.867856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.864886+0000","last_change":"2024-09-21T02:13:23.864886+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.864886+0000","last_undegraded":"2024-09-21T02:13:23.864886+0000","last_fullsized":"2024-09-21T02:13:23.864886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.868952+0000","last_change":"2024-09-21T02:13:23.868952+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.868952+0000","last_undegraded":"2024-09-21T02:13:23.868952+0000","last_fullsized":"2024-09-21T02:13:23.868952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866765+0000","last_change":"2024-09-21T02:13:23.866765+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866765+0000","last_undegraded":"2024-09-21T02:13:23.866765+0000","last_fullsized":"2024-09-21T02:13:23.866765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861508+0000","last_change":"2024-09-21T02:13:23.861508+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861508+0000","last_undegraded":"2024-09-21T02:13:23.861508+0000","last_fullsized":"2024-09-21T02:13:23.861508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861470+0000","last_change":"2024-09-21T02:13:23.861470+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861470+0000","last_undegraded":"2024-09-21T02:13:23.861470+0000","last_fullsized":"2024-09-21T02:13:23.861470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.862094+0000","last_change":"2024-09-21T02:13:23.862094+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.862094+0000","last_undegraded":"2024-09-21T02:13:23.862094+0000","last_fullsized":"2024-09-21T02:13:23.862094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867804+0000","last_change":"2024-09-21T02:13:23.867804+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867804+0000","last_undegraded":"2024-09-21T02:13:23.867804+0000","last_fullsized":"2024-09-21T02:13:23.867804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:13:23.866625+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:23.866625+0000","last_peered":"2024-09-21T02:13:23.866625+0000","last_clean":"2024-09-21T02:13:23.866625+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:23.866625+0000","last_undegraded":"2024-09-21T02:13:23.866625+0000","last_fullsized":"2024-09-21T02:13:23.866625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866753+0000","last_change":"2024-09-21T02:13:23.866753+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866753+0000","last_undegraded":"2024-09-21T02:13:23.866753+0000","last_fullsized":"2024-09-21T02:13:23.866753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866314+0000","last_change":"2024-09-21T02:13:23.866314+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866314+0000","last_undegraded":"2024-09-21T02:13:23.866314+0000","last_fullsized":"2024-09-21T02:13:23.866314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867831+0000","last_change":"2024-09-21T02:13:23.867831+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867831+0000","last_undegraded":"2024-09-21T02:13:23.867831+0000","last_fullsized":"2024-09-21T02:13:23.867831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866015+0000","last_change":"2024-09-21T02:13:23.866015+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866015+0000","last_undegraded":"2024-09-21T02:13:23.866015+0000","last_fullsized":"2024-09-21T02:13:23.866015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861727+0000","last_change":"2024-09-21T02:13:23.861727+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861727+0000","last_undegraded":"2024-09-21T02:13:23.861727+0000","last_fullsized":"2024-09-21T02:13:23.861727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.865234+0000","last_change":"2024-09-21T02:13:23.865234+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.865234+0000","last_undegraded":"2024-09-21T02:13:23.865234+0000","last_fullsized":"2024-09-21T02:13:23.865234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.865116+0000","last_change":"2024-09-21T02:13:23.865116+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.865116+0000","last_undegraded":"2024-09-21T02:13:23.865116+0000","last_fullsized":"2024-09-21T02:13:23.865116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.862156+0000","last_change":"2024-09-21T02:13:23.862156+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.862156+0000","last_undegraded":"2024-09-21T02:13:23.862156+0000","last_fullsized":"2024-09-21T02:13:23.862156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869393+0000","last_change":"2024-09-21T02:13:23.869393+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869393+0000","last_undegraded":"2024-09-21T02:13:23.869393+0000","last_fullsized":"2024-09-21T02:13:23.869393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869455+0000","last_change":"2024-09-21T02:13:23.869455+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869455+0000","last_undegraded":"2024-09-21T02:13:23.869455+0000","last_fullsized":"2024-09-21T02:13:23.869455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861621+0000","last_change":"2024-09-21T02:13:23.861621+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861621+0000","last_undegraded":"2024-09-21T02:13:23.861621+0000","last_fullsized":"2024-09-21T02:13:23.861621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861576+0000","last_change":"2024-09-21T02:13:23.861576+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861576+0000","last_undegraded":"2024-09-21T02:13:23.861576+0000","last_fullsized":"2024-09-21T02:13:23.861576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.863225+0000","last_change":"2024-09-21T02:13:23.863225+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.863225+0000","last_undegraded":"2024-09-21T02:13:23.863225+0000","last_fullsized":"2024-09-21T02:13:23.863225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":646954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":62403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36120,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335720,"statfs":{"total":96636764160,"available":96599777280,"internally_reserved":0,"allocated":790528,"data_stored":652771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:30.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.654+0000 7fdeecfc5640 1 -- 172.21.15.170:0/2006094409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cf7e0 msgr2=0x7fdee8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.654+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/2006094409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cf7e0 0x7fdee8070340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdee8070cc0 tx=0x7fded801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.654+0000 7fdeecfc5640 1 -- 172.21.15.170:0/2006094409 shutdown_connections 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.654+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/2006094409 >> v2:172.21.15.191:3300/0 conn(0x7fdee80cb460 0x7fdee80cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.654+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/2006094409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cf7e0 0x7fdee8070340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.654+0000 7fdeecfc5640 1 -- 172.21.15.170:0/2006094409 >> 172.21.15.170:0/2006094409 conn(0x7fdee80b1c40 msgr2=0x7fdee80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- 172.21.15.170:0/2006094409 shutdown_connections 2024-09-21T02:13:30.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- 172.21.15.170:0/2006094409 wait complete. 2024-09-21T02:13:30.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 Processor -- start 2024-09-21T02:13:30.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- start start 2024-09-21T02:13:30.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 0x7fdee8148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee80cc8c0 0x7fdee81657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee80cf7e0 0x7fdee80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81690d0 con 0x7fdee80cb460 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8168f50 con 0x7fdee80cc8c0 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fdee8169250 con 0x7fdee80cf7e0 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 0x7fdee8148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 0x7fdee8148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42214/0 (socket says 172.21.15.170:42214) 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 -- 172.21.15.170:0/4293429269 learned_addr learned my addr 172.21.15.170:0/4293429269 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee6d9d640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee80cf7e0 0x7fdee80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee5d9b640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee80cc8c0 0x7fdee81657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 -- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee80cf7e0 msgr2=0x7fdee80c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee80cf7e0 0x7fdee80c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 -- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee80cc8c0 msgr2=0x7fdee81657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee80cc8c0 0x7fdee81657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 -- 172.21.15.170:0/4293429269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee80c7920 con 0x7fdee80cb460 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee5d9b640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee80cc8c0 0x7fdee81657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdee659c640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 0x7fdee8148d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fded801af70 tx=0x7fded80a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80bb070 con 0x7fdee80cb460 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.658+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee80c7c00 con 0x7fdee80cb460 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fded8004430 con 0x7fdee80cb460 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee80c81c0 con 0x7fdee80cb460 2024-09-21T02:13:30.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded80b2040 con 0x7fdee80cb460 2024-09-21T02:13:30.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee80cb870 con 0x7fdee80cb460 2024-09-21T02:13:30.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fded80052d0 con 0x7fdee80cb460 2024-09-21T02:13:30.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fded6ffd640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdec8073200 0x7fdec8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:30.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fded812d930 con 0x7fdee80cb460 2024-09-21T02:13:30.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fdee5d9b640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdec8073200 0x7fdec8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:30.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.662+0000 7fdee5d9b640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdec8073200 0x7fdec8075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005fd0 tx=0x7fdedc005de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:30.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.666+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fded80164b0 con 0x7fdee80cb460 2024-09-21T02:13:30.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.830+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdee80d0db0 con 0x7fdec8073200 2024-09-21T02:13:30.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.842+0000 7fded6ffd640 1 -- 172.21.15.170:0/4293429269 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241530 (secure 0 0 0) 0x7fdee80d0db0 con 0x7fdec8073200 2024-09-21T02:13:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:30.845 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:30.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdec8073200 msgr2=0x7fdec8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdec8073200 0x7fdec8075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005fd0 tx=0x7fdedc005de0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 msgr2=0x7fdee8148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 0x7fdee8148d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fded801af70 tx=0x7fded80a57b0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 shutdown_connections 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdec8073200 0x7fdec8075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee80cf7e0 0x7fdee80c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee80cc8c0 0x7fdee81657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 --2- 172.21.15.170:0/4293429269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee80cb460 0x7fdee8148d30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 >> 172.21.15.170:0/4293429269 conn(0x7fdee80b1c40 msgr2=0x7fdee80ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 shutdown_connections 2024-09-21T02:13:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:30.846+0000 7fdeecfc5640 1 -- 172.21.15.170:0/4293429269 wait complete. 2024-09-21T02:13:30.864 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T02:13:28.863929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244152,"kb_used_data":2692,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730568,"statfs":{"total":773094113280,"available":772844101632,"internally_reserved":0,"allocated":2756608,"data_stored":1672509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.009170"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.863531+0000","last_change":"2024-09-21T02:13:23.863531+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.863531+0000","last_undegraded":"2024-09-21T02:13:23.863531+0000","last_fullsized":"2024-09-21T02:13:23.863531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869051+0000","last_change":"2024-09-21T02:13:23.869051+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869051+0000","last_undegraded":"2024-09-21T02:13:23.869051+0000","last_fullsized":"2024-09-21T02:13:23.869051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867709+0000","last_change":"2024-09-21T02:13:23.867709+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867709+0000","last_undegraded":"2024-09-21T02:13:23.867709+0000","last_fullsized":"2024-09-21T02:13:23.867709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867719+0000","last_change":"2024-09-21T02:13:23.867719+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867719+0000","last_undegraded":"2024-09-21T02:13:23.867719+0000","last_fullsized":"2024-09-21T02:13:23.867719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866060+0000","last_change":"2024-09-21T02:13:23.866060+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866060+0000","last_undegraded":"2024-09-21T02:13:23.866060+0000","last_fullsized":"2024-09-21T02:13:23.866060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861766+0000","last_change":"2024-09-21T02:13:23.861766+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861766+0000","last_undegraded":"2024-09-21T02:13:23.861766+0000","last_fullsized":"2024-09-21T02:13:23.861766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867856+0000","last_change":"2024-09-21T02:13:23.867856+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867856+0000","last_undegraded":"2024-09-21T02:13:23.867856+0000","last_fullsized":"2024-09-21T02:13:23.867856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.864886+0000","last_change":"2024-09-21T02:13:23.864886+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.864886+0000","last_undegraded":"2024-09-21T02:13:23.864886+0000","last_fullsized":"2024-09-21T02:13:23.864886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.868952+0000","last_change":"2024-09-21T02:13:23.868952+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.868952+0000","last_undegraded":"2024-09-21T02:13:23.868952+0000","last_fullsized":"2024-09-21T02:13:23.868952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866765+0000","last_change":"2024-09-21T02:13:23.866765+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866765+0000","last_undegraded":"2024-09-21T02:13:23.866765+0000","last_fullsized":"2024-09-21T02:13:23.866765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861508+0000","last_change":"2024-09-21T02:13:23.861508+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861508+0000","last_undegraded":"2024-09-21T02:13:23.861508+0000","last_fullsized":"2024-09-21T02:13:23.861508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861470+0000","last_change":"2024-09-21T02:13:23.861470+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861470+0000","last_undegraded":"2024-09-21T02:13:23.861470+0000","last_fullsized":"2024-09-21T02:13:23.861470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.862094+0000","last_change":"2024-09-21T02:13:23.862094+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.862094+0000","last_undegraded":"2024-09-21T02:13:23.862094+0000","last_fullsized":"2024-09-21T02:13:23.862094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867804+0000","last_change":"2024-09-21T02:13:23.867804+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867804+0000","last_undegraded":"2024-09-21T02:13:23.867804+0000","last_fullsized":"2024-09-21T02:13:23.867804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T02:13:23.866625+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:23.866625+0000","last_peered":"2024-09-21T02:13:23.866625+0000","last_clean":"2024-09-21T02:13:23.866625+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:23.866625+0000","last_undegraded":"2024-09-21T02:13:23.866625+0000","last_fullsized":"2024-09-21T02:13:23.866625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866753+0000","last_change":"2024-09-21T02:13:23.866753+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866753+0000","last_undegraded":"2024-09-21T02:13:23.866753+0000","last_fullsized":"2024-09-21T02:13:23.866753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866314+0000","last_change":"2024-09-21T02:13:23.866314+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866314+0000","last_undegraded":"2024-09-21T02:13:23.866314+0000","last_fullsized":"2024-09-21T02:13:23.866314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.867831+0000","last_change":"2024-09-21T02:13:23.867831+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.867831+0000","last_undegraded":"2024-09-21T02:13:23.867831+0000","last_fullsized":"2024-09-21T02:13:23.867831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.866015+0000","last_change":"2024-09-21T02:13:23.866015+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.866015+0000","last_undegraded":"2024-09-21T02:13:23.866015+0000","last_fullsized":"2024-09-21T02:13:23.866015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861727+0000","last_change":"2024-09-21T02:13:23.861727+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861727+0000","last_undegraded":"2024-09-21T02:13:23.861727+0000","last_fullsized":"2024-09-21T02:13:23.861727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.865234+0000","last_change":"2024-09-21T02:13:23.865234+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.865234+0000","last_undegraded":"2024-09-21T02:13:23.865234+0000","last_fullsized":"2024-09-21T02:13:23.865234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.865116+0000","last_change":"2024-09-21T02:13:23.865116+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.865116+0000","last_undegraded":"2024-09-21T02:13:23.865116+0000","last_fullsized":"2024-09-21T02:13:23.865116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.862156+0000","last_change":"2024-09-21T02:13:23.862156+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.862156+0000","last_undegraded":"2024-09-21T02:13:23.862156+0000","last_fullsized":"2024-09-21T02:13:23.862156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869393+0000","last_change":"2024-09-21T02:13:23.869393+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869393+0000","last_undegraded":"2024-09-21T02:13:23.869393+0000","last_fullsized":"2024-09-21T02:13:23.869393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.869455+0000","last_change":"2024-09-21T02:13:23.869455+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.869455+0000","last_undegraded":"2024-09-21T02:13:23.869455+0000","last_fullsized":"2024-09-21T02:13:23.869455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861621+0000","last_change":"2024-09-21T02:13:23.861621+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861621+0000","last_undegraded":"2024-09-21T02:13:23.861621+0000","last_fullsized":"2024-09-21T02:13:23.861621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.861576+0000","last_change":"2024-09-21T02:13:23.861576+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.861576+0000","last_undegraded":"2024-09-21T02:13:23.861576+0000","last_fullsized":"2024-09-21T02:13:23.861576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T02:13:23.863225+0000","last_change":"2024-09-21T02:13:23.863225+0000","last_active":"2024-09-21T02:13:23.844820+0000","last_peered":"2024-09-21T02:13:23.844820+0000","last_clean":"2024-09-21T02:13:23.844820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:23.863225+0000","last_undegraded":"2024-09-21T02:13:23.863225+0000","last_fullsized":"2024-09-21T02:13:23.863225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:13:24.855267+0000","last_change":"2024-09-21T02:13:24.855267+0000","last_active":"2024-09-21T02:13:24.855267+0000","last_peered":"2024-09-21T02:13:24.855267+0000","last_clean":"2024-09-21T02:13:24.855267+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:13:24.855267+0000","last_undegraded":"2024-09-21T02:13:24.855267+0000","last_fullsized":"2024-09-21T02:13:24.855267+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":646954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":62403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36120,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335720,"statfs":{"total":96636764160,"available":96599777280,"internally_reserved":0,"allocated":790528,"data_stored":652771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":56586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:30.867 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-21T02:13:30.867 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.868 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-21T02:13:30.869 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.870 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-21T02:13:30.870 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.871 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.872 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.873 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.863531+0000', 'last_change': '2024-09-21T02:13:23.863531+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.863531+0000', 'last_undegraded': '2024-09-21T02:13:23.863531+0000', 'last_fullsized': '2024-09-21T02:13:23.863531+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.869051+0000', 'last_change': '2024-09-21T02:13:23.869051+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.869051+0000', 'last_undegraded': '2024-09-21T02:13:23.869051+0000', 'last_fullsized': '2024-09-21T02:13:23.869051+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.867709+0000', 'last_change': '2024-09-21T02:13:23.867709+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.867709+0000', 'last_undegraded': '2024-09-21T02:13:23.867709+0000', 'last_fullsized': '2024-09-21T02:13:23.867709+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.867719+0000', 'last_change': '2024-09-21T02:13:23.867719+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.867719+0000', 'last_undegraded': '2024-09-21T02:13:23.867719+0000', 'last_fullsized': '2024-09-21T02:13:23.867719+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-21T02:13:30.874 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.866060+0000', 'last_change': '2024-09-21T02:13:23.866060+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.866060+0000', 'last_undegraded': '2024-09-21T02:13:23.866060+0000', 'last_fullsized': '2024-09-21T02:13:23.866060+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.861766+0000', 'last_change': '2024-09-21T02:13:23.861766+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.861766+0000', 'last_undegraded': '2024-09-21T02:13:23.861766+0000', 'last_fullsized': '2024-09-21T02:13:23.861766+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.867856+0000', 'last_change': '2024-09-21T02:13:23.867856+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.867856+0000', 'last_undegraded': '2024-09-21T02:13:23.867856+0000', 'last_fullsized': '2024-09-21T02:13:23.867856+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-21T02:13:30.875 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.864886+0000', 'last_change': '2024-09-21T02:13:23.864886+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.864886+0000', 'last_undegraded': '2024-09-21T02:13:23.864886+0000', 'last_fullsized': '2024-09-21T02:13:23.864886+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.868952+0000', 'last_change': '2024-09-21T02:13:23.868952+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.868952+0000', 'last_undegraded': '2024-09-21T02:13:23.868952+0000', 'last_fullsized': '2024-09-21T02:13:23.868952+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.866765+0000', 'last_change': '2024-09-21T02:13:23.866765+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.866765+0000', 'last_undegraded': '2024-09-21T02:13:23.866765+0000', 'last_fullsized': '2024-09-21T02:13:23.866765+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-21T02:13:30.876 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.861508+0000', 'last_change': '2024-09-21T02:13:23.861508+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.861508+0000', 'last_undegraded': '2024-09-21T02:13:23.861508+0000', 'last_fullsized': '2024-09-21T02:13:23.861508+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.861470+0000', 'last_change': '2024-09-21T02:13:23.861470+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.861470+0000', 'last_undegraded': '2024-09-21T02:13:23.861470+0000', 'last_fullsized': '2024-09-21T02:13:23.861470+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-21T02:13:30.877 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.862094+0000', 'last_change': '2024-09-21T02:13:23.862094+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.862094+0000', 'last_undegraded': '2024-09-21T02:13:23.862094+0000', 'last_fullsized': '2024-09-21T02:13:23.862094+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.867804+0000', 'last_change': '2024-09-21T02:13:23.867804+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.867804+0000', 'last_undegraded': '2024-09-21T02:13:23.867804+0000', 'last_fullsized': '2024-09-21T02:13:23.867804+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.866753+0000', 'last_change': '2024-09-21T02:13:23.866753+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.866753+0000', 'last_undegraded': '2024-09-21T02:13:23.866753+0000', 'last_fullsized': '2024-09-21T02:13:23.866753+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.878 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.866314+0000', 'last_change': '2024-09-21T02:13:23.866314+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.866314+0000', 'last_undegraded': '2024-09-21T02:13:23.866314+0000', 'last_fullsized': '2024-09-21T02:13:23.866314+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.867831+0000', 'last_change': '2024-09-21T02:13:23.867831+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.867831+0000', 'last_undegraded': '2024-09-21T02:13:23.867831+0000', 'last_fullsized': '2024-09-21T02:13:23.867831+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.866015+0000', 'last_change': '2024-09-21T02:13:23.866015+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.866015+0000', 'last_undegraded': '2024-09-21T02:13:23.866015+0000', 'last_fullsized': '2024-09-21T02:13:23.866015+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.879 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.861727+0000', 'last_change': '2024-09-21T02:13:23.861727+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.861727+0000', 'last_undegraded': '2024-09-21T02:13:23.861727+0000', 'last_fullsized': '2024-09-21T02:13:23.861727+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.865234+0000', 'last_change': '2024-09-21T02:13:23.865234+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.865234+0000', 'last_undegraded': '2024-09-21T02:13:23.865234+0000', 'last_fullsized': '2024-09-21T02:13:23.865234+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.865116+0000', 'last_change': '2024-09-21T02:13:23.865116+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.865116+0000', 'last_undegraded': '2024-09-21T02:13:23.865116+0000', 'last_fullsized': '2024-09-21T02:13:23.865116+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.880 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.862156+0000', 'last_change': '2024-09-21T02:13:23.862156+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.862156+0000', 'last_undegraded': '2024-09-21T02:13:23.862156+0000', 'last_fullsized': '2024-09-21T02:13:23.862156+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.869393+0000', 'last_change': '2024-09-21T02:13:23.869393+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.869393+0000', 'last_undegraded': '2024-09-21T02:13:23.869393+0000', 'last_fullsized': '2024-09-21T02:13:23.869393+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.869455+0000', 'last_change': '2024-09-21T02:13:23.869455+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.869455+0000', 'last_undegraded': '2024-09-21T02:13:23.869455+0000', 'last_fullsized': '2024-09-21T02:13:23.869455+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-21T02:13:30.881 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.861621+0000', 'last_change': '2024-09-21T02:13:23.861621+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.861621+0000', 'last_undegraded': '2024-09-21T02:13:23.861621+0000', 'last_fullsized': '2024-09-21T02:13:23.861621+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.861576+0000', 'last_change': '2024-09-21T02:13:23.861576+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.861576+0000', 'last_undegraded': '2024-09-21T02:13:23.861576+0000', 'last_fullsized': '2024-09-21T02:13:23.861576+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:13:23.863225+0000', 'last_change': '2024-09-21T02:13:23.863225+0000', 'last_active': '2024-09-21T02:13:23.844820+0000', 'last_peered': '2024-09-21T02:13:23.844820+0000', 'last_clean': '2024-09-21T02:13:23.844820+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:23.863225+0000', 'last_undegraded': '2024-09-21T02:13:23.863225+0000', 'last_fullsized': '2024-09-21T02:13:23.863225+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:23.844820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-21T02:13:30.882 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:13:24.855267+0000', 'last_change': '2024-09-21T02:13:24.855267+0000', 'last_active': '2024-09-21T02:13:24.855267+0000', 'last_peered': '2024-09-21T02:13:24.855267+0000', 'last_clean': '2024-09-21T02:13:24.855267+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:13:24.855267+0000', 'last_undegraded': '2024-09-21T02:13:24.855267+0000', 'last_fullsized': '2024-09-21T02:13:24.855267+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'last_clean_scrub_stamp': '2024-09-21T02:13:24.855267+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:13:30.883 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 -- 172.21.15.170:0/2974049987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cf7e0 msgr2=0x7f62d4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2974049987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cf7e0 0x7f62d4070340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009900 tx=0x7f62d001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 -- 172.21.15.170:0/2974049987 shutdown_connections 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2974049987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62d40cb460 0x7f62d40cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2974049987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cf7e0 0x7f62d4070340 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 -- 172.21.15.170:0/2974049987 >> 172.21.15.170:0/2974049987 conn(0x7f62d40b1c40 msgr2=0x7f62d40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 -- 172.21.15.170:0/2974049987 shutdown_connections 2024-09-21T02:13:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 -- 172.21.15.170:0/2974049987 wait complete. 2024-09-21T02:13:31.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 Processor -- start 2024-09-21T02:13:31.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.046+0000 7f62dad4c640 1 -- start start 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 0x7f62d414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62d40cc8c0 0x7f62d41fa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62d40cf7e0 0x7f62d41fac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4169180 con 0x7f62d40cb460 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4169000 con 0x7f62d40cf7e0 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f62d4169300 con 0x7f62d40cc8c0 2024-09-21T02:13:31.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 0x7f62d414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 0x7f62d414b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42240/0 (socket says 172.21.15.170:42240) 2024-09-21T02:13:31.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 -- 172.21.15.170:0/2973091078 learned_addr learned my addr 172.21.15.170:0/2973091078 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:31.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62cbfff640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62d40cc8c0 0x7f62d41fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d92e9640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62d40cf7e0 0x7f62d41fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 -- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62d40cc8c0 msgr2=0x7f62d41fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62d40cc8c0 0x7f62d41fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 -- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62d40cf7e0 msgr2=0x7f62d41fac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62d40cf7e0 0x7f62d41fac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 -- 172.21.15.170:0/2973091078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d40c59f0 con 0x7f62d40cb460 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62d8ae8640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 0x7f62d414b900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009850 tx=0x7f62d0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00b2040 con 0x7f62d40cb460 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62d0003600 con 0x7f62d40cb460 2024-09-21T02:13:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00039f0 con 0x7f62d40cb460 2024-09-21T02:13:31.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d40c5cd0 con 0x7f62d40cb460 2024-09-21T02:13:31.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d40c7830 con 0x7f62d40cb460 2024-09-21T02:13:31.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d40cb870 con 0x7f62d40cb460 2024-09-21T02:13:31.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f62d0003b90 con 0x7f62d40cb460 2024-09-21T02:13:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62c97fa640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f62c0073200 0x7f62c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f62d012dff0 con 0x7f62d40cb460 2024-09-21T02:13:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.050+0000 7f62cbfff640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f62c0073200 0x7f62c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.054+0000 7f62cbfff640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f62c0073200 0x7f62c0075710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62d40af0b0 tx=0x7f62c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.054+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f62d00bd060 con 0x7f62d40cb460 2024-09-21T02:13:31.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.274+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f62d40ad0a0 con 0x7f62d40cb460 2024-09-21T02:13:31.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.274+0000 7f62c97fa640 1 -- 172.21.15.170:0/2973091078 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1053 (secure 0 0 0) 0x7f62d00164d0 con 0x7f62d40cb460 2024-09-21T02:13:31.276 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:31.276 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1726884793,"num_in_osds":8,"osd_in_since":1726884783,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":250130432,"bytes_avail":772843982848,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"c","status":"up:active","gid":4381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-21T02:12:59.969618+0000","services":{}},"progress_events":{}} 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f62c0073200 msgr2=0x7f62c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f62c0073200 0x7f62c0075710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62d40af0b0 tx=0x7f62c4008040 comp rx=0 tx=0).stop 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 msgr2=0x7f62d414b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 0x7f62d414b900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009850 tx=0x7f62d0004040 comp rx=0 tx=0).stop 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 shutdown_connections 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f62c0073200 0x7f62c0075710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62d40cc8c0 0x7f62d41fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62d40cf7e0 0x7f62d41fac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 --2- 172.21.15.170:0/2973091078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d40cb460 0x7f62d414b900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 >> 172.21.15.170:0/2973091078 conn(0x7f62d40b1c40 msgr2=0x7f62d415a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 shutdown_connections 2024-09-21T02:13:31.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.278+0000 7f62dad4c640 1 -- 172.21.15.170:0/2973091078 wait complete. 2024-09-21T02:13:31.296 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T02:13:31.296 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:31.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 -- 172.21.15.170:0/1485075313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cf7e0 msgr2=0x7f44e8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 --2- 172.21.15.170:0/1485075313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cf7e0 0x7f44e8070340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009900 tx=0x7f44e401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 -- 172.21.15.170:0/1485075313 shutdown_connections 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 --2- 172.21.15.170:0/1485075313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44e80cb460 0x7f44e80cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 --2- 172.21.15.170:0/1485075313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cf7e0 0x7f44e8070340 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 -- 172.21.15.170:0/1485075313 >> 172.21.15.170:0/1485075313 conn(0x7f44e80b1c40 msgr2=0x7f44e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 -- 172.21.15.170:0/1485075313 shutdown_connections 2024-09-21T02:13:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 -- 172.21.15.170:0/1485075313 wait complete. 2024-09-21T02:13:31.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 Processor -- start 2024-09-21T02:13:31.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.438+0000 7f44f05bc640 1 -- start start 2024-09-21T02:13:31.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80cb460 0x7f44e815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 0x7f44e81f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44e80cf7e0 0x7f44e80afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8169180 con 0x7f44e80cc8c0 2024-09-21T02:13:31.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8169000 con 0x7f44e80cb460 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f44e8169300 con 0x7f44e80cf7e0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 0x7f44e81f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 0x7f44e81f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42256/0 (socket says 172.21.15.170:42256) 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 -- 172.21.15.170:0/3356102305 learned_addr learned my addr 172.21.15.170:0/3356102305 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 -- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44e80cf7e0 msgr2=0x7f44e80afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44ee358640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80cb460 0x7f44e815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44e80cf7e0 0x7f44e80afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 -- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80cb460 msgr2=0x7f44e815b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80cb460 0x7f44e815b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 -- 172.21.15.170:0/3356102305 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e80c6fb0 con 0x7f44e80cc8c0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44edb57640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 0x7f44e81f9a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f44d800b440 tx=0x7f44d800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8015020 con 0x7f44e80cc8c0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44d80119a0 con 0x7f44e80cc8c0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e80c7290 con 0x7f44e80cc8c0 2024-09-21T02:13:31.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8011df0 con 0x7f44e80cc8c0 2024-09-21T02:13:31.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e80c7850 con 0x7f44e80cc8c0 2024-09-21T02:13:31.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e80cb870 con 0x7f44e80cc8c0 2024-09-21T02:13:31.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f44d8018020 con 0x7f44e80cc8c0 2024-09-21T02:13:31.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44deffd640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f44cc073200 0x7f44cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:31.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f44d8094320 con 0x7f44e80cc8c0 2024-09-21T02:13:31.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.442+0000 7f44ee358640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f44cc073200 0x7f44cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:31.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.446+0000 7f44ee358640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f44cc073200 0x7f44cc075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44e40af000 tx=0x7f44e401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:31.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.446+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f44d801f020 con 0x7f44e80cc8c0 2024-09-21T02:13:31.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.602+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44e81f9f70 con 0x7f44cc073200 2024-09-21T02:13:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.614+0000 7f44deffd640 1 -- 172.21.15.170:0/3356102305 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f44e40f9840 con 0x7f44cc073200 2024-09-21T02:13:31.619 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:31.620 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f44cc073200 msgr2=0x7f44cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f44cc073200 0x7f44cc075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44e40af000 tx=0x7f44e401b120 comp rx=0 tx=0).stop 2024-09-21T02:13:31.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 msgr2=0x7f44e81f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 0x7f44e81f9a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f44d800b440 tx=0x7f44d800b900 comp rx=0 tx=0).stop 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 shutdown_connections 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f44cc073200 0x7f44cc075710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44e80cf7e0 0x7f44e80afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44e80cc8c0 0x7f44e81f9a30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 --2- 172.21.15.170:0/3356102305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80cb460 0x7f44e815b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.618+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 >> 172.21.15.170:0/3356102305 conn(0x7f44e80b1c40 msgr2=0x7f44e80cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.622+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 shutdown_connections 2024-09-21T02:13:31.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:31.622+0000 7f44f05bc640 1 -- 172.21.15.170:0/3356102305 wait complete. 2024-09-21T02:13:31.638 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-21T02:13:30.864787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244268,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730452,"statfs":{"total":773094113280,"available":772843982848,"internally_reserved":0,"allocated":3121152,"data_stored":1839580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002335"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964977+0000","last_change":"2024-09-21T02:13:25.918477+0000","last_active":"2024-09-21T02:13:26.964977+0000","last_peered":"2024-09-21T02:13:26.964977+0000","last_clean":"2024-09-21T02:13:26.964977+0000","last_became_active":"2024-09-21T02:13:25.917930+0000","last_became_peered":"2024-09-21T02:13:25.917930+0000","last_unstale":"2024-09-21T02:13:26.964977+0000","last_undegraded":"2024-09-21T02:13:26.964977+0000","last_fullsized":"2024-09-21T02:13:26.964977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:08.714546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883166+0000","last_change":"2024-09-21T02:13:25.893989+0000","last_active":"2024-09-21T02:13:26.883166+0000","last_peered":"2024-09-21T02:13:26.883166+0000","last_clean":"2024-09-21T02:13:26.883166+0000","last_became_active":"2024-09-21T02:13:25.893173+0000","last_became_peered":"2024-09-21T02:13:25.893173+0000","last_unstale":"2024-09-21T02:13:26.883166+0000","last_undegraded":"2024-09-21T02:13:26.883166+0000","last_fullsized":"2024-09-21T02:13:26.883166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:49.876135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879431+0000","last_change":"2024-09-21T02:13:25.932549+0000","last_active":"2024-09-21T02:13:26.879431+0000","last_peered":"2024-09-21T02:13:26.879431+0000","last_clean":"2024-09-21T02:13:26.879431+0000","last_became_active":"2024-09-21T02:13:25.931843+0000","last_became_peered":"2024-09-21T02:13:25.931843+0000","last_unstale":"2024-09-21T02:13:26.879431+0000","last_undegraded":"2024-09-21T02:13:26.879431+0000","last_fullsized":"2024-09-21T02:13:26.879431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:19.382874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879618+0000","last_change":"2024-09-21T02:13:25.941857+0000","last_active":"2024-09-21T02:13:26.879618+0000","last_peered":"2024-09-21T02:13:26.879618+0000","last_clean":"2024-09-21T02:13:26.879618+0000","last_became_active":"2024-09-21T02:13:25.941422+0000","last_became_peered":"2024-09-21T02:13:25.941422+0000","last_unstale":"2024-09-21T02:13:26.879618+0000","last_undegraded":"2024-09-21T02:13:26.879618+0000","last_fullsized":"2024-09-21T02:13:26.879618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:02.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965518+0000","last_change":"2024-09-21T02:13:25.911062+0000","last_active":"2024-09-21T02:13:26.965518+0000","last_peered":"2024-09-21T02:13:26.965518+0000","last_clean":"2024-09-21T02:13:26.965518+0000","last_became_active":"2024-09-21T02:13:25.899879+0000","last_became_peered":"2024-09-21T02:13:25.899879+0000","last_unstale":"2024-09-21T02:13:26.965518+0000","last_undegraded":"2024-09-21T02:13:26.965518+0000","last_fullsized":"2024-09-21T02:13:26.965518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:40.011102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878421+0000","last_change":"2024-09-21T02:13:25.942318+0000","last_active":"2024-09-21T02:13:26.878421+0000","last_peered":"2024-09-21T02:13:26.878421+0000","last_clean":"2024-09-21T02:13:26.878421+0000","last_became_active":"2024-09-21T02:13:25.941452+0000","last_became_peered":"2024-09-21T02:13:25.941452+0000","last_unstale":"2024-09-21T02:13:26.878421+0000","last_undegraded":"2024-09-21T02:13:26.878421+0000","last_fullsized":"2024-09-21T02:13:26.878421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:08.840148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963534+0000","last_change":"2024-09-21T02:13:25.942111+0000","last_active":"2024-09-21T02:13:26.963534+0000","last_peered":"2024-09-21T02:13:26.963534+0000","last_clean":"2024-09-21T02:13:26.963534+0000","last_became_active":"2024-09-21T02:13:25.941400+0000","last_became_peered":"2024-09-21T02:13:25.941400+0000","last_unstale":"2024-09-21T02:13:26.963534+0000","last_undegraded":"2024-09-21T02:13:26.963534+0000","last_fullsized":"2024-09-21T02:13:26.963534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878836+0000","last_change":"2024-09-21T02:13:25.914747+0000","last_active":"2024-09-21T02:13:26.878836+0000","last_peered":"2024-09-21T02:13:26.878836+0000","last_clean":"2024-09-21T02:13:26.878836+0000","last_became_active":"2024-09-21T02:13:25.914058+0000","last_became_peered":"2024-09-21T02:13:25.914058+0000","last_unstale":"2024-09-21T02:13:26.878836+0000","last_undegraded":"2024-09-21T02:13:26.878836+0000","last_fullsized":"2024-09-21T02:13:26.878836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:21.986443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967211+0000","last_change":"2024-09-21T02:13:25.909803+0000","last_active":"2024-09-21T02:13:26.967211+0000","last_peered":"2024-09-21T02:13:26.967211+0000","last_clean":"2024-09-21T02:13:26.967211+0000","last_became_active":"2024-09-21T02:13:25.909116+0000","last_became_peered":"2024-09-21T02:13:25.909116+0000","last_unstale":"2024-09-21T02:13:26.967211+0000","last_undegraded":"2024-09-21T02:13:26.967211+0000","last_fullsized":"2024-09-21T02:13:26.967211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:00.174710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965018+0000","last_change":"2024-09-21T02:13:25.915682+0000","last_active":"2024-09-21T02:13:26.965018+0000","last_peered":"2024-09-21T02:13:26.965018+0000","last_clean":"2024-09-21T02:13:26.965018+0000","last_became_active":"2024-09-21T02:13:25.915194+0000","last_became_peered":"2024-09-21T02:13:25.915194+0000","last_unstale":"2024-09-21T02:13:26.965018+0000","last_undegraded":"2024-09-21T02:13:26.965018+0000","last_fullsized":"2024-09-21T02:13:26.965018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.966309+0000","last_change":"2024-09-21T02:13:25.914960+0000","last_active":"2024-09-21T02:13:26.966309+0000","last_peered":"2024-09-21T02:13:26.966309+0000","last_clean":"2024-09-21T02:13:26.966309+0000","last_became_active":"2024-09-21T02:13:25.914610+0000","last_became_peered":"2024-09-21T02:13:25.914610+0000","last_unstale":"2024-09-21T02:13:26.966309+0000","last_undegraded":"2024-09-21T02:13:26.966309+0000","last_fullsized":"2024-09-21T02:13:26.966309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965589+0000","last_change":"2024-09-21T02:13:25.924430+0000","last_active":"2024-09-21T02:13:26.965589+0000","last_peered":"2024-09-21T02:13:26.965589+0000","last_clean":"2024-09-21T02:13:26.965589+0000","last_became_active":"2024-09-21T02:13:25.923881+0000","last_became_peered":"2024-09-21T02:13:25.923881+0000","last_unstale":"2024-09-21T02:13:26.965589+0000","last_undegraded":"2024-09-21T02:13:26.965589+0000","last_fullsized":"2024-09-21T02:13:26.965589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963608+0000","last_change":"2024-09-21T02:13:25.920280+0000","last_active":"2024-09-21T02:13:26.963608+0000","last_peered":"2024-09-21T02:13:26.963608+0000","last_clean":"2024-09-21T02:13:26.963608+0000","last_became_active":"2024-09-21T02:13:25.919389+0000","last_became_peered":"2024-09-21T02:13:25.919389+0000","last_unstale":"2024-09-21T02:13:26.963608+0000","last_undegraded":"2024-09-21T02:13:26.963608+0000","last_fullsized":"2024-09-21T02:13:26.963608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:40.761905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963494+0000","last_change":"2024-09-21T02:13:25.919658+0000","last_active":"2024-09-21T02:13:26.963494+0000","last_peered":"2024-09-21T02:13:26.963494+0000","last_clean":"2024-09-21T02:13:26.963494+0000","last_became_active":"2024-09-21T02:13:25.918063+0000","last_became_peered":"2024-09-21T02:13:25.918063+0000","last_unstale":"2024-09-21T02:13:26.963494+0000","last_undegraded":"2024-09-21T02:13:26.963494+0000","last_fullsized":"2024-09-21T02:13:26.963494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.069956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882761+0000","last_change":"2024-09-21T02:13:25.915232+0000","last_active":"2024-09-21T02:13:26.882761+0000","last_peered":"2024-09-21T02:13:26.882761+0000","last_clean":"2024-09-21T02:13:26.882761+0000","last_became_active":"2024-09-21T02:13:25.914387+0000","last_became_peered":"2024-09-21T02:13:25.914387+0000","last_unstale":"2024-09-21T02:13:26.882761+0000","last_undegraded":"2024-09-21T02:13:26.882761+0000","last_fullsized":"2024-09-21T02:13:26.882761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878878+0000","last_change":"2024-09-21T02:13:25.902781+0000","last_active":"2024-09-21T02:13:26.878878+0000","last_peered":"2024-09-21T02:13:26.878878+0000","last_clean":"2024-09-21T02:13:26.878878+0000","last_became_active":"2024-09-21T02:13:25.902445+0000","last_became_peered":"2024-09-21T02:13:25.902445+0000","last_unstale":"2024-09-21T02:13:26.878878+0000","last_undegraded":"2024-09-21T02:13:26.878878+0000","last_fullsized":"2024-09-21T02:13:26.878878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881922+0000","last_change":"2024-09-21T02:13:25.915843+0000","last_active":"2024-09-21T02:13:26.881922+0000","last_peered":"2024-09-21T02:13:26.881922+0000","last_clean":"2024-09-21T02:13:26.881922+0000","last_became_active":"2024-09-21T02:13:25.915560+0000","last_became_peered":"2024-09-21T02:13:25.915560+0000","last_unstale":"2024-09-21T02:13:26.881922+0000","last_undegraded":"2024-09-21T02:13:26.881922+0000","last_fullsized":"2024-09-21T02:13:26.881922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:41.816297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965307+0000","last_change":"2024-09-21T02:13:25.926096+0000","last_active":"2024-09-21T02:13:26.965307+0000","last_peered":"2024-09-21T02:13:26.965307+0000","last_clean":"2024-09-21T02:13:26.965307+0000","last_became_active":"2024-09-21T02:13:25.925408+0000","last_became_peered":"2024-09-21T02:13:25.925408+0000","last_unstale":"2024-09-21T02:13:26.965307+0000","last_undegraded":"2024-09-21T02:13:26.965307+0000","last_fullsized":"2024-09-21T02:13:26.965307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.244537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881981+0000","last_change":"2024-09-21T02:13:25.940050+0000","last_active":"2024-09-21T02:13:26.881981+0000","last_peered":"2024-09-21T02:13:26.881981+0000","last_clean":"2024-09-21T02:13:26.881981+0000","last_became_active":"2024-09-21T02:13:25.939688+0000","last_became_peered":"2024-09-21T02:13:25.939688+0000","last_unstale":"2024-09-21T02:13:26.881981+0000","last_undegraded":"2024-09-21T02:13:26.881981+0000","last_fullsized":"2024-09-21T02:13:26.881981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:47.887251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965669+0000","last_change":"2024-09-21T02:13:25.913323+0000","last_active":"2024-09-21T02:13:26.965669+0000","last_peered":"2024-09-21T02:13:26.965669+0000","last_clean":"2024-09-21T02:13:26.965669+0000","last_became_active":"2024-09-21T02:13:25.912945+0000","last_became_peered":"2024-09-21T02:13:25.912945+0000","last_unstale":"2024-09-21T02:13:26.965669+0000","last_undegraded":"2024-09-21T02:13:26.965669+0000","last_fullsized":"2024-09-21T02:13:26.965669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964449+0000","last_change":"2024-09-21T02:13:25.926590+0000","last_active":"2024-09-21T02:13:26.964449+0000","last_peered":"2024-09-21T02:13:26.964449+0000","last_clean":"2024-09-21T02:13:26.964449+0000","last_became_active":"2024-09-21T02:13:25.926154+0000","last_became_peered":"2024-09-21T02:13:25.926154+0000","last_unstale":"2024-09-21T02:13:26.964449+0000","last_undegraded":"2024-09-21T02:13:26.964449+0000","last_fullsized":"2024-09-21T02:13:26.964449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:00.641354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882548+0000","last_change":"2024-09-21T02:13:25.896024+0000","last_active":"2024-09-21T02:13:26.882548+0000","last_peered":"2024-09-21T02:13:26.882548+0000","last_clean":"2024-09-21T02:13:26.882548+0000","last_became_active":"2024-09-21T02:13:25.895216+0000","last_became_peered":"2024-09-21T02:13:25.895216+0000","last_unstale":"2024-09-21T02:13:26.882548+0000","last_undegraded":"2024-09-21T02:13:26.882548+0000","last_fullsized":"2024-09-21T02:13:26.882548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879525+0000","last_change":"2024-09-21T02:13:25.928846+0000","last_active":"2024-09-21T02:13:26.879525+0000","last_peered":"2024-09-21T02:13:26.879525+0000","last_clean":"2024-09-21T02:13:26.879525+0000","last_became_active":"2024-09-21T02:13:25.928527+0000","last_became_peered":"2024-09-21T02:13:25.928527+0000","last_unstale":"2024-09-21T02:13:26.879525+0000","last_undegraded":"2024-09-21T02:13:26.879525+0000","last_fullsized":"2024-09-21T02:13:26.879525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:38.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883432+0000","last_change":"2024-09-21T02:13:25.911388+0000","last_active":"2024-09-21T02:13:26.883432+0000","last_peered":"2024-09-21T02:13:26.883432+0000","last_clean":"2024-09-21T02:13:26.883432+0000","last_became_active":"2024-09-21T02:13:25.910631+0000","last_became_peered":"2024-09-21T02:13:25.910631+0000","last_unstale":"2024-09-21T02:13:26.883432+0000","last_undegraded":"2024-09-21T02:13:26.883432+0000","last_fullsized":"2024-09-21T02:13:26.883432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:01.276714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882752+0000","last_change":"2024-09-21T02:13:25.908215+0000","last_active":"2024-09-21T02:13:26.882752+0000","last_peered":"2024-09-21T02:13:26.882752+0000","last_clean":"2024-09-21T02:13:26.882752+0000","last_became_active":"2024-09-21T02:13:25.907764+0000","last_became_peered":"2024-09-21T02:13:25.907764+0000","last_unstale":"2024-09-21T02:13:26.882752+0000","last_undegraded":"2024-09-21T02:13:26.882752+0000","last_fullsized":"2024-09-21T02:13:26.882752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:31.448658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.918292+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.917267+0000","last_became_peered":"2024-09-21T02:13:25.917267+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:22.501288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963998+0000","last_change":"2024-09-21T02:13:25.918530+0000","last_active":"2024-09-21T02:13:26.963998+0000","last_peered":"2024-09-21T02:13:26.963998+0000","last_clean":"2024-09-21T02:13:26.963998+0000","last_became_active":"2024-09-21T02:13:25.917735+0000","last_became_peered":"2024-09-21T02:13:25.917735+0000","last_unstale":"2024-09-21T02:13:26.963998+0000","last_undegraded":"2024-09-21T02:13:26.963998+0000","last_fullsized":"2024-09-21T02:13:26.963998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:03.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880008+0000","last_change":"2024-09-21T02:13:25.913497+0000","last_active":"2024-09-21T02:13:26.880008+0000","last_peered":"2024-09-21T02:13:26.880008+0000","last_clean":"2024-09-21T02:13:26.880008+0000","last_became_active":"2024-09-21T02:13:25.912567+0000","last_became_peered":"2024-09-21T02:13:25.912567+0000","last_unstale":"2024-09-21T02:13:26.880008+0000","last_undegraded":"2024-09-21T02:13:26.880008+0000","last_fullsized":"2024-09-21T02:13:26.880008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879152+0000","last_change":"2024-09-21T02:13:25.917618+0000","last_active":"2024-09-21T02:13:26.879152+0000","last_peered":"2024-09-21T02:13:26.879152+0000","last_clean":"2024-09-21T02:13:26.879152+0000","last_became_active":"2024-09-21T02:13:25.916325+0000","last_became_peered":"2024-09-21T02:13:25.916325+0000","last_unstale":"2024-09-21T02:13:26.879152+0000","last_undegraded":"2024-09-21T02:13:26.879152+0000","last_fullsized":"2024-09-21T02:13:26.879152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.915861+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.912257+0000","last_became_peered":"2024-09-21T02:13:25.912257+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:56.675098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878068+0000","last_change":"2024-09-21T02:13:25.894915+0000","last_active":"2024-09-21T02:13:26.878068+0000","last_peered":"2024-09-21T02:13:26.878068+0000","last_clean":"2024-09-21T02:13:26.878068+0000","last_became_active":"2024-09-21T02:13:25.894440+0000","last_became_peered":"2024-09-21T02:13:25.894440+0000","last_unstale":"2024-09-21T02:13:26.878068+0000","last_undegraded":"2024-09-21T02:13:26.878068+0000","last_fullsized":"2024-09-21T02:13:26.878068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.857628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964697+0000","last_change":"2024-09-21T02:13:25.930047+0000","last_active":"2024-09-21T02:13:26.964697+0000","last_peered":"2024-09-21T02:13:26.964697+0000","last_clean":"2024-09-21T02:13:26.964697+0000","last_became_active":"2024-09-21T02:13:25.929729+0000","last_became_peered":"2024-09-21T02:13:25.929729+0000","last_unstale":"2024-09-21T02:13:26.964697+0000","last_undegraded":"2024-09-21T02:13:26.964697+0000","last_fullsized":"2024-09-21T02:13:26.964697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:26.319373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880194+0000","last_change":"2024-09-21T02:13:24.903640+0000","last_active":"2024-09-21T02:13:26.880194+0000","last_peered":"2024-09-21T02:13:26.880194+0000","last_clean":"2024-09-21T02:13:26.880194+0000","last_became_active":"2024-09-21T02:13:24.899880+0000","last_became_peered":"2024-09-21T02:13:24.899880+0000","last_unstale":"2024-09-21T02:13:26.880194+0000","last_undegraded":"2024-09-21T02:13:26.880194+0000","last_fullsized":"2024-09-21T02:13:26.880194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879277+0000","last_change":"2024-09-21T02:13:24.906393+0000","last_active":"2024-09-21T02:13:26.879277+0000","last_peered":"2024-09-21T02:13:26.879277+0000","last_clean":"2024-09-21T02:13:26.879277+0000","last_became_active":"2024-09-21T02:13:24.905410+0000","last_became_peered":"2024-09-21T02:13:24.905410+0000","last_unstale":"2024-09-21T02:13:26.879277+0000","last_undegraded":"2024-09-21T02:13:26.879277+0000","last_fullsized":"2024-09-21T02:13:26.879277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:13.262457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.962957+0000","last_change":"2024-09-21T02:13:24.909821+0000","last_active":"2024-09-21T02:13:26.962957+0000","last_peered":"2024-09-21T02:13:26.962957+0000","last_clean":"2024-09-21T02:13:26.962957+0000","last_became_active":"2024-09-21T02:13:24.907797+0000","last_became_peered":"2024-09-21T02:13:24.907797+0000","last_unstale":"2024-09-21T02:13:26.962957+0000","last_undegraded":"2024-09-21T02:13:26.962957+0000","last_fullsized":"2024-09-21T02:13:26.962957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879469+0000","last_change":"2024-09-21T02:13:25.927047+0000","last_active":"2024-09-21T02:13:26.879469+0000","last_peered":"2024-09-21T02:13:26.879469+0000","last_clean":"2024-09-21T02:13:26.879469+0000","last_became_active":"2024-09-21T02:13:25.926307+0000","last_became_peered":"2024-09-21T02:13:25.926307+0000","last_unstale":"2024-09-21T02:13:26.879469+0000","last_undegraded":"2024-09-21T02:13:26.879469+0000","last_fullsized":"2024-09-21T02:13:26.879469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879564+0000","last_change":"2024-09-21T02:13:24.896260+0000","last_active":"2024-09-21T02:13:26.879564+0000","last_peered":"2024-09-21T02:13:26.879564+0000","last_clean":"2024-09-21T02:13:26.879564+0000","last_became_active":"2024-09-21T02:13:24.895867+0000","last_became_peered":"2024-09-21T02:13:24.895867+0000","last_unstale":"2024-09-21T02:13:26.879564+0000","last_undegraded":"2024-09-21T02:13:26.879564+0000","last_fullsized":"2024-09-21T02:13:26.879564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:14.510200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967072+0000","last_change":"2024-09-21T02:13:24.899492+0000","last_active":"2024-09-21T02:13:26.967072+0000","last_peered":"2024-09-21T02:13:26.967072+0000","last_clean":"2024-09-21T02:13:26.967072+0000","last_became_active":"2024-09-21T02:13:24.899044+0000","last_became_peered":"2024-09-21T02:13:24.899044+0000","last_unstale":"2024-09-21T02:13:26.967072+0000","last_undegraded":"2024-09-21T02:13:26.967072+0000","last_fullsized":"2024-09-21T02:13:26.967072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882998+0000","last_change":"2024-09-21T02:13:25.908074+0000","last_active":"2024-09-21T02:13:26.882998+0000","last_peered":"2024-09-21T02:13:26.882998+0000","last_clean":"2024-09-21T02:13:26.882998+0000","last_became_active":"2024-09-21T02:13:25.907240+0000","last_became_peered":"2024-09-21T02:13:25.907240+0000","last_unstale":"2024-09-21T02:13:26.882998+0000","last_undegraded":"2024-09-21T02:13:26.882998+0000","last_fullsized":"2024-09-21T02:13:26.882998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.065966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965202+0000","last_change":"2024-09-21T02:13:24.898945+0000","last_active":"2024-09-21T02:13:26.965202+0000","last_peered":"2024-09-21T02:13:26.965202+0000","last_clean":"2024-09-21T02:13:26.965202+0000","last_became_active":"2024-09-21T02:13:24.897945+0000","last_became_peered":"2024-09-21T02:13:24.897945+0000","last_unstale":"2024-09-21T02:13:26.965202+0000","last_undegraded":"2024-09-21T02:13:26.965202+0000","last_fullsized":"2024-09-21T02:13:26.965202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:22.784881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883777+0000","last_change":"2024-09-21T02:13:25.919085+0000","last_active":"2024-09-21T02:13:26.883777+0000","last_peered":"2024-09-21T02:13:26.883777+0000","last_clean":"2024-09-21T02:13:26.883777+0000","last_became_active":"2024-09-21T02:13:25.918349+0000","last_became_peered":"2024-09-21T02:13:25.918349+0000","last_unstale":"2024-09-21T02:13:26.883777+0000","last_undegraded":"2024-09-21T02:13:26.883777+0000","last_fullsized":"2024-09-21T02:13:26.883777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965045+0000","last_change":"2024-09-21T02:13:24.919633+0000","last_active":"2024-09-21T02:13:26.965045+0000","last_peered":"2024-09-21T02:13:26.965045+0000","last_clean":"2024-09-21T02:13:26.965045+0000","last_became_active":"2024-09-21T02:13:24.918940+0000","last_became_peered":"2024-09-21T02:13:24.918940+0000","last_unstale":"2024-09-21T02:13:26.965045+0000","last_undegraded":"2024-09-21T02:13:26.965045+0000","last_fullsized":"2024-09-21T02:13:26.965045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880486+0000","last_change":"2024-09-21T02:13:25.901393+0000","last_active":"2024-09-21T02:13:26.880486+0000","last_peered":"2024-09-21T02:13:26.880486+0000","last_clean":"2024-09-21T02:13:26.880486+0000","last_became_active":"2024-09-21T02:13:25.901081+0000","last_became_peered":"2024-09-21T02:13:25.901081+0000","last_unstale":"2024-09-21T02:13:26.880486+0000","last_undegraded":"2024-09-21T02:13:26.880486+0000","last_fullsized":"2024-09-21T02:13:26.880486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:05.811740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879810+0000","last_change":"2024-09-21T02:13:24.901069+0000","last_active":"2024-09-21T02:13:26.879810+0000","last_peered":"2024-09-21T02:13:26.879810+0000","last_clean":"2024-09-21T02:13:26.879810+0000","last_became_active":"2024-09-21T02:13:24.900566+0000","last_became_peered":"2024-09-21T02:13:24.900566+0000","last_unstale":"2024-09-21T02:13:26.879810+0000","last_undegraded":"2024-09-21T02:13:26.879810+0000","last_fullsized":"2024-09-21T02:13:26.879810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:33.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964606+0000","last_change":"2024-09-21T02:13:25.923864+0000","last_active":"2024-09-21T02:13:26.964606+0000","last_peered":"2024-09-21T02:13:26.964606+0000","last_clean":"2024-09-21T02:13:26.964606+0000","last_became_active":"2024-09-21T02:13:25.923315+0000","last_became_peered":"2024-09-21T02:13:25.923315+0000","last_unstale":"2024-09-21T02:13:26.964606+0000","last_undegraded":"2024-09-21T02:13:26.964606+0000","last_fullsized":"2024-09-21T02:13:26.964606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965043+0000","last_change":"2024-09-21T02:13:24.918203+0000","last_active":"2024-09-21T02:13:26.965043+0000","last_peered":"2024-09-21T02:13:26.965043+0000","last_clean":"2024-09-21T02:13:26.965043+0000","last_became_active":"2024-09-21T02:13:24.907150+0000","last_became_peered":"2024-09-21T02:13:24.907150+0000","last_unstale":"2024-09-21T02:13:26.965043+0000","last_undegraded":"2024-09-21T02:13:26.965043+0000","last_fullsized":"2024-09-21T02:13:26.965043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963777+0000","last_change":"2024-09-21T02:13:25.906616+0000","last_active":"2024-09-21T02:13:26.963777+0000","last_peered":"2024-09-21T02:13:26.963777+0000","last_clean":"2024-09-21T02:13:26.963777+0000","last_became_active":"2024-09-21T02:13:25.905776+0000","last_became_peered":"2024-09-21T02:13:25.905776+0000","last_unstale":"2024-09-21T02:13:26.963777+0000","last_undegraded":"2024-09-21T02:13:26.963777+0000","last_fullsized":"2024-09-21T02:13:26.963777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:37.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879412+0000","last_change":"2024-09-21T02:13:24.906862+0000","last_active":"2024-09-21T02:13:26.879412+0000","last_peered":"2024-09-21T02:13:26.879412+0000","last_clean":"2024-09-21T02:13:26.879412+0000","last_became_active":"2024-09-21T02:13:24.905954+0000","last_became_peered":"2024-09-21T02:13:24.905954+0000","last_unstale":"2024-09-21T02:13:26.879412+0000","last_undegraded":"2024-09-21T02:13:26.879412+0000","last_fullsized":"2024-09-21T02:13:26.879412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:34.713354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880043+0000","last_change":"2024-09-21T02:13:25.919387+0000","last_active":"2024-09-21T02:13:26.880043+0000","last_peered":"2024-09-21T02:13:26.880043+0000","last_clean":"2024-09-21T02:13:26.880043+0000","last_became_active":"2024-09-21T02:13:25.919065+0000","last_became_peered":"2024-09-21T02:13:25.919065+0000","last_unstale":"2024-09-21T02:13:26.880043+0000","last_undegraded":"2024-09-21T02:13:26.880043+0000","last_fullsized":"2024-09-21T02:13:26.880043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:30.665668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963542+0000","last_change":"2024-09-21T02:13:24.906999+0000","last_active":"2024-09-21T02:13:26.963542+0000","last_peered":"2024-09-21T02:13:26.963542+0000","last_clean":"2024-09-21T02:13:26.963542+0000","last_became_active":"2024-09-21T02:13:24.897714+0000","last_became_peered":"2024-09-21T02:13:24.897714+0000","last_unstale":"2024-09-21T02:13:26.963542+0000","last_undegraded":"2024-09-21T02:13:26.963542+0000","last_fullsized":"2024-09-21T02:13:26.963542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963997+0000","last_change":"2024-09-21T02:13:25.928157+0000","last_active":"2024-09-21T02:13:26.963997+0000","last_peered":"2024-09-21T02:13:26.963997+0000","last_clean":"2024-09-21T02:13:26.963997+0000","last_became_active":"2024-09-21T02:13:25.927747+0000","last_became_peered":"2024-09-21T02:13:25.927747+0000","last_unstale":"2024-09-21T02:13:26.963997+0000","last_undegraded":"2024-09-21T02:13:26.963997+0000","last_fullsized":"2024-09-21T02:13:26.963997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:42.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878788+0000","last_change":"2024-09-21T02:13:25.902089+0000","last_active":"2024-09-21T02:13:26.878788+0000","last_peered":"2024-09-21T02:13:26.878788+0000","last_clean":"2024-09-21T02:13:26.878788+0000","last_became_active":"2024-09-21T02:13:25.901778+0000","last_became_peered":"2024-09-21T02:13:25.901778+0000","last_unstale":"2024-09-21T02:13:26.878788+0000","last_undegraded":"2024-09-21T02:13:26.878788+0000","last_fullsized":"2024-09-21T02:13:26.878788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965582+0000","last_change":"2024-09-21T02:13:24.914792+0000","last_active":"2024-09-21T02:13:26.965582+0000","last_peered":"2024-09-21T02:13:26.965582+0000","last_clean":"2024-09-21T02:13:26.965582+0000","last_became_active":"2024-09-21T02:13:24.914298+0000","last_became_peered":"2024-09-21T02:13:24.914298+0000","last_unstale":"2024-09-21T02:13:26.965582+0000","last_undegraded":"2024-09-21T02:13:26.965582+0000","last_fullsized":"2024-09-21T02:13:26.965582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883347+0000","last_change":"2024-09-21T02:13:25.924238+0000","last_active":"2024-09-21T02:13:26.883347+0000","last_peered":"2024-09-21T02:13:26.883347+0000","last_clean":"2024-09-21T02:13:26.883347+0000","last_became_active":"2024-09-21T02:13:25.922155+0000","last_became_peered":"2024-09-21T02:13:25.922155+0000","last_unstale":"2024-09-21T02:13:26.883347+0000","last_undegraded":"2024-09-21T02:13:26.883347+0000","last_fullsized":"2024-09-21T02:13:26.883347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:27.528134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965211+0000","last_change":"2024-09-21T02:13:25.924993+0000","last_active":"2024-09-21T02:13:26.965211+0000","last_peered":"2024-09-21T02:13:26.965211+0000","last_clean":"2024-09-21T02:13:26.965211+0000","last_became_active":"2024-09-21T02:13:25.924677+0000","last_became_peered":"2024-09-21T02:13:25.924677+0000","last_unstale":"2024-09-21T02:13:26.965211+0000","last_undegraded":"2024-09-21T02:13:26.965211+0000","last_fullsized":"2024-09-21T02:13:26.965211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879653+0000","last_change":"2024-09-21T02:13:24.893497+0000","last_active":"2024-09-21T02:13:26.879653+0000","last_peered":"2024-09-21T02:13:26.879653+0000","last_clean":"2024-09-21T02:13:26.879653+0000","last_became_active":"2024-09-21T02:13:24.892789+0000","last_became_peered":"2024-09-21T02:13:24.892789+0000","last_unstale":"2024-09-21T02:13:26.879653+0000","last_undegraded":"2024-09-21T02:13:26.879653+0000","last_fullsized":"2024-09-21T02:13:26.879653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965821+0000","last_change":"2024-09-21T02:13:25.927918+0000","last_active":"2024-09-21T02:13:26.965821+0000","last_peered":"2024-09-21T02:13:26.965821+0000","last_clean":"2024-09-21T02:13:26.965821+0000","last_became_active":"2024-09-21T02:13:25.927608+0000","last_became_peered":"2024-09-21T02:13:25.927608+0000","last_unstale":"2024-09-21T02:13:26.965821+0000","last_undegraded":"2024-09-21T02:13:26.965821+0000","last_fullsized":"2024-09-21T02:13:26.965821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964326+0000","last_change":"2024-09-21T02:13:24.904899+0000","last_active":"2024-09-21T02:13:26.964326+0000","last_peered":"2024-09-21T02:13:26.964326+0000","last_clean":"2024-09-21T02:13:26.964326+0000","last_became_active":"2024-09-21T02:13:24.904137+0000","last_became_peered":"2024-09-21T02:13:24.904137+0000","last_unstale":"2024-09-21T02:13:26.964326+0000","last_undegraded":"2024-09-21T02:13:26.964326+0000","last_fullsized":"2024-09-21T02:13:26.964326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:06.764948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881399+0000","last_change":"2024-09-21T02:13:25.908087+0000","last_active":"2024-09-21T02:13:26.881399+0000","last_peered":"2024-09-21T02:13:26.881399+0000","last_clean":"2024-09-21T02:13:26.881399+0000","last_became_active":"2024-09-21T02:13:25.907748+0000","last_became_peered":"2024-09-21T02:13:25.907748+0000","last_unstale":"2024-09-21T02:13:26.881399+0000","last_undegraded":"2024-09-21T02:13:26.881399+0000","last_fullsized":"2024-09-21T02:13:26.881399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:24.998552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964107+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:26.964107+0000","last_peered":"2024-09-21T02:13:26.964107+0000","last_clean":"2024-09-21T02:13:26.964107+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:26.964107+0000","last_undegraded":"2024-09-21T02:13:26.964107+0000","last_fullsized":"2024-09-21T02:13:26.964107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880025+0000","last_change":"2024-09-21T02:13:24.893632+0000","last_active":"2024-09-21T02:13:26.880025+0000","last_peered":"2024-09-21T02:13:26.880025+0000","last_clean":"2024-09-21T02:13:26.880025+0000","last_became_active":"2024-09-21T02:13:24.892990+0000","last_became_peered":"2024-09-21T02:13:24.892990+0000","last_unstale":"2024-09-21T02:13:26.880025+0000","last_undegraded":"2024-09-21T02:13:26.880025+0000","last_fullsized":"2024-09-21T02:13:26.880025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879715+0000","last_change":"2024-09-21T02:13:24.914198+0000","last_active":"2024-09-21T02:13:26.879715+0000","last_peered":"2024-09-21T02:13:26.879715+0000","last_clean":"2024-09-21T02:13:26.879715+0000","last_became_active":"2024-09-21T02:13:24.913815+0000","last_became_peered":"2024-09-21T02:13:24.913815+0000","last_unstale":"2024-09-21T02:13:26.879715+0000","last_undegraded":"2024-09-21T02:13:26.879715+0000","last_fullsized":"2024-09-21T02:13:26.879715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:35.245420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963392+0000","last_change":"2024-09-21T02:13:25.906344+0000","last_active":"2024-09-21T02:13:26.963392+0000","last_peered":"2024-09-21T02:13:26.963392+0000","last_clean":"2024-09-21T02:13:26.963392+0000","last_became_active":"2024-09-21T02:13:25.905571+0000","last_became_peered":"2024-09-21T02:13:25.905571+0000","last_unstale":"2024-09-21T02:13:26.963392+0000","last_undegraded":"2024-09-21T02:13:26.963392+0000","last_fullsized":"2024-09-21T02:13:26.963392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:01.266262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964760+0000","last_change":"2024-09-21T02:13:24.919598+0000","last_active":"2024-09-21T02:13:26.964760+0000","last_peered":"2024-09-21T02:13:26.964760+0000","last_clean":"2024-09-21T02:13:26.964760+0000","last_became_active":"2024-09-21T02:13:24.918909+0000","last_became_peered":"2024-09-21T02:13:24.918909+0000","last_unstale":"2024-09-21T02:13:26.964760+0000","last_undegraded":"2024-09-21T02:13:26.964760+0000","last_fullsized":"2024-09-21T02:13:26.964760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:19.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878944+0000","last_change":"2024-09-21T02:13:25.913216+0000","last_active":"2024-09-21T02:13:26.878944+0000","last_peered":"2024-09-21T02:13:26.878944+0000","last_clean":"2024-09-21T02:13:26.878944+0000","last_became_active":"2024-09-21T02:13:25.912412+0000","last_became_peered":"2024-09-21T02:13:25.912412+0000","last_unstale":"2024-09-21T02:13:26.878944+0000","last_undegraded":"2024-09-21T02:13:26.878944+0000","last_fullsized":"2024-09-21T02:13:26.878944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:16.249421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879536+0000","last_change":"2024-09-21T02:13:24.900311+0000","last_active":"2024-09-21T02:13:26.879536+0000","last_peered":"2024-09-21T02:13:26.879536+0000","last_clean":"2024-09-21T02:13:26.879536+0000","last_became_active":"2024-09-21T02:13:24.899845+0000","last_became_peered":"2024-09-21T02:13:24.899845+0000","last_unstale":"2024-09-21T02:13:26.879536+0000","last_undegraded":"2024-09-21T02:13:26.879536+0000","last_fullsized":"2024-09-21T02:13:26.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:34.513381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964099+0000","last_change":"2024-09-21T02:13:25.899982+0000","last_active":"2024-09-21T02:13:26.964099+0000","last_peered":"2024-09-21T02:13:26.964099+0000","last_clean":"2024-09-21T02:13:26.964099+0000","last_became_active":"2024-09-21T02:13:25.899671+0000","last_became_peered":"2024-09-21T02:13:25.899671+0000","last_unstale":"2024-09-21T02:13:26.964099+0000","last_undegraded":"2024-09-21T02:13:26.964099+0000","last_fullsized":"2024-09-21T02:13:26.964099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879852+0000","last_change":"2024-09-21T02:13:24.903776+0000","last_active":"2024-09-21T02:13:26.879852+0000","last_peered":"2024-09-21T02:13:26.879852+0000","last_clean":"2024-09-21T02:13:26.879852+0000","last_became_active":"2024-09-21T02:13:24.902681+0000","last_became_peered":"2024-09-21T02:13:24.902681+0000","last_unstale":"2024-09-21T02:13:26.879852+0000","last_undegraded":"2024-09-21T02:13:26.879852+0000","last_fullsized":"2024-09-21T02:13:26.879852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880542+0000","last_change":"2024-09-21T02:13:25.893632+0000","last_active":"2024-09-21T02:13:26.880542+0000","last_peered":"2024-09-21T02:13:26.880542+0000","last_clean":"2024-09-21T02:13:26.880542+0000","last_became_active":"2024-09-21T02:13:25.893137+0000","last_became_peered":"2024-09-21T02:13:25.893137+0000","last_unstale":"2024-09-21T02:13:26.880542+0000","last_undegraded":"2024-09-21T02:13:26.880542+0000","last_fullsized":"2024-09-21T02:13:26.880542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.630932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882169+0000","last_change":"2024-09-21T02:13:24.899031+0000","last_active":"2024-09-21T02:13:26.882169+0000","last_peered":"2024-09-21T02:13:26.882169+0000","last_clean":"2024-09-21T02:13:26.882169+0000","last_became_active":"2024-09-21T02:13:24.898526+0000","last_became_peered":"2024-09-21T02:13:24.898526+0000","last_unstale":"2024-09-21T02:13:26.882169+0000","last_undegraded":"2024-09-21T02:13:26.882169+0000","last_fullsized":"2024-09-21T02:13:26.882169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:15.323641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964112+0000","last_change":"2024-09-21T02:13:25.906716+0000","last_active":"2024-09-21T02:13:26.964112+0000","last_peered":"2024-09-21T02:13:26.964112+0000","last_clean":"2024-09-21T02:13:26.964112+0000","last_became_active":"2024-09-21T02:13:25.905889+0000","last_became_peered":"2024-09-21T02:13:25.905889+0000","last_unstale":"2024-09-21T02:13:26.964112+0000","last_undegraded":"2024-09-21T02:13:26.964112+0000","last_fullsized":"2024-09-21T02:13:26.964112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:56.985078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882102+0000","last_change":"2024-09-21T02:13:24.905877+0000","last_active":"2024-09-21T02:13:26.882102+0000","last_peered":"2024-09-21T02:13:26.882102+0000","last_clean":"2024-09-21T02:13:26.882102+0000","last_became_active":"2024-09-21T02:13:24.905486+0000","last_became_peered":"2024-09-21T02:13:24.905486+0000","last_unstale":"2024-09-21T02:13:26.882102+0000","last_undegraded":"2024-09-21T02:13:26.882102+0000","last_fullsized":"2024-09-21T02:13:26.882102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963169+0000","last_change":"2024-09-21T02:13:24.898279+0000","last_active":"2024-09-21T02:13:26.963169+0000","last_peered":"2024-09-21T02:13:26.963169+0000","last_clean":"2024-09-21T02:13:26.963169+0000","last_became_active":"2024-09-21T02:13:24.897369+0000","last_became_peered":"2024-09-21T02:13:24.897369+0000","last_unstale":"2024-09-21T02:13:26.963169+0000","last_undegraded":"2024-09-21T02:13:26.963169+0000","last_fullsized":"2024-09-21T02:13:26.963169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879396+0000","last_change":"2024-09-21T02:13:25.921611+0000","last_active":"2024-09-21T02:13:26.879396+0000","last_peered":"2024-09-21T02:13:26.879396+0000","last_clean":"2024-09-21T02:13:26.879396+0000","last_became_active":"2024-09-21T02:13:25.921165+0000","last_became_peered":"2024-09-21T02:13:25.921165+0000","last_unstale":"2024-09-21T02:13:26.879396+0000","last_undegraded":"2024-09-21T02:13:26.879396+0000","last_fullsized":"2024-09-21T02:13:26.879396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963689+0000","last_change":"2024-09-21T02:13:24.913917+0000","last_active":"2024-09-21T02:13:26.963689+0000","last_peered":"2024-09-21T02:13:26.963689+0000","last_clean":"2024-09-21T02:13:26.963689+0000","last_became_active":"2024-09-21T02:13:24.913151+0000","last_became_peered":"2024-09-21T02:13:24.913151+0000","last_unstale":"2024-09-21T02:13:26.963689+0000","last_undegraded":"2024-09-21T02:13:26.963689+0000","last_fullsized":"2024-09-21T02:13:26.963689+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:57.921560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965191+0000","last_change":"2024-09-21T02:13:25.937217+0000","last_active":"2024-09-21T02:13:26.965191+0000","last_peered":"2024-09-21T02:13:26.965191+0000","last_clean":"2024-09-21T02:13:26.965191+0000","last_became_active":"2024-09-21T02:13:25.936656+0000","last_became_peered":"2024-09-21T02:13:25.936656+0000","last_unstale":"2024-09-21T02:13:26.965191+0000","last_undegraded":"2024-09-21T02:13:26.965191+0000","last_fullsized":"2024-09-21T02:13:26.965191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:55.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879890+0000","last_change":"2024-09-21T02:13:24.893809+0000","last_active":"2024-09-21T02:13:26.879890+0000","last_peered":"2024-09-21T02:13:26.879890+0000","last_clean":"2024-09-21T02:13:26.879890+0000","last_became_active":"2024-09-21T02:13:24.893342+0000","last_became_peered":"2024-09-21T02:13:24.893342+0000","last_unstale":"2024-09-21T02:13:26.879890+0000","last_undegraded":"2024-09-21T02:13:26.879890+0000","last_fullsized":"2024-09-21T02:13:26.879890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880297+0000","last_change":"2024-09-21T02:13:25.909794+0000","last_active":"2024-09-21T02:13:26.880297+0000","last_peered":"2024-09-21T02:13:26.880297+0000","last_clean":"2024-09-21T02:13:26.880297+0000","last_became_active":"2024-09-21T02:13:25.908961+0000","last_became_peered":"2024-09-21T02:13:25.908961+0000","last_unstale":"2024-09-21T02:13:26.880297+0000","last_undegraded":"2024-09-21T02:13:26.880297+0000","last_fullsized":"2024-09-21T02:13:26.880297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:37.003277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880824+0000","last_change":"2024-09-21T02:13:25.915575+0000","last_active":"2024-09-21T02:13:26.880824+0000","last_peered":"2024-09-21T02:13:26.880824+0000","last_clean":"2024-09-21T02:13:26.880824+0000","last_became_active":"2024-09-21T02:13:25.915035+0000","last_became_peered":"2024-09-21T02:13:25.915035+0000","last_unstale":"2024-09-21T02:13:26.880824+0000","last_undegraded":"2024-09-21T02:13:26.880824+0000","last_fullsized":"2024-09-21T02:13:26.880824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:42.771535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965874+0000","last_change":"2024-09-21T02:13:24.903167+0000","last_active":"2024-09-21T02:13:26.965874+0000","last_peered":"2024-09-21T02:13:26.965874+0000","last_clean":"2024-09-21T02:13:26.965874+0000","last_became_active":"2024-09-21T02:13:24.902698+0000","last_became_peered":"2024-09-21T02:13:24.902698+0000","last_unstale":"2024-09-21T02:13:26.965874+0000","last_undegraded":"2024-09-21T02:13:26.965874+0000","last_fullsized":"2024-09-21T02:13:26.965874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964514+0000","last_change":"2024-09-21T02:13:25.925357+0000","last_active":"2024-09-21T02:13:26.964514+0000","last_peered":"2024-09-21T02:13:26.964514+0000","last_clean":"2024-09-21T02:13:26.964514+0000","last_became_active":"2024-09-21T02:13:25.924899+0000","last_became_peered":"2024-09-21T02:13:25.924899+0000","last_unstale":"2024-09-21T02:13:26.964514+0000","last_undegraded":"2024-09-21T02:13:26.964514+0000","last_fullsized":"2024-09-21T02:13:26.964514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:35.680962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964793+0000","last_change":"2024-09-21T02:13:24.898153+0000","last_active":"2024-09-21T02:13:26.964793+0000","last_peered":"2024-09-21T02:13:26.964793+0000","last_clean":"2024-09-21T02:13:26.964793+0000","last_became_active":"2024-09-21T02:13:24.897451+0000","last_became_peered":"2024-09-21T02:13:24.897451+0000","last_unstale":"2024-09-21T02:13:26.964793+0000","last_undegraded":"2024-09-21T02:13:26.964793+0000","last_fullsized":"2024-09-21T02:13:26.964793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883658+0000","last_change":"2024-09-21T02:13:25.911108+0000","last_active":"2024-09-21T02:13:26.883658+0000","last_peered":"2024-09-21T02:13:26.883658+0000","last_clean":"2024-09-21T02:13:26.883658+0000","last_became_active":"2024-09-21T02:13:25.910368+0000","last_became_peered":"2024-09-21T02:13:25.910368+0000","last_unstale":"2024-09-21T02:13:26.883658+0000","last_undegraded":"2024-09-21T02:13:26.883658+0000","last_fullsized":"2024-09-21T02:13:26.883658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883288+0000","last_change":"2024-09-21T02:13:24.905477+0000","last_active":"2024-09-21T02:13:26.883288+0000","last_peered":"2024-09-21T02:13:26.883288+0000","last_clean":"2024-09-21T02:13:26.883288+0000","last_became_active":"2024-09-21T02:13:24.904718+0000","last_became_peered":"2024-09-21T02:13:24.904718+0000","last_unstale":"2024-09-21T02:13:26.883288+0000","last_undegraded":"2024-09-21T02:13:26.883288+0000","last_fullsized":"2024-09-21T02:13:26.883288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883454+0000","last_change":"2024-09-21T02:13:25.897102+0000","last_active":"2024-09-21T02:13:26.883454+0000","last_peered":"2024-09-21T02:13:26.883454+0000","last_clean":"2024-09-21T02:13:26.883454+0000","last_became_active":"2024-09-21T02:13:25.896342+0000","last_became_peered":"2024-09-21T02:13:25.896342+0000","last_unstale":"2024-09-21T02:13:26.883454+0000","last_undegraded":"2024-09-21T02:13:26.883454+0000","last_fullsized":"2024-09-21T02:13:26.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":81753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":835584,"data_stored":672213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":237568,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":831488,"data_stored":672191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:34.645 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:34.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7cd206640 1 -- 172.21.15.170:0/130304420 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00a5e40 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 -- 172.21.15.170:0/130304420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cc5a0 msgr2=0x7fb7c80b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 --2- 172.21.15.170:0/130304420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80b44a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009840 tx=0x7fb7c001c820 comp rx=0 tx=0).stop 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 -- 172.21.15.170:0/130304420 shutdown_connections 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 --2- 172.21.15.170:0/130304420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80b44a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 --2- 172.21.15.170:0/130304420 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80cbc50 0x7fb7c80cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 -- 172.21.15.170:0/130304420 >> 172.21.15.170:0/130304420 conn(0x7fb7c809a9c0 msgr2=0x7fb7c80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 -- 172.21.15.170:0/130304420 shutdown_connections 2024-09-21T02:13:34.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.810+0000 7fb7d046c640 1 -- 172.21.15.170:0/130304420 wait complete. 2024-09-21T02:13:34.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 Processor -- start 2024-09-21T02:13:34.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 -- start start 2024-09-21T02:13:34.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80b25f0 0x7fb7c8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:34.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cbc50 0x7fb7c8155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:34.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:34.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8168f40 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8168dc0 con 0x7fb7c80cbc50 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c81690c0 con 0x7fb7c80b25f0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42266/0 (socket says 172.21.15.170:42266) 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 -- 172.21.15.170:0/3653908454 learned_addr learned my addr 172.21.15.170:0/3653908454 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cda07640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cbc50 0x7fb7c8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7ce208640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80b25f0 0x7fb7c8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 -- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80b25f0 msgr2=0x7fb7c8150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80b25f0 0x7fb7c8150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 -- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cbc50 msgr2=0x7fb7c8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cbc50 0x7fb7c8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 -- 172.21.15.170:0/3653908454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c81fc5d0 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7ce208640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80b25f0 0x7fb7c8150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7cea09640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80c9f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb7c001a450 tx=0x7fb7c00a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00bb070 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c81fc8b0 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7c0004430 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00b2040 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.814+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c81fce70 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.818+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c80cc060 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.818+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb7c0005290 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.818+0000 7fb7baffd640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb7ac073200 0x7fb7ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:34.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.818+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb7c012dbc0 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.818+0000 7fb7ce208640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb7ac073200 0x7fb7ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:34.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.818+0000 7fb7ce208640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb7ac073200 0x7fb7ac075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc001700 tx=0x7fb7bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:34.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.822+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb7c00f2290 con 0x7fb7c80cc5a0 2024-09-21T02:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:34.994+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7c80ce9d0 con 0x7fb7ac073200 2024-09-21T02:13:35.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.006+0000 7fb7baffd640 1 -- 172.21.15.170:0/3653908454 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251821 (secure 0 0 0) 0x7fb7c80ce9d0 con 0x7fb7ac073200 2024-09-21T02:13:35.011 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:35.012 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:35.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb7ac073200 msgr2=0x7fb7ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb7ac073200 0x7fb7ac075710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc001700 tx=0x7fb7bc009040 comp rx=0 tx=0).stop 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 msgr2=0x7fb7c80c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80c9f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb7c001a450 tx=0x7fb7c00a5810 comp rx=0 tx=0).stop 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 shutdown_connections 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb7ac073200 0x7fb7ac075710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c80cc5a0 0x7fb7c80c9f90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7c80cbc50 0x7fb7c8155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 --2- 172.21.15.170:0/3653908454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7c80b25f0 0x7fb7c8150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 >> 172.21.15.170:0/3653908454 conn(0x7fb7c809a9c0 msgr2=0x7fb7c80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 shutdown_connections 2024-09-21T02:13:35.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.014+0000 7fb7d046c640 1 -- 172.21.15.170:0/3653908454 wait complete. 2024-09-21T02:13:35.033 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-21T02:13:34.866394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244268,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730452,"statfs":{"total":773094113280,"available":772843982848,"internally_reserved":0,"allocated":3121152,"data_stored":1839580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991209"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964977+0000","last_change":"2024-09-21T02:13:25.918477+0000","last_active":"2024-09-21T02:13:26.964977+0000","last_peered":"2024-09-21T02:13:26.964977+0000","last_clean":"2024-09-21T02:13:26.964977+0000","last_became_active":"2024-09-21T02:13:25.917930+0000","last_became_peered":"2024-09-21T02:13:25.917930+0000","last_unstale":"2024-09-21T02:13:26.964977+0000","last_undegraded":"2024-09-21T02:13:26.964977+0000","last_fullsized":"2024-09-21T02:13:26.964977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:08.714546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883166+0000","last_change":"2024-09-21T02:13:25.893989+0000","last_active":"2024-09-21T02:13:26.883166+0000","last_peered":"2024-09-21T02:13:26.883166+0000","last_clean":"2024-09-21T02:13:26.883166+0000","last_became_active":"2024-09-21T02:13:25.893173+0000","last_became_peered":"2024-09-21T02:13:25.893173+0000","last_unstale":"2024-09-21T02:13:26.883166+0000","last_undegraded":"2024-09-21T02:13:26.883166+0000","last_fullsized":"2024-09-21T02:13:26.883166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:49.876135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879431+0000","last_change":"2024-09-21T02:13:25.932549+0000","last_active":"2024-09-21T02:13:26.879431+0000","last_peered":"2024-09-21T02:13:26.879431+0000","last_clean":"2024-09-21T02:13:26.879431+0000","last_became_active":"2024-09-21T02:13:25.931843+0000","last_became_peered":"2024-09-21T02:13:25.931843+0000","last_unstale":"2024-09-21T02:13:26.879431+0000","last_undegraded":"2024-09-21T02:13:26.879431+0000","last_fullsized":"2024-09-21T02:13:26.879431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:19.382874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879618+0000","last_change":"2024-09-21T02:13:25.941857+0000","last_active":"2024-09-21T02:13:26.879618+0000","last_peered":"2024-09-21T02:13:26.879618+0000","last_clean":"2024-09-21T02:13:26.879618+0000","last_became_active":"2024-09-21T02:13:25.941422+0000","last_became_peered":"2024-09-21T02:13:25.941422+0000","last_unstale":"2024-09-21T02:13:26.879618+0000","last_undegraded":"2024-09-21T02:13:26.879618+0000","last_fullsized":"2024-09-21T02:13:26.879618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:02.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965518+0000","last_change":"2024-09-21T02:13:25.911062+0000","last_active":"2024-09-21T02:13:26.965518+0000","last_peered":"2024-09-21T02:13:26.965518+0000","last_clean":"2024-09-21T02:13:26.965518+0000","last_became_active":"2024-09-21T02:13:25.899879+0000","last_became_peered":"2024-09-21T02:13:25.899879+0000","last_unstale":"2024-09-21T02:13:26.965518+0000","last_undegraded":"2024-09-21T02:13:26.965518+0000","last_fullsized":"2024-09-21T02:13:26.965518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:40.011102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878421+0000","last_change":"2024-09-21T02:13:25.942318+0000","last_active":"2024-09-21T02:13:26.878421+0000","last_peered":"2024-09-21T02:13:26.878421+0000","last_clean":"2024-09-21T02:13:26.878421+0000","last_became_active":"2024-09-21T02:13:25.941452+0000","last_became_peered":"2024-09-21T02:13:25.941452+0000","last_unstale":"2024-09-21T02:13:26.878421+0000","last_undegraded":"2024-09-21T02:13:26.878421+0000","last_fullsized":"2024-09-21T02:13:26.878421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:08.840148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963534+0000","last_change":"2024-09-21T02:13:25.942111+0000","last_active":"2024-09-21T02:13:26.963534+0000","last_peered":"2024-09-21T02:13:26.963534+0000","last_clean":"2024-09-21T02:13:26.963534+0000","last_became_active":"2024-09-21T02:13:25.941400+0000","last_became_peered":"2024-09-21T02:13:25.941400+0000","last_unstale":"2024-09-21T02:13:26.963534+0000","last_undegraded":"2024-09-21T02:13:26.963534+0000","last_fullsized":"2024-09-21T02:13:26.963534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878836+0000","last_change":"2024-09-21T02:13:25.914747+0000","last_active":"2024-09-21T02:13:26.878836+0000","last_peered":"2024-09-21T02:13:26.878836+0000","last_clean":"2024-09-21T02:13:26.878836+0000","last_became_active":"2024-09-21T02:13:25.914058+0000","last_became_peered":"2024-09-21T02:13:25.914058+0000","last_unstale":"2024-09-21T02:13:26.878836+0000","last_undegraded":"2024-09-21T02:13:26.878836+0000","last_fullsized":"2024-09-21T02:13:26.878836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:21.986443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967211+0000","last_change":"2024-09-21T02:13:25.909803+0000","last_active":"2024-09-21T02:13:26.967211+0000","last_peered":"2024-09-21T02:13:26.967211+0000","last_clean":"2024-09-21T02:13:26.967211+0000","last_became_active":"2024-09-21T02:13:25.909116+0000","last_became_peered":"2024-09-21T02:13:25.909116+0000","last_unstale":"2024-09-21T02:13:26.967211+0000","last_undegraded":"2024-09-21T02:13:26.967211+0000","last_fullsized":"2024-09-21T02:13:26.967211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:00.174710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965018+0000","last_change":"2024-09-21T02:13:25.915682+0000","last_active":"2024-09-21T02:13:26.965018+0000","last_peered":"2024-09-21T02:13:26.965018+0000","last_clean":"2024-09-21T02:13:26.965018+0000","last_became_active":"2024-09-21T02:13:25.915194+0000","last_became_peered":"2024-09-21T02:13:25.915194+0000","last_unstale":"2024-09-21T02:13:26.965018+0000","last_undegraded":"2024-09-21T02:13:26.965018+0000","last_fullsized":"2024-09-21T02:13:26.965018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.966309+0000","last_change":"2024-09-21T02:13:25.914960+0000","last_active":"2024-09-21T02:13:26.966309+0000","last_peered":"2024-09-21T02:13:26.966309+0000","last_clean":"2024-09-21T02:13:26.966309+0000","last_became_active":"2024-09-21T02:13:25.914610+0000","last_became_peered":"2024-09-21T02:13:25.914610+0000","last_unstale":"2024-09-21T02:13:26.966309+0000","last_undegraded":"2024-09-21T02:13:26.966309+0000","last_fullsized":"2024-09-21T02:13:26.966309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965589+0000","last_change":"2024-09-21T02:13:25.924430+0000","last_active":"2024-09-21T02:13:26.965589+0000","last_peered":"2024-09-21T02:13:26.965589+0000","last_clean":"2024-09-21T02:13:26.965589+0000","last_became_active":"2024-09-21T02:13:25.923881+0000","last_became_peered":"2024-09-21T02:13:25.923881+0000","last_unstale":"2024-09-21T02:13:26.965589+0000","last_undegraded":"2024-09-21T02:13:26.965589+0000","last_fullsized":"2024-09-21T02:13:26.965589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963608+0000","last_change":"2024-09-21T02:13:25.920280+0000","last_active":"2024-09-21T02:13:26.963608+0000","last_peered":"2024-09-21T02:13:26.963608+0000","last_clean":"2024-09-21T02:13:26.963608+0000","last_became_active":"2024-09-21T02:13:25.919389+0000","last_became_peered":"2024-09-21T02:13:25.919389+0000","last_unstale":"2024-09-21T02:13:26.963608+0000","last_undegraded":"2024-09-21T02:13:26.963608+0000","last_fullsized":"2024-09-21T02:13:26.963608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:40.761905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963494+0000","last_change":"2024-09-21T02:13:25.919658+0000","last_active":"2024-09-21T02:13:26.963494+0000","last_peered":"2024-09-21T02:13:26.963494+0000","last_clean":"2024-09-21T02:13:26.963494+0000","last_became_active":"2024-09-21T02:13:25.918063+0000","last_became_peered":"2024-09-21T02:13:25.918063+0000","last_unstale":"2024-09-21T02:13:26.963494+0000","last_undegraded":"2024-09-21T02:13:26.963494+0000","last_fullsized":"2024-09-21T02:13:26.963494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.069956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882761+0000","last_change":"2024-09-21T02:13:25.915232+0000","last_active":"2024-09-21T02:13:26.882761+0000","last_peered":"2024-09-21T02:13:26.882761+0000","last_clean":"2024-09-21T02:13:26.882761+0000","last_became_active":"2024-09-21T02:13:25.914387+0000","last_became_peered":"2024-09-21T02:13:25.914387+0000","last_unstale":"2024-09-21T02:13:26.882761+0000","last_undegraded":"2024-09-21T02:13:26.882761+0000","last_fullsized":"2024-09-21T02:13:26.882761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878878+0000","last_change":"2024-09-21T02:13:25.902781+0000","last_active":"2024-09-21T02:13:26.878878+0000","last_peered":"2024-09-21T02:13:26.878878+0000","last_clean":"2024-09-21T02:13:26.878878+0000","last_became_active":"2024-09-21T02:13:25.902445+0000","last_became_peered":"2024-09-21T02:13:25.902445+0000","last_unstale":"2024-09-21T02:13:26.878878+0000","last_undegraded":"2024-09-21T02:13:26.878878+0000","last_fullsized":"2024-09-21T02:13:26.878878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881922+0000","last_change":"2024-09-21T02:13:25.915843+0000","last_active":"2024-09-21T02:13:26.881922+0000","last_peered":"2024-09-21T02:13:26.881922+0000","last_clean":"2024-09-21T02:13:26.881922+0000","last_became_active":"2024-09-21T02:13:25.915560+0000","last_became_peered":"2024-09-21T02:13:25.915560+0000","last_unstale":"2024-09-21T02:13:26.881922+0000","last_undegraded":"2024-09-21T02:13:26.881922+0000","last_fullsized":"2024-09-21T02:13:26.881922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:41.816297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965307+0000","last_change":"2024-09-21T02:13:25.926096+0000","last_active":"2024-09-21T02:13:26.965307+0000","last_peered":"2024-09-21T02:13:26.965307+0000","last_clean":"2024-09-21T02:13:26.965307+0000","last_became_active":"2024-09-21T02:13:25.925408+0000","last_became_peered":"2024-09-21T02:13:25.925408+0000","last_unstale":"2024-09-21T02:13:26.965307+0000","last_undegraded":"2024-09-21T02:13:26.965307+0000","last_fullsized":"2024-09-21T02:13:26.965307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.244537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881981+0000","last_change":"2024-09-21T02:13:25.940050+0000","last_active":"2024-09-21T02:13:26.881981+0000","last_peered":"2024-09-21T02:13:26.881981+0000","last_clean":"2024-09-21T02:13:26.881981+0000","last_became_active":"2024-09-21T02:13:25.939688+0000","last_became_peered":"2024-09-21T02:13:25.939688+0000","last_unstale":"2024-09-21T02:13:26.881981+0000","last_undegraded":"2024-09-21T02:13:26.881981+0000","last_fullsized":"2024-09-21T02:13:26.881981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:47.887251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965669+0000","last_change":"2024-09-21T02:13:25.913323+0000","last_active":"2024-09-21T02:13:26.965669+0000","last_peered":"2024-09-21T02:13:26.965669+0000","last_clean":"2024-09-21T02:13:26.965669+0000","last_became_active":"2024-09-21T02:13:25.912945+0000","last_became_peered":"2024-09-21T02:13:25.912945+0000","last_unstale":"2024-09-21T02:13:26.965669+0000","last_undegraded":"2024-09-21T02:13:26.965669+0000","last_fullsized":"2024-09-21T02:13:26.965669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964449+0000","last_change":"2024-09-21T02:13:25.926590+0000","last_active":"2024-09-21T02:13:26.964449+0000","last_peered":"2024-09-21T02:13:26.964449+0000","last_clean":"2024-09-21T02:13:26.964449+0000","last_became_active":"2024-09-21T02:13:25.926154+0000","last_became_peered":"2024-09-21T02:13:25.926154+0000","last_unstale":"2024-09-21T02:13:26.964449+0000","last_undegraded":"2024-09-21T02:13:26.964449+0000","last_fullsized":"2024-09-21T02:13:26.964449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:00.641354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882548+0000","last_change":"2024-09-21T02:13:25.896024+0000","last_active":"2024-09-21T02:13:26.882548+0000","last_peered":"2024-09-21T02:13:26.882548+0000","last_clean":"2024-09-21T02:13:26.882548+0000","last_became_active":"2024-09-21T02:13:25.895216+0000","last_became_peered":"2024-09-21T02:13:25.895216+0000","last_unstale":"2024-09-21T02:13:26.882548+0000","last_undegraded":"2024-09-21T02:13:26.882548+0000","last_fullsized":"2024-09-21T02:13:26.882548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879525+0000","last_change":"2024-09-21T02:13:25.928846+0000","last_active":"2024-09-21T02:13:26.879525+0000","last_peered":"2024-09-21T02:13:26.879525+0000","last_clean":"2024-09-21T02:13:26.879525+0000","last_became_active":"2024-09-21T02:13:25.928527+0000","last_became_peered":"2024-09-21T02:13:25.928527+0000","last_unstale":"2024-09-21T02:13:26.879525+0000","last_undegraded":"2024-09-21T02:13:26.879525+0000","last_fullsized":"2024-09-21T02:13:26.879525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:38.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883432+0000","last_change":"2024-09-21T02:13:25.911388+0000","last_active":"2024-09-21T02:13:26.883432+0000","last_peered":"2024-09-21T02:13:26.883432+0000","last_clean":"2024-09-21T02:13:26.883432+0000","last_became_active":"2024-09-21T02:13:25.910631+0000","last_became_peered":"2024-09-21T02:13:25.910631+0000","last_unstale":"2024-09-21T02:13:26.883432+0000","last_undegraded":"2024-09-21T02:13:26.883432+0000","last_fullsized":"2024-09-21T02:13:26.883432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:01.276714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882752+0000","last_change":"2024-09-21T02:13:25.908215+0000","last_active":"2024-09-21T02:13:26.882752+0000","last_peered":"2024-09-21T02:13:26.882752+0000","last_clean":"2024-09-21T02:13:26.882752+0000","last_became_active":"2024-09-21T02:13:25.907764+0000","last_became_peered":"2024-09-21T02:13:25.907764+0000","last_unstale":"2024-09-21T02:13:26.882752+0000","last_undegraded":"2024-09-21T02:13:26.882752+0000","last_fullsized":"2024-09-21T02:13:26.882752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:31.448658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.918292+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.917267+0000","last_became_peered":"2024-09-21T02:13:25.917267+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:22.501288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963998+0000","last_change":"2024-09-21T02:13:25.918530+0000","last_active":"2024-09-21T02:13:26.963998+0000","last_peered":"2024-09-21T02:13:26.963998+0000","last_clean":"2024-09-21T02:13:26.963998+0000","last_became_active":"2024-09-21T02:13:25.917735+0000","last_became_peered":"2024-09-21T02:13:25.917735+0000","last_unstale":"2024-09-21T02:13:26.963998+0000","last_undegraded":"2024-09-21T02:13:26.963998+0000","last_fullsized":"2024-09-21T02:13:26.963998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:03.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880008+0000","last_change":"2024-09-21T02:13:25.913497+0000","last_active":"2024-09-21T02:13:26.880008+0000","last_peered":"2024-09-21T02:13:26.880008+0000","last_clean":"2024-09-21T02:13:26.880008+0000","last_became_active":"2024-09-21T02:13:25.912567+0000","last_became_peered":"2024-09-21T02:13:25.912567+0000","last_unstale":"2024-09-21T02:13:26.880008+0000","last_undegraded":"2024-09-21T02:13:26.880008+0000","last_fullsized":"2024-09-21T02:13:26.880008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879152+0000","last_change":"2024-09-21T02:13:25.917618+0000","last_active":"2024-09-21T02:13:26.879152+0000","last_peered":"2024-09-21T02:13:26.879152+0000","last_clean":"2024-09-21T02:13:26.879152+0000","last_became_active":"2024-09-21T02:13:25.916325+0000","last_became_peered":"2024-09-21T02:13:25.916325+0000","last_unstale":"2024-09-21T02:13:26.879152+0000","last_undegraded":"2024-09-21T02:13:26.879152+0000","last_fullsized":"2024-09-21T02:13:26.879152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.915861+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.912257+0000","last_became_peered":"2024-09-21T02:13:25.912257+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:56.675098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878068+0000","last_change":"2024-09-21T02:13:25.894915+0000","last_active":"2024-09-21T02:13:26.878068+0000","last_peered":"2024-09-21T02:13:26.878068+0000","last_clean":"2024-09-21T02:13:26.878068+0000","last_became_active":"2024-09-21T02:13:25.894440+0000","last_became_peered":"2024-09-21T02:13:25.894440+0000","last_unstale":"2024-09-21T02:13:26.878068+0000","last_undegraded":"2024-09-21T02:13:26.878068+0000","last_fullsized":"2024-09-21T02:13:26.878068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.857628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964697+0000","last_change":"2024-09-21T02:13:25.930047+0000","last_active":"2024-09-21T02:13:26.964697+0000","last_peered":"2024-09-21T02:13:26.964697+0000","last_clean":"2024-09-21T02:13:26.964697+0000","last_became_active":"2024-09-21T02:13:25.929729+0000","last_became_peered":"2024-09-21T02:13:25.929729+0000","last_unstale":"2024-09-21T02:13:26.964697+0000","last_undegraded":"2024-09-21T02:13:26.964697+0000","last_fullsized":"2024-09-21T02:13:26.964697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:26.319373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880194+0000","last_change":"2024-09-21T02:13:24.903640+0000","last_active":"2024-09-21T02:13:26.880194+0000","last_peered":"2024-09-21T02:13:26.880194+0000","last_clean":"2024-09-21T02:13:26.880194+0000","last_became_active":"2024-09-21T02:13:24.899880+0000","last_became_peered":"2024-09-21T02:13:24.899880+0000","last_unstale":"2024-09-21T02:13:26.880194+0000","last_undegraded":"2024-09-21T02:13:26.880194+0000","last_fullsized":"2024-09-21T02:13:26.880194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879277+0000","last_change":"2024-09-21T02:13:24.906393+0000","last_active":"2024-09-21T02:13:26.879277+0000","last_peered":"2024-09-21T02:13:26.879277+0000","last_clean":"2024-09-21T02:13:26.879277+0000","last_became_active":"2024-09-21T02:13:24.905410+0000","last_became_peered":"2024-09-21T02:13:24.905410+0000","last_unstale":"2024-09-21T02:13:26.879277+0000","last_undegraded":"2024-09-21T02:13:26.879277+0000","last_fullsized":"2024-09-21T02:13:26.879277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:13.262457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.962957+0000","last_change":"2024-09-21T02:13:24.909821+0000","last_active":"2024-09-21T02:13:26.962957+0000","last_peered":"2024-09-21T02:13:26.962957+0000","last_clean":"2024-09-21T02:13:26.962957+0000","last_became_active":"2024-09-21T02:13:24.907797+0000","last_became_peered":"2024-09-21T02:13:24.907797+0000","last_unstale":"2024-09-21T02:13:26.962957+0000","last_undegraded":"2024-09-21T02:13:26.962957+0000","last_fullsized":"2024-09-21T02:13:26.962957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879469+0000","last_change":"2024-09-21T02:13:25.927047+0000","last_active":"2024-09-21T02:13:26.879469+0000","last_peered":"2024-09-21T02:13:26.879469+0000","last_clean":"2024-09-21T02:13:26.879469+0000","last_became_active":"2024-09-21T02:13:25.926307+0000","last_became_peered":"2024-09-21T02:13:25.926307+0000","last_unstale":"2024-09-21T02:13:26.879469+0000","last_undegraded":"2024-09-21T02:13:26.879469+0000","last_fullsized":"2024-09-21T02:13:26.879469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879564+0000","last_change":"2024-09-21T02:13:24.896260+0000","last_active":"2024-09-21T02:13:26.879564+0000","last_peered":"2024-09-21T02:13:26.879564+0000","last_clean":"2024-09-21T02:13:26.879564+0000","last_became_active":"2024-09-21T02:13:24.895867+0000","last_became_peered":"2024-09-21T02:13:24.895867+0000","last_unstale":"2024-09-21T02:13:26.879564+0000","last_undegraded":"2024-09-21T02:13:26.879564+0000","last_fullsized":"2024-09-21T02:13:26.879564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:14.510200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967072+0000","last_change":"2024-09-21T02:13:24.899492+0000","last_active":"2024-09-21T02:13:26.967072+0000","last_peered":"2024-09-21T02:13:26.967072+0000","last_clean":"2024-09-21T02:13:26.967072+0000","last_became_active":"2024-09-21T02:13:24.899044+0000","last_became_peered":"2024-09-21T02:13:24.899044+0000","last_unstale":"2024-09-21T02:13:26.967072+0000","last_undegraded":"2024-09-21T02:13:26.967072+0000","last_fullsized":"2024-09-21T02:13:26.967072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882998+0000","last_change":"2024-09-21T02:13:25.908074+0000","last_active":"2024-09-21T02:13:26.882998+0000","last_peered":"2024-09-21T02:13:26.882998+0000","last_clean":"2024-09-21T02:13:26.882998+0000","last_became_active":"2024-09-21T02:13:25.907240+0000","last_became_peered":"2024-09-21T02:13:25.907240+0000","last_unstale":"2024-09-21T02:13:26.882998+0000","last_undegraded":"2024-09-21T02:13:26.882998+0000","last_fullsized":"2024-09-21T02:13:26.882998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.065966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965202+0000","last_change":"2024-09-21T02:13:24.898945+0000","last_active":"2024-09-21T02:13:26.965202+0000","last_peered":"2024-09-21T02:13:26.965202+0000","last_clean":"2024-09-21T02:13:26.965202+0000","last_became_active":"2024-09-21T02:13:24.897945+0000","last_became_peered":"2024-09-21T02:13:24.897945+0000","last_unstale":"2024-09-21T02:13:26.965202+0000","last_undegraded":"2024-09-21T02:13:26.965202+0000","last_fullsized":"2024-09-21T02:13:26.965202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:22.784881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883777+0000","last_change":"2024-09-21T02:13:25.919085+0000","last_active":"2024-09-21T02:13:26.883777+0000","last_peered":"2024-09-21T02:13:26.883777+0000","last_clean":"2024-09-21T02:13:26.883777+0000","last_became_active":"2024-09-21T02:13:25.918349+0000","last_became_peered":"2024-09-21T02:13:25.918349+0000","last_unstale":"2024-09-21T02:13:26.883777+0000","last_undegraded":"2024-09-21T02:13:26.883777+0000","last_fullsized":"2024-09-21T02:13:26.883777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965045+0000","last_change":"2024-09-21T02:13:24.919633+0000","last_active":"2024-09-21T02:13:26.965045+0000","last_peered":"2024-09-21T02:13:26.965045+0000","last_clean":"2024-09-21T02:13:26.965045+0000","last_became_active":"2024-09-21T02:13:24.918940+0000","last_became_peered":"2024-09-21T02:13:24.918940+0000","last_unstale":"2024-09-21T02:13:26.965045+0000","last_undegraded":"2024-09-21T02:13:26.965045+0000","last_fullsized":"2024-09-21T02:13:26.965045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880486+0000","last_change":"2024-09-21T02:13:25.901393+0000","last_active":"2024-09-21T02:13:26.880486+0000","last_peered":"2024-09-21T02:13:26.880486+0000","last_clean":"2024-09-21T02:13:26.880486+0000","last_became_active":"2024-09-21T02:13:25.901081+0000","last_became_peered":"2024-09-21T02:13:25.901081+0000","last_unstale":"2024-09-21T02:13:26.880486+0000","last_undegraded":"2024-09-21T02:13:26.880486+0000","last_fullsized":"2024-09-21T02:13:26.880486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:05.811740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879810+0000","last_change":"2024-09-21T02:13:24.901069+0000","last_active":"2024-09-21T02:13:26.879810+0000","last_peered":"2024-09-21T02:13:26.879810+0000","last_clean":"2024-09-21T02:13:26.879810+0000","last_became_active":"2024-09-21T02:13:24.900566+0000","last_became_peered":"2024-09-21T02:13:24.900566+0000","last_unstale":"2024-09-21T02:13:26.879810+0000","last_undegraded":"2024-09-21T02:13:26.879810+0000","last_fullsized":"2024-09-21T02:13:26.879810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:33.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964606+0000","last_change":"2024-09-21T02:13:25.923864+0000","last_active":"2024-09-21T02:13:26.964606+0000","last_peered":"2024-09-21T02:13:26.964606+0000","last_clean":"2024-09-21T02:13:26.964606+0000","last_became_active":"2024-09-21T02:13:25.923315+0000","last_became_peered":"2024-09-21T02:13:25.923315+0000","last_unstale":"2024-09-21T02:13:26.964606+0000","last_undegraded":"2024-09-21T02:13:26.964606+0000","last_fullsized":"2024-09-21T02:13:26.964606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965043+0000","last_change":"2024-09-21T02:13:24.918203+0000","last_active":"2024-09-21T02:13:26.965043+0000","last_peered":"2024-09-21T02:13:26.965043+0000","last_clean":"2024-09-21T02:13:26.965043+0000","last_became_active":"2024-09-21T02:13:24.907150+0000","last_became_peered":"2024-09-21T02:13:24.907150+0000","last_unstale":"2024-09-21T02:13:26.965043+0000","last_undegraded":"2024-09-21T02:13:26.965043+0000","last_fullsized":"2024-09-21T02:13:26.965043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963777+0000","last_change":"2024-09-21T02:13:25.906616+0000","last_active":"2024-09-21T02:13:26.963777+0000","last_peered":"2024-09-21T02:13:26.963777+0000","last_clean":"2024-09-21T02:13:26.963777+0000","last_became_active":"2024-09-21T02:13:25.905776+0000","last_became_peered":"2024-09-21T02:13:25.905776+0000","last_unstale":"2024-09-21T02:13:26.963777+0000","last_undegraded":"2024-09-21T02:13:26.963777+0000","last_fullsized":"2024-09-21T02:13:26.963777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:37.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879412+0000","last_change":"2024-09-21T02:13:24.906862+0000","last_active":"2024-09-21T02:13:26.879412+0000","last_peered":"2024-09-21T02:13:26.879412+0000","last_clean":"2024-09-21T02:13:26.879412+0000","last_became_active":"2024-09-21T02:13:24.905954+0000","last_became_peered":"2024-09-21T02:13:24.905954+0000","last_unstale":"2024-09-21T02:13:26.879412+0000","last_undegraded":"2024-09-21T02:13:26.879412+0000","last_fullsized":"2024-09-21T02:13:26.879412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:34.713354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880043+0000","last_change":"2024-09-21T02:13:25.919387+0000","last_active":"2024-09-21T02:13:26.880043+0000","last_peered":"2024-09-21T02:13:26.880043+0000","last_clean":"2024-09-21T02:13:26.880043+0000","last_became_active":"2024-09-21T02:13:25.919065+0000","last_became_peered":"2024-09-21T02:13:25.919065+0000","last_unstale":"2024-09-21T02:13:26.880043+0000","last_undegraded":"2024-09-21T02:13:26.880043+0000","last_fullsized":"2024-09-21T02:13:26.880043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:30.665668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963542+0000","last_change":"2024-09-21T02:13:24.906999+0000","last_active":"2024-09-21T02:13:26.963542+0000","last_peered":"2024-09-21T02:13:26.963542+0000","last_clean":"2024-09-21T02:13:26.963542+0000","last_became_active":"2024-09-21T02:13:24.897714+0000","last_became_peered":"2024-09-21T02:13:24.897714+0000","last_unstale":"2024-09-21T02:13:26.963542+0000","last_undegraded":"2024-09-21T02:13:26.963542+0000","last_fullsized":"2024-09-21T02:13:26.963542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963997+0000","last_change":"2024-09-21T02:13:25.928157+0000","last_active":"2024-09-21T02:13:26.963997+0000","last_peered":"2024-09-21T02:13:26.963997+0000","last_clean":"2024-09-21T02:13:26.963997+0000","last_became_active":"2024-09-21T02:13:25.927747+0000","last_became_peered":"2024-09-21T02:13:25.927747+0000","last_unstale":"2024-09-21T02:13:26.963997+0000","last_undegraded":"2024-09-21T02:13:26.963997+0000","last_fullsized":"2024-09-21T02:13:26.963997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:42.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878788+0000","last_change":"2024-09-21T02:13:25.902089+0000","last_active":"2024-09-21T02:13:26.878788+0000","last_peered":"2024-09-21T02:13:26.878788+0000","last_clean":"2024-09-21T02:13:26.878788+0000","last_became_active":"2024-09-21T02:13:25.901778+0000","last_became_peered":"2024-09-21T02:13:25.901778+0000","last_unstale":"2024-09-21T02:13:26.878788+0000","last_undegraded":"2024-09-21T02:13:26.878788+0000","last_fullsized":"2024-09-21T02:13:26.878788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965582+0000","last_change":"2024-09-21T02:13:24.914792+0000","last_active":"2024-09-21T02:13:26.965582+0000","last_peered":"2024-09-21T02:13:26.965582+0000","last_clean":"2024-09-21T02:13:26.965582+0000","last_became_active":"2024-09-21T02:13:24.914298+0000","last_became_peered":"2024-09-21T02:13:24.914298+0000","last_unstale":"2024-09-21T02:13:26.965582+0000","last_undegraded":"2024-09-21T02:13:26.965582+0000","last_fullsized":"2024-09-21T02:13:26.965582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883347+0000","last_change":"2024-09-21T02:13:25.924238+0000","last_active":"2024-09-21T02:13:26.883347+0000","last_peered":"2024-09-21T02:13:26.883347+0000","last_clean":"2024-09-21T02:13:26.883347+0000","last_became_active":"2024-09-21T02:13:25.922155+0000","last_became_peered":"2024-09-21T02:13:25.922155+0000","last_unstale":"2024-09-21T02:13:26.883347+0000","last_undegraded":"2024-09-21T02:13:26.883347+0000","last_fullsized":"2024-09-21T02:13:26.883347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:27.528134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965211+0000","last_change":"2024-09-21T02:13:25.924993+0000","last_active":"2024-09-21T02:13:26.965211+0000","last_peered":"2024-09-21T02:13:26.965211+0000","last_clean":"2024-09-21T02:13:26.965211+0000","last_became_active":"2024-09-21T02:13:25.924677+0000","last_became_peered":"2024-09-21T02:13:25.924677+0000","last_unstale":"2024-09-21T02:13:26.965211+0000","last_undegraded":"2024-09-21T02:13:26.965211+0000","last_fullsized":"2024-09-21T02:13:26.965211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879653+0000","last_change":"2024-09-21T02:13:24.893497+0000","last_active":"2024-09-21T02:13:26.879653+0000","last_peered":"2024-09-21T02:13:26.879653+0000","last_clean":"2024-09-21T02:13:26.879653+0000","last_became_active":"2024-09-21T02:13:24.892789+0000","last_became_peered":"2024-09-21T02:13:24.892789+0000","last_unstale":"2024-09-21T02:13:26.879653+0000","last_undegraded":"2024-09-21T02:13:26.879653+0000","last_fullsized":"2024-09-21T02:13:26.879653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965821+0000","last_change":"2024-09-21T02:13:25.927918+0000","last_active":"2024-09-21T02:13:26.965821+0000","last_peered":"2024-09-21T02:13:26.965821+0000","last_clean":"2024-09-21T02:13:26.965821+0000","last_became_active":"2024-09-21T02:13:25.927608+0000","last_became_peered":"2024-09-21T02:13:25.927608+0000","last_unstale":"2024-09-21T02:13:26.965821+0000","last_undegraded":"2024-09-21T02:13:26.965821+0000","last_fullsized":"2024-09-21T02:13:26.965821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964326+0000","last_change":"2024-09-21T02:13:24.904899+0000","last_active":"2024-09-21T02:13:26.964326+0000","last_peered":"2024-09-21T02:13:26.964326+0000","last_clean":"2024-09-21T02:13:26.964326+0000","last_became_active":"2024-09-21T02:13:24.904137+0000","last_became_peered":"2024-09-21T02:13:24.904137+0000","last_unstale":"2024-09-21T02:13:26.964326+0000","last_undegraded":"2024-09-21T02:13:26.964326+0000","last_fullsized":"2024-09-21T02:13:26.964326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:06.764948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881399+0000","last_change":"2024-09-21T02:13:25.908087+0000","last_active":"2024-09-21T02:13:26.881399+0000","last_peered":"2024-09-21T02:13:26.881399+0000","last_clean":"2024-09-21T02:13:26.881399+0000","last_became_active":"2024-09-21T02:13:25.907748+0000","last_became_peered":"2024-09-21T02:13:25.907748+0000","last_unstale":"2024-09-21T02:13:26.881399+0000","last_undegraded":"2024-09-21T02:13:26.881399+0000","last_fullsized":"2024-09-21T02:13:26.881399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:24.998552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964107+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:26.964107+0000","last_peered":"2024-09-21T02:13:26.964107+0000","last_clean":"2024-09-21T02:13:26.964107+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:26.964107+0000","last_undegraded":"2024-09-21T02:13:26.964107+0000","last_fullsized":"2024-09-21T02:13:26.964107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880025+0000","last_change":"2024-09-21T02:13:24.893632+0000","last_active":"2024-09-21T02:13:26.880025+0000","last_peered":"2024-09-21T02:13:26.880025+0000","last_clean":"2024-09-21T02:13:26.880025+0000","last_became_active":"2024-09-21T02:13:24.892990+0000","last_became_peered":"2024-09-21T02:13:24.892990+0000","last_unstale":"2024-09-21T02:13:26.880025+0000","last_undegraded":"2024-09-21T02:13:26.880025+0000","last_fullsized":"2024-09-21T02:13:26.880025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879715+0000","last_change":"2024-09-21T02:13:24.914198+0000","last_active":"2024-09-21T02:13:26.879715+0000","last_peered":"2024-09-21T02:13:26.879715+0000","last_clean":"2024-09-21T02:13:26.879715+0000","last_became_active":"2024-09-21T02:13:24.913815+0000","last_became_peered":"2024-09-21T02:13:24.913815+0000","last_unstale":"2024-09-21T02:13:26.879715+0000","last_undegraded":"2024-09-21T02:13:26.879715+0000","last_fullsized":"2024-09-21T02:13:26.879715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:35.245420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963392+0000","last_change":"2024-09-21T02:13:25.906344+0000","last_active":"2024-09-21T02:13:26.963392+0000","last_peered":"2024-09-21T02:13:26.963392+0000","last_clean":"2024-09-21T02:13:26.963392+0000","last_became_active":"2024-09-21T02:13:25.905571+0000","last_became_peered":"2024-09-21T02:13:25.905571+0000","last_unstale":"2024-09-21T02:13:26.963392+0000","last_undegraded":"2024-09-21T02:13:26.963392+0000","last_fullsized":"2024-09-21T02:13:26.963392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:01.266262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964760+0000","last_change":"2024-09-21T02:13:24.919598+0000","last_active":"2024-09-21T02:13:26.964760+0000","last_peered":"2024-09-21T02:13:26.964760+0000","last_clean":"2024-09-21T02:13:26.964760+0000","last_became_active":"2024-09-21T02:13:24.918909+0000","last_became_peered":"2024-09-21T02:13:24.918909+0000","last_unstale":"2024-09-21T02:13:26.964760+0000","last_undegraded":"2024-09-21T02:13:26.964760+0000","last_fullsized":"2024-09-21T02:13:26.964760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:19.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878944+0000","last_change":"2024-09-21T02:13:25.913216+0000","last_active":"2024-09-21T02:13:26.878944+0000","last_peered":"2024-09-21T02:13:26.878944+0000","last_clean":"2024-09-21T02:13:26.878944+0000","last_became_active":"2024-09-21T02:13:25.912412+0000","last_became_peered":"2024-09-21T02:13:25.912412+0000","last_unstale":"2024-09-21T02:13:26.878944+0000","last_undegraded":"2024-09-21T02:13:26.878944+0000","last_fullsized":"2024-09-21T02:13:26.878944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:16.249421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879536+0000","last_change":"2024-09-21T02:13:24.900311+0000","last_active":"2024-09-21T02:13:26.879536+0000","last_peered":"2024-09-21T02:13:26.879536+0000","last_clean":"2024-09-21T02:13:26.879536+0000","last_became_active":"2024-09-21T02:13:24.899845+0000","last_became_peered":"2024-09-21T02:13:24.899845+0000","last_unstale":"2024-09-21T02:13:26.879536+0000","last_undegraded":"2024-09-21T02:13:26.879536+0000","last_fullsized":"2024-09-21T02:13:26.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:34.513381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964099+0000","last_change":"2024-09-21T02:13:25.899982+0000","last_active":"2024-09-21T02:13:26.964099+0000","last_peered":"2024-09-21T02:13:26.964099+0000","last_clean":"2024-09-21T02:13:26.964099+0000","last_became_active":"2024-09-21T02:13:25.899671+0000","last_became_peered":"2024-09-21T02:13:25.899671+0000","last_unstale":"2024-09-21T02:13:26.964099+0000","last_undegraded":"2024-09-21T02:13:26.964099+0000","last_fullsized":"2024-09-21T02:13:26.964099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879852+0000","last_change":"2024-09-21T02:13:24.903776+0000","last_active":"2024-09-21T02:13:26.879852+0000","last_peered":"2024-09-21T02:13:26.879852+0000","last_clean":"2024-09-21T02:13:26.879852+0000","last_became_active":"2024-09-21T02:13:24.902681+0000","last_became_peered":"2024-09-21T02:13:24.902681+0000","last_unstale":"2024-09-21T02:13:26.879852+0000","last_undegraded":"2024-09-21T02:13:26.879852+0000","last_fullsized":"2024-09-21T02:13:26.879852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880542+0000","last_change":"2024-09-21T02:13:25.893632+0000","last_active":"2024-09-21T02:13:26.880542+0000","last_peered":"2024-09-21T02:13:26.880542+0000","last_clean":"2024-09-21T02:13:26.880542+0000","last_became_active":"2024-09-21T02:13:25.893137+0000","last_became_peered":"2024-09-21T02:13:25.893137+0000","last_unstale":"2024-09-21T02:13:26.880542+0000","last_undegraded":"2024-09-21T02:13:26.880542+0000","last_fullsized":"2024-09-21T02:13:26.880542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.630932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882169+0000","last_change":"2024-09-21T02:13:24.899031+0000","last_active":"2024-09-21T02:13:26.882169+0000","last_peered":"2024-09-21T02:13:26.882169+0000","last_clean":"2024-09-21T02:13:26.882169+0000","last_became_active":"2024-09-21T02:13:24.898526+0000","last_became_peered":"2024-09-21T02:13:24.898526+0000","last_unstale":"2024-09-21T02:13:26.882169+0000","last_undegraded":"2024-09-21T02:13:26.882169+0000","last_fullsized":"2024-09-21T02:13:26.882169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:15.323641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964112+0000","last_change":"2024-09-21T02:13:25.906716+0000","last_active":"2024-09-21T02:13:26.964112+0000","last_peered":"2024-09-21T02:13:26.964112+0000","last_clean":"2024-09-21T02:13:26.964112+0000","last_became_active":"2024-09-21T02:13:25.905889+0000","last_became_peered":"2024-09-21T02:13:25.905889+0000","last_unstale":"2024-09-21T02:13:26.964112+0000","last_undegraded":"2024-09-21T02:13:26.964112+0000","last_fullsized":"2024-09-21T02:13:26.964112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:56.985078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882102+0000","last_change":"2024-09-21T02:13:24.905877+0000","last_active":"2024-09-21T02:13:26.882102+0000","last_peered":"2024-09-21T02:13:26.882102+0000","last_clean":"2024-09-21T02:13:26.882102+0000","last_became_active":"2024-09-21T02:13:24.905486+0000","last_became_peered":"2024-09-21T02:13:24.905486+0000","last_unstale":"2024-09-21T02:13:26.882102+0000","last_undegraded":"2024-09-21T02:13:26.882102+0000","last_fullsized":"2024-09-21T02:13:26.882102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963169+0000","last_change":"2024-09-21T02:13:24.898279+0000","last_active":"2024-09-21T02:13:26.963169+0000","last_peered":"2024-09-21T02:13:26.963169+0000","last_clean":"2024-09-21T02:13:26.963169+0000","last_became_active":"2024-09-21T02:13:24.897369+0000","last_became_peered":"2024-09-21T02:13:24.897369+0000","last_unstale":"2024-09-21T02:13:26.963169+0000","last_undegraded":"2024-09-21T02:13:26.963169+0000","last_fullsized":"2024-09-21T02:13:26.963169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879396+0000","last_change":"2024-09-21T02:13:25.921611+0000","last_active":"2024-09-21T02:13:26.879396+0000","last_peered":"2024-09-21T02:13:26.879396+0000","last_clean":"2024-09-21T02:13:26.879396+0000","last_became_active":"2024-09-21T02:13:25.921165+0000","last_became_peered":"2024-09-21T02:13:25.921165+0000","last_unstale":"2024-09-21T02:13:26.879396+0000","last_undegraded":"2024-09-21T02:13:26.879396+0000","last_fullsized":"2024-09-21T02:13:26.879396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963689+0000","last_change":"2024-09-21T02:13:24.913917+0000","last_active":"2024-09-21T02:13:26.963689+0000","last_peered":"2024-09-21T02:13:26.963689+0000","last_clean":"2024-09-21T02:13:26.963689+0000","last_became_active":"2024-09-21T02:13:24.913151+0000","last_became_peered":"2024-09-21T02:13:24.913151+0000","last_unstale":"2024-09-21T02:13:26.963689+0000","last_undegraded":"2024-09-21T02:13:26.963689+0000","last_fullsized":"2024-09-21T02:13:26.963689+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:57.921560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965191+0000","last_change":"2024-09-21T02:13:25.937217+0000","last_active":"2024-09-21T02:13:26.965191+0000","last_peered":"2024-09-21T02:13:26.965191+0000","last_clean":"2024-09-21T02:13:26.965191+0000","last_became_active":"2024-09-21T02:13:25.936656+0000","last_became_peered":"2024-09-21T02:13:25.936656+0000","last_unstale":"2024-09-21T02:13:26.965191+0000","last_undegraded":"2024-09-21T02:13:26.965191+0000","last_fullsized":"2024-09-21T02:13:26.965191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:55.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879890+0000","last_change":"2024-09-21T02:13:24.893809+0000","last_active":"2024-09-21T02:13:26.879890+0000","last_peered":"2024-09-21T02:13:26.879890+0000","last_clean":"2024-09-21T02:13:26.879890+0000","last_became_active":"2024-09-21T02:13:24.893342+0000","last_became_peered":"2024-09-21T02:13:24.893342+0000","last_unstale":"2024-09-21T02:13:26.879890+0000","last_undegraded":"2024-09-21T02:13:26.879890+0000","last_fullsized":"2024-09-21T02:13:26.879890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880297+0000","last_change":"2024-09-21T02:13:25.909794+0000","last_active":"2024-09-21T02:13:26.880297+0000","last_peered":"2024-09-21T02:13:26.880297+0000","last_clean":"2024-09-21T02:13:26.880297+0000","last_became_active":"2024-09-21T02:13:25.908961+0000","last_became_peered":"2024-09-21T02:13:25.908961+0000","last_unstale":"2024-09-21T02:13:26.880297+0000","last_undegraded":"2024-09-21T02:13:26.880297+0000","last_fullsized":"2024-09-21T02:13:26.880297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:37.003277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880824+0000","last_change":"2024-09-21T02:13:25.915575+0000","last_active":"2024-09-21T02:13:26.880824+0000","last_peered":"2024-09-21T02:13:26.880824+0000","last_clean":"2024-09-21T02:13:26.880824+0000","last_became_active":"2024-09-21T02:13:25.915035+0000","last_became_peered":"2024-09-21T02:13:25.915035+0000","last_unstale":"2024-09-21T02:13:26.880824+0000","last_undegraded":"2024-09-21T02:13:26.880824+0000","last_fullsized":"2024-09-21T02:13:26.880824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:42.771535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965874+0000","last_change":"2024-09-21T02:13:24.903167+0000","last_active":"2024-09-21T02:13:26.965874+0000","last_peered":"2024-09-21T02:13:26.965874+0000","last_clean":"2024-09-21T02:13:26.965874+0000","last_became_active":"2024-09-21T02:13:24.902698+0000","last_became_peered":"2024-09-21T02:13:24.902698+0000","last_unstale":"2024-09-21T02:13:26.965874+0000","last_undegraded":"2024-09-21T02:13:26.965874+0000","last_fullsized":"2024-09-21T02:13:26.965874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964514+0000","last_change":"2024-09-21T02:13:25.925357+0000","last_active":"2024-09-21T02:13:26.964514+0000","last_peered":"2024-09-21T02:13:26.964514+0000","last_clean":"2024-09-21T02:13:26.964514+0000","last_became_active":"2024-09-21T02:13:25.924899+0000","last_became_peered":"2024-09-21T02:13:25.924899+0000","last_unstale":"2024-09-21T02:13:26.964514+0000","last_undegraded":"2024-09-21T02:13:26.964514+0000","last_fullsized":"2024-09-21T02:13:26.964514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:35.680962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964793+0000","last_change":"2024-09-21T02:13:24.898153+0000","last_active":"2024-09-21T02:13:26.964793+0000","last_peered":"2024-09-21T02:13:26.964793+0000","last_clean":"2024-09-21T02:13:26.964793+0000","last_became_active":"2024-09-21T02:13:24.897451+0000","last_became_peered":"2024-09-21T02:13:24.897451+0000","last_unstale":"2024-09-21T02:13:26.964793+0000","last_undegraded":"2024-09-21T02:13:26.964793+0000","last_fullsized":"2024-09-21T02:13:26.964793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883658+0000","last_change":"2024-09-21T02:13:25.911108+0000","last_active":"2024-09-21T02:13:26.883658+0000","last_peered":"2024-09-21T02:13:26.883658+0000","last_clean":"2024-09-21T02:13:26.883658+0000","last_became_active":"2024-09-21T02:13:25.910368+0000","last_became_peered":"2024-09-21T02:13:25.910368+0000","last_unstale":"2024-09-21T02:13:26.883658+0000","last_undegraded":"2024-09-21T02:13:26.883658+0000","last_fullsized":"2024-09-21T02:13:26.883658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883288+0000","last_change":"2024-09-21T02:13:24.905477+0000","last_active":"2024-09-21T02:13:26.883288+0000","last_peered":"2024-09-21T02:13:26.883288+0000","last_clean":"2024-09-21T02:13:26.883288+0000","last_became_active":"2024-09-21T02:13:24.904718+0000","last_became_peered":"2024-09-21T02:13:24.904718+0000","last_unstale":"2024-09-21T02:13:26.883288+0000","last_undegraded":"2024-09-21T02:13:26.883288+0000","last_fullsized":"2024-09-21T02:13:26.883288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883454+0000","last_change":"2024-09-21T02:13:25.897102+0000","last_active":"2024-09-21T02:13:26.883454+0000","last_peered":"2024-09-21T02:13:26.883454+0000","last_clean":"2024-09-21T02:13:26.883454+0000","last_became_active":"2024-09-21T02:13:25.896342+0000","last_became_peered":"2024-09-21T02:13:25.896342+0000","last_unstale":"2024-09-21T02:13:26.883454+0000","last_undegraded":"2024-09-21T02:13:26.883454+0000","last_fullsized":"2024-09-21T02:13:26.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":81753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":835584,"data_stored":672213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":237568,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":831488,"data_stored":672191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:35.037 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:13:35.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T02:13:35.037 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1379620189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80b5410 msgr2=0x7f2de806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1379620189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80b5410 0x7f2de806fa30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2de8070e90 tx=0x7f2de001cd80 comp rx=0 tx=0).stop 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1379620189 shutdown_connections 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1379620189 >> v2:172.21.15.191:3300/0 conn(0x7f2de80b2f40 0x7f2de80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1379620189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80b5410 0x7f2de806fa30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1379620189 >> 172.21.15.170:0/1379620189 conn(0x7f2de80b1c40 msgr2=0x7f2de80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1379620189 shutdown_connections 2024-09-21T02:13:35.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1379620189 wait complete. 2024-09-21T02:13:35.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 Processor -- start 2024-09-21T02:13:35.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- start start 2024-09-21T02:13:35.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de80b2f40 0x7f2de8157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de80b5410 0x7f2de80acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 0x7f2de80c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8169fd0 con 0x7f2de80ccb00 2024-09-21T02:13:35.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2de8169e50 con 0x7f2de80b5410 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2ded1fc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2de816a150 con 0x7f2de80b2f40 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 0x7f2de80c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 0x7f2de80c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42290/0 (socket says 172.21.15.170:42290) 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 -- 172.21.15.170:0/1798824923 learned_addr learned my addr 172.21.15.170:0/1798824923 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de659c640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de80b5410 0x7f2de80acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de6d9d640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de80b2f40 0x7f2de8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 -- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de80b2f40 msgr2=0x7f2de8157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de80b2f40 0x7f2de8157130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 -- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de80b5410 msgr2=0x7f2de80acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de80b5410 0x7f2de80acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de759e640 1 -- 172.21.15.170:0/1798824923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de80ada90 con 0x7f2de80ccb00 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.226+0000 7f2de659c640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de80b5410 0x7f2de80acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2de759e640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 0x7f2de80c4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c3c0 tx=0x7f2dd800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8017070 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd8003600 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de81f6ab0 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8003a50 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de81f7070 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de80ad7e0 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2dd801a070 con 0x7f2de80ccb00 2024-09-21T02:13:35.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ddf7fe640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2dc8073200 0x7f2dc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2dd8093930 con 0x7f2de80ccb00 2024-09-21T02:13:35.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2de6d9d640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2dc8073200 0x7f2dc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.230+0000 7f2de6d9d640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2dc8073200 0x7f2dc8075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2de00af000 tx=0x7f2de001b5c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.234+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2dd8021020 con 0x7f2de80ccb00 2024-09-21T02:13:35.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.406+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f2de80b3350 con 0x7f2de80ccb00 2024-09-21T02:13:35.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.406+0000 7f2ddf7fe640 1 -- 172.21.15.170:0/1798824923 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7f2dd805ea30 con 0x7f2de80ccb00 2024-09-21T02:13:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2dc8073200 msgr2=0x7f2dc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2dc8073200 0x7f2dc8075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2de00af000 tx=0x7f2de001b5c0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 msgr2=0x7f2de80c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 0x7f2de80c4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c3c0 tx=0x7f2dd800c7c0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 shutdown_connections 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2dc8073200 0x7f2dc8075710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de80ccb00 0x7f2de80c4dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de80b5410 0x7f2de80acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 --2- 172.21.15.170:0/1798824923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de80b2f40 0x7f2de8157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 >> 172.21.15.170:0/1798824923 conn(0x7f2de80b1c40 msgr2=0x7f2de80b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 shutdown_connections 2024-09-21T02:13:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.410+0000 7f2ded1fc640 1 -- 172.21.15.170:0/1798824923 wait complete. 2024-09-21T02:13:35.427 INFO:teuthology.orchestra.run.smithi170.stdout:[{"version":31,"timestamp":"2024-09-21T02:13:13.321697+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"6178.054356"}]}] 2024-09-21T02:13:35.427 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T02:13:35.427 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T02:13:35.427 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T02:13:35.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 -- 172.21.15.170:0/3550167225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 msgr2=0x7f42440b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 --2- 172.21.15.170:0/3550167225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440b1730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f423c005fd0 tx=0x7f423c01c820 comp rx=0 tx=0).stop 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 -- 172.21.15.170:0/3550167225 shutdown_connections 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 --2- 172.21.15.170:0/3550167225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f424406d510 0x7f4244160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 --2- 172.21.15.170:0/3550167225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440b1730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 -- 172.21.15.170:0/3550167225 >> 172.21.15.170:0/3550167225 conn(0x7f424409a9c0 msgr2=0x7f42440d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 -- 172.21.15.170:0/3550167225 shutdown_connections 2024-09-21T02:13:35.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.566+0000 7f424b67e640 1 -- 172.21.15.170:0/3550167225 wait complete. 2024-09-21T02:13:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 Processor -- start 2024-09-21T02:13:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 -- start start 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42440b6930 0x7f4244155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4244155de0 0x7f42441fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f424415c670 con 0x7f42440b22b0 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f424415c4f0 con 0x7f42440b6930 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424b67e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f424415c7f0 con 0x7f4244155de0 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53926/0 (socket says 172.21.15.170:53926) 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 -- 172.21.15.170:0/4015688621 learned_addr learned my addr 172.21.15.170:0/4015688621 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f4249c1b640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4244155de0 0x7f42441fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 -- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4244155de0 msgr2=0x7f42441fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f4248c19640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42440b6930 0x7f4244155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4244155de0 0x7f42441fbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 -- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42440b6930 msgr2=0x7f4244155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42440b6930 0x7f4244155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 -- 172.21.15.170:0/4015688621 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440cfec0 con 0x7f42440b22b0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.570+0000 7f424941a640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440c7a20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f423c009790 tx=0x7f423c005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c0048d0 con 0x7f42440b22b0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f423c0bd070 con 0x7f42440b22b0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42440d01a0 con 0x7f42440b22b0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c004de0 con 0x7f42440b22b0 2024-09-21T02:13:35.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441fcc70 con 0x7f42440b22b0 2024-09-21T02:13:35.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244075140 con 0x7f42440b22b0 2024-09-21T02:13:35.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f423c003600 con 0x7f42440b22b0 2024-09-21T02:13:35.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4239ffb640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4228073230 0x7f4228075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f423c12ee10 con 0x7f42440b22b0 2024-09-21T02:13:35.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4248c19640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4228073230 0x7f4228075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.574+0000 7f4248c19640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4228073230 0x7f4228075740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4234001230 tx=0x7f4234009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:35.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.578+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f423c016410 con 0x7f42440b22b0 2024-09-21T02:13:35.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.778+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f4244160390 con 0x7f42440b22b0 2024-09-21T02:13:35.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.782+0000 7f4239ffb640 1 -- 172.21.15.170:0/4015688621 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186463 (secure 0 0 0) 0x7f423c0041f0 con 0x7f42440b22b0 2024-09-21T02:13:35.786 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4228073230 msgr2=0x7f4228075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4228073230 0x7f4228075740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4234001230 tx=0x7f4234009040 comp rx=0 tx=0).stop 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 msgr2=0x7f42440c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440c7a20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f423c009790 tx=0x7f423c005cc0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 shutdown_connections 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4228073230 0x7f4228075740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4244155de0 0x7f42441fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42440b6930 0x7f4244155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 --2- 172.21.15.170:0/4015688621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42440b22b0 0x7f42440c7a20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.786+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 >> 172.21.15.170:0/4015688621 conn(0x7f424409a9c0 msgr2=0x7f42440affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.790+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 shutdown_connections 2024-09-21T02:13:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.790+0000 7f424b67e640 1 -- 172.21.15.170:0/4015688621 wait complete. 2024-09-21T02:13:35.806 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2615148678},{"type":"v1","addr":"172.21.15.191:6801","nonce":2615148678}]},"active_addr":"172.21.15.191:6801/2615148678","active_change":"2024-09-21T02:13:04.850225+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4122,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":1852913239}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":543979301}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":1634809445}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":2890432817}]}]} 2024-09-21T02:13:35.809 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T02:13:35.810 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T02:13:35.810 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:35.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 -- 172.21.15.170:0/1705734967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e415faf0 msgr2=0x7f84e415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 --2- 172.21.15.170:0/1705734967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e415faf0 0x7f84e415ff00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f84e40d2370 tx=0x7f84d801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 -- 172.21.15.170:0/1705734967 shutdown_connections 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 --2- 172.21.15.170:0/1705734967 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e4160440 0x7f84e40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 --2- 172.21.15.170:0/1705734967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e415faf0 0x7f84e415ff00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 --2- 172.21.15.170:0/1705734967 >> v2:172.21.15.191:3300/0 conn(0x7f84e40b25f0 0x7f84e40cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 -- 172.21.15.170:0/1705734967 >> 172.21.15.170:0/1705734967 conn(0x7f84e409a9c0 msgr2=0x7f84e40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 -- 172.21.15.170:0/1705734967 shutdown_connections 2024-09-21T02:13:35.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.962+0000 7f84ea47d640 1 -- 172.21.15.170:0/1705734967 wait complete. 2024-09-21T02:13:35.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 Processor -- start 2024-09-21T02:13:35.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- start start 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 0x7f84e41fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e415faf0 0x7f84e40c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84e4160440 0x7f84e415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40b4bc0 con 0x7f84e40b25f0 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40b4a40 con 0x7f84e4160440 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f84e40b4d40 con 0x7f84e415faf0 2024-09-21T02:13:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 0x7f84e41fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 0x7f84e41fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53946/0 (socket says 172.21.15.170:53946) 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 -- 172.21.15.170:0/3580255334 learned_addr learned my addr 172.21.15.170:0/3580255334 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e37fe640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e415faf0 0x7f84e40c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e8a1a640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84e4160440 0x7f84e415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 -- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e415faf0 msgr2=0x7f84e40c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e415faf0 0x7f84e40c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 -- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84e4160440 msgr2=0x7f84e415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84e4160440 0x7f84e415eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 -- 172.21.15.170:0/3580255334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e40c4800 con 0x7f84e40b25f0 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e37fe640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e415faf0 0x7f84e40c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:35.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e8a1a640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84e4160440 0x7f84e415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:35.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e3fff640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 0x7f84e41fae30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00cce0 tx=0x7f84dc007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:35.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004b30 con 0x7f84e40b25f0 2024-09-21T02:13:35.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84dc004dc0 con 0x7f84e40b25f0 2024-09-21T02:13:35.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e414e700 con 0x7f84e40b25f0 2024-09-21T02:13:35.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc005290 con 0x7f84e40b25f0 2024-09-21T02:13:35.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e4150450 con 0x7f84e40b25f0 2024-09-21T02:13:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.966+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e415ff00 con 0x7f84e40b25f0 2024-09-21T02:13:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.970+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f84dc005430 con 0x7f84e40b25f0 2024-09-21T02:13:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.970+0000 7f84e0ff9640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f84d0073230 0x7f84d0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.970+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f84dc094950 con 0x7f84e40b25f0 2024-09-21T02:13:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.970+0000 7f84e37fe640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f84d0073230 0x7f84d0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.970+0000 7f84e37fe640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f84d0073230 0x7f84d0075740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84d80af000 tx=0x7f84d801b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:35.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:35.974+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f84dc014070 con 0x7f84e40b25f0 2024-09-21T02:13:36.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.146+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84e40ce9d0 con 0x7f84e40b25f0 2024-09-21T02:13:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.146+0000 7f84e0ff9640 1 -- 172.21.15.170:0/3580255334 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7f84dc05fa00 con 0x7f84e40b25f0 2024-09-21T02:13:36.147 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:36.147 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f84d0073230 msgr2=0x7f84d0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f84d0073230 0x7f84d0075740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84d80af000 tx=0x7f84d801b100 comp rx=0 tx=0).stop 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 msgr2=0x7f84e41fae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 0x7f84e41fae30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00cce0 tx=0x7f84dc007610 comp rx=0 tx=0).stop 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 shutdown_connections 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f84d0073230 0x7f84d0075740 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84e415faf0 0x7f84e40c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84e4160440 0x7f84e415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 --2- 172.21.15.170:0/3580255334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e40b25f0 0x7f84e41fae30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 >> 172.21.15.170:0/3580255334 conn(0x7f84e409a9c0 msgr2=0x7f84e4157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 shutdown_connections 2024-09-21T02:13:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.150+0000 7f84ea47d640 1 -- 172.21.15.170:0/3580255334 wait complete. 2024-09-21T02:13:36.167 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T02:13:36.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 -- 172.21.15.170:0/1572520522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f8164510 msgr2=0x7f30f8166930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 --2- 172.21.15.170:0/1572520522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f8164510 0x7f30f8166930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009bc0 tx=0x7f30f001cba0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 -- 172.21.15.170:0/1572520522 shutdown_connections 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 --2- 172.21.15.170:0/1572520522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f8164510 0x7f30f8166930 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 -- 172.21.15.170:0/1572520522 >> 172.21.15.170:0/1572520522 conn(0x7f30f80b1c40 msgr2=0x7f30f80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 -- 172.21.15.170:0/1572520522 shutdown_connections 2024-09-21T02:13:36.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 -- 172.21.15.170:0/1572520522 wait complete. 2024-09-21T02:13:36.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 Processor -- start 2024-09-21T02:13:36.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.306+0000 7f30fe95f640 1 -- start start 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 0x7f30f8155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30f80cf7e0 0x7f30f81fa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30f8164510 0x7f30f81f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8169f30 con 0x7f30f80cb460 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8169db0 con 0x7f30f80cf7e0 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f30f816a0b0 con 0x7f30f8164510 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 0x7f30f8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 0x7f30f8155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53972/0 (socket says 172.21.15.170:53972) 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 -- 172.21.15.170:0/2422487429 learned_addr learned my addr 172.21.15.170:0/2422487429 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 -- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30f8164510 msgr2=0x7f30f81f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30f8164510 0x7f30f81f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 -- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30f80cf7e0 msgr2=0x7f30f81fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f77fe640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30f80cf7e0 0x7f30f81fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30f80cf7e0 0x7f30f81fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 -- 172.21.15.170:0/2422487429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f81fb1d0 con 0x7f30f80cb460 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f7fff640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 0x7f30f8155310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c400 tx=0x7f30e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8011790 con 0x7f30f80cb460 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e8017070 con 0x7f30f80cb460 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f81fac10 con 0x7f30f80cb460 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8011cc0 con 0x7f30f80cb460 2024-09-21T02:13:36.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81f6e00 con 0x7f30f80cb460 2024-09-21T02:13:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f80cb870 con 0x7f30f80cb460 2024-09-21T02:13:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30e801c460 con 0x7f30f80cb460 2024-09-21T02:13:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f4ff9640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30e4073200 0x7f30e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.310+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f30e8094050 con 0x7f30f80cb460 2024-09-21T02:13:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.314+0000 7f30f77fe640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30e4073200 0x7f30e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.314+0000 7f30f77fe640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30e4073200 0x7f30e4075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30f81f4870 tx=0x7f30ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.318+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f30e805f070 con 0x7f30f80cb460 2024-09-21T02:13:36.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.474+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30f80ad130 con 0x7f30f80cb460 2024-09-21T02:13:36.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30f4ff9640 1 -- 172.21.15.170:0/2422487429 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7f30e805e970 con 0x7f30f80cb460 2024-09-21T02:13:36.478 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:36.478 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30e4073200 msgr2=0x7f30e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30e4073200 0x7f30e4075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30f81f4870 tx=0x7f30ec008040 comp rx=0 tx=0).stop 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 msgr2=0x7f30f8155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 0x7f30f8155310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c400 tx=0x7f30e800c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 shutdown_connections 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30e4073200 0x7f30e4075710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30f8164510 0x7f30f81f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30f80cf7e0 0x7f30f81fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 --2- 172.21.15.170:0/2422487429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30f80cb460 0x7f30f8155310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 >> 172.21.15.170:0/2422487429 conn(0x7f30f80b1c40 msgr2=0x7f30f80afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 shutdown_connections 2024-09-21T02:13:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.478+0000 7f30fe95f640 1 -- 172.21.15.170:0/2422487429 wait complete. 2024-09-21T02:13:36.499 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T02:13:36.499 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T02:13:36.500 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T02:13:36.500 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T02:13:36.500 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T02:13:36.501 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T02:13:36.501 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T02:13:36.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T02:13:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 -- 172.21.15.170:0/3929219754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1623a0 msgr2=0x7f97dc1627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 --2- 172.21.15.170:0/3929219754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1623a0 0x7f97dc1627b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009840 tx=0x7f97d001c6b0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 -- 172.21.15.170:0/3929219754 shutdown_connections 2024-09-21T02:13:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 --2- 172.21.15.170:0/3929219754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97dc073b40 0x7f97dc073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 --2- 172.21.15.170:0/3929219754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1623a0 0x7f97dc1627b0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009840 tx=0x7f97d001c6b0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 -- 172.21.15.170:0/3929219754 >> 172.21.15.170:0/3929219754 conn(0x7f97dc0cffa0 msgr2=0x7f97dc0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 -- 172.21.15.170:0/3929219754 shutdown_connections 2024-09-21T02:13:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.718+0000 7f97e1281640 1 -- 172.21.15.170:0/3929219754 wait complete. 2024-09-21T02:13:36.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 Processor -- start 2024-09-21T02:13:36.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 -- start start 2024-09-21T02:13:36.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97dc073b40 0x7f97dc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97dc155210 0x7f97dc1fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1542b0 0x7f97dc1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc155b50 con 0x7f97dc155210 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1559d0 con 0x7f97dc1542b0 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97daffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97dc073b40 0x7f97dc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97e1281640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc070800 con 0x7f97dc073b40 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97daffd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97dc073b40 0x7f97dc0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:51940/0 (socket says 172.21.15.170:51940) 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97da7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97dc155210 0x7f97dc1fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.722+0000 7f97daffd640 1 -- 172.21.15.170:0/3570464867 learned_addr learned my addr 172.21.15.170:0/3570464867 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97db7fe640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1542b0 0x7f97dc1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97da7fc640 1 -- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97dc073b40 msgr2=0x7f97dc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97da7fc640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97dc073b40 0x7f97dc0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97da7fc640 1 -- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1542b0 msgr2=0x7f97dc1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97da7fc640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97dc1542b0 0x7f97dc1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97da7fc640 1 -- 172.21.15.170:0/3570464867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc154c00 con 0x7f97dc155210 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97da7fc640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97dc155210 0x7f97dc1fcc80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e6c0 tx=0x7f97cc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00cab0 con 0x7f97dc155210 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc0ce950 con 0x7f97dc155210 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97cc00cd40 con 0x7f97dc155210 2024-09-21T02:13:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc010690 con 0x7f97dc155210 2024-09-21T02:13:36.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc1592a0 con 0x7f97dc155210 2024-09-21T02:13:36.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.726+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97dc070ce0 con 0x7f97dc155210 2024-09-21T02:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f97cc010830 con 0x7f97dc155210 2024-09-21T02:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97c37fe640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f97c8073200 0x7f97c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45ae427640 1 -- 172.21.15.170:0/1825525969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4594003c20 msgr2=0x7f4594004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45ae427640 1 --2- 172.21.15.170:0/1825525969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4594003c20 0x7f4594004070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c2e0 tx=0x7f45a401cfb0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45adc26640 1 -- 172.21.15.170:0/1825525969 reap_dead start 2024-09-21T02:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45ae427640 1 -- 172.21.15.170:0/1825525969 shutdown_connections 2024-09-21T02:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45ae427640 1 -- 172.21.15.170:0/1825525969 >> 172.21.15.170:0/1825525969 conn(0x7f45a80b1a50 msgr2=0x7f45a80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f97cc094ba0 con 0x7f97dc155210 2024-09-21T02:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97c37fe640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7f97c807b3f0 0x7f97c807d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 --> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97cc023e50 con 0x7f97c807b3f0 2024-09-21T02:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97daffd640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f97c8073200 0x7f97c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45ae427640 1 -- 172.21.15.170:0/1825525969 shutdown_connections 2024-09-21T02:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97daffd640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f97c8073200 0x7f97c8075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009840 tx=0x7f97d001aef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97db7fe640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7f97c807b3f0 0x7f97c807d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f97cc05bba0 con 0x7f97dc155210 2024-09-21T02:13:36.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f97db7fe640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7f97c807b3f0 0x7f97c807d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.730+0000 7f45ae427640 1 -- 172.21.15.170:0/1825525969 wait complete. 2024-09-21T02:13:36.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== osd.2 v2:172.21.15.170:6808/2727456551 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97cc023e50 con 0x7f97c807b3f0 2024-09-21T02:13:36.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 -- 172.21.15.170:0/451537640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40b40f0 msgr2=0x7f69a4073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 --2- 172.21.15.170:0/451537640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40b40f0 0x7f69a4073cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69a406e900 tx=0x7f69a0025080 comp rx=0 tx=0).stop 2024-09-21T02:13:36.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 Processor -- start 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 -- 172.21.15.170:0/451537640 shutdown_connections 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 --2- 172.21.15.170:0/451537640 >> v2:172.21.15.170:3300/0 conn(0x7f69a4162710 0x7f69a40700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 --2- 172.21.15.170:0/451537640 >> v2:172.21.15.191:3300/0 conn(0x7f69a40741f0 0x7f69a41621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 --2- 172.21.15.170:0/451537640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40b40f0 0x7f69a4073cb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 -- 172.21.15.170:0/451537640 >> 172.21.15.170:0/451537640 conn(0x7f69a40cffa0 msgr2=0x7f69a40d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb606cc9640 1 -- 172.21.15.170:0/2615378557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec004d50 msgr2=0x7fb5ec0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb606cc9640 1 --2- 172.21.15.170:0/2615378557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec004d50 0x7fb5ec0051a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6000aed50 tx=0x7fb5f801f9e0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 -- 172.21.15.170:0/451537640 shutdown_connections 2024-09-21T02:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 -- 172.21.15.170:0/451537640 wait complete. 2024-09-21T02:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 -- 172.21.15.170:0/2448735503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 msgr2=0x7f5b98034d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 --2- 172.21.15.170:0/2448735503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98034d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009890 tx=0x7f5b9001c6f0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb6064c8640 1 -- 172.21.15.170:0/2615378557 reap_dead start 2024-09-21T02:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb606cc9640 1 -- 172.21.15.170:0/2615378557 shutdown_connections 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb606cc9640 1 -- 172.21.15.170:0/2615378557 >> 172.21.15.170:0/2615378557 conn(0x7fb60006d400 msgr2=0x7fb6000b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb606cc9640 1 -- 172.21.15.170:0/2615378557 shutdown_connections 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 -- 172.21.15.170:0/2448735503 shutdown_connections 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 --2- 172.21.15.170:0/2448735503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b980e1a80 0x7f5b98035270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 --2- 172.21.15.170:0/2448735503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98034d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 -- 172.21.15.170:0/2448735503 >> 172.21.15.170:0/2448735503 conn(0x7f5b9801eaf0 msgr2=0x7f5b9803a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 -- 172.21.15.170:0/2448735503 shutdown_connections 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 -- start start 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45a80b3310 0x7f45a8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45a8168360 0x7f45a8168770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45acc24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45a8168360 0x7f45a8168770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45acc24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45a8168360 0x7f45a8168770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54018/0 (socket says 172.21.15.170:54018) 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ad425640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45a80b3310 0x7f45a8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45acc24640 1 -- 172.21.15.170:0/1535084179 learned_addr learned my addr 172.21.15.170:0/1535084179 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45a815e0d0 0x7f45a815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f69ab86d640 1 Processor -- start 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45a816aed0 con 0x7f45a8168360 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f45a816ad50 con 0x7f45a80b3310 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f45a816b050 con 0x7f45a815e0d0 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45adc26640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45a815e0d0 0x7f45a815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45acc24640 1 -- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45a815e0d0 msgr2=0x7f45a815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45acc24640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45a815e0d0 0x7f45a815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45acc24640 1 -- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45a80b3310 msgr2=0x7f45a8156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45acc24640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45a80b3310 0x7f45a8156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45acc24640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81574c0 con 0x7f45a8168360 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45acc24640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45a8168360 0x7f45a8168770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f459800e6f0 tx=0x7f459800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598005220 con 0x7f45a8168360 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a8168cb0 con 0x7f45a8168360 2024-09-21T02:13:36.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a8072cf0 con 0x7f45a8168360 2024-09-21T02:13:36.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7fb606cc9640 1 -- 172.21.15.170:0/2615378557 wait complete. 2024-09-21T02:13:36.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 Processor -- start 2024-09-21T02:13:36.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 -- start start 2024-09-21T02:13:36.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.734+0000 7f5ba652a640 1 -- 172.21.15.170:0/2448735503 wait complete. 2024-09-21T02:13:36.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f5ba652a640 1 Processor -- start 2024-09-21T02:13:36.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f5ba652a640 1 -- start start 2024-09-21T02:13:36.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 -- start start 2024-09-21T02:13:36.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40741f0 0x7f69a414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69a40b40f0 0x7f69a415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69a4162710 0x7f69a41e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69a415f6e0 con 0x7f69a40b40f0 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f69a415f560 con 0x7f69a4162710 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69ab86d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f69a415f860 con 0x7f69a40741f0 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40741f0 0x7f69a414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40741f0 0x7f69a414c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52038/0 (socket says 172.21.15.170:52038) 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a8e08640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69a40b40f0 0x7f69a415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 -- 172.21.15.170:0/905707659 learned_addr learned my addr 172.21.15.170:0/905707659 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9e0a640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69a4162710 0x7f69a41e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45980053c0 con 0x7f45a8168360 2024-09-21T02:13:36.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980047b0 con 0x7f45a8168360 2024-09-21T02:13:36.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4598004a10 con 0x7f45a8168360 2024-09-21T02:13:36.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6000cde60 0x7fb600148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb60014b130 0x7fb600155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb60014b540 0x7fb6001564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb600074610 con 0x7fb6000cde60 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb600074490 con 0x7fb60014b130 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7fb606cc9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb600074790 con 0x7fb60014b540 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 -- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69a4162710 msgr2=0x7f69a41e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69a4162710 0x7f69a41e97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 -- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69a40b40f0 msgr2=0x7f69a415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69a40b40f0 0x7f69a415be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.738+0000 7f69a9609640 1 -- 172.21.15.170:0/905707659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a41f59e0 con 0x7f69a40741f0 2024-09-21T02:13:36.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6000cde60 0x7fb600148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6000cde60 0x7fb600148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54026/0 (socket says 172.21.15.170:54026) 2024-09-21T02:13:36.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 -- 172.21.15.170:0/4154798512 learned_addr learned my addr 172.21.15.170:0/4154798512 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f69a9609640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40741f0 0x7f69a414c850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69a0023730 tx=0x7f69a00b4480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba652a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b980e0890 0x7f5b98033d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a00b9070 con 0x7f69a40741f0 2024-09-21T02:13:36.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba652a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb6064c8640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb60014b540 0x7fb6001564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 -- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb60014b540 msgr2=0x7fb6001564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb60014b540 0x7fb6001564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f459dffb640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f458c0732f0 0x7f458c075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 -- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb60014b130 msgr2=0x7fb600155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb60014b130 0x7fb600155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 -- 172.21.15.170:0/4154798512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6000ae860 con 0x7fb6000cde60 2024-09-21T02:13:36.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba652a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b980e1a80 0x7f5b980329b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb6064c8640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb60014b540 0x7fb6001564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:36.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9ffff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b980e0890 0x7f5b98033d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9ffff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b980e0890 0x7f5b98033d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52054/0 (socket says 172.21.15.170:52054) 2024-09-21T02:13:36.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4598094a80 con 0x7f45a8168360 2024-09-21T02:13:36.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98032470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54030/0 (socket says 172.21.15.170:54030) 2024-09-21T02:13:36.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb605cc7640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6000cde60 0x7fb600148350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0011ae0 tx=0x7fb5f0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f45ad425640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f458c0732f0 0x7f458c075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9ffff640 1 -- 172.21.15.170:0/388908916 learned_addr learned my addr 172.21.15.170:0/388908916 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b98034af0 con 0x7f5b980e1670 2024-09-21T02:13:36.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5b98034970 con 0x7f5b980e1a80 2024-09-21T02:13:36.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f001e070 con 0x7fb6000cde60 2024-09-21T02:13:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba4ac7640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b980e1a80 0x7f5b980329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5b98006830 con 0x7f5b980e0890 2024-09-21T02:13:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f0004460 con 0x7fb6000cde60 2024-09-21T02:13:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000fc30 con 0x7fb6000cde60 2024-09-21T02:13:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a40ccd20 con 0x7f69a40741f0 2024-09-21T02:13:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a40cd2e0 con 0x7f69a40741f0 2024-09-21T02:13:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6000ce270 con 0x7fb6000cde60 2024-09-21T02:13:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69a4073cb0 con 0x7f69a40741f0 2024-09-21T02:13:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69a00c3070 con 0x7f69a40741f0 2024-09-21T02:13:36.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f45ad425640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f458c0732f0 0x7f458c075800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45a8071db0 tx=0x7f45a4005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 -- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b980e0890 msgr2=0x7f5b98033d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b980e0890 0x7f5b98033d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 -- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b980e1a80 msgr2=0x7f5b980329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b980e1a80 0x7f5b980329b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b98054f90 con 0x7f5b980e1670 2024-09-21T02:13:36.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0003600 con 0x7f69a40741f0 2024-09-21T02:13:36.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.742+0000 7f5b9f7fe640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98032470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800dea0 tx=0x7f5b9400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5f0021070 con 0x7fb6000cde60 2024-09-21T02:13:36.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f45937fe640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7f4588001660 0x7f4588003b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f45937fe640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4588006c10 con 0x7f4588001660 2024-09-21T02:13:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001f6ff0 con 0x7fb6000cde60 2024-09-21T02:13:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/2343628709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d8005d50 msgr2=0x7ff3d80363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 --2- 172.21.15.170:0/2343628709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d8005d50 0x7ff3d80363e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009840 tx=0x7ff3d401c610 comp rx=0 tx=0).stop 2024-09-21T02:13:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/2343628709 shutdown_connections 2024-09-21T02:13:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 --2- 172.21.15.170:0/2343628709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d8005d50 0x7ff3d80363e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 --2- 172.21.15.170:0/2343628709 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d80e2bd0 0x7ff3d800b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/2343628709 >> 172.21.15.170:0/2343628709 conn(0x7ff3d8051420 msgr2=0x7ff3d8051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f69a00037a0 con 0x7f69a40741f0 2024-09-21T02:13:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f45adc26640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7f4588001660 0x7f4588003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/2343628709 shutdown_connections 2024-09-21T02:13:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/2343628709 wait complete. 2024-09-21T02:13:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb5f67fc640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb5e4073200 0x7fb5e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb5f0096cc0 con 0x7fb6000cde60 2024-09-21T02:13:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb6054c6640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb5e4073200 0x7fb5e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6988073200 0x7f6988075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb606cc9640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fb6001f72d0 0x7fb6001f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 --> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb60006a270 con 0x7fb6001f72d0 2024-09-21T02:13:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f69a0137ea0 con 0x7f69a40741f0 2024-09-21T02:13:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f69a8e08640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6988073200 0x7f6988075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f698807b4d0 0x7f698807d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 --> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69a00b46f0 con 0x7f698807b4d0 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f69a0138290 con 0x7f69a40741f0 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f69a9e0a640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f698807b4d0 0x7f698807d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f69a8e08640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6988073200 0x7f6988075710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69a41eace0 tx=0x7f6994009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f69a9e0a640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f698807b4d0 0x7f698807d920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb6054c6640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb5e4073200 0x7fb5e4075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6000d0eb0 tx=0x7fb5fc0067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400ca80 con 0x7f5b980e1670 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b98055270 con 0x7f5b980e1670 2024-09-21T02:13:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b98055830 con 0x7f5b980e1670 2024-09-21T02:13:36.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 Processor -- start 2024-09-21T02:13:36.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7ff3e5fbc640 1 -- start start 2024-09-21T02:13:36.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== osd.3 v2:172.21.15.170:6816/708481897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69a00b46f0 con 0x7f698807b4d0 2024-09-21T02:13:36.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b980e0ca0 con 0x7f5b980e1670 2024-09-21T02:13:36.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f45adc26640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7f4588001660 0x7f4588003b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b9400cc20 con 0x7f5b980e1670 2024-09-21T02:13:36.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94010430 con 0x7f5b980e1670 2024-09-21T02:13:36.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.746+0000 7fb6064c8640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fb6001f72d0 0x7fb6001f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d8005d50 0x7ff3d817cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d8036920 0x7ff3d80d72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d80e2bd0 0x7ff3d817ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80ec950 con 0x7ff3d8036920 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80ec7d0 con 0x7ff3d80e2bd0 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d80ecad0 con 0x7ff3d8005d50 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d8005d50 0x7ff3d817cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d8005d50 0x7ff3d817cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52062/0 (socket says 172.21.15.170:52062) 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 -- 172.21.15.170:0/1765811582 learned_addr learned my addr 172.21.15.170:0/1765811582 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 -- 172.21.15.170:0/532697479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc072f30 msgr2=0x7f71bc073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 --2- 172.21.15.170:0/532697479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc072f30 0x7f71bc073340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009900 tx=0x7f71b001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 -- 172.21.15.170:0/532697479 shutdown_connections 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 --2- 172.21.15.170:0/532697479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc072f30 0x7f71bc073340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 --2- 172.21.15.170:0/532697479 >> v2:172.21.15.170:3300/0 conn(0x7f71bc15ebc0 0x7f71bc0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 -- 172.21.15.170:0/532697479 >> 172.21.15.170:0/532697479 conn(0x7f71bc0d00f0 msgr2=0x7f71bc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3dffff640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d80e2bd0 0x7ff3d817ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 -- 172.21.15.170:0/532697479 shutdown_connections 2024-09-21T02:13:36.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 -- 172.21.15.170:0/532697479 wait complete. 2024-09-21T02:13:36.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== osd.4 v2:172.21.15.191:6802/1771265885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4588006c10 con 0x7f4588001660 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 -- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d80e2bd0 msgr2=0x7ff3d817ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d80e2bd0 0x7ff3d817ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 -- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d8036920 msgr2=0x7ff3d80d72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d8036920 0x7ff3d80d72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 -- 172.21.15.170:0/1765811582 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d817c4d0 con 0x7ff3d8005d50 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3dffff640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d80e2bd0 0x7ff3d817ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:36.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5b940105d0 con 0x7f5b980e1670 2024-09-21T02:13:36.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3df7fe640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d8005d50 0x7ff3d817cae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3e006aac0 tx=0x7ff3e0075500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0098070 con 0x7ff3d8005d50 2024-09-21T02:13:36.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3e00717a0 con 0x7ff3d8005d50 2024-09-21T02:13:36.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d817c7b0 con 0x7ff3d8005d50 2024-09-21T02:13:36.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00719b0 con 0x7ff3d8005d50 2024-09-21T02:13:36.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f5b9cff9640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5b8c073300 0x7f5b8c075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7fb6064c8640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fb6001f72d0 0x7fb6001f3a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d80d6ad0 con 0x7ff3d8005d50 2024-09-21T02:13:36.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5b94095510 con 0x7f5b980e1670 2024-09-21T02:13:36.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3d80363e0 con 0x7ff3d8005d50 2024-09-21T02:13:36.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f5b9ffff640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5b8c073300 0x7f5b8c075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 Processor -- start 2024-09-21T02:13:36.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f5b9cff9640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f5b8c07b5d0 0x7f5b8c07da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.750+0000 7f71c1fc7640 1 -- start start 2024-09-21T02:13:36.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f5b9ffff640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5b8c073300 0x7f5b8c075810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009890 tx=0x7f5b9001af80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b94019ec0 con 0x7f5b8c07b5d0 2024-09-21T02:13:36.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== osd.5 v2:172.21.15.191:6818/2854287101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5f80ad040 con 0x7fb6001f72d0 2024-09-21T02:13:36.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5b94095900 con 0x7f5b980e1670 2024-09-21T02:13:36.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f5ba4ac7640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f5b8c07b5d0 0x7f5b8c07da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71c1fc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71bc072f30 0x7f71bc0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71c1fc7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc0cea40 0x7f71bc200160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71c1fc7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc15ebc0 0x7f71bc2006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71c1fc7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc16bfc0 con 0x7f71bc072f30 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71c1fc7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc0700d0 con 0x7f71bc0cea40 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71c1fc7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f71bc16c140 con 0x7f71bc15ebc0 2024-09-21T02:13:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71bc072f30 0x7f71bc0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f5ba4ac7640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f5b8c07b5d0 0x7f5b8c07da20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff3e0071b50 con 0x7ff3d8005d50 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3cbfff640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff3c4073230 0x7ff3c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff3e01101a0 con 0x7ff3d8005d50 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3cbfff640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7ff3c407b480 0x7ff3c407d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 --> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3e008ce50 con 0x7ff3c407b480 2024-09-21T02:13:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff3e0110590 con 0x7ff3d8005d50 2024-09-21T02:13:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71bc072f30 0x7f71bc0c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54058/0 (socket says 172.21.15.170:54058) 2024-09-21T02:13:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 -- 172.21.15.170:0/424605958 learned_addr learned my addr 172.21.15.170:0/424605958 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71baffd640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc0cea40 0x7f71bc200160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bbfff640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc15ebc0 0x7f71bc2006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3dffff640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7ff3c407b480 0x7ff3c407d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3deffd640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff3c4073230 0x7ff3c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 -- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc15ebc0 msgr2=0x7f71bc2006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71bc15ebc0 0x7f71bc2006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 -- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc0cea40 msgr2=0x7f71bc200160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc0cea40 0x7f71bc200160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 -- 172.21.15.170:0/424605958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc0b5e10 con 0x7f71bc072f30 2024-09-21T02:13:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3dffff640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7ff3c407b480 0x7ff3c407d8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7f71bb7fe640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71bc072f30 0x7f71bc0c6020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71b400cce0 tx=0x7f71b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.754+0000 7ff3deffd640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff3c4073230 0x7ff3c4075740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40af000 tx=0x7ff3d40193b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.758+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== osd.0 v2:172.21.15.170:6800/1810219020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3e008ce50 con 0x7ff3c407b480 2024-09-21T02:13:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.758+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== osd.6 v2:172.21.15.191:6826/283530163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b94019ec0 con 0x7f5b8c07b5d0 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 --> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97dc15f9d0 con 0x7f97c807b3f0 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400f450 con 0x7f71bc072f30 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc0c6560 con 0x7f71bc072f30 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc0b3f70 con 0x7f71bc072f30 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71b400f6e0 con 0x7f71bc072f30 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b400fb10 con 0x7f71bc072f30 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f719c001040 con 0x7f71bc072f30 2024-09-21T02:13:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97c37fe640 1 -- 172.21.15.170:0/3570464867 <== osd.2 v2:172.21.15.170:6808/2727456551 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f97dc15f9d0 con 0x7f97c807b3f0 2024-09-21T02:13:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7f97c807b3f0 msgr2=0x7f97c807d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7f97c807b3f0 0x7f97c807d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f97c8073200 msgr2=0x7f97c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f97c8073200 0x7f97c8075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009840 tx=0x7f97d001aef0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97dc155210 msgr2=0x7f97dc1fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97e1281640 1 --2- 172.21.15.170:0/3570464867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97dc155210 0x7f97dc1fcc80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e6c0 tx=0x7f97cc00eb80 comp rx=0 tx=0).stop 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.762+0000 7f97db7fe640 1 -- 172.21.15.170:0/3570464867 reap_dead start 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 shutdown_connections 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 >> 172.21.15.170:0/3570464867 conn(0x7f97dc0cffa0 msgr2=0x7f97dc15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 shutdown_connections 2024-09-21T02:13:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f97e1281640 1 -- 172.21.15.170:0/3570464867 wait complete. 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 -- 172.21.15.170:0/86442568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e1670 msgr2=0x7f4be0034d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 --2- 172.21.15.170:0/86442568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e1670 0x7f4be0034d30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4be000ec40 tx=0x7f4bd801c770 comp rx=0 tx=0).stop 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 -- 172.21.15.170:0/86442568 shutdown_connections 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 --2- 172.21.15.170:0/86442568 >> v2:172.21.15.191:3300/0 conn(0x7f4be00e0fa0 0x7f4be0035270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 --2- 172.21.15.170:0/86442568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e1670 0x7f4be0034d30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 -- 172.21.15.170:0/86442568 >> 172.21.15.170:0/86442568 conn(0x7f4be001eaf0 msgr2=0x7f4be003a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 -- 172.21.15.170:0/86442568 shutdown_connections 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71b4018400 con 0x7f71bc072f30 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71a3fff640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f71a8073300 0x7f71a8075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f71b4094c90 con 0x7f71bc072f30 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71a3fff640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7f71a807b570 0x7f71a807d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 --> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71b400fe50 con 0x7f71a807b570 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f71b40972f0 con 0x7f71bc072f30 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71baffd640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f71a8073300 0x7f71a8075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 -- 172.21.15.170:0/86442568 wait complete. 2024-09-21T02:13:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f4be7fff640 1 Processor -- start 2024-09-21T02:13:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.766+0000 7f71bbfff640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7f71a807b570 0x7f71a807d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be7fff640 1 -- start start 2024-09-21T02:13:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f71baffd640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f71a8073300 0x7f71a8075810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71bc1561a0 tx=0x7f71ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e0fa0 0x7f4be01810f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4be00e1670 0x7f4be018e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be00e1a80 0x7f4be00dee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e0fa0 0x7f4be01810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e0fa0 0x7f4be01810f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54062/0 (socket says 172.21.15.170:54062) 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be67fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4be00e1670 0x7f4be018e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f4be67fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4be00e1670 0x7f4be018e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52082/0 (socket says 172.21.15.170:52082) 2024-09-21T02:13:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f71bbfff640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7f71a807b570 0x7f71a807d9c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.770+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 --> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69a40cda60 con 0x7f698807b4d0 2024-09-21T02:13:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 -- 172.21.15.170:0/2624653433 learned_addr learned my addr 172.21.15.170:0/2624653433 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00ed7d0 con 0x7f4be00e0fa0 2024-09-21T02:13:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00ed650 con 0x7f4be00e1a80 2024-09-21T02:13:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4be00ed950 con 0x7f4be00e1670 2024-09-21T02:13:36.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f6999ffb640 1 -- 172.21.15.170:0/905707659 <== osd.3 v2:172.21.15.170:6816/708481897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f69a40cda60 con 0x7f698807b4d0 2024-09-21T02:13:36.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 --> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6000cc750 con 0x7fb6001f72d0 2024-09-21T02:13:36.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be77fe640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be00e1a80 0x7f4be00dee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 -- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4be00e1670 msgr2=0x7f4be018e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4be00e1670 0x7f4be018e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 -- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be00e1a80 msgr2=0x7f4be00dee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4be00e1a80 0x7f4be00dee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be018eb80 con 0x7f4be00e0fa0 2024-09-21T02:13:36.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7fb5f67fc640 1 -- 172.21.15.170:0/4154798512 <== osd.5 v2:172.21.15.191:6818/2854287101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb6000cc750 con 0x7fb6001f72d0 2024-09-21T02:13:36.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be6ffd640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e0fa0 0x7f4be01810f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8019380 tx=0x7f4bd80a6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8004030 con 0x7f4be00e0fa0 2024-09-21T02:13:36.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bd80041d0 con 0x7f4be00e0fa0 2024-09-21T02:13:36.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be018ab80 con 0x7f4be00e0fa0 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f698807b4d0 msgr2=0x7f698807d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f698807b4d0 0x7f698807d920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6988073200 msgr2=0x7f6988075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6988073200 0x7f6988075710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69a41eace0 tx=0x7f6994009060 comp rx=0 tx=0).stop 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40741f0 msgr2=0x7f69a414c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 --2- 172.21.15.170:0/905707659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69a40741f0 0x7f69a414c850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69a0023730 tx=0x7f69a00b4480 comp rx=0 tx=0).stop 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69a9e0a640 1 -- 172.21.15.170:0/905707659 reap_dead start 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 shutdown_connections 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 >> 172.21.15.170:0/905707659 conn(0x7f69a40cffa0 msgr2=0x7f69a415df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 shutdown_connections 2024-09-21T02:13:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.774+0000 7f69ab86d640 1 -- 172.21.15.170:0/905707659 wait complete. 2024-09-21T02:13:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== osd.1 v2:172.21.15.170:6824/2831250627 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71b400fe50 con 0x7f71a807b570 2024-09-21T02:13:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80050c0 con 0x7f4be00e0fa0 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4bd80b8030 con 0x7f4be00e0fa0 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be018b140 con 0x7f4be00e0fa0 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4bcf7fe640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4bd40731e0 0x7f4bd40756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd80f46c0 con 0x7f4be00e0fa0 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4be67fc640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4bd40731e0 0x7f4bd40756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4be7fff640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f4bc8001660 0x7f4bc8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bc8006c10 con 0x7f4bc8001660 2024-09-21T02:13:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fb6001f72d0 msgr2=0x7fb6001f3a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fb6001f72d0 0x7fb6001f3a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb5e4073200 msgr2=0x7fb5e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb5e4073200 0x7fb5e4075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6000d0eb0 tx=0x7fb5fc0067b0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6000cde60 msgr2=0x7fb600148350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 --2- 172.21.15.170:0/4154798512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6000cde60 0x7fb600148350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0011ae0 tx=0x7fb5f0009ea0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb6064c8640 1 -- 172.21.15.170:0/4154798512 reap_dead start 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 shutdown_connections 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 >> 172.21.15.170:0/4154798512 conn(0x7fb60006d400 msgr2=0x7fb600151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 shutdown_connections 2024-09-21T02:13:36.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f4be77fe640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f4bc8001660 0x7f4bc8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:36.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7fb606cc9640 1 -- 172.21.15.170:0/4154798512 wait complete. 2024-09-21T02:13:36.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.778+0000 7f45937fe640 1 -- 172.21.15.170:0/1535084179 --> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4588005ca0 con 0x7f4588001660 2024-09-21T02:13:36.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f4be67fc640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4bd40731e0 0x7f4bd40756f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0046c0 tx=0x7f4bdc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f459dffb640 1 -- 172.21.15.170:0/1535084179 <== osd.4 v2:172.21.15.191:6802/1771265885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4588005ca0 con 0x7f4588001660 2024-09-21T02:13:36.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f4be77fe640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f4bc8001660 0x7f4bc8003b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7f4588001660 msgr2=0x7f4588003b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7f4588001660 0x7f4588003b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f458c0732f0 msgr2=0x7f458c075800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f458c0732f0 0x7f458c075800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45a8071db0 tx=0x7f45a4005040 comp rx=0 tx=0).stop 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45a8168360 msgr2=0x7f45a8168770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 --2- 172.21.15.170:0/1535084179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45a8168360 0x7f45a8168770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f459800e6f0 tx=0x7f459800ebb0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45adc26640 1 -- 172.21.15.170:0/1535084179 reap_dead start 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 shutdown_connections 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 >> 172.21.15.170:0/1535084179 conn(0x7f45a80b1a50 msgr2=0x7f45a8169b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 shutdown_connections 2024-09-21T02:13:36.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f45ae427640 1 -- 172.21.15.170:0/1535084179 wait complete. 2024-09-21T02:13:36.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== osd.7 v2:172.21.15.191:6810/1521218345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc8006c10 con 0x7f4bc8001660 2024-09-21T02:13:36.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.782+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 --> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b98054d40 con 0x7f5b8c07b5d0 2024-09-21T02:13:36.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.786+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 --> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff3d80363e0 con 0x7ff3c407b480 2024-09-21T02:13:36.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.794+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 --> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f719c002de0 con 0x7f71a807b570 2024-09-21T02:13:36.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7ff3cbfff640 1 -- 172.21.15.170:0/1765811582 <== osd.0 v2:172.21.15.170:6800/1810219020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff3d80363e0 con 0x7ff3c407b480 2024-09-21T02:13:36.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71a3fff640 1 -- 172.21.15.170:0/424605958 <== osd.1 v2:172.21.15.170:6824/2831250627 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f719c002de0 con 0x7f71a807b570 2024-09-21T02:13:36.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5b9cff9640 1 -- 172.21.15.170:0/388908916 <== osd.6 v2:172.21.15.191:6826/283530163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b98054d40 con 0x7f5b8c07b5d0 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7f71a807b570 msgr2=0x7f71a807d9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7f71a807b570 0x7f71a807d9c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f71a8073300 msgr2=0x7f71a8075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f71a8073300 0x7f71a8075810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71bc1561a0 tx=0x7f71ac009040 comp rx=0 tx=0).stop 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71bc072f30 msgr2=0x7f71bc0c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 --2- 172.21.15.170:0/424605958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71bc072f30 0x7f71bc0c6020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71b400cce0 tx=0x7f71b400a620 comp rx=0 tx=0).stop 2024-09-21T02:13:36.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71bbfff640 1 -- 172.21.15.170:0/424605958 reap_dead start 2024-09-21T02:13:36.799 INFO:teuthology.orchestra.run.smithi170.stdout:64424509448 2024-09-21T02:13:36.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T02:13:36.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 shutdown_connections 2024-09-21T02:13:36.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 >> 172.21.15.170:0/424605958 conn(0x7f71bc0d00f0 msgr2=0x7f71bc15d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 shutdown_connections 2024-09-21T02:13:36.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f71c1fc7640 1 -- 172.21.15.170:0/424605958 wait complete. 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f5b8c07b5d0 msgr2=0x7f5b8c07da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f5b8c07b5d0 0x7f5b8c07da20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5b8c073300 msgr2=0x7f5b8c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5b8c073300 0x7f5b8c075810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009890 tx=0x7f5b9001af80 comp rx=0 tx=0).stop 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 msgr2=0x7f5b98032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 --2- 172.21.15.170:0/388908916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b980e1670 0x7f5b98032470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800dea0 tx=0x7f5b9400eb50 comp rx=0 tx=0).stop 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba4ac7640 1 -- 172.21.15.170:0/388908916 reap_dead start 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 shutdown_connections 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 >> 172.21.15.170:0/388908916 conn(0x7f5b9801eaf0 msgr2=0x7f5b980e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 shutdown_connections 2024-09-21T02:13:36.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.798+0000 7f5ba652a640 1 -- 172.21.15.170:0/388908916 wait complete. 2024-09-21T02:13:36.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7ff3c407b480 msgr2=0x7ff3c407d8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7ff3c407b480 0x7ff3c407d8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff3c4073230 msgr2=0x7ff3c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff3c4073230 0x7ff3c4075740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40af000 tx=0x7ff3d40193b0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d8005d50 msgr2=0x7ff3d817cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 --2- 172.21.15.170:0/1765811582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3d8005d50 0x7ff3d817cae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3e006aac0 tx=0x7ff3e0075500 comp rx=0 tx=0).stop 2024-09-21T02:13:36.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3dffff640 1 -- 172.21.15.170:0/1765811582 reap_dead start 2024-09-21T02:13:36.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 shutdown_connections 2024-09-21T02:13:36.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.802+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 >> 172.21.15.170:0/1765811582 conn(0x7ff3d8051420 msgr2=0x7ff3d804a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.804 INFO:teuthology.orchestra.run.smithi170.stdout:64424509448 2024-09-21T02:13:36.804 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T02:13:36.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.806+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 shutdown_connections 2024-09-21T02:13:36.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.806+0000 7ff3e5fbc640 1 -- 172.21.15.170:0/1765811582 wait complete. 2024-09-21T02:13:36.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.810+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 --> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bc8005ca0 con 0x7f4bc8001660 2024-09-21T02:13:36.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.810+0000 7f4bcf7fe640 1 -- 172.21.15.170:0/2624653433 <== osd.7 v2:172.21.15.191:6810/1521218345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4bc8005ca0 con 0x7f4bc8001660 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f4bc8001660 msgr2=0x7f4bc8003b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f4bc8001660 0x7f4bc8003b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4bd40731e0 msgr2=0x7f4bd40756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4bd40731e0 0x7f4bd40756f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0046c0 tx=0x7f4bdc007ec0 comp rx=0 tx=0).stop 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e0fa0 msgr2=0x7f4be01810f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 --2- 172.21.15.170:0/2624653433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be00e0fa0 0x7f4be01810f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8019380 tx=0x7f4bd80a6690 comp rx=0 tx=0).stop 2024-09-21T02:13:36.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be77fe640 1 -- 172.21.15.170:0/2624653433 reap_dead start 2024-09-21T02:13:36.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 shutdown_connections 2024-09-21T02:13:36.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.814+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 >> 172.21.15.170:0/2624653433 conn(0x7f4be001eaf0 msgr2=0x7f4be00372d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:36.817 INFO:teuthology.orchestra.run.smithi170.stdout:64424509447 2024-09-21T02:13:36.817 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T02:13:36.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.818+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 shutdown_connections 2024-09-21T02:13:36.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:36.818+0000 7f4be7fff640 1 -- 172.21.15.170:0/2624653433 wait complete. 2024-09-21T02:13:36.818 INFO:teuthology.orchestra.run.smithi170.stdout:60129542151 2024-09-21T02:13:36.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T02:13:36.835 INFO:teuthology.orchestra.run.smithi170.stdout:64424509447 2024-09-21T02:13:36.836 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T02:13:36.840 INFO:teuthology.orchestra.run.smithi170.stdout:64424509446 2024-09-21T02:13:36.840 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T02:13:36.842 INFO:teuthology.orchestra.run.smithi170.stdout:64424509449 2024-09-21T02:13:36.842 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T02:13:36.845 INFO:teuthology.orchestra.run.smithi170.stdout:64424509448 2024-09-21T02:13:36.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 -- 172.21.15.170:0/4088209992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70073b40 msgr2=0x7f4c70073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 --2- 172.21.15.170:0/4088209992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70073b40 0x7f4c70073f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c700724b0 tx=0x7f4c6801cd50 comp rx=0 tx=0).stop 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 -- 172.21.15.170:0/4088209992 shutdown_connections 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 --2- 172.21.15.170:0/4088209992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70073b40 0x7f4c70073f50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 --2- 172.21.15.170:0/4088209992 >> v2:172.21.15.191:3300/0 conn(0x7f4c701623a0 0x7f4c701627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 -- 172.21.15.170:0/4088209992 >> 172.21.15.170:0/4088209992 conn(0x7f4c700cffa0 msgr2=0x7f4c700d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 -- 172.21.15.170:0/4088209992 shutdown_connections 2024-09-21T02:13:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.010+0000 7f4c7573d640 1 -- 172.21.15.170:0/4088209992 wait complete. 2024-09-21T02:13:37.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 Processor -- start 2024-09-21T02:13:37.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 -- start start 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c70073b40 0x7f4c701523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 0x7f4c702001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c701623a0 0x7f4c701fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701702d0 con 0x7f4c70155210 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70170150 con 0x7f4c701623a0 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c7573d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4c70170450 con 0x7f4c70073b40 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 0x7f4c702001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 0x7f4c702001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54070/0 (socket says 172.21.15.170:54070) 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6effd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c70073b40 0x7f4c701523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 -- 172.21.15.170:0/1968978277 learned_addr learned my addr 172.21.15.170:0/1968978277 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6f7fe640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c701623a0 0x7f4c701fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 -- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c70073b40 msgr2=0x7f4c701523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c70073b40 0x7f4c701523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 -- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c701623a0 msgr2=0x7f4c701fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c701623a0 0x7f4c701fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.014+0000 7f4c6e7fc640 1 -- 172.21.15.170:0/1968978277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c700cba10 con 0x7f4c70155210 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c6effd640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c70073b40 0x7f4c701523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c6e7fc640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 0x7f4c702001c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c68000f90 tx=0x7f4c680a5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680b2040 con 0x7f4c70155210 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c7573d640 1 -- 172.21.15.170:0/1968978277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c700cbcf0 con 0x7f4c70155210 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c7573d640 1 -- 172.21.15.170:0/1968978277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701fce20 con 0x7f4c70155210 2024-09-21T02:13:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c680ae830 con 0x7f4c70155210 2024-09-21T02:13:37.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680aec60 con 0x7f4c70155210 2024-09-21T02:13:37.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.018+0000 7f4c7573d640 1 -- 172.21.15.170:0/1968978277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c500052a0 con 0x7f4c70155210 2024-09-21T02:13:37.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.030+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4c680c0030 con 0x7f4c70155210 2024-09-21T02:13:37.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.030+0000 7f4c577fe640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c58073420 0x7f4c58075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.030+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4c6812e5a0 con 0x7f4c70155210 2024-09-21T02:13:37.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.030+0000 7f4c6effd640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c58073420 0x7f4c58075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.030+0000 7f4c6effd640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c58073420 0x7f4c58075930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009f30 tx=0x7f4c640091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.030+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4c68016990 con 0x7f4c70155210 2024-09-21T02:13:37.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 -- 172.21.15.170:0/26555869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0b49e0 msgr2=0x7f9a0c16d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 --2- 172.21.15.170:0/26555869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c16d780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009bc0 tx=0x7f9a0401cac0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 -- 172.21.15.170:0/26555869 shutdown_connections 2024-09-21T02:13:37.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 --2- 172.21.15.170:0/26555869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c16d780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 -- 172.21.15.170:0/26555869 >> 172.21.15.170:0/26555869 conn(0x7f9a0c06ce00 msgr2=0x7f9a0c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 -- 172.21.15.170:0/26555869 shutdown_connections 2024-09-21T02:13:37.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.042+0000 7f9a13ba0640 1 -- 172.21.15.170:0/26555869 wait complete. 2024-09-21T02:13:37.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.046+0000 7f9a13ba0640 1 Processor -- start 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.046+0000 7f9a13ba0640 1 -- start start 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 0x7f9a0c1ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c14aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c15d0c0 0x7f9a0c1fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c170cc0 con 0x7f9a0c0b49e0 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c170b40 con 0x7f9a0c15d0c0 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c170e40 con 0x7f9a0c06ed60 2024-09-21T02:13:37.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 0x7f9a0c1ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1213d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c15d0c0 0x7f9a0c1fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 0x7f9a0c1ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52102/0 (socket says 172.21.15.170:52102) 2024-09-21T02:13:37.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 -- 172.21.15.170:0/2424605484 learned_addr learned my addr 172.21.15.170:0/2424605484 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1113b640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c14aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 -- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c15d0c0 msgr2=0x7f9a0c1fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c15d0c0 0x7f9a0c1fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 -- 172.21.15.170:0/2424605484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0c0b49e0 msgr2=0x7f9a0c14aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c14aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 -- 172.21.15.170:0/2424605484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c1fd4d0 con 0x7f9a0c06ed60 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1213d640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c15d0c0 0x7f9a0c1fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1113b640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c14aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:13:37.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a1193c640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 0x7f9a0c1ffbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c420 tx=0x7f9a0800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08017070 con 0x7f9a0c06ed60 2024-09-21T02:13:37.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 -- 172.21.15.170:0/2424605484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c2004c0 con 0x7f9a0c06ed60 2024-09-21T02:13:37.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a080053c0 con 0x7f9a0c06ed60 2024-09-21T02:13:37.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08003860 con 0x7f9a0c06ed60 2024-09-21T02:13:37.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.050+0000 7f9a13ba0640 1 -- 172.21.15.170:0/2424605484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c1fc790 con 0x7f9a0c06ed60 2024-09-21T02:13:37.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 -- 172.21.15.170:0/3115997317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 msgr2=0x7f202c162e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 --2- 172.21.15.170:0/3115997317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 0x7f202c162e70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2024009bc0 tx=0x7f202401cbc0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 -- 172.21.15.170:0/3115997317 shutdown_connections 2024-09-21T02:13:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 --2- 172.21.15.170:0/3115997317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 0x7f202c162e70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 -- 172.21.15.170:0/3115997317 >> 172.21.15.170:0/3115997317 conn(0x7f202c0d00f0 msgr2=0x7f202c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 -- 172.21.15.170:0/3115997317 shutdown_connections 2024-09-21T02:13:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c072ce0 con 0x7f9a0c06ed60 2024-09-21T02:13:37.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a0801a070 con 0x7f9a0c06ed60 2024-09-21T02:13:37.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f9a027fc640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99f0073230 0x7f99f0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9a080937d0 con 0x7f9a0c06ed60 2024-09-21T02:13:37.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 -- 172.21.15.170:0/3115997317 wait complete. 2024-09-21T02:13:37.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 Processor -- start 2024-09-21T02:13:37.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f9a1113b640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99f0073230 0x7f99f0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f9a1113b640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99f0073230 0x7f99f0075740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c15c1a0 tx=0x7f99fc007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.054+0000 7f20317c3640 1 -- start start 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20317c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 0x7f202c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20317c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 0x7f202c09ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20317c3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c15ed70 0x7f202c09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 0x7f202c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 0x7f202c1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54082/0 (socket says 172.21.15.170:54082) 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 -- 172.21.15.170:0/2966548582 learned_addr learned my addr 172.21.15.170:0/2966548582 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202a7fc640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 0x7f202c09ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f202c16f810 con 0x7f202c0cf740 2024-09-21T02:13:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f202c16f690 con 0x7f202c0cfb50 2024-09-21T02:13:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f202c16f990 con 0x7f202c15ed70 2024-09-21T02:13:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202b7fe640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c15ed70 0x7f202c09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 -- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c15ed70 msgr2=0x7f202c09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c15ed70 0x7f202c09cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 -- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 msgr2=0x7f202c09ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 0x7f202c09ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c0b4200 con 0x7f202c0cf740 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f202affd640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 0x7f202c1571a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f202000c440 tx=0x7f202000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020016020 con 0x7f202c0cf740 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2020012920 con 0x7f202c0cf740 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.058+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020012d70 con 0x7f202c0cf740 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c0b44e0 con 0x7f202c0cf740 2024-09-21T02:13:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c0b4aa0 con 0x7f202c0cf740 2024-09-21T02:13:37.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c16fe40 con 0x7f202c0cf740 2024-09-21T02:13:37.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2020019020 con 0x7f202c0cf740 2024-09-21T02:13:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f20137fe640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2018073230 0x7f2018075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2020093fc0 con 0x7f202c0cf740 2024-09-21T02:13:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.062+0000 7f202a7fc640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2018073230 0x7f2018075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.066+0000 7f202a7fc640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2018073230 0x7f2018075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f20240af000 tx=0x7f202401b400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.070+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2020020020 con 0x7f202c0cf740 2024-09-21T02:13:37.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.070+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9a0805e230 con 0x7f9a0c06ed60 2024-09-21T02:13:37.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 -- 172.21.15.170:0/286398255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 msgr2=0x7f05640d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 --2- 172.21.15.170:0/286398255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f05640d0990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f055c009bc0 tx=0x7f055c01cab0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 -- 172.21.15.170:0/286398255 shutdown_connections 2024-09-21T02:13:37.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 --2- 172.21.15.170:0/286398255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f05640d0990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 --2- 172.21.15.170:0/286398255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640ccaa0 0x7f05640cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 -- 172.21.15.170:0/286398255 >> 172.21.15.170:0/286398255 conn(0x7f05640b3ca0 msgr2=0x7f05640b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 -- 172.21.15.170:0/286398255 shutdown_connections 2024-09-21T02:13:37.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 -- 172.21.15.170:0/286398255 wait complete. 2024-09-21T02:13:37.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 -- 172.21.15.170:0/788326853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02300cfb90 msgr2=0x7f02300cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 --2- 172.21.15.170:0/788326853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02300cfb90 0x7f02300cffa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f023006eec0 tx=0x7f022001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 -- 172.21.15.170:0/788326853 shutdown_connections 2024-09-21T02:13:37.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 --2- 172.21.15.170:0/788326853 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f023015f100 0x7f023015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 --2- 172.21.15.170:0/788326853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02300cfb90 0x7f02300cffa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 --2- 172.21.15.170:0/788326853 >> v2:172.21.15.191:3300/0 conn(0x7f0230071c30 0x7f02300cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 -- 172.21.15.170:0/788326853 >> 172.21.15.170:0/788326853 conn(0x7f02300b2cc0 msgr2=0x7f02300b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 Processor -- start 2024-09-21T02:13:37.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f0237061640 1 -- 172.21.15.170:0/788326853 shutdown_connections 2024-09-21T02:13:37.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.074+0000 7f056ac09640 1 -- start start 2024-09-21T02:13:37.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640711a0 0x7f05640b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05640ccaa0 0x7f056415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f056415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0564169f30 con 0x7f05640d0580 2024-09-21T02:13:37.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0564169db0 con 0x7f05640ccaa0 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f056416a0b0 con 0x7f05640711a0 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0569406640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05640ccaa0 0x7f056415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0569c07640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640711a0 0x7f05640b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0569c07640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640711a0 0x7f05640b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52124/0 (socket says 172.21.15.170:52124) 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0569406640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05640ccaa0 0x7f056415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:39986/0 (socket says 172.21.15.170:39986) 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0569c07640 1 -- 172.21.15.170:0/1411486591 learned_addr learned my addr 172.21.15.170:0/1411486591 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f056415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 -- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640711a0 msgr2=0x7f05640b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640711a0 0x7f05640b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 -- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05640ccaa0 msgr2=0x7f056415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05640ccaa0 0x7f056415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 -- 172.21.15.170:0/1411486591 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05640b73d0 con 0x7f05640d0580 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056a408640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f056415ef40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05640b4b50 tx=0x7f055c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0237061640 1 -- 172.21.15.170:0/788326853 wait complete. 2024-09-21T02:13:37.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0237061640 1 Processor -- start 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f0237061640 1 -- start start 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0237061640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 0x7f02301fc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0b2040 con 0x7f05640d0580 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05640b76b0 con 0x7f05640d0580 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.078+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05640b7c70 con 0x7f05640d0580 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f055c0bb070 con 0x7f05640d0580 2024-09-21T02:13:37.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554001700 con 0x7f05640d0580 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 0x7f02301fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 0x7f02301fc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54138/0 (socket says 172.21.15.170:54138) 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 -- 172.21.15.170:0/4247797795 learned_addr learned my addr 172.21.15.170:0/4247797795 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02300cfb90 0x7f02301551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f023015f100 0x7f0230150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f023015c920 con 0x7f0230071c30 2024-09-21T02:13:37.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023585e640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02300cfb90 0x7f02301551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f055c004040 con 0x7f05640d0580 2024-09-21T02:13:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0236860640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f023015f100 0x7f0230150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f023015c7a0 con 0x7f02300cfb90 2024-09-21T02:13:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f023006ce00 con 0x7f023015f100 2024-09-21T02:13:37.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 -- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f023015f100 msgr2=0x7f0230150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f023015f100 0x7f0230150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 -- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02300cfb90 msgr2=0x7f02301551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02300cfb90 0x7f02301551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230206ba0 con 0x7f0230071c30 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f055a7fc640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05480732f0 0x7f0548075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f055c12e4d0 con 0x7f05640d0580 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f0569c07640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05480732f0 0x7f0548075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.082+0000 7f023605f640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 0x7f02301fc840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f022c00d6d0 tx=0x7f022c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0569c07640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05480732f0 0x7f0548075800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05540010a0 tx=0x7f0554001c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c017070 con 0x7f0230071c30 2024-09-21T02:13:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301546e0 con 0x7f0230071c30 2024-09-21T02:13:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0230154ca0 con 0x7f0230071c30 2024-09-21T02:13:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f022c009b10 con 0x7f0230071c30 2024-09-21T02:13:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c010460 con 0x7f0230071c30 2024-09-21T02:13:37.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c0052a0 con 0x7f05640d0580 2024-09-21T02:13:37.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f022c01a070 con 0x7f0230071c30 2024-09-21T02:13:37.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.086+0000 7f0226ffd640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0214073250 0x7f0214075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f022c094100 con 0x7f0230071c30 2024-09-21T02:13:37.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f023585e640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0214073250 0x7f0214075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f023585e640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0214073250 0x7f0214075760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02200af000 tx=0x7f0220019660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f55ed324640 1 -- 172.21.15.170:0/42831605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e80ae500 msgr2=0x7f55e8169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f55ed324640 1 --2- 172.21.15.170:0/42831605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e80ae500 0x7f55e8169870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009c80 tx=0x7f55e001cd10 comp rx=0 tx=0).stop 2024-09-21T02:13:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f55ed324640 1 -- 172.21.15.170:0/42831605 shutdown_connections 2024-09-21T02:13:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f55ed324640 1 --2- 172.21.15.170:0/42831605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e80ae500 0x7f55e8169870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f55ed324640 1 --2- 172.21.15.170:0/42831605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e806d510 0x7f55e80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f55ed324640 1 -- 172.21.15.170:0/42831605 >> 172.21.15.170:0/42831605 conn(0x7f55e809a9c0 msgr2=0x7f55e80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.090+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02180052a0 con 0x7f0230071c30 2024-09-21T02:13:37.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.094+0000 7f55ed324640 1 -- 172.21.15.170:0/42831605 shutdown_connections 2024-09-21T02:13:37.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.094+0000 7f55ed324640 1 -- 172.21.15.170:0/42831605 wait complete. 2024-09-21T02:13:37.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.094+0000 7f55ed324640 1 Processor -- start 2024-09-21T02:13:37.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.094+0000 7f55ed324640 1 -- start start 2024-09-21T02:13:37.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55ed324640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e80ae500 0x7f55e8209f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55ed324640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e8162e80 0x7f55e8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f055c0166d0 con 0x7f05640d0580 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55ed324640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 0x7f55e8071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e67fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e8162e80 0x7f55e8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e6ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e80ae500 0x7f55e8209f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e6ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e80ae500 0x7f55e8209f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:39998/0 (socket says 172.21.15.170:39998) 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e67fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e8162e80 0x7f55e8070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52146/0 (socket says 172.21.15.170:52146) 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 0x7f55e8071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 0x7f55e8071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54140/0 (socket says 172.21.15.170:54140) 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55ed324640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55e816c750 con 0x7f55e81596b0 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55ed324640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f55e816c5d0 con 0x7f55e80ae500 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55ed324640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f55e816c8d0 con 0x7f55e8162e80 2024-09-21T02:13:37.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e6ffd640 1 -- 172.21.15.170:0/1875367864 learned_addr learned my addr 172.21.15.170:0/1875367864 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 -- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e8162e80 msgr2=0x7f55e8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e8162e80 0x7f55e8070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 -- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e80ae500 msgr2=0x7f55e8209f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e80ae500 0x7f55e8209f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 -- 172.21.15.170:0/1875367864 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e82062b0 con 0x7f55e81596b0 2024-09-21T02:13:37.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.098+0000 7f55e77fe640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 0x7f55e8071200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f55e80d11d0 tx=0x7f55e00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.106+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f022c05ec40 con 0x7f0230071c30 2024-09-21T02:13:37.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.106+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00044d0 con 0x7f55e81596b0 2024-09-21T02:13:37.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.106+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e8206590 con 0x7f55e81596b0 2024-09-21T02:13:37.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.106+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e8206b50 con 0x7f55e81596b0 2024-09-21T02:13:37.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.106+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e00bb070 con 0x7f55e81596b0 2024-09-21T02:13:37.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c80052a0 con 0x7f55e81596b0 2024-09-21T02:13:37.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00a5d60 con 0x7f55e81596b0 2024-09-21T02:13:37.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 -- 172.21.15.170:0/3613145845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb540cd930 msgr2=0x7fbb5415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 --2- 172.21.15.170:0/3613145845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb540cd930 0x7fbb5415ef10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009c80 tx=0x7fbb4c01d7c0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 -- 172.21.15.170:0/3613145845 shutdown_connections 2024-09-21T02:13:37.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 --2- 172.21.15.170:0/3613145845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb540cd930 0x7fbb5415ef10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 --2- 172.21.15.170:0/3613145845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb540ccb50 0x7fbb540cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 -- 172.21.15.170:0/3613145845 >> 172.21.15.170:0/3613145845 conn(0x7fbb540b23e0 msgr2=0x7fbb540b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 -- 172.21.15.170:0/3613145845 shutdown_connections 2024-09-21T02:13:37.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7f987c756640 1 -- 172.21.15.170:0/4054352048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c035440 msgr2=0x7f986c0e0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7f987c756640 1 --2- 172.21.15.170:0/4054352048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c035440 0x7f986c0e0890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9870009870 tx=0x7f987001c810 comp rx=0 tx=0).stop 2024-09-21T02:13:37.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7f987c756640 1 -- 172.21.15.170:0/4054352048 shutdown_connections 2024-09-21T02:13:37.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7f987c756640 1 --2- 172.21.15.170:0/4054352048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c035440 0x7f986c0e0890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7f987c756640 1 -- 172.21.15.170:0/4054352048 >> 172.21.15.170:0/4054352048 conn(0x7f986c01eaf0 msgr2=0x7f986c03a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.110+0000 7fbb589cf640 1 -- 172.21.15.170:0/3613145845 wait complete. 2024-09-21T02:13:37.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7fbb589cf640 1 Processor -- start 2024-09-21T02:13:37.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.114+0000 7fbb589cf640 1 -- start start 2024-09-21T02:13:37.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f987c756640 1 -- 172.21.15.170:0/4054352048 shutdown_connections 2024-09-21T02:13:37.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f987c756640 1 -- 172.21.15.170:0/4054352048 wait complete. 2024-09-21T02:13:37.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f987c756640 1 Processor -- start 2024-09-21T02:13:37.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f987c756640 1 -- start start 2024-09-21T02:13:37.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55e0004670 con 0x7f55e81596b0 2024-09-21T02:13:37.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f55cf7fe640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55d40733a0 0x7f55d40758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.118+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f55e012f810 con 0x7f55e81596b0 2024-09-21T02:13:37.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f55e6ffd640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55d40733a0 0x7f55d40758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb589cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 0x7fbb541fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 0x7fbb541fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 0x7fbb541fd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54168/0 (socket says 172.21.15.170:54168) 2024-09-21T02:13:37.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 -- 172.21.15.170:0/1750711686 learned_addr learned my addr 172.21.15.170:0/1750711686 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f55e6ffd640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55d40733a0 0x7f55d40758b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0099f0 tx=0x7f55dc0090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb540ccb50 0x7fbb541535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54157280 0x7fbb54154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54170280 con 0x7fbb54070f30 2024-09-21T02:13:37.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb52ffd640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb540ccb50 0x7fbb541535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb53fff640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54157280 0x7fbb54154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987c756640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c0e1670 0x7f986c055eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54170100 con 0x7fbb54157280 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987c756640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f986c0e1a80 0x7f986c17d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987c756640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c17b7f0 0x7f986c0dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987c756640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0eeef0 con 0x7f986c17b7f0 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987c756640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0eed70 con 0x7f986c0e1a80 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987a4f2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c0e1670 0x7f986c055eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987a4f2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c0e1670 0x7f986c055eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52180/0 (socket says 172.21.15.170:52180) 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987a4f2640 1 -- 172.21.15.170:0/3122159813 learned_addr learned my addr 172.21.15.170:0/3122159813 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c17b7f0 0x7f986c0dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f9879cf1640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f986c0e1a80 0x7f986c17d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987c756640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f986c0ef070 con 0x7f986c0e1670 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbb54170400 con 0x7fbb540ccb50 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 -- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb540ccb50 msgr2=0x7fbb541535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb540ccb50 0x7fbb541535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 -- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54157280 msgr2=0x7fbb54154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54157280 0x7fbb54154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7fbb537fe640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb54157010 con 0x7fbb54070f30 2024-09-21T02:13:37.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.126+0000 7fbb537fe640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 0x7fbb541fd8a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c970 tx=0x7fbb4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 -- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c0e1670 msgr2=0x7f986c055eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c0e1670 0x7f986c055eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 -- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f986c0e1a80 msgr2=0x7f986c17d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f986c0e1a80 0x7f986c17d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 -- 172.21.15.170:0/3122159813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c0525b0 con 0x7f986c17b7f0 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f987acf3640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c17b7f0 0x7f986c0dc5e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f987406c6e0 tx=0x7f98740720e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.122+0000 7f9879cf1640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f986c0e1a80 0x7f986c17d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:37.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.126+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48004220 con 0x7fbb54070f30 2024-09-21T02:13:37.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.126+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f55e0016990 con 0x7f55e81596b0 2024-09-21T02:13:37.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.126+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb54156a50 con 0x7fbb54070f30 2024-09-21T02:13:37.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.126+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb54153f60 con 0x7fbb54070f30 2024-09-21T02:13:37.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.126+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb480079b0 con 0x7fbb54070f30 2024-09-21T02:13:37.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874090070 con 0x7f986c17b7f0 2024-09-21T02:13:37.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb480044b0 con 0x7fbb54070f30 2024-09-21T02:13:37.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c052870 con 0x7f986c17b7f0 2024-09-21T02:13:37.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb340052a0 con 0x7fbb54070f30 2024-09-21T02:13:37.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c17bc30 con 0x7f986c17b7f0 2024-09-21T02:13:37.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9874072a50 con 0x7f986c17b7f0 2024-09-21T02:13:37.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98740943b0 con 0x7f986c17b7f0 2024-09-21T02:13:37.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.130+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985c0052a0 con 0x7f986c17b7f0 2024-09-21T02:13:37.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.134+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9874093070 con 0x7f986c17b7f0 2024-09-21T02:13:37.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7f986affd640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9858073300 0x7f9858075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f987410f6c0 con 0x7f986c17b7f0 2024-09-21T02:13:37.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7f987a4f2640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9858073300 0x7f9858075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb48003600 con 0x7fbb54070f30 2024-09-21T02:13:37.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7f987a4f2640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9858073300 0x7f9858075810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98640046c0 tx=0x7f9864007f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7fbb3bfff640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbb40073230 0x7fbb40075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbb48094430 con 0x7fbb54070f30 2024-09-21T02:13:37.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7fbb52ffd640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbb40073230 0x7fbb40075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.138+0000 7fbb52ffd640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbb40073230 0x7fbb40075740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb540afbf0 tx=0x7fbb44009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.146+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f98740da190 con 0x7f986c17b7f0 2024-09-21T02:13:37.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.158+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbb48014030 con 0x7fbb54070f30 2024-09-21T02:13:37.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.370+0000 7f4c7573d640 1 -- 172.21.15.170:0/1968978277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4c50005590 con 0x7f4c70155210 2024-09-21T02:13:37.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.370+0000 7f4c577fe640 1 -- 172.21.15.170:0/1968978277 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c680f9070 con 0x7f4c70155210 2024-09-21T02:13:37.370 INFO:teuthology.orchestra.run.smithi170.stdout:64424509448 2024-09-21T02:13:37.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.374+0000 7f4c557fa640 1 -- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c58073420 msgr2=0x7f4c58075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.374+0000 7f4c557fa640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c58073420 0x7f4c58075930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009f30 tx=0x7f4c640091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.374+0000 7f4c557fa640 1 -- 172.21.15.170:0/1968978277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 msgr2=0x7f4c702001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.374+0000 7f4c557fa640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 0x7f4c702001c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c68000f90 tx=0x7f4c680a5c20 comp rx=0 tx=0).stop 2024-09-21T02:13:37.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 -- 172.21.15.170:0/1968978277 shutdown_connections 2024-09-21T02:13:37.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c58073420 0x7f4c58075930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c701623a0 0x7f4c701fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c70155210 0x7f4c702001c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 --2- 172.21.15.170:0/1968978277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c70073b40 0x7f4c701523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 -- 172.21.15.170:0/1968978277 >> 172.21.15.170:0/1968978277 conn(0x7f4c700cffa0 msgr2=0x7f4c701627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 -- 172.21.15.170:0/1968978277 shutdown_connections 2024-09-21T02:13:37.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.378+0000 7f4c557fa640 1 -- 172.21.15.170:0/1968978277 wait complete. 2024-09-21T02:13:37.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-09-21T02:13:37.403 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.410+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9a0c0b4fd0 con 0x7f9a0c06ed60 2024-09-21T02:13:37.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.410+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f202c0b0e00 con 0x7f202c0cf740 2024-09-21T02:13:37.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.410+0000 7f9a027fc640 1 -- 172.21.15.170:0/2424605484 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a08062c60 con 0x7f9a0c06ed60 2024-09-21T02:13:37.411 INFO:teuthology.orchestra.run.smithi170.stdout:64424509448 2024-09-21T02:13:37.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.410+0000 7f20137fe640 1 -- 172.21.15.170:0/2966548582 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f202005eae0 con 0x7f202c0cf740 2024-09-21T02:13:37.412 INFO:teuthology.orchestra.run.smithi170.stdout:64424509447 2024-09-21T02:13:37.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99f0073230 msgr2=0x7f99f0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99f0073230 0x7f99f0075740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c15c1a0 tx=0x7f99fc007ea0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 msgr2=0x7f9a0c1ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 0x7f9a0c1ffbe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c420 tx=0x7f9a0800c8e0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 shutdown_connections 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f99f0073230 0x7f99f0075740 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c15d0c0 0x7f9a0c1fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a0c0b49e0 0x7f9a0c14aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 --2- 172.21.15.170:0/2424605484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a0c06ed60 0x7f9a0c1ffbe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 >> 172.21.15.170:0/2424605484 conn(0x7f9a0c06ce00 msgr2=0x7f9a0c0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 shutdown_connections 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f99effff640 1 -- 172.21.15.170:0/2424605484 wait complete. 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2018073230 msgr2=0x7f2018075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f20317c3640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2018073230 0x7f2018075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f20240af000 tx=0x7f202401b400 comp rx=0 tx=0).stop 2024-09-21T02:13:37.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 msgr2=0x7f202c1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.414+0000 7f20317c3640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 0x7f202c1571a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f202000c440 tx=0x7f202000c900 comp rx=0 tx=0).stop 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 shutdown_connections 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2018073230 0x7f2018075740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c15ed70 0x7f202c09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c0cfb50 0x7f202c09ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 --2- 172.21.15.170:0/2966548582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c0cf740 0x7f202c1571a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 >> 172.21.15.170:0/2966548582 conn(0x7f202c0d00f0 msgr2=0x7f202c0b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 shutdown_connections 2024-09-21T02:13:37.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.418+0000 7f20317c3640 1 -- 172.21.15.170:0/2966548582 wait complete. 2024-09-21T02:13:37.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.430+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0218005860 con 0x7f0230071c30 2024-09-21T02:13:37.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.430+0000 7f0226ffd640 1 -- 172.21.15.170:0/4247797795 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f022c063670 con 0x7f0230071c30 2024-09-21T02:13:37.430 INFO:teuthology.orchestra.run.smithi170.stdout:64424509447 2024-09-21T02:13:37.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0214073250 msgr2=0x7f0214075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0214073250 0x7f0214075760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02200af000 tx=0x7f0220019660 comp rx=0 tx=0).stop 2024-09-21T02:13:37.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 msgr2=0x7f02301fc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 0x7f02301fc840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f022c00d6d0 tx=0x7f022c00db90 comp rx=0 tx=0).stop 2024-09-21T02:13:37.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 shutdown_connections 2024-09-21T02:13:37.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0214073250 0x7f0214075760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f023015f100 0x7f0230150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02300cfb90 0x7f02301551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 --2- 172.21.15.170:0/4247797795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0230071c30 0x7f02301fc840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 >> 172.21.15.170:0/4247797795 conn(0x7f02300b2cc0 msgr2=0x7f023006d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 shutdown_connections 2024-09-21T02:13:37.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f0237061640 1 -- 172.21.15.170:0/4247797795 wait complete. 2024-09-21T02:13:37.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-09-21T02:13:37.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f054c005590 con 0x7f05640d0580 2024-09-21T02:13:37.436 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.434+0000 7f055a7fc640 1 -- 172.21.15.170:0/1411486591 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f055c0f9030 con 0x7f05640d0580 2024-09-21T02:13:37.437 INFO:teuthology.orchestra.run.smithi170.stdout:60129542151 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05480732f0 msgr2=0x7f0548075800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05480732f0 0x7f0548075800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05540010a0 tx=0x7f0554001c00 comp rx=0 tx=0).stop 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 msgr2=0x7f056415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f056415ef40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05640b4b50 tx=0x7f055c009220 comp rx=0 tx=0).stop 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 shutdown_connections 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05480732f0 0x7f0548075800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05640d0580 0x7f056415ef40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05640ccaa0 0x7f056415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 --2- 172.21.15.170:0/1411486591 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05640711a0 0x7f05640b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 >> 172.21.15.170:0/1411486591 conn(0x7f05640b3ca0 msgr2=0x7f05640c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 shutdown_connections 2024-09-21T02:13:37.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.438+0000 7f056ac09640 1 -- 172.21.15.170:0/1411486591 wait complete. 2024-09-21T02:13:37.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-21T02:13:37.448 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-21T02:13:37.460 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2024-09-21T02:13:37.470 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.470+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f55c8005860 con 0x7f55e81596b0 2024-09-21T02:13:37.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55cf7fe640 1 -- 172.21.15.170:0/1875367864 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55e00fa1d0 con 0x7f55e81596b0 2024-09-21T02:13:37.473 INFO:teuthology.orchestra.run.smithi170.stdout:64424509446 2024-09-21T02:13:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55d40733a0 msgr2=0x7f55d40758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55d40733a0 0x7f55d40758b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0099f0 tx=0x7f55dc0090e0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 msgr2=0x7f55e8071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 0x7f55e8071200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f55e80d11d0 tx=0x7f55e00092c0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 shutdown_connections 2024-09-21T02:13:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55d40733a0 0x7f55d40758b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55e8162e80 0x7f55e8070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55e81596b0 0x7f55e8071200 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 --2- 172.21.15.170:0/1875367864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e80ae500 0x7f55e8209f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 >> 172.21.15.170:0/1875367864 conn(0x7f55e809a9c0 msgr2=0x7f55e8157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 shutdown_connections 2024-09-21T02:13:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.474+0000 7f55ed324640 1 -- 172.21.15.170:0/1875367864 wait complete. 2024-09-21T02:13:37.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.478+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f985c005860 con 0x7f986c17b7f0 2024-09-21T02:13:37.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.478+0000 7f986affd640 1 -- 172.21.15.170:0/3122159813 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f98740debc0 con 0x7f986c17b7f0 2024-09-21T02:13:37.480 INFO:teuthology.orchestra.run.smithi170.stdout:64424509448 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9858073300 msgr2=0x7f9858075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9858073300 0x7f9858075810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98640046c0 tx=0x7f9864007f60 comp rx=0 tx=0).stop 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c17b7f0 msgr2=0x7f986c0dc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c17b7f0 0x7f986c0dc5e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f987406c6e0 tx=0x7f98740720e0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 shutdown_connections 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9858073300 0x7f9858075810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c17b7f0 0x7f986c0dc5e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f986c0e1a80 0x7f986c17d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 --2- 172.21.15.170:0/3122159813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f986c0e1670 0x7f986c055eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 >> 172.21.15.170:0/3122159813 conn(0x7f986c01eaf0 msgr2=0x7f986c008090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 shutdown_connections 2024-09-21T02:13:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.482+0000 7f987c756640 1 -- 172.21.15.170:0/3122159813 wait complete. 2024-09-21T02:13:37.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.486+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbb34005860 con 0x7fbb54070f30 2024-09-21T02:13:37.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.486+0000 7fbb3bfff640 1 -- 172.21.15.170:0/1750711686 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbb4805ef20 con 0x7fbb54070f30 2024-09-21T02:13:37.487 INFO:teuthology.orchestra.run.smithi170.stdout:64424509449 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbb40073230 msgr2=0x7fbb40075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbb40073230 0x7fbb40075740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb540afbf0 tx=0x7fbb44009060 comp rx=0 tx=0).stop 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 msgr2=0x7fbb541fd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 0x7fbb541fd8a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c970 tx=0x7fbb4800ce30 comp rx=0 tx=0).stop 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 shutdown_connections 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbb40073230 0x7fbb40075740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb540ccb50 0x7fbb541535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb54157280 0x7fbb54154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 --2- 172.21.15.170:0/1750711686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb54070f30 0x7fbb541fd8a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 >> 172.21.15.170:0/1750711686 conn(0x7fbb540b23e0 msgr2=0x7fbb5416d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 shutdown_connections 2024-09-21T02:13:37.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.490+0000 7fbb589cf640 1 -- 172.21.15.170:0/1750711686 wait complete. 2024-09-21T02:13:37.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-09-21T02:13:37.496 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-09-21T02:13:37.504 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-09-21T02:13:37.509 DEBUG:teuthology.parallel:result is None 2024-09-21T02:13:37.509 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:13:37.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439c021640 1 -- 172.21.15.170:0/978431131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4380003a40 msgr2=0x7f4380003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439c021640 1 --2- 172.21.15.170:0/978431131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4380003a40 0x7f4380003eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f439000c2e0 tx=0x7f439001cf60 comp rx=0 tx=0).stop 2024-09-21T02:13:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439a5be640 1 -- 172.21.15.170:0/978431131 reap_dead start 2024-09-21T02:13:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439c021640 1 -- 172.21.15.170:0/978431131 shutdown_connections 2024-09-21T02:13:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439c021640 1 -- 172.21.15.170:0/978431131 >> 172.21.15.170:0/978431131 conn(0x7f43940b1c40 msgr2=0x7f43940b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439c021640 1 -- 172.21.15.170:0/978431131 shutdown_connections 2024-09-21T02:13:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.690+0000 7f439c021640 1 -- 172.21.15.170:0/978431131 wait complete. 2024-09-21T02:13:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 Processor -- start 2024-09-21T02:13:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 -- start start 2024-09-21T02:13:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4380003a40 0x7f4394159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43940ca230 0x7f43941fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 0x7f439406f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43941679a0 con 0x7f439406f080 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4394167820 con 0x7f4380003a40 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4394167b20 con 0x7f43940ca230 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 0x7f439406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 0x7f439406f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54194/0 (socket says 172.21.15.170:54194) 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 -- 172.21.15.170:0/444269605 learned_addr learned my addr 172.21.15.170:0/444269605 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f43995bc640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43940ca230 0x7f43941fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f4399dbd640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4380003a40 0x7f4394159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 -- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43940ca230 msgr2=0x7f43941fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43940ca230 0x7f43941fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 -- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4380003a40 msgr2=0x7f4394159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4380003a40 0x7f4394159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 -- 172.21.15.170:0/444269605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4394162360 con 0x7f439406f080 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439a5be640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 0x7f439406f490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43940b5250 tx=0x7f438c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c003980 con 0x7f439406f080 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f438c015070 con 0x7f439406f080 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4394162640 con 0x7f439406f080 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c004030 con 0x7f439406f080 2024-09-21T02:13:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.694+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439409d460 con 0x7f439406f080 2024-09-21T02:13:37.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.698+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43940cb870 con 0x7f439406f080 2024-09-21T02:13:37.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.698+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f438c0041d0 con 0x7f439406f080 2024-09-21T02:13:37.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.698+0000 7f438a7fc640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073200 0x7f4378075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:37.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.698+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f438c094b90 con 0x7f439406f080 2024-09-21T02:13:37.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.698+0000 7f4399dbd640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073200 0x7f4378075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:37.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.698+0000 7f4399dbd640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073200 0x7f4378075710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f43900ab000 tx=0x7f4390019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:37.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.702+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f438c05fcd0 con 0x7f439406f080 2024-09-21T02:13:37.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.858+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43940cce50 con 0x7f4378073200 2024-09-21T02:13:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.870+0000 7f438a7fc640 1 -- 172.21.15.170:0/444269605 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f43940cce50 con 0x7f4378073200 2024-09-21T02:13:37.874 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073200 msgr2=0x7f4378075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073200 0x7f4378075710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f43900ab000 tx=0x7f4390019da0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 msgr2=0x7f439406f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 0x7f439406f490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43940b5250 tx=0x7f438c010f40 comp rx=0 tx=0).stop 2024-09-21T02:13:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 shutdown_connections 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4378073200 0x7f4378075710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43940ca230 0x7f43941fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f439406f080 0x7f439406f490 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 --2- 172.21.15.170:0/444269605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4380003a40 0x7f4394159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 >> 172.21.15.170:0/444269605 conn(0x7f43940b1c40 msgr2=0x7f43940b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 shutdown_connections 2024-09-21T02:13:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:37.874+0000 7f439c021640 1 -- 172.21.15.170:0/444269605 wait complete. 2024-09-21T02:13:37.894 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T02:13:36.867365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244268,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730452,"statfs":{"total":773094113280,"available":772843982848,"internally_reserved":0,"allocated":3121152,"data_stored":1839580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004130"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964977+0000","last_change":"2024-09-21T02:13:25.918477+0000","last_active":"2024-09-21T02:13:26.964977+0000","last_peered":"2024-09-21T02:13:26.964977+0000","last_clean":"2024-09-21T02:13:26.964977+0000","last_became_active":"2024-09-21T02:13:25.917930+0000","last_became_peered":"2024-09-21T02:13:25.917930+0000","last_unstale":"2024-09-21T02:13:26.964977+0000","last_undegraded":"2024-09-21T02:13:26.964977+0000","last_fullsized":"2024-09-21T02:13:26.964977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:08.714546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883166+0000","last_change":"2024-09-21T02:13:25.893989+0000","last_active":"2024-09-21T02:13:26.883166+0000","last_peered":"2024-09-21T02:13:26.883166+0000","last_clean":"2024-09-21T02:13:26.883166+0000","last_became_active":"2024-09-21T02:13:25.893173+0000","last_became_peered":"2024-09-21T02:13:25.893173+0000","last_unstale":"2024-09-21T02:13:26.883166+0000","last_undegraded":"2024-09-21T02:13:26.883166+0000","last_fullsized":"2024-09-21T02:13:26.883166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:49.876135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879431+0000","last_change":"2024-09-21T02:13:25.932549+0000","last_active":"2024-09-21T02:13:26.879431+0000","last_peered":"2024-09-21T02:13:26.879431+0000","last_clean":"2024-09-21T02:13:26.879431+0000","last_became_active":"2024-09-21T02:13:25.931843+0000","last_became_peered":"2024-09-21T02:13:25.931843+0000","last_unstale":"2024-09-21T02:13:26.879431+0000","last_undegraded":"2024-09-21T02:13:26.879431+0000","last_fullsized":"2024-09-21T02:13:26.879431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:19.382874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879618+0000","last_change":"2024-09-21T02:13:25.941857+0000","last_active":"2024-09-21T02:13:26.879618+0000","last_peered":"2024-09-21T02:13:26.879618+0000","last_clean":"2024-09-21T02:13:26.879618+0000","last_became_active":"2024-09-21T02:13:25.941422+0000","last_became_peered":"2024-09-21T02:13:25.941422+0000","last_unstale":"2024-09-21T02:13:26.879618+0000","last_undegraded":"2024-09-21T02:13:26.879618+0000","last_fullsized":"2024-09-21T02:13:26.879618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:02.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965518+0000","last_change":"2024-09-21T02:13:25.911062+0000","last_active":"2024-09-21T02:13:26.965518+0000","last_peered":"2024-09-21T02:13:26.965518+0000","last_clean":"2024-09-21T02:13:26.965518+0000","last_became_active":"2024-09-21T02:13:25.899879+0000","last_became_peered":"2024-09-21T02:13:25.899879+0000","last_unstale":"2024-09-21T02:13:26.965518+0000","last_undegraded":"2024-09-21T02:13:26.965518+0000","last_fullsized":"2024-09-21T02:13:26.965518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:40.011102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878421+0000","last_change":"2024-09-21T02:13:25.942318+0000","last_active":"2024-09-21T02:13:26.878421+0000","last_peered":"2024-09-21T02:13:26.878421+0000","last_clean":"2024-09-21T02:13:26.878421+0000","last_became_active":"2024-09-21T02:13:25.941452+0000","last_became_peered":"2024-09-21T02:13:25.941452+0000","last_unstale":"2024-09-21T02:13:26.878421+0000","last_undegraded":"2024-09-21T02:13:26.878421+0000","last_fullsized":"2024-09-21T02:13:26.878421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:08.840148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963534+0000","last_change":"2024-09-21T02:13:25.942111+0000","last_active":"2024-09-21T02:13:26.963534+0000","last_peered":"2024-09-21T02:13:26.963534+0000","last_clean":"2024-09-21T02:13:26.963534+0000","last_became_active":"2024-09-21T02:13:25.941400+0000","last_became_peered":"2024-09-21T02:13:25.941400+0000","last_unstale":"2024-09-21T02:13:26.963534+0000","last_undegraded":"2024-09-21T02:13:26.963534+0000","last_fullsized":"2024-09-21T02:13:26.963534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878836+0000","last_change":"2024-09-21T02:13:25.914747+0000","last_active":"2024-09-21T02:13:26.878836+0000","last_peered":"2024-09-21T02:13:26.878836+0000","last_clean":"2024-09-21T02:13:26.878836+0000","last_became_active":"2024-09-21T02:13:25.914058+0000","last_became_peered":"2024-09-21T02:13:25.914058+0000","last_unstale":"2024-09-21T02:13:26.878836+0000","last_undegraded":"2024-09-21T02:13:26.878836+0000","last_fullsized":"2024-09-21T02:13:26.878836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:21.986443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967211+0000","last_change":"2024-09-21T02:13:25.909803+0000","last_active":"2024-09-21T02:13:26.967211+0000","last_peered":"2024-09-21T02:13:26.967211+0000","last_clean":"2024-09-21T02:13:26.967211+0000","last_became_active":"2024-09-21T02:13:25.909116+0000","last_became_peered":"2024-09-21T02:13:25.909116+0000","last_unstale":"2024-09-21T02:13:26.967211+0000","last_undegraded":"2024-09-21T02:13:26.967211+0000","last_fullsized":"2024-09-21T02:13:26.967211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:00.174710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965018+0000","last_change":"2024-09-21T02:13:25.915682+0000","last_active":"2024-09-21T02:13:26.965018+0000","last_peered":"2024-09-21T02:13:26.965018+0000","last_clean":"2024-09-21T02:13:26.965018+0000","last_became_active":"2024-09-21T02:13:25.915194+0000","last_became_peered":"2024-09-21T02:13:25.915194+0000","last_unstale":"2024-09-21T02:13:26.965018+0000","last_undegraded":"2024-09-21T02:13:26.965018+0000","last_fullsized":"2024-09-21T02:13:26.965018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.966309+0000","last_change":"2024-09-21T02:13:25.914960+0000","last_active":"2024-09-21T02:13:26.966309+0000","last_peered":"2024-09-21T02:13:26.966309+0000","last_clean":"2024-09-21T02:13:26.966309+0000","last_became_active":"2024-09-21T02:13:25.914610+0000","last_became_peered":"2024-09-21T02:13:25.914610+0000","last_unstale":"2024-09-21T02:13:26.966309+0000","last_undegraded":"2024-09-21T02:13:26.966309+0000","last_fullsized":"2024-09-21T02:13:26.966309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965589+0000","last_change":"2024-09-21T02:13:25.924430+0000","last_active":"2024-09-21T02:13:26.965589+0000","last_peered":"2024-09-21T02:13:26.965589+0000","last_clean":"2024-09-21T02:13:26.965589+0000","last_became_active":"2024-09-21T02:13:25.923881+0000","last_became_peered":"2024-09-21T02:13:25.923881+0000","last_unstale":"2024-09-21T02:13:26.965589+0000","last_undegraded":"2024-09-21T02:13:26.965589+0000","last_fullsized":"2024-09-21T02:13:26.965589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963608+0000","last_change":"2024-09-21T02:13:25.920280+0000","last_active":"2024-09-21T02:13:26.963608+0000","last_peered":"2024-09-21T02:13:26.963608+0000","last_clean":"2024-09-21T02:13:26.963608+0000","last_became_active":"2024-09-21T02:13:25.919389+0000","last_became_peered":"2024-09-21T02:13:25.919389+0000","last_unstale":"2024-09-21T02:13:26.963608+0000","last_undegraded":"2024-09-21T02:13:26.963608+0000","last_fullsized":"2024-09-21T02:13:26.963608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:40.761905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963494+0000","last_change":"2024-09-21T02:13:25.919658+0000","last_active":"2024-09-21T02:13:26.963494+0000","last_peered":"2024-09-21T02:13:26.963494+0000","last_clean":"2024-09-21T02:13:26.963494+0000","last_became_active":"2024-09-21T02:13:25.918063+0000","last_became_peered":"2024-09-21T02:13:25.918063+0000","last_unstale":"2024-09-21T02:13:26.963494+0000","last_undegraded":"2024-09-21T02:13:26.963494+0000","last_fullsized":"2024-09-21T02:13:26.963494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.069956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882761+0000","last_change":"2024-09-21T02:13:25.915232+0000","last_active":"2024-09-21T02:13:26.882761+0000","last_peered":"2024-09-21T02:13:26.882761+0000","last_clean":"2024-09-21T02:13:26.882761+0000","last_became_active":"2024-09-21T02:13:25.914387+0000","last_became_peered":"2024-09-21T02:13:25.914387+0000","last_unstale":"2024-09-21T02:13:26.882761+0000","last_undegraded":"2024-09-21T02:13:26.882761+0000","last_fullsized":"2024-09-21T02:13:26.882761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878878+0000","last_change":"2024-09-21T02:13:25.902781+0000","last_active":"2024-09-21T02:13:26.878878+0000","last_peered":"2024-09-21T02:13:26.878878+0000","last_clean":"2024-09-21T02:13:26.878878+0000","last_became_active":"2024-09-21T02:13:25.902445+0000","last_became_peered":"2024-09-21T02:13:25.902445+0000","last_unstale":"2024-09-21T02:13:26.878878+0000","last_undegraded":"2024-09-21T02:13:26.878878+0000","last_fullsized":"2024-09-21T02:13:26.878878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881922+0000","last_change":"2024-09-21T02:13:25.915843+0000","last_active":"2024-09-21T02:13:26.881922+0000","last_peered":"2024-09-21T02:13:26.881922+0000","last_clean":"2024-09-21T02:13:26.881922+0000","last_became_active":"2024-09-21T02:13:25.915560+0000","last_became_peered":"2024-09-21T02:13:25.915560+0000","last_unstale":"2024-09-21T02:13:26.881922+0000","last_undegraded":"2024-09-21T02:13:26.881922+0000","last_fullsized":"2024-09-21T02:13:26.881922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:41.816297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965307+0000","last_change":"2024-09-21T02:13:25.926096+0000","last_active":"2024-09-21T02:13:26.965307+0000","last_peered":"2024-09-21T02:13:26.965307+0000","last_clean":"2024-09-21T02:13:26.965307+0000","last_became_active":"2024-09-21T02:13:25.925408+0000","last_became_peered":"2024-09-21T02:13:25.925408+0000","last_unstale":"2024-09-21T02:13:26.965307+0000","last_undegraded":"2024-09-21T02:13:26.965307+0000","last_fullsized":"2024-09-21T02:13:26.965307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.244537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881981+0000","last_change":"2024-09-21T02:13:25.940050+0000","last_active":"2024-09-21T02:13:26.881981+0000","last_peered":"2024-09-21T02:13:26.881981+0000","last_clean":"2024-09-21T02:13:26.881981+0000","last_became_active":"2024-09-21T02:13:25.939688+0000","last_became_peered":"2024-09-21T02:13:25.939688+0000","last_unstale":"2024-09-21T02:13:26.881981+0000","last_undegraded":"2024-09-21T02:13:26.881981+0000","last_fullsized":"2024-09-21T02:13:26.881981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:47.887251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965669+0000","last_change":"2024-09-21T02:13:25.913323+0000","last_active":"2024-09-21T02:13:26.965669+0000","last_peered":"2024-09-21T02:13:26.965669+0000","last_clean":"2024-09-21T02:13:26.965669+0000","last_became_active":"2024-09-21T02:13:25.912945+0000","last_became_peered":"2024-09-21T02:13:25.912945+0000","last_unstale":"2024-09-21T02:13:26.965669+0000","last_undegraded":"2024-09-21T02:13:26.965669+0000","last_fullsized":"2024-09-21T02:13:26.965669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964449+0000","last_change":"2024-09-21T02:13:25.926590+0000","last_active":"2024-09-21T02:13:26.964449+0000","last_peered":"2024-09-21T02:13:26.964449+0000","last_clean":"2024-09-21T02:13:26.964449+0000","last_became_active":"2024-09-21T02:13:25.926154+0000","last_became_peered":"2024-09-21T02:13:25.926154+0000","last_unstale":"2024-09-21T02:13:26.964449+0000","last_undegraded":"2024-09-21T02:13:26.964449+0000","last_fullsized":"2024-09-21T02:13:26.964449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:00.641354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882548+0000","last_change":"2024-09-21T02:13:25.896024+0000","last_active":"2024-09-21T02:13:26.882548+0000","last_peered":"2024-09-21T02:13:26.882548+0000","last_clean":"2024-09-21T02:13:26.882548+0000","last_became_active":"2024-09-21T02:13:25.895216+0000","last_became_peered":"2024-09-21T02:13:25.895216+0000","last_unstale":"2024-09-21T02:13:26.882548+0000","last_undegraded":"2024-09-21T02:13:26.882548+0000","last_fullsized":"2024-09-21T02:13:26.882548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879525+0000","last_change":"2024-09-21T02:13:25.928846+0000","last_active":"2024-09-21T02:13:26.879525+0000","last_peered":"2024-09-21T02:13:26.879525+0000","last_clean":"2024-09-21T02:13:26.879525+0000","last_became_active":"2024-09-21T02:13:25.928527+0000","last_became_peered":"2024-09-21T02:13:25.928527+0000","last_unstale":"2024-09-21T02:13:26.879525+0000","last_undegraded":"2024-09-21T02:13:26.879525+0000","last_fullsized":"2024-09-21T02:13:26.879525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:38.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883432+0000","last_change":"2024-09-21T02:13:25.911388+0000","last_active":"2024-09-21T02:13:26.883432+0000","last_peered":"2024-09-21T02:13:26.883432+0000","last_clean":"2024-09-21T02:13:26.883432+0000","last_became_active":"2024-09-21T02:13:25.910631+0000","last_became_peered":"2024-09-21T02:13:25.910631+0000","last_unstale":"2024-09-21T02:13:26.883432+0000","last_undegraded":"2024-09-21T02:13:26.883432+0000","last_fullsized":"2024-09-21T02:13:26.883432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:01.276714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882752+0000","last_change":"2024-09-21T02:13:25.908215+0000","last_active":"2024-09-21T02:13:26.882752+0000","last_peered":"2024-09-21T02:13:26.882752+0000","last_clean":"2024-09-21T02:13:26.882752+0000","last_became_active":"2024-09-21T02:13:25.907764+0000","last_became_peered":"2024-09-21T02:13:25.907764+0000","last_unstale":"2024-09-21T02:13:26.882752+0000","last_undegraded":"2024-09-21T02:13:26.882752+0000","last_fullsized":"2024-09-21T02:13:26.882752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:31.448658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.918292+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.917267+0000","last_became_peered":"2024-09-21T02:13:25.917267+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:22.501288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963998+0000","last_change":"2024-09-21T02:13:25.918530+0000","last_active":"2024-09-21T02:13:26.963998+0000","last_peered":"2024-09-21T02:13:26.963998+0000","last_clean":"2024-09-21T02:13:26.963998+0000","last_became_active":"2024-09-21T02:13:25.917735+0000","last_became_peered":"2024-09-21T02:13:25.917735+0000","last_unstale":"2024-09-21T02:13:26.963998+0000","last_undegraded":"2024-09-21T02:13:26.963998+0000","last_fullsized":"2024-09-21T02:13:26.963998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:03.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880008+0000","last_change":"2024-09-21T02:13:25.913497+0000","last_active":"2024-09-21T02:13:26.880008+0000","last_peered":"2024-09-21T02:13:26.880008+0000","last_clean":"2024-09-21T02:13:26.880008+0000","last_became_active":"2024-09-21T02:13:25.912567+0000","last_became_peered":"2024-09-21T02:13:25.912567+0000","last_unstale":"2024-09-21T02:13:26.880008+0000","last_undegraded":"2024-09-21T02:13:26.880008+0000","last_fullsized":"2024-09-21T02:13:26.880008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879152+0000","last_change":"2024-09-21T02:13:25.917618+0000","last_active":"2024-09-21T02:13:26.879152+0000","last_peered":"2024-09-21T02:13:26.879152+0000","last_clean":"2024-09-21T02:13:26.879152+0000","last_became_active":"2024-09-21T02:13:25.916325+0000","last_became_peered":"2024-09-21T02:13:25.916325+0000","last_unstale":"2024-09-21T02:13:26.879152+0000","last_undegraded":"2024-09-21T02:13:26.879152+0000","last_fullsized":"2024-09-21T02:13:26.879152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.915861+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.912257+0000","last_became_peered":"2024-09-21T02:13:25.912257+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:56.675098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878068+0000","last_change":"2024-09-21T02:13:25.894915+0000","last_active":"2024-09-21T02:13:26.878068+0000","last_peered":"2024-09-21T02:13:26.878068+0000","last_clean":"2024-09-21T02:13:26.878068+0000","last_became_active":"2024-09-21T02:13:25.894440+0000","last_became_peered":"2024-09-21T02:13:25.894440+0000","last_unstale":"2024-09-21T02:13:26.878068+0000","last_undegraded":"2024-09-21T02:13:26.878068+0000","last_fullsized":"2024-09-21T02:13:26.878068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.857628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964697+0000","last_change":"2024-09-21T02:13:25.930047+0000","last_active":"2024-09-21T02:13:26.964697+0000","last_peered":"2024-09-21T02:13:26.964697+0000","last_clean":"2024-09-21T02:13:26.964697+0000","last_became_active":"2024-09-21T02:13:25.929729+0000","last_became_peered":"2024-09-21T02:13:25.929729+0000","last_unstale":"2024-09-21T02:13:26.964697+0000","last_undegraded":"2024-09-21T02:13:26.964697+0000","last_fullsized":"2024-09-21T02:13:26.964697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:26.319373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880194+0000","last_change":"2024-09-21T02:13:24.903640+0000","last_active":"2024-09-21T02:13:26.880194+0000","last_peered":"2024-09-21T02:13:26.880194+0000","last_clean":"2024-09-21T02:13:26.880194+0000","last_became_active":"2024-09-21T02:13:24.899880+0000","last_became_peered":"2024-09-21T02:13:24.899880+0000","last_unstale":"2024-09-21T02:13:26.880194+0000","last_undegraded":"2024-09-21T02:13:26.880194+0000","last_fullsized":"2024-09-21T02:13:26.880194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879277+0000","last_change":"2024-09-21T02:13:24.906393+0000","last_active":"2024-09-21T02:13:26.879277+0000","last_peered":"2024-09-21T02:13:26.879277+0000","last_clean":"2024-09-21T02:13:26.879277+0000","last_became_active":"2024-09-21T02:13:24.905410+0000","last_became_peered":"2024-09-21T02:13:24.905410+0000","last_unstale":"2024-09-21T02:13:26.879277+0000","last_undegraded":"2024-09-21T02:13:26.879277+0000","last_fullsized":"2024-09-21T02:13:26.879277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:13.262457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.962957+0000","last_change":"2024-09-21T02:13:24.909821+0000","last_active":"2024-09-21T02:13:26.962957+0000","last_peered":"2024-09-21T02:13:26.962957+0000","last_clean":"2024-09-21T02:13:26.962957+0000","last_became_active":"2024-09-21T02:13:24.907797+0000","last_became_peered":"2024-09-21T02:13:24.907797+0000","last_unstale":"2024-09-21T02:13:26.962957+0000","last_undegraded":"2024-09-21T02:13:26.962957+0000","last_fullsized":"2024-09-21T02:13:26.962957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879469+0000","last_change":"2024-09-21T02:13:25.927047+0000","last_active":"2024-09-21T02:13:26.879469+0000","last_peered":"2024-09-21T02:13:26.879469+0000","last_clean":"2024-09-21T02:13:26.879469+0000","last_became_active":"2024-09-21T02:13:25.926307+0000","last_became_peered":"2024-09-21T02:13:25.926307+0000","last_unstale":"2024-09-21T02:13:26.879469+0000","last_undegraded":"2024-09-21T02:13:26.879469+0000","last_fullsized":"2024-09-21T02:13:26.879469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879564+0000","last_change":"2024-09-21T02:13:24.896260+0000","last_active":"2024-09-21T02:13:26.879564+0000","last_peered":"2024-09-21T02:13:26.879564+0000","last_clean":"2024-09-21T02:13:26.879564+0000","last_became_active":"2024-09-21T02:13:24.895867+0000","last_became_peered":"2024-09-21T02:13:24.895867+0000","last_unstale":"2024-09-21T02:13:26.879564+0000","last_undegraded":"2024-09-21T02:13:26.879564+0000","last_fullsized":"2024-09-21T02:13:26.879564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:14.510200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967072+0000","last_change":"2024-09-21T02:13:24.899492+0000","last_active":"2024-09-21T02:13:26.967072+0000","last_peered":"2024-09-21T02:13:26.967072+0000","last_clean":"2024-09-21T02:13:26.967072+0000","last_became_active":"2024-09-21T02:13:24.899044+0000","last_became_peered":"2024-09-21T02:13:24.899044+0000","last_unstale":"2024-09-21T02:13:26.967072+0000","last_undegraded":"2024-09-21T02:13:26.967072+0000","last_fullsized":"2024-09-21T02:13:26.967072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882998+0000","last_change":"2024-09-21T02:13:25.908074+0000","last_active":"2024-09-21T02:13:26.882998+0000","last_peered":"2024-09-21T02:13:26.882998+0000","last_clean":"2024-09-21T02:13:26.882998+0000","last_became_active":"2024-09-21T02:13:25.907240+0000","last_became_peered":"2024-09-21T02:13:25.907240+0000","last_unstale":"2024-09-21T02:13:26.882998+0000","last_undegraded":"2024-09-21T02:13:26.882998+0000","last_fullsized":"2024-09-21T02:13:26.882998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.065966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965202+0000","last_change":"2024-09-21T02:13:24.898945+0000","last_active":"2024-09-21T02:13:26.965202+0000","last_peered":"2024-09-21T02:13:26.965202+0000","last_clean":"2024-09-21T02:13:26.965202+0000","last_became_active":"2024-09-21T02:13:24.897945+0000","last_became_peered":"2024-09-21T02:13:24.897945+0000","last_unstale":"2024-09-21T02:13:26.965202+0000","last_undegraded":"2024-09-21T02:13:26.965202+0000","last_fullsized":"2024-09-21T02:13:26.965202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:22.784881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883777+0000","last_change":"2024-09-21T02:13:25.919085+0000","last_active":"2024-09-21T02:13:26.883777+0000","last_peered":"2024-09-21T02:13:26.883777+0000","last_clean":"2024-09-21T02:13:26.883777+0000","last_became_active":"2024-09-21T02:13:25.918349+0000","last_became_peered":"2024-09-21T02:13:25.918349+0000","last_unstale":"2024-09-21T02:13:26.883777+0000","last_undegraded":"2024-09-21T02:13:26.883777+0000","last_fullsized":"2024-09-21T02:13:26.883777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965045+0000","last_change":"2024-09-21T02:13:24.919633+0000","last_active":"2024-09-21T02:13:26.965045+0000","last_peered":"2024-09-21T02:13:26.965045+0000","last_clean":"2024-09-21T02:13:26.965045+0000","last_became_active":"2024-09-21T02:13:24.918940+0000","last_became_peered":"2024-09-21T02:13:24.918940+0000","last_unstale":"2024-09-21T02:13:26.965045+0000","last_undegraded":"2024-09-21T02:13:26.965045+0000","last_fullsized":"2024-09-21T02:13:26.965045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880486+0000","last_change":"2024-09-21T02:13:25.901393+0000","last_active":"2024-09-21T02:13:26.880486+0000","last_peered":"2024-09-21T02:13:26.880486+0000","last_clean":"2024-09-21T02:13:26.880486+0000","last_became_active":"2024-09-21T02:13:25.901081+0000","last_became_peered":"2024-09-21T02:13:25.901081+0000","last_unstale":"2024-09-21T02:13:26.880486+0000","last_undegraded":"2024-09-21T02:13:26.880486+0000","last_fullsized":"2024-09-21T02:13:26.880486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:05.811740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879810+0000","last_change":"2024-09-21T02:13:24.901069+0000","last_active":"2024-09-21T02:13:26.879810+0000","last_peered":"2024-09-21T02:13:26.879810+0000","last_clean":"2024-09-21T02:13:26.879810+0000","last_became_active":"2024-09-21T02:13:24.900566+0000","last_became_peered":"2024-09-21T02:13:24.900566+0000","last_unstale":"2024-09-21T02:13:26.879810+0000","last_undegraded":"2024-09-21T02:13:26.879810+0000","last_fullsized":"2024-09-21T02:13:26.879810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:33.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964606+0000","last_change":"2024-09-21T02:13:25.923864+0000","last_active":"2024-09-21T02:13:26.964606+0000","last_peered":"2024-09-21T02:13:26.964606+0000","last_clean":"2024-09-21T02:13:26.964606+0000","last_became_active":"2024-09-21T02:13:25.923315+0000","last_became_peered":"2024-09-21T02:13:25.923315+0000","last_unstale":"2024-09-21T02:13:26.964606+0000","last_undegraded":"2024-09-21T02:13:26.964606+0000","last_fullsized":"2024-09-21T02:13:26.964606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965043+0000","last_change":"2024-09-21T02:13:24.918203+0000","last_active":"2024-09-21T02:13:26.965043+0000","last_peered":"2024-09-21T02:13:26.965043+0000","last_clean":"2024-09-21T02:13:26.965043+0000","last_became_active":"2024-09-21T02:13:24.907150+0000","last_became_peered":"2024-09-21T02:13:24.907150+0000","last_unstale":"2024-09-21T02:13:26.965043+0000","last_undegraded":"2024-09-21T02:13:26.965043+0000","last_fullsized":"2024-09-21T02:13:26.965043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963777+0000","last_change":"2024-09-21T02:13:25.906616+0000","last_active":"2024-09-21T02:13:26.963777+0000","last_peered":"2024-09-21T02:13:26.963777+0000","last_clean":"2024-09-21T02:13:26.963777+0000","last_became_active":"2024-09-21T02:13:25.905776+0000","last_became_peered":"2024-09-21T02:13:25.905776+0000","last_unstale":"2024-09-21T02:13:26.963777+0000","last_undegraded":"2024-09-21T02:13:26.963777+0000","last_fullsized":"2024-09-21T02:13:26.963777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:37.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879412+0000","last_change":"2024-09-21T02:13:24.906862+0000","last_active":"2024-09-21T02:13:26.879412+0000","last_peered":"2024-09-21T02:13:26.879412+0000","last_clean":"2024-09-21T02:13:26.879412+0000","last_became_active":"2024-09-21T02:13:24.905954+0000","last_became_peered":"2024-09-21T02:13:24.905954+0000","last_unstale":"2024-09-21T02:13:26.879412+0000","last_undegraded":"2024-09-21T02:13:26.879412+0000","last_fullsized":"2024-09-21T02:13:26.879412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:34.713354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880043+0000","last_change":"2024-09-21T02:13:25.919387+0000","last_active":"2024-09-21T02:13:26.880043+0000","last_peered":"2024-09-21T02:13:26.880043+0000","last_clean":"2024-09-21T02:13:26.880043+0000","last_became_active":"2024-09-21T02:13:25.919065+0000","last_became_peered":"2024-09-21T02:13:25.919065+0000","last_unstale":"2024-09-21T02:13:26.880043+0000","last_undegraded":"2024-09-21T02:13:26.880043+0000","last_fullsized":"2024-09-21T02:13:26.880043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:30.665668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963542+0000","last_change":"2024-09-21T02:13:24.906999+0000","last_active":"2024-09-21T02:13:26.963542+0000","last_peered":"2024-09-21T02:13:26.963542+0000","last_clean":"2024-09-21T02:13:26.963542+0000","last_became_active":"2024-09-21T02:13:24.897714+0000","last_became_peered":"2024-09-21T02:13:24.897714+0000","last_unstale":"2024-09-21T02:13:26.963542+0000","last_undegraded":"2024-09-21T02:13:26.963542+0000","last_fullsized":"2024-09-21T02:13:26.963542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963997+0000","last_change":"2024-09-21T02:13:25.928157+0000","last_active":"2024-09-21T02:13:26.963997+0000","last_peered":"2024-09-21T02:13:26.963997+0000","last_clean":"2024-09-21T02:13:26.963997+0000","last_became_active":"2024-09-21T02:13:25.927747+0000","last_became_peered":"2024-09-21T02:13:25.927747+0000","last_unstale":"2024-09-21T02:13:26.963997+0000","last_undegraded":"2024-09-21T02:13:26.963997+0000","last_fullsized":"2024-09-21T02:13:26.963997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:42.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878788+0000","last_change":"2024-09-21T02:13:25.902089+0000","last_active":"2024-09-21T02:13:26.878788+0000","last_peered":"2024-09-21T02:13:26.878788+0000","last_clean":"2024-09-21T02:13:26.878788+0000","last_became_active":"2024-09-21T02:13:25.901778+0000","last_became_peered":"2024-09-21T02:13:25.901778+0000","last_unstale":"2024-09-21T02:13:26.878788+0000","last_undegraded":"2024-09-21T02:13:26.878788+0000","last_fullsized":"2024-09-21T02:13:26.878788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965582+0000","last_change":"2024-09-21T02:13:24.914792+0000","last_active":"2024-09-21T02:13:26.965582+0000","last_peered":"2024-09-21T02:13:26.965582+0000","last_clean":"2024-09-21T02:13:26.965582+0000","last_became_active":"2024-09-21T02:13:24.914298+0000","last_became_peered":"2024-09-21T02:13:24.914298+0000","last_unstale":"2024-09-21T02:13:26.965582+0000","last_undegraded":"2024-09-21T02:13:26.965582+0000","last_fullsized":"2024-09-21T02:13:26.965582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883347+0000","last_change":"2024-09-21T02:13:25.924238+0000","last_active":"2024-09-21T02:13:26.883347+0000","last_peered":"2024-09-21T02:13:26.883347+0000","last_clean":"2024-09-21T02:13:26.883347+0000","last_became_active":"2024-09-21T02:13:25.922155+0000","last_became_peered":"2024-09-21T02:13:25.922155+0000","last_unstale":"2024-09-21T02:13:26.883347+0000","last_undegraded":"2024-09-21T02:13:26.883347+0000","last_fullsized":"2024-09-21T02:13:26.883347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:27.528134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965211+0000","last_change":"2024-09-21T02:13:25.924993+0000","last_active":"2024-09-21T02:13:26.965211+0000","last_peered":"2024-09-21T02:13:26.965211+0000","last_clean":"2024-09-21T02:13:26.965211+0000","last_became_active":"2024-09-21T02:13:25.924677+0000","last_became_peered":"2024-09-21T02:13:25.924677+0000","last_unstale":"2024-09-21T02:13:26.965211+0000","last_undegraded":"2024-09-21T02:13:26.965211+0000","last_fullsized":"2024-09-21T02:13:26.965211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879653+0000","last_change":"2024-09-21T02:13:24.893497+0000","last_active":"2024-09-21T02:13:26.879653+0000","last_peered":"2024-09-21T02:13:26.879653+0000","last_clean":"2024-09-21T02:13:26.879653+0000","last_became_active":"2024-09-21T02:13:24.892789+0000","last_became_peered":"2024-09-21T02:13:24.892789+0000","last_unstale":"2024-09-21T02:13:26.879653+0000","last_undegraded":"2024-09-21T02:13:26.879653+0000","last_fullsized":"2024-09-21T02:13:26.879653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965821+0000","last_change":"2024-09-21T02:13:25.927918+0000","last_active":"2024-09-21T02:13:26.965821+0000","last_peered":"2024-09-21T02:13:26.965821+0000","last_clean":"2024-09-21T02:13:26.965821+0000","last_became_active":"2024-09-21T02:13:25.927608+0000","last_became_peered":"2024-09-21T02:13:25.927608+0000","last_unstale":"2024-09-21T02:13:26.965821+0000","last_undegraded":"2024-09-21T02:13:26.965821+0000","last_fullsized":"2024-09-21T02:13:26.965821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964326+0000","last_change":"2024-09-21T02:13:24.904899+0000","last_active":"2024-09-21T02:13:26.964326+0000","last_peered":"2024-09-21T02:13:26.964326+0000","last_clean":"2024-09-21T02:13:26.964326+0000","last_became_active":"2024-09-21T02:13:24.904137+0000","last_became_peered":"2024-09-21T02:13:24.904137+0000","last_unstale":"2024-09-21T02:13:26.964326+0000","last_undegraded":"2024-09-21T02:13:26.964326+0000","last_fullsized":"2024-09-21T02:13:26.964326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:06.764948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881399+0000","last_change":"2024-09-21T02:13:25.908087+0000","last_active":"2024-09-21T02:13:26.881399+0000","last_peered":"2024-09-21T02:13:26.881399+0000","last_clean":"2024-09-21T02:13:26.881399+0000","last_became_active":"2024-09-21T02:13:25.907748+0000","last_became_peered":"2024-09-21T02:13:25.907748+0000","last_unstale":"2024-09-21T02:13:26.881399+0000","last_undegraded":"2024-09-21T02:13:26.881399+0000","last_fullsized":"2024-09-21T02:13:26.881399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:24.998552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964107+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:26.964107+0000","last_peered":"2024-09-21T02:13:26.964107+0000","last_clean":"2024-09-21T02:13:26.964107+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:26.964107+0000","last_undegraded":"2024-09-21T02:13:26.964107+0000","last_fullsized":"2024-09-21T02:13:26.964107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880025+0000","last_change":"2024-09-21T02:13:24.893632+0000","last_active":"2024-09-21T02:13:26.880025+0000","last_peered":"2024-09-21T02:13:26.880025+0000","last_clean":"2024-09-21T02:13:26.880025+0000","last_became_active":"2024-09-21T02:13:24.892990+0000","last_became_peered":"2024-09-21T02:13:24.892990+0000","last_unstale":"2024-09-21T02:13:26.880025+0000","last_undegraded":"2024-09-21T02:13:26.880025+0000","last_fullsized":"2024-09-21T02:13:26.880025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879715+0000","last_change":"2024-09-21T02:13:24.914198+0000","last_active":"2024-09-21T02:13:26.879715+0000","last_peered":"2024-09-21T02:13:26.879715+0000","last_clean":"2024-09-21T02:13:26.879715+0000","last_became_active":"2024-09-21T02:13:24.913815+0000","last_became_peered":"2024-09-21T02:13:24.913815+0000","last_unstale":"2024-09-21T02:13:26.879715+0000","last_undegraded":"2024-09-21T02:13:26.879715+0000","last_fullsized":"2024-09-21T02:13:26.879715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:35.245420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963392+0000","last_change":"2024-09-21T02:13:25.906344+0000","last_active":"2024-09-21T02:13:26.963392+0000","last_peered":"2024-09-21T02:13:26.963392+0000","last_clean":"2024-09-21T02:13:26.963392+0000","last_became_active":"2024-09-21T02:13:25.905571+0000","last_became_peered":"2024-09-21T02:13:25.905571+0000","last_unstale":"2024-09-21T02:13:26.963392+0000","last_undegraded":"2024-09-21T02:13:26.963392+0000","last_fullsized":"2024-09-21T02:13:26.963392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:01.266262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964760+0000","last_change":"2024-09-21T02:13:24.919598+0000","last_active":"2024-09-21T02:13:26.964760+0000","last_peered":"2024-09-21T02:13:26.964760+0000","last_clean":"2024-09-21T02:13:26.964760+0000","last_became_active":"2024-09-21T02:13:24.918909+0000","last_became_peered":"2024-09-21T02:13:24.918909+0000","last_unstale":"2024-09-21T02:13:26.964760+0000","last_undegraded":"2024-09-21T02:13:26.964760+0000","last_fullsized":"2024-09-21T02:13:26.964760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:19.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878944+0000","last_change":"2024-09-21T02:13:25.913216+0000","last_active":"2024-09-21T02:13:26.878944+0000","last_peered":"2024-09-21T02:13:26.878944+0000","last_clean":"2024-09-21T02:13:26.878944+0000","last_became_active":"2024-09-21T02:13:25.912412+0000","last_became_peered":"2024-09-21T02:13:25.912412+0000","last_unstale":"2024-09-21T02:13:26.878944+0000","last_undegraded":"2024-09-21T02:13:26.878944+0000","last_fullsized":"2024-09-21T02:13:26.878944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:16.249421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879536+0000","last_change":"2024-09-21T02:13:24.900311+0000","last_active":"2024-09-21T02:13:26.879536+0000","last_peered":"2024-09-21T02:13:26.879536+0000","last_clean":"2024-09-21T02:13:26.879536+0000","last_became_active":"2024-09-21T02:13:24.899845+0000","last_became_peered":"2024-09-21T02:13:24.899845+0000","last_unstale":"2024-09-21T02:13:26.879536+0000","last_undegraded":"2024-09-21T02:13:26.879536+0000","last_fullsized":"2024-09-21T02:13:26.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:34.513381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964099+0000","last_change":"2024-09-21T02:13:25.899982+0000","last_active":"2024-09-21T02:13:26.964099+0000","last_peered":"2024-09-21T02:13:26.964099+0000","last_clean":"2024-09-21T02:13:26.964099+0000","last_became_active":"2024-09-21T02:13:25.899671+0000","last_became_peered":"2024-09-21T02:13:25.899671+0000","last_unstale":"2024-09-21T02:13:26.964099+0000","last_undegraded":"2024-09-21T02:13:26.964099+0000","last_fullsized":"2024-09-21T02:13:26.964099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879852+0000","last_change":"2024-09-21T02:13:24.903776+0000","last_active":"2024-09-21T02:13:26.879852+0000","last_peered":"2024-09-21T02:13:26.879852+0000","last_clean":"2024-09-21T02:13:26.879852+0000","last_became_active":"2024-09-21T02:13:24.902681+0000","last_became_peered":"2024-09-21T02:13:24.902681+0000","last_unstale":"2024-09-21T02:13:26.879852+0000","last_undegraded":"2024-09-21T02:13:26.879852+0000","last_fullsized":"2024-09-21T02:13:26.879852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880542+0000","last_change":"2024-09-21T02:13:25.893632+0000","last_active":"2024-09-21T02:13:26.880542+0000","last_peered":"2024-09-21T02:13:26.880542+0000","last_clean":"2024-09-21T02:13:26.880542+0000","last_became_active":"2024-09-21T02:13:25.893137+0000","last_became_peered":"2024-09-21T02:13:25.893137+0000","last_unstale":"2024-09-21T02:13:26.880542+0000","last_undegraded":"2024-09-21T02:13:26.880542+0000","last_fullsized":"2024-09-21T02:13:26.880542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.630932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882169+0000","last_change":"2024-09-21T02:13:24.899031+0000","last_active":"2024-09-21T02:13:26.882169+0000","last_peered":"2024-09-21T02:13:26.882169+0000","last_clean":"2024-09-21T02:13:26.882169+0000","last_became_active":"2024-09-21T02:13:24.898526+0000","last_became_peered":"2024-09-21T02:13:24.898526+0000","last_unstale":"2024-09-21T02:13:26.882169+0000","last_undegraded":"2024-09-21T02:13:26.882169+0000","last_fullsized":"2024-09-21T02:13:26.882169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:15.323641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964112+0000","last_change":"2024-09-21T02:13:25.906716+0000","last_active":"2024-09-21T02:13:26.964112+0000","last_peered":"2024-09-21T02:13:26.964112+0000","last_clean":"2024-09-21T02:13:26.964112+0000","last_became_active":"2024-09-21T02:13:25.905889+0000","last_became_peered":"2024-09-21T02:13:25.905889+0000","last_unstale":"2024-09-21T02:13:26.964112+0000","last_undegraded":"2024-09-21T02:13:26.964112+0000","last_fullsized":"2024-09-21T02:13:26.964112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:56.985078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882102+0000","last_change":"2024-09-21T02:13:24.905877+0000","last_active":"2024-09-21T02:13:26.882102+0000","last_peered":"2024-09-21T02:13:26.882102+0000","last_clean":"2024-09-21T02:13:26.882102+0000","last_became_active":"2024-09-21T02:13:24.905486+0000","last_became_peered":"2024-09-21T02:13:24.905486+0000","last_unstale":"2024-09-21T02:13:26.882102+0000","last_undegraded":"2024-09-21T02:13:26.882102+0000","last_fullsized":"2024-09-21T02:13:26.882102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963169+0000","last_change":"2024-09-21T02:13:24.898279+0000","last_active":"2024-09-21T02:13:26.963169+0000","last_peered":"2024-09-21T02:13:26.963169+0000","last_clean":"2024-09-21T02:13:26.963169+0000","last_became_active":"2024-09-21T02:13:24.897369+0000","last_became_peered":"2024-09-21T02:13:24.897369+0000","last_unstale":"2024-09-21T02:13:26.963169+0000","last_undegraded":"2024-09-21T02:13:26.963169+0000","last_fullsized":"2024-09-21T02:13:26.963169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879396+0000","last_change":"2024-09-21T02:13:25.921611+0000","last_active":"2024-09-21T02:13:26.879396+0000","last_peered":"2024-09-21T02:13:26.879396+0000","last_clean":"2024-09-21T02:13:26.879396+0000","last_became_active":"2024-09-21T02:13:25.921165+0000","last_became_peered":"2024-09-21T02:13:25.921165+0000","last_unstale":"2024-09-21T02:13:26.879396+0000","last_undegraded":"2024-09-21T02:13:26.879396+0000","last_fullsized":"2024-09-21T02:13:26.879396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963689+0000","last_change":"2024-09-21T02:13:24.913917+0000","last_active":"2024-09-21T02:13:26.963689+0000","last_peered":"2024-09-21T02:13:26.963689+0000","last_clean":"2024-09-21T02:13:26.963689+0000","last_became_active":"2024-09-21T02:13:24.913151+0000","last_became_peered":"2024-09-21T02:13:24.913151+0000","last_unstale":"2024-09-21T02:13:26.963689+0000","last_undegraded":"2024-09-21T02:13:26.963689+0000","last_fullsized":"2024-09-21T02:13:26.963689+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:57.921560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965191+0000","last_change":"2024-09-21T02:13:25.937217+0000","last_active":"2024-09-21T02:13:26.965191+0000","last_peered":"2024-09-21T02:13:26.965191+0000","last_clean":"2024-09-21T02:13:26.965191+0000","last_became_active":"2024-09-21T02:13:25.936656+0000","last_became_peered":"2024-09-21T02:13:25.936656+0000","last_unstale":"2024-09-21T02:13:26.965191+0000","last_undegraded":"2024-09-21T02:13:26.965191+0000","last_fullsized":"2024-09-21T02:13:26.965191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:55.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879890+0000","last_change":"2024-09-21T02:13:24.893809+0000","last_active":"2024-09-21T02:13:26.879890+0000","last_peered":"2024-09-21T02:13:26.879890+0000","last_clean":"2024-09-21T02:13:26.879890+0000","last_became_active":"2024-09-21T02:13:24.893342+0000","last_became_peered":"2024-09-21T02:13:24.893342+0000","last_unstale":"2024-09-21T02:13:26.879890+0000","last_undegraded":"2024-09-21T02:13:26.879890+0000","last_fullsized":"2024-09-21T02:13:26.879890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880297+0000","last_change":"2024-09-21T02:13:25.909794+0000","last_active":"2024-09-21T02:13:26.880297+0000","last_peered":"2024-09-21T02:13:26.880297+0000","last_clean":"2024-09-21T02:13:26.880297+0000","last_became_active":"2024-09-21T02:13:25.908961+0000","last_became_peered":"2024-09-21T02:13:25.908961+0000","last_unstale":"2024-09-21T02:13:26.880297+0000","last_undegraded":"2024-09-21T02:13:26.880297+0000","last_fullsized":"2024-09-21T02:13:26.880297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:37.003277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880824+0000","last_change":"2024-09-21T02:13:25.915575+0000","last_active":"2024-09-21T02:13:26.880824+0000","last_peered":"2024-09-21T02:13:26.880824+0000","last_clean":"2024-09-21T02:13:26.880824+0000","last_became_active":"2024-09-21T02:13:25.915035+0000","last_became_peered":"2024-09-21T02:13:25.915035+0000","last_unstale":"2024-09-21T02:13:26.880824+0000","last_undegraded":"2024-09-21T02:13:26.880824+0000","last_fullsized":"2024-09-21T02:13:26.880824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:42.771535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965874+0000","last_change":"2024-09-21T02:13:24.903167+0000","last_active":"2024-09-21T02:13:26.965874+0000","last_peered":"2024-09-21T02:13:26.965874+0000","last_clean":"2024-09-21T02:13:26.965874+0000","last_became_active":"2024-09-21T02:13:24.902698+0000","last_became_peered":"2024-09-21T02:13:24.902698+0000","last_unstale":"2024-09-21T02:13:26.965874+0000","last_undegraded":"2024-09-21T02:13:26.965874+0000","last_fullsized":"2024-09-21T02:13:26.965874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964514+0000","last_change":"2024-09-21T02:13:25.925357+0000","last_active":"2024-09-21T02:13:26.964514+0000","last_peered":"2024-09-21T02:13:26.964514+0000","last_clean":"2024-09-21T02:13:26.964514+0000","last_became_active":"2024-09-21T02:13:25.924899+0000","last_became_peered":"2024-09-21T02:13:25.924899+0000","last_unstale":"2024-09-21T02:13:26.964514+0000","last_undegraded":"2024-09-21T02:13:26.964514+0000","last_fullsized":"2024-09-21T02:13:26.964514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:35.680962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964793+0000","last_change":"2024-09-21T02:13:24.898153+0000","last_active":"2024-09-21T02:13:26.964793+0000","last_peered":"2024-09-21T02:13:26.964793+0000","last_clean":"2024-09-21T02:13:26.964793+0000","last_became_active":"2024-09-21T02:13:24.897451+0000","last_became_peered":"2024-09-21T02:13:24.897451+0000","last_unstale":"2024-09-21T02:13:26.964793+0000","last_undegraded":"2024-09-21T02:13:26.964793+0000","last_fullsized":"2024-09-21T02:13:26.964793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883658+0000","last_change":"2024-09-21T02:13:25.911108+0000","last_active":"2024-09-21T02:13:26.883658+0000","last_peered":"2024-09-21T02:13:26.883658+0000","last_clean":"2024-09-21T02:13:26.883658+0000","last_became_active":"2024-09-21T02:13:25.910368+0000","last_became_peered":"2024-09-21T02:13:25.910368+0000","last_unstale":"2024-09-21T02:13:26.883658+0000","last_undegraded":"2024-09-21T02:13:26.883658+0000","last_fullsized":"2024-09-21T02:13:26.883658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883288+0000","last_change":"2024-09-21T02:13:24.905477+0000","last_active":"2024-09-21T02:13:26.883288+0000","last_peered":"2024-09-21T02:13:26.883288+0000","last_clean":"2024-09-21T02:13:26.883288+0000","last_became_active":"2024-09-21T02:13:24.904718+0000","last_became_peered":"2024-09-21T02:13:24.904718+0000","last_unstale":"2024-09-21T02:13:26.883288+0000","last_undegraded":"2024-09-21T02:13:26.883288+0000","last_fullsized":"2024-09-21T02:13:26.883288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883454+0000","last_change":"2024-09-21T02:13:25.897102+0000","last_active":"2024-09-21T02:13:26.883454+0000","last_peered":"2024-09-21T02:13:26.883454+0000","last_clean":"2024-09-21T02:13:26.883454+0000","last_became_active":"2024-09-21T02:13:25.896342+0000","last_became_peered":"2024-09-21T02:13:25.896342+0000","last_unstale":"2024-09-21T02:13:26.883454+0000","last_undegraded":"2024-09-21T02:13:26.883454+0000","last_fullsized":"2024-09-21T02:13:26.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":81753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":835584,"data_stored":672213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":237568,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":831488,"data_stored":672191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:37.898 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 -- 172.21.15.170:0/3122905491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80b3fa0 msgr2=0x7f40b80cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/3122905491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80b3fa0 0x7f40b80cf970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009bc0 tx=0x7f40b001cae0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 -- 172.21.15.170:0/3122905491 shutdown_connections 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/3122905491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80d0e50 0x7f40b80d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/3122905491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80b3fa0 0x7f40b80cf970 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 -- 172.21.15.170:0/3122905491 >> 172.21.15.170:0/3122905491 conn(0x7f40b808bf50 msgr2=0x7f40b809d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 -- 172.21.15.170:0/3122905491 shutdown_connections 2024-09-21T02:13:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 -- 172.21.15.170:0/3122905491 wait complete. 2024-09-21T02:13:38.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.078+0000 7f40bcf2b640 1 Processor -- start 2024-09-21T02:13:38.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- start start 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80b3fa0 0x7f40b81596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80d0e50 0x7f40b8157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 0x7f40b80b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40b816c390 con 0x7f40b80d2440 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40b816c210 con 0x7f40b80d0e50 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f40b816c510 con 0x7f40b80b3fa0 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 0x7f40b80b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 0x7f40b80b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54222/0 (socket says 172.21.15.170:54222) 2024-09-21T02:13:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 -- 172.21.15.170:0/317516908 learned_addr learned my addr 172.21.15.170:0/317516908 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b659c640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80b3fa0 0x7f40b81596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b5d9b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80d0e50 0x7f40b8157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 -- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80b3fa0 msgr2=0x7f40b81596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80b3fa0 0x7f40b81596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 -- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80d0e50 msgr2=0x7f40b8157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80d0e50 0x7f40b8157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 -- 172.21.15.170:0/317516908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80cab90 con 0x7f40b80d2440 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b659c640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80b3fa0 0x7f40b81596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b5d9b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80d0e50 0x7f40b8157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40b6d9d640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 0x7f40b80b8380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c3f0 tx=0x7f40a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8017070 con 0x7f40b80d2440 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b815cd10 con 0x7f40b80d2440 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40a8003600 con 0x7f40b80d2440 2024-09-21T02:13:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003a50 con 0x7f40b80d2440 2024-09-21T02:13:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b80cae70 con 0x7f40b80d2440 2024-09-21T02:13:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b80cf970 con 0x7f40b80d2440 2024-09-21T02:13:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.082+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f40a801a070 con 0x7f40b80d2440 2024-09-21T02:13:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.086+0000 7f40a6ffd640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4098073230 0x7f4098075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.086+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f40a80934c0 con 0x7f40b80d2440 2024-09-21T02:13:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.086+0000 7f40b659c640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4098073230 0x7f4098075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.086+0000 7f40b659c640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4098073230 0x7f4098075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40b00af000 tx=0x7f40b001b320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.090+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f40b80cfeb0 con 0x7f40b80d2440 2024-09-21T02:13:38.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.246+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40b80ca940 con 0x7f4098073230 2024-09-21T02:13:38.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.258+0000 7f40a6ffd640 1 -- 172.21.15.170:0/317516908 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f40b80ca940 con 0x7f4098073230 2024-09-21T02:13:38.260 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:38.262 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:13:38.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4098073230 msgr2=0x7f4098075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4098073230 0x7f4098075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40b00af000 tx=0x7f40b001b320 comp rx=0 tx=0).stop 2024-09-21T02:13:38.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 msgr2=0x7f40b80b8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 0x7f40b80b8380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c3f0 tx=0x7f40a800c7f0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 shutdown_connections 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4098073230 0x7f4098075740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80d2440 0x7f40b80b8380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80d0e50 0x7f40b8157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 --2- 172.21.15.170:0/317516908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80b3fa0 0x7f40b81596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 >> 172.21.15.170:0/317516908 conn(0x7f40b808bf50 msgr2=0x7f40b815c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 shutdown_connections 2024-09-21T02:13:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.262+0000 7f40bcf2b640 1 -- 172.21.15.170:0/317516908 wait complete. 2024-09-21T02:13:38.281 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T02:13:36.867365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244268,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730452,"statfs":{"total":773094113280,"available":772843982848,"internally_reserved":0,"allocated":3121152,"data_stored":1839580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004130"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964977+0000","last_change":"2024-09-21T02:13:25.918477+0000","last_active":"2024-09-21T02:13:26.964977+0000","last_peered":"2024-09-21T02:13:26.964977+0000","last_clean":"2024-09-21T02:13:26.964977+0000","last_became_active":"2024-09-21T02:13:25.917930+0000","last_became_peered":"2024-09-21T02:13:25.917930+0000","last_unstale":"2024-09-21T02:13:26.964977+0000","last_undegraded":"2024-09-21T02:13:26.964977+0000","last_fullsized":"2024-09-21T02:13:26.964977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:08.714546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883166+0000","last_change":"2024-09-21T02:13:25.893989+0000","last_active":"2024-09-21T02:13:26.883166+0000","last_peered":"2024-09-21T02:13:26.883166+0000","last_clean":"2024-09-21T02:13:26.883166+0000","last_became_active":"2024-09-21T02:13:25.893173+0000","last_became_peered":"2024-09-21T02:13:25.893173+0000","last_unstale":"2024-09-21T02:13:26.883166+0000","last_undegraded":"2024-09-21T02:13:26.883166+0000","last_fullsized":"2024-09-21T02:13:26.883166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:49.876135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879431+0000","last_change":"2024-09-21T02:13:25.932549+0000","last_active":"2024-09-21T02:13:26.879431+0000","last_peered":"2024-09-21T02:13:26.879431+0000","last_clean":"2024-09-21T02:13:26.879431+0000","last_became_active":"2024-09-21T02:13:25.931843+0000","last_became_peered":"2024-09-21T02:13:25.931843+0000","last_unstale":"2024-09-21T02:13:26.879431+0000","last_undegraded":"2024-09-21T02:13:26.879431+0000","last_fullsized":"2024-09-21T02:13:26.879431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:19.382874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879618+0000","last_change":"2024-09-21T02:13:25.941857+0000","last_active":"2024-09-21T02:13:26.879618+0000","last_peered":"2024-09-21T02:13:26.879618+0000","last_clean":"2024-09-21T02:13:26.879618+0000","last_became_active":"2024-09-21T02:13:25.941422+0000","last_became_peered":"2024-09-21T02:13:25.941422+0000","last_unstale":"2024-09-21T02:13:26.879618+0000","last_undegraded":"2024-09-21T02:13:26.879618+0000","last_fullsized":"2024-09-21T02:13:26.879618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:02.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965518+0000","last_change":"2024-09-21T02:13:25.911062+0000","last_active":"2024-09-21T02:13:26.965518+0000","last_peered":"2024-09-21T02:13:26.965518+0000","last_clean":"2024-09-21T02:13:26.965518+0000","last_became_active":"2024-09-21T02:13:25.899879+0000","last_became_peered":"2024-09-21T02:13:25.899879+0000","last_unstale":"2024-09-21T02:13:26.965518+0000","last_undegraded":"2024-09-21T02:13:26.965518+0000","last_fullsized":"2024-09-21T02:13:26.965518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:40.011102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878421+0000","last_change":"2024-09-21T02:13:25.942318+0000","last_active":"2024-09-21T02:13:26.878421+0000","last_peered":"2024-09-21T02:13:26.878421+0000","last_clean":"2024-09-21T02:13:26.878421+0000","last_became_active":"2024-09-21T02:13:25.941452+0000","last_became_peered":"2024-09-21T02:13:25.941452+0000","last_unstale":"2024-09-21T02:13:26.878421+0000","last_undegraded":"2024-09-21T02:13:26.878421+0000","last_fullsized":"2024-09-21T02:13:26.878421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:08.840148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963534+0000","last_change":"2024-09-21T02:13:25.942111+0000","last_active":"2024-09-21T02:13:26.963534+0000","last_peered":"2024-09-21T02:13:26.963534+0000","last_clean":"2024-09-21T02:13:26.963534+0000","last_became_active":"2024-09-21T02:13:25.941400+0000","last_became_peered":"2024-09-21T02:13:25.941400+0000","last_unstale":"2024-09-21T02:13:26.963534+0000","last_undegraded":"2024-09-21T02:13:26.963534+0000","last_fullsized":"2024-09-21T02:13:26.963534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878836+0000","last_change":"2024-09-21T02:13:25.914747+0000","last_active":"2024-09-21T02:13:26.878836+0000","last_peered":"2024-09-21T02:13:26.878836+0000","last_clean":"2024-09-21T02:13:26.878836+0000","last_became_active":"2024-09-21T02:13:25.914058+0000","last_became_peered":"2024-09-21T02:13:25.914058+0000","last_unstale":"2024-09-21T02:13:26.878836+0000","last_undegraded":"2024-09-21T02:13:26.878836+0000","last_fullsized":"2024-09-21T02:13:26.878836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:21.986443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967211+0000","last_change":"2024-09-21T02:13:25.909803+0000","last_active":"2024-09-21T02:13:26.967211+0000","last_peered":"2024-09-21T02:13:26.967211+0000","last_clean":"2024-09-21T02:13:26.967211+0000","last_became_active":"2024-09-21T02:13:25.909116+0000","last_became_peered":"2024-09-21T02:13:25.909116+0000","last_unstale":"2024-09-21T02:13:26.967211+0000","last_undegraded":"2024-09-21T02:13:26.967211+0000","last_fullsized":"2024-09-21T02:13:26.967211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:00.174710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965018+0000","last_change":"2024-09-21T02:13:25.915682+0000","last_active":"2024-09-21T02:13:26.965018+0000","last_peered":"2024-09-21T02:13:26.965018+0000","last_clean":"2024-09-21T02:13:26.965018+0000","last_became_active":"2024-09-21T02:13:25.915194+0000","last_became_peered":"2024-09-21T02:13:25.915194+0000","last_unstale":"2024-09-21T02:13:26.965018+0000","last_undegraded":"2024-09-21T02:13:26.965018+0000","last_fullsized":"2024-09-21T02:13:26.965018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.966309+0000","last_change":"2024-09-21T02:13:25.914960+0000","last_active":"2024-09-21T02:13:26.966309+0000","last_peered":"2024-09-21T02:13:26.966309+0000","last_clean":"2024-09-21T02:13:26.966309+0000","last_became_active":"2024-09-21T02:13:25.914610+0000","last_became_peered":"2024-09-21T02:13:25.914610+0000","last_unstale":"2024-09-21T02:13:26.966309+0000","last_undegraded":"2024-09-21T02:13:26.966309+0000","last_fullsized":"2024-09-21T02:13:26.966309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965589+0000","last_change":"2024-09-21T02:13:25.924430+0000","last_active":"2024-09-21T02:13:26.965589+0000","last_peered":"2024-09-21T02:13:26.965589+0000","last_clean":"2024-09-21T02:13:26.965589+0000","last_became_active":"2024-09-21T02:13:25.923881+0000","last_became_peered":"2024-09-21T02:13:25.923881+0000","last_unstale":"2024-09-21T02:13:26.965589+0000","last_undegraded":"2024-09-21T02:13:26.965589+0000","last_fullsized":"2024-09-21T02:13:26.965589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963608+0000","last_change":"2024-09-21T02:13:25.920280+0000","last_active":"2024-09-21T02:13:26.963608+0000","last_peered":"2024-09-21T02:13:26.963608+0000","last_clean":"2024-09-21T02:13:26.963608+0000","last_became_active":"2024-09-21T02:13:25.919389+0000","last_became_peered":"2024-09-21T02:13:25.919389+0000","last_unstale":"2024-09-21T02:13:26.963608+0000","last_undegraded":"2024-09-21T02:13:26.963608+0000","last_fullsized":"2024-09-21T02:13:26.963608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:40.761905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963494+0000","last_change":"2024-09-21T02:13:25.919658+0000","last_active":"2024-09-21T02:13:26.963494+0000","last_peered":"2024-09-21T02:13:26.963494+0000","last_clean":"2024-09-21T02:13:26.963494+0000","last_became_active":"2024-09-21T02:13:25.918063+0000","last_became_peered":"2024-09-21T02:13:25.918063+0000","last_unstale":"2024-09-21T02:13:26.963494+0000","last_undegraded":"2024-09-21T02:13:26.963494+0000","last_fullsized":"2024-09-21T02:13:26.963494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.069956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882761+0000","last_change":"2024-09-21T02:13:25.915232+0000","last_active":"2024-09-21T02:13:26.882761+0000","last_peered":"2024-09-21T02:13:26.882761+0000","last_clean":"2024-09-21T02:13:26.882761+0000","last_became_active":"2024-09-21T02:13:25.914387+0000","last_became_peered":"2024-09-21T02:13:25.914387+0000","last_unstale":"2024-09-21T02:13:26.882761+0000","last_undegraded":"2024-09-21T02:13:26.882761+0000","last_fullsized":"2024-09-21T02:13:26.882761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:42.752352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878878+0000","last_change":"2024-09-21T02:13:25.902781+0000","last_active":"2024-09-21T02:13:26.878878+0000","last_peered":"2024-09-21T02:13:26.878878+0000","last_clean":"2024-09-21T02:13:26.878878+0000","last_became_active":"2024-09-21T02:13:25.902445+0000","last_became_peered":"2024-09-21T02:13:25.902445+0000","last_unstale":"2024-09-21T02:13:26.878878+0000","last_undegraded":"2024-09-21T02:13:26.878878+0000","last_fullsized":"2024-09-21T02:13:26.878878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881922+0000","last_change":"2024-09-21T02:13:25.915843+0000","last_active":"2024-09-21T02:13:26.881922+0000","last_peered":"2024-09-21T02:13:26.881922+0000","last_clean":"2024-09-21T02:13:26.881922+0000","last_became_active":"2024-09-21T02:13:25.915560+0000","last_became_peered":"2024-09-21T02:13:25.915560+0000","last_unstale":"2024-09-21T02:13:26.881922+0000","last_undegraded":"2024-09-21T02:13:26.881922+0000","last_fullsized":"2024-09-21T02:13:26.881922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:41.816297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965307+0000","last_change":"2024-09-21T02:13:25.926096+0000","last_active":"2024-09-21T02:13:26.965307+0000","last_peered":"2024-09-21T02:13:26.965307+0000","last_clean":"2024-09-21T02:13:26.965307+0000","last_became_active":"2024-09-21T02:13:25.925408+0000","last_became_peered":"2024-09-21T02:13:25.925408+0000","last_unstale":"2024-09-21T02:13:26.965307+0000","last_undegraded":"2024-09-21T02:13:26.965307+0000","last_fullsized":"2024-09-21T02:13:26.965307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:11.244537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881981+0000","last_change":"2024-09-21T02:13:25.940050+0000","last_active":"2024-09-21T02:13:26.881981+0000","last_peered":"2024-09-21T02:13:26.881981+0000","last_clean":"2024-09-21T02:13:26.881981+0000","last_became_active":"2024-09-21T02:13:25.939688+0000","last_became_peered":"2024-09-21T02:13:25.939688+0000","last_unstale":"2024-09-21T02:13:26.881981+0000","last_undegraded":"2024-09-21T02:13:26.881981+0000","last_fullsized":"2024-09-21T02:13:26.881981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:47.887251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965669+0000","last_change":"2024-09-21T02:13:25.913323+0000","last_active":"2024-09-21T02:13:26.965669+0000","last_peered":"2024-09-21T02:13:26.965669+0000","last_clean":"2024-09-21T02:13:26.965669+0000","last_became_active":"2024-09-21T02:13:25.912945+0000","last_became_peered":"2024-09-21T02:13:25.912945+0000","last_unstale":"2024-09-21T02:13:26.965669+0000","last_undegraded":"2024-09-21T02:13:26.965669+0000","last_fullsized":"2024-09-21T02:13:26.965669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880485+0000","last_change":"2024-09-21T02:13:25.939114+0000","last_active":"2024-09-21T02:13:26.880485+0000","last_peered":"2024-09-21T02:13:26.880485+0000","last_clean":"2024-09-21T02:13:26.880485+0000","last_became_active":"2024-09-21T02:13:25.938180+0000","last_became_peered":"2024-09-21T02:13:25.938180+0000","last_unstale":"2024-09-21T02:13:26.880485+0000","last_undegraded":"2024-09-21T02:13:26.880485+0000","last_fullsized":"2024-09-21T02:13:26.880485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:37.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964449+0000","last_change":"2024-09-21T02:13:25.926590+0000","last_active":"2024-09-21T02:13:26.964449+0000","last_peered":"2024-09-21T02:13:26.964449+0000","last_clean":"2024-09-21T02:13:26.964449+0000","last_became_active":"2024-09-21T02:13:25.926154+0000","last_became_peered":"2024-09-21T02:13:25.926154+0000","last_unstale":"2024-09-21T02:13:26.964449+0000","last_undegraded":"2024-09-21T02:13:26.964449+0000","last_fullsized":"2024-09-21T02:13:26.964449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:03:00.641354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882548+0000","last_change":"2024-09-21T02:13:25.896024+0000","last_active":"2024-09-21T02:13:26.882548+0000","last_peered":"2024-09-21T02:13:26.882548+0000","last_clean":"2024-09-21T02:13:26.882548+0000","last_became_active":"2024-09-21T02:13:25.895216+0000","last_became_peered":"2024-09-21T02:13:25.895216+0000","last_unstale":"2024-09-21T02:13:26.882548+0000","last_undegraded":"2024-09-21T02:13:26.882548+0000","last_fullsized":"2024-09-21T02:13:26.882548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879525+0000","last_change":"2024-09-21T02:13:25.928846+0000","last_active":"2024-09-21T02:13:26.879525+0000","last_peered":"2024-09-21T02:13:26.879525+0000","last_clean":"2024-09-21T02:13:26.879525+0000","last_became_active":"2024-09-21T02:13:25.928527+0000","last_became_peered":"2024-09-21T02:13:25.928527+0000","last_unstale":"2024-09-21T02:13:26.879525+0000","last_undegraded":"2024-09-21T02:13:26.879525+0000","last_fullsized":"2024-09-21T02:13:26.879525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:38.551392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883432+0000","last_change":"2024-09-21T02:13:25.911388+0000","last_active":"2024-09-21T02:13:26.883432+0000","last_peered":"2024-09-21T02:13:26.883432+0000","last_clean":"2024-09-21T02:13:26.883432+0000","last_became_active":"2024-09-21T02:13:25.910631+0000","last_became_peered":"2024-09-21T02:13:25.910631+0000","last_unstale":"2024-09-21T02:13:26.883432+0000","last_undegraded":"2024-09-21T02:13:26.883432+0000","last_fullsized":"2024-09-21T02:13:26.883432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:01.276714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882752+0000","last_change":"2024-09-21T02:13:25.908215+0000","last_active":"2024-09-21T02:13:26.882752+0000","last_peered":"2024-09-21T02:13:26.882752+0000","last_clean":"2024-09-21T02:13:26.882752+0000","last_became_active":"2024-09-21T02:13:25.907764+0000","last_became_peered":"2024-09-21T02:13:25.907764+0000","last_unstale":"2024-09-21T02:13:26.882752+0000","last_undegraded":"2024-09-21T02:13:26.882752+0000","last_fullsized":"2024-09-21T02:13:26.882752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:31.448658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.918292+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.917267+0000","last_became_peered":"2024-09-21T02:13:25.917267+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:22.501288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963998+0000","last_change":"2024-09-21T02:13:25.918530+0000","last_active":"2024-09-21T02:13:26.963998+0000","last_peered":"2024-09-21T02:13:26.963998+0000","last_clean":"2024-09-21T02:13:26.963998+0000","last_became_active":"2024-09-21T02:13:25.917735+0000","last_became_peered":"2024-09-21T02:13:25.917735+0000","last_unstale":"2024-09-21T02:13:26.963998+0000","last_undegraded":"2024-09-21T02:13:26.963998+0000","last_fullsized":"2024-09-21T02:13:26.963998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:03.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880008+0000","last_change":"2024-09-21T02:13:25.913497+0000","last_active":"2024-09-21T02:13:26.880008+0000","last_peered":"2024-09-21T02:13:26.880008+0000","last_clean":"2024-09-21T02:13:26.880008+0000","last_became_active":"2024-09-21T02:13:25.912567+0000","last_became_peered":"2024-09-21T02:13:25.912567+0000","last_unstale":"2024-09-21T02:13:26.880008+0000","last_undegraded":"2024-09-21T02:13:26.880008+0000","last_fullsized":"2024-09-21T02:13:26.880008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879152+0000","last_change":"2024-09-21T02:13:25.917618+0000","last_active":"2024-09-21T02:13:26.879152+0000","last_peered":"2024-09-21T02:13:26.879152+0000","last_clean":"2024-09-21T02:13:26.879152+0000","last_became_active":"2024-09-21T02:13:25.916325+0000","last_became_peered":"2024-09-21T02:13:25.916325+0000","last_unstale":"2024-09-21T02:13:26.879152+0000","last_undegraded":"2024-09-21T02:13:26.879152+0000","last_fullsized":"2024-09-21T02:13:26.879152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964562+0000","last_change":"2024-09-21T02:13:25.915861+0000","last_active":"2024-09-21T02:13:26.964562+0000","last_peered":"2024-09-21T02:13:26.964562+0000","last_clean":"2024-09-21T02:13:26.964562+0000","last_became_active":"2024-09-21T02:13:25.912257+0000","last_became_peered":"2024-09-21T02:13:25.912257+0000","last_unstale":"2024-09-21T02:13:26.964562+0000","last_undegraded":"2024-09-21T02:13:26.964562+0000","last_fullsized":"2024-09-21T02:13:26.964562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:56.675098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878068+0000","last_change":"2024-09-21T02:13:25.894915+0000","last_active":"2024-09-21T02:13:26.878068+0000","last_peered":"2024-09-21T02:13:26.878068+0000","last_clean":"2024-09-21T02:13:26.878068+0000","last_became_active":"2024-09-21T02:13:25.894440+0000","last_became_peered":"2024-09-21T02:13:25.894440+0000","last_unstale":"2024-09-21T02:13:26.878068+0000","last_undegraded":"2024-09-21T02:13:26.878068+0000","last_fullsized":"2024-09-21T02:13:26.878068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:28.857628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964697+0000","last_change":"2024-09-21T02:13:25.930047+0000","last_active":"2024-09-21T02:13:26.964697+0000","last_peered":"2024-09-21T02:13:26.964697+0000","last_clean":"2024-09-21T02:13:26.964697+0000","last_became_active":"2024-09-21T02:13:25.929729+0000","last_became_peered":"2024-09-21T02:13:25.929729+0000","last_unstale":"2024-09-21T02:13:26.964697+0000","last_undegraded":"2024-09-21T02:13:26.964697+0000","last_fullsized":"2024-09-21T02:13:26.964697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:26.319373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880194+0000","last_change":"2024-09-21T02:13:24.903640+0000","last_active":"2024-09-21T02:13:26.880194+0000","last_peered":"2024-09-21T02:13:26.880194+0000","last_clean":"2024-09-21T02:13:26.880194+0000","last_became_active":"2024-09-21T02:13:24.899880+0000","last_became_peered":"2024-09-21T02:13:24.899880+0000","last_unstale":"2024-09-21T02:13:26.880194+0000","last_undegraded":"2024-09-21T02:13:26.880194+0000","last_fullsized":"2024-09-21T02:13:26.880194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880861+0000","last_change":"2024-09-21T02:13:25.932469+0000","last_active":"2024-09-21T02:13:26.880861+0000","last_peered":"2024-09-21T02:13:26.880861+0000","last_clean":"2024-09-21T02:13:26.880861+0000","last_became_active":"2024-09-21T02:13:25.931740+0000","last_became_peered":"2024-09-21T02:13:25.931740+0000","last_unstale":"2024-09-21T02:13:26.880861+0000","last_undegraded":"2024-09-21T02:13:26.880861+0000","last_fullsized":"2024-09-21T02:13:26.880861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:32.988566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879277+0000","last_change":"2024-09-21T02:13:24.906393+0000","last_active":"2024-09-21T02:13:26.879277+0000","last_peered":"2024-09-21T02:13:26.879277+0000","last_clean":"2024-09-21T02:13:26.879277+0000","last_became_active":"2024-09-21T02:13:24.905410+0000","last_became_peered":"2024-09-21T02:13:24.905410+0000","last_unstale":"2024-09-21T02:13:26.879277+0000","last_undegraded":"2024-09-21T02:13:26.879277+0000","last_fullsized":"2024-09-21T02:13:26.879277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:13.262457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.962957+0000","last_change":"2024-09-21T02:13:24.909821+0000","last_active":"2024-09-21T02:13:26.962957+0000","last_peered":"2024-09-21T02:13:26.962957+0000","last_clean":"2024-09-21T02:13:26.962957+0000","last_became_active":"2024-09-21T02:13:24.907797+0000","last_became_peered":"2024-09-21T02:13:24.907797+0000","last_unstale":"2024-09-21T02:13:26.962957+0000","last_undegraded":"2024-09-21T02:13:26.962957+0000","last_fullsized":"2024-09-21T02:13:26.962957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879469+0000","last_change":"2024-09-21T02:13:25.927047+0000","last_active":"2024-09-21T02:13:26.879469+0000","last_peered":"2024-09-21T02:13:26.879469+0000","last_clean":"2024-09-21T02:13:26.879469+0000","last_became_active":"2024-09-21T02:13:25.926307+0000","last_became_peered":"2024-09-21T02:13:25.926307+0000","last_unstale":"2024-09-21T02:13:26.879469+0000","last_undegraded":"2024-09-21T02:13:26.879469+0000","last_fullsized":"2024-09-21T02:13:26.879469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:26.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879564+0000","last_change":"2024-09-21T02:13:24.896260+0000","last_active":"2024-09-21T02:13:26.879564+0000","last_peered":"2024-09-21T02:13:26.879564+0000","last_clean":"2024-09-21T02:13:26.879564+0000","last_became_active":"2024-09-21T02:13:24.895867+0000","last_became_peered":"2024-09-21T02:13:24.895867+0000","last_unstale":"2024-09-21T02:13:26.879564+0000","last_undegraded":"2024-09-21T02:13:26.879564+0000","last_fullsized":"2024-09-21T02:13:26.879564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:14.510200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880811+0000","last_change":"2024-09-21T02:13:25.917250+0000","last_active":"2024-09-21T02:13:26.880811+0000","last_peered":"2024-09-21T02:13:26.880811+0000","last_clean":"2024-09-21T02:13:26.880811+0000","last_became_active":"2024-09-21T02:13:25.916424+0000","last_became_peered":"2024-09-21T02:13:25.916424+0000","last_unstale":"2024-09-21T02:13:26.880811+0000","last_undegraded":"2024-09-21T02:13:26.880811+0000","last_fullsized":"2024-09-21T02:13:26.880811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:05.715180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.967072+0000","last_change":"2024-09-21T02:13:24.899492+0000","last_active":"2024-09-21T02:13:26.967072+0000","last_peered":"2024-09-21T02:13:26.967072+0000","last_clean":"2024-09-21T02:13:26.967072+0000","last_became_active":"2024-09-21T02:13:24.899044+0000","last_became_peered":"2024-09-21T02:13:24.899044+0000","last_unstale":"2024-09-21T02:13:26.967072+0000","last_undegraded":"2024-09-21T02:13:26.967072+0000","last_fullsized":"2024-09-21T02:13:26.967072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.278287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882998+0000","last_change":"2024-09-21T02:13:25.908074+0000","last_active":"2024-09-21T02:13:26.882998+0000","last_peered":"2024-09-21T02:13:26.882998+0000","last_clean":"2024-09-21T02:13:26.882998+0000","last_became_active":"2024-09-21T02:13:25.907240+0000","last_became_peered":"2024-09-21T02:13:25.907240+0000","last_unstale":"2024-09-21T02:13:26.882998+0000","last_undegraded":"2024-09-21T02:13:26.882998+0000","last_fullsized":"2024-09-21T02:13:26.882998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.065966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965202+0000","last_change":"2024-09-21T02:13:24.898945+0000","last_active":"2024-09-21T02:13:26.965202+0000","last_peered":"2024-09-21T02:13:26.965202+0000","last_clean":"2024-09-21T02:13:26.965202+0000","last_became_active":"2024-09-21T02:13:24.897945+0000","last_became_peered":"2024-09-21T02:13:24.897945+0000","last_unstale":"2024-09-21T02:13:26.965202+0000","last_undegraded":"2024-09-21T02:13:26.965202+0000","last_fullsized":"2024-09-21T02:13:26.965202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:22.784881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883777+0000","last_change":"2024-09-21T02:13:25.919085+0000","last_active":"2024-09-21T02:13:26.883777+0000","last_peered":"2024-09-21T02:13:26.883777+0000","last_clean":"2024-09-21T02:13:26.883777+0000","last_became_active":"2024-09-21T02:13:25.918349+0000","last_became_peered":"2024-09-21T02:13:25.918349+0000","last_unstale":"2024-09-21T02:13:26.883777+0000","last_undegraded":"2024-09-21T02:13:26.883777+0000","last_fullsized":"2024-09-21T02:13:26.883777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:21.580694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965045+0000","last_change":"2024-09-21T02:13:24.919633+0000","last_active":"2024-09-21T02:13:26.965045+0000","last_peered":"2024-09-21T02:13:26.965045+0000","last_clean":"2024-09-21T02:13:26.965045+0000","last_became_active":"2024-09-21T02:13:24.918940+0000","last_became_peered":"2024-09-21T02:13:24.918940+0000","last_unstale":"2024-09-21T02:13:26.965045+0000","last_undegraded":"2024-09-21T02:13:26.965045+0000","last_fullsized":"2024-09-21T02:13:26.965045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880486+0000","last_change":"2024-09-21T02:13:25.901393+0000","last_active":"2024-09-21T02:13:26.880486+0000","last_peered":"2024-09-21T02:13:26.880486+0000","last_clean":"2024-09-21T02:13:26.880486+0000","last_became_active":"2024-09-21T02:13:25.901081+0000","last_became_peered":"2024-09-21T02:13:25.901081+0000","last_unstale":"2024-09-21T02:13:26.880486+0000","last_undegraded":"2024-09-21T02:13:26.880486+0000","last_fullsized":"2024-09-21T02:13:26.880486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:05.811740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879810+0000","last_change":"2024-09-21T02:13:24.901069+0000","last_active":"2024-09-21T02:13:26.879810+0000","last_peered":"2024-09-21T02:13:26.879810+0000","last_clean":"2024-09-21T02:13:26.879810+0000","last_became_active":"2024-09-21T02:13:24.900566+0000","last_became_peered":"2024-09-21T02:13:24.900566+0000","last_unstale":"2024-09-21T02:13:26.879810+0000","last_undegraded":"2024-09-21T02:13:26.879810+0000","last_fullsized":"2024-09-21T02:13:26.879810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:33.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964606+0000","last_change":"2024-09-21T02:13:25.923864+0000","last_active":"2024-09-21T02:13:26.964606+0000","last_peered":"2024-09-21T02:13:26.964606+0000","last_clean":"2024-09-21T02:13:26.964606+0000","last_became_active":"2024-09-21T02:13:25.923315+0000","last_became_peered":"2024-09-21T02:13:25.923315+0000","last_unstale":"2024-09-21T02:13:26.964606+0000","last_undegraded":"2024-09-21T02:13:26.964606+0000","last_fullsized":"2024-09-21T02:13:26.964606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:22.325276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965043+0000","last_change":"2024-09-21T02:13:24.918203+0000","last_active":"2024-09-21T02:13:26.965043+0000","last_peered":"2024-09-21T02:13:26.965043+0000","last_clean":"2024-09-21T02:13:26.965043+0000","last_became_active":"2024-09-21T02:13:24.907150+0000","last_became_peered":"2024-09-21T02:13:24.907150+0000","last_unstale":"2024-09-21T02:13:26.965043+0000","last_undegraded":"2024-09-21T02:13:26.965043+0000","last_fullsized":"2024-09-21T02:13:26.965043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963777+0000","last_change":"2024-09-21T02:13:25.906616+0000","last_active":"2024-09-21T02:13:26.963777+0000","last_peered":"2024-09-21T02:13:26.963777+0000","last_clean":"2024-09-21T02:13:26.963777+0000","last_became_active":"2024-09-21T02:13:25.905776+0000","last_became_peered":"2024-09-21T02:13:25.905776+0000","last_unstale":"2024-09-21T02:13:26.963777+0000","last_undegraded":"2024-09-21T02:13:26.963777+0000","last_fullsized":"2024-09-21T02:13:26.963777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:37.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879412+0000","last_change":"2024-09-21T02:13:24.906862+0000","last_active":"2024-09-21T02:13:26.879412+0000","last_peered":"2024-09-21T02:13:26.879412+0000","last_clean":"2024-09-21T02:13:26.879412+0000","last_became_active":"2024-09-21T02:13:24.905954+0000","last_became_peered":"2024-09-21T02:13:24.905954+0000","last_unstale":"2024-09-21T02:13:26.879412+0000","last_undegraded":"2024-09-21T02:13:26.879412+0000","last_fullsized":"2024-09-21T02:13:26.879412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:34.713354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880043+0000","last_change":"2024-09-21T02:13:25.919387+0000","last_active":"2024-09-21T02:13:26.880043+0000","last_peered":"2024-09-21T02:13:26.880043+0000","last_clean":"2024-09-21T02:13:26.880043+0000","last_became_active":"2024-09-21T02:13:25.919065+0000","last_became_peered":"2024-09-21T02:13:25.919065+0000","last_unstale":"2024-09-21T02:13:26.880043+0000","last_undegraded":"2024-09-21T02:13:26.880043+0000","last_fullsized":"2024-09-21T02:13:26.880043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:30.665668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963542+0000","last_change":"2024-09-21T02:13:24.906999+0000","last_active":"2024-09-21T02:13:26.963542+0000","last_peered":"2024-09-21T02:13:26.963542+0000","last_clean":"2024-09-21T02:13:26.963542+0000","last_became_active":"2024-09-21T02:13:24.897714+0000","last_became_peered":"2024-09-21T02:13:24.897714+0000","last_unstale":"2024-09-21T02:13:26.963542+0000","last_undegraded":"2024-09-21T02:13:26.963542+0000","last_fullsized":"2024-09-21T02:13:26.963542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963997+0000","last_change":"2024-09-21T02:13:25.928157+0000","last_active":"2024-09-21T02:13:26.963997+0000","last_peered":"2024-09-21T02:13:26.963997+0000","last_clean":"2024-09-21T02:13:26.963997+0000","last_became_active":"2024-09-21T02:13:25.927747+0000","last_became_peered":"2024-09-21T02:13:25.927747+0000","last_unstale":"2024-09-21T02:13:26.963997+0000","last_undegraded":"2024-09-21T02:13:26.963997+0000","last_fullsized":"2024-09-21T02:13:26.963997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:42.363071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881672+0000","last_change":"2024-09-21T02:13:24.911216+0000","last_active":"2024-09-21T02:13:26.881672+0000","last_peered":"2024-09-21T02:13:26.881672+0000","last_clean":"2024-09-21T02:13:26.881672+0000","last_became_active":"2024-09-21T02:13:24.909015+0000","last_became_peered":"2024-09-21T02:13:24.909015+0000","last_unstale":"2024-09-21T02:13:26.881672+0000","last_undegraded":"2024-09-21T02:13:26.881672+0000","last_fullsized":"2024-09-21T02:13:26.881672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.393881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878788+0000","last_change":"2024-09-21T02:13:25.902089+0000","last_active":"2024-09-21T02:13:26.878788+0000","last_peered":"2024-09-21T02:13:26.878788+0000","last_clean":"2024-09-21T02:13:26.878788+0000","last_became_active":"2024-09-21T02:13:25.901778+0000","last_became_peered":"2024-09-21T02:13:25.901778+0000","last_unstale":"2024-09-21T02:13:26.878788+0000","last_undegraded":"2024-09-21T02:13:26.878788+0000","last_fullsized":"2024-09-21T02:13:26.878788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965582+0000","last_change":"2024-09-21T02:13:24.914792+0000","last_active":"2024-09-21T02:13:26.965582+0000","last_peered":"2024-09-21T02:13:26.965582+0000","last_clean":"2024-09-21T02:13:26.965582+0000","last_became_active":"2024-09-21T02:13:24.914298+0000","last_became_peered":"2024-09-21T02:13:24.914298+0000","last_unstale":"2024-09-21T02:13:26.965582+0000","last_undegraded":"2024-09-21T02:13:26.965582+0000","last_fullsized":"2024-09-21T02:13:26.965582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883347+0000","last_change":"2024-09-21T02:13:25.924238+0000","last_active":"2024-09-21T02:13:26.883347+0000","last_peered":"2024-09-21T02:13:26.883347+0000","last_clean":"2024-09-21T02:13:26.883347+0000","last_became_active":"2024-09-21T02:13:25.922155+0000","last_became_peered":"2024-09-21T02:13:25.922155+0000","last_unstale":"2024-09-21T02:13:26.883347+0000","last_undegraded":"2024-09-21T02:13:26.883347+0000","last_fullsized":"2024-09-21T02:13:26.883347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:27.528134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881619+0000","last_change":"2024-09-21T02:13:24.915381+0000","last_active":"2024-09-21T02:13:26.881619+0000","last_peered":"2024-09-21T02:13:26.881619+0000","last_clean":"2024-09-21T02:13:26.881619+0000","last_became_active":"2024-09-21T02:13:24.914348+0000","last_became_peered":"2024-09-21T02:13:24.914348+0000","last_unstale":"2024-09-21T02:13:26.881619+0000","last_undegraded":"2024-09-21T02:13:26.881619+0000","last_fullsized":"2024-09-21T02:13:26.881619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:51.545636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965211+0000","last_change":"2024-09-21T02:13:25.924993+0000","last_active":"2024-09-21T02:13:26.965211+0000","last_peered":"2024-09-21T02:13:26.965211+0000","last_clean":"2024-09-21T02:13:26.965211+0000","last_became_active":"2024-09-21T02:13:25.924677+0000","last_became_peered":"2024-09-21T02:13:25.924677+0000","last_unstale":"2024-09-21T02:13:26.965211+0000","last_undegraded":"2024-09-21T02:13:26.965211+0000","last_fullsized":"2024-09-21T02:13:26.965211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879653+0000","last_change":"2024-09-21T02:13:24.893497+0000","last_active":"2024-09-21T02:13:26.879653+0000","last_peered":"2024-09-21T02:13:26.879653+0000","last_clean":"2024-09-21T02:13:26.879653+0000","last_became_active":"2024-09-21T02:13:24.892789+0000","last_became_peered":"2024-09-21T02:13:24.892789+0000","last_unstale":"2024-09-21T02:13:26.879653+0000","last_undegraded":"2024-09-21T02:13:26.879653+0000","last_fullsized":"2024-09-21T02:13:26.879653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965821+0000","last_change":"2024-09-21T02:13:25.927918+0000","last_active":"2024-09-21T02:13:26.965821+0000","last_peered":"2024-09-21T02:13:26.965821+0000","last_clean":"2024-09-21T02:13:26.965821+0000","last_became_active":"2024-09-21T02:13:25.927608+0000","last_became_peered":"2024-09-21T02:13:25.927608+0000","last_unstale":"2024-09-21T02:13:26.965821+0000","last_undegraded":"2024-09-21T02:13:26.965821+0000","last_fullsized":"2024-09-21T02:13:26.965821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.891802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964326+0000","last_change":"2024-09-21T02:13:24.904899+0000","last_active":"2024-09-21T02:13:26.964326+0000","last_peered":"2024-09-21T02:13:26.964326+0000","last_clean":"2024-09-21T02:13:26.964326+0000","last_became_active":"2024-09-21T02:13:24.904137+0000","last_became_peered":"2024-09-21T02:13:24.904137+0000","last_unstale":"2024-09-21T02:13:26.964326+0000","last_undegraded":"2024-09-21T02:13:26.964326+0000","last_fullsized":"2024-09-21T02:13:26.964326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:06.764948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881399+0000","last_change":"2024-09-21T02:13:25.908087+0000","last_active":"2024-09-21T02:13:26.881399+0000","last_peered":"2024-09-21T02:13:26.881399+0000","last_clean":"2024-09-21T02:13:26.881399+0000","last_became_active":"2024-09-21T02:13:25.907748+0000","last_became_peered":"2024-09-21T02:13:25.907748+0000","last_unstale":"2024-09-21T02:13:26.881399+0000","last_undegraded":"2024-09-21T02:13:26.881399+0000","last_fullsized":"2024-09-21T02:13:26.881399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:24.998552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964107+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:13:26.964107+0000","last_peered":"2024-09-21T02:13:26.964107+0000","last_clean":"2024-09-21T02:13:26.964107+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:13:26.964107+0000","last_undegraded":"2024-09-21T02:13:26.964107+0000","last_fullsized":"2024-09-21T02:13:26.964107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880025+0000","last_change":"2024-09-21T02:13:24.893632+0000","last_active":"2024-09-21T02:13:26.880025+0000","last_peered":"2024-09-21T02:13:26.880025+0000","last_clean":"2024-09-21T02:13:26.880025+0000","last_became_active":"2024-09-21T02:13:24.892990+0000","last_became_peered":"2024-09-21T02:13:24.892990+0000","last_unstale":"2024-09-21T02:13:26.880025+0000","last_undegraded":"2024-09-21T02:13:26.880025+0000","last_fullsized":"2024-09-21T02:13:26.880025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880851+0000","last_change":"2024-09-21T02:13:25.923436+0000","last_active":"2024-09-21T02:13:26.880851+0000","last_peered":"2024-09-21T02:13:26.880851+0000","last_clean":"2024-09-21T02:13:26.880851+0000","last_became_active":"2024-09-21T02:13:25.922491+0000","last_became_peered":"2024-09-21T02:13:25.922491+0000","last_unstale":"2024-09-21T02:13:26.880851+0000","last_undegraded":"2024-09-21T02:13:26.880851+0000","last_fullsized":"2024-09-21T02:13:26.880851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:58:13.239789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879715+0000","last_change":"2024-09-21T02:13:24.914198+0000","last_active":"2024-09-21T02:13:26.879715+0000","last_peered":"2024-09-21T02:13:26.879715+0000","last_clean":"2024-09-21T02:13:26.879715+0000","last_became_active":"2024-09-21T02:13:24.913815+0000","last_became_peered":"2024-09-21T02:13:24.913815+0000","last_unstale":"2024-09-21T02:13:26.879715+0000","last_undegraded":"2024-09-21T02:13:26.879715+0000","last_fullsized":"2024-09-21T02:13:26.879715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:35.245420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963392+0000","last_change":"2024-09-21T02:13:25.906344+0000","last_active":"2024-09-21T02:13:26.963392+0000","last_peered":"2024-09-21T02:13:26.963392+0000","last_clean":"2024-09-21T02:13:26.963392+0000","last_became_active":"2024-09-21T02:13:25.905571+0000","last_became_peered":"2024-09-21T02:13:25.905571+0000","last_unstale":"2024-09-21T02:13:26.963392+0000","last_undegraded":"2024-09-21T02:13:26.963392+0000","last_fullsized":"2024-09-21T02:13:26.963392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:01.266262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964760+0000","last_change":"2024-09-21T02:13:24.919598+0000","last_active":"2024-09-21T02:13:26.964760+0000","last_peered":"2024-09-21T02:13:26.964760+0000","last_clean":"2024-09-21T02:13:26.964760+0000","last_became_active":"2024-09-21T02:13:24.918909+0000","last_became_peered":"2024-09-21T02:13:24.918909+0000","last_unstale":"2024-09-21T02:13:26.964760+0000","last_undegraded":"2024-09-21T02:13:26.964760+0000","last_fullsized":"2024-09-21T02:13:26.964760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:19.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.878944+0000","last_change":"2024-09-21T02:13:25.913216+0000","last_active":"2024-09-21T02:13:26.878944+0000","last_peered":"2024-09-21T02:13:26.878944+0000","last_clean":"2024-09-21T02:13:26.878944+0000","last_became_active":"2024-09-21T02:13:25.912412+0000","last_became_peered":"2024-09-21T02:13:25.912412+0000","last_unstale":"2024-09-21T02:13:26.878944+0000","last_undegraded":"2024-09-21T02:13:26.878944+0000","last_fullsized":"2024-09-21T02:13:26.878944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:16.249421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879536+0000","last_change":"2024-09-21T02:13:24.900311+0000","last_active":"2024-09-21T02:13:26.879536+0000","last_peered":"2024-09-21T02:13:26.879536+0000","last_clean":"2024-09-21T02:13:26.879536+0000","last_became_active":"2024-09-21T02:13:24.899845+0000","last_became_peered":"2024-09-21T02:13:24.899845+0000","last_unstale":"2024-09-21T02:13:26.879536+0000","last_undegraded":"2024-09-21T02:13:26.879536+0000","last_fullsized":"2024-09-21T02:13:26.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:34.513381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964099+0000","last_change":"2024-09-21T02:13:25.899982+0000","last_active":"2024-09-21T02:13:26.964099+0000","last_peered":"2024-09-21T02:13:26.964099+0000","last_clean":"2024-09-21T02:13:26.964099+0000","last_became_active":"2024-09-21T02:13:25.899671+0000","last_became_peered":"2024-09-21T02:13:25.899671+0000","last_unstale":"2024-09-21T02:13:26.964099+0000","last_undegraded":"2024-09-21T02:13:26.964099+0000","last_fullsized":"2024-09-21T02:13:26.964099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:42.309402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879852+0000","last_change":"2024-09-21T02:13:24.903776+0000","last_active":"2024-09-21T02:13:26.879852+0000","last_peered":"2024-09-21T02:13:26.879852+0000","last_clean":"2024-09-21T02:13:26.879852+0000","last_became_active":"2024-09-21T02:13:24.902681+0000","last_became_peered":"2024-09-21T02:13:24.902681+0000","last_unstale":"2024-09-21T02:13:26.879852+0000","last_undegraded":"2024-09-21T02:13:26.879852+0000","last_fullsized":"2024-09-21T02:13:26.879852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:46.752514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880542+0000","last_change":"2024-09-21T02:13:25.893632+0000","last_active":"2024-09-21T02:13:26.880542+0000","last_peered":"2024-09-21T02:13:26.880542+0000","last_clean":"2024-09-21T02:13:26.880542+0000","last_became_active":"2024-09-21T02:13:25.893137+0000","last_became_peered":"2024-09-21T02:13:25.893137+0000","last_unstale":"2024-09-21T02:13:26.880542+0000","last_undegraded":"2024-09-21T02:13:26.880542+0000","last_fullsized":"2024-09-21T02:13:26.880542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:45.630932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882169+0000","last_change":"2024-09-21T02:13:24.899031+0000","last_active":"2024-09-21T02:13:26.882169+0000","last_peered":"2024-09-21T02:13:26.882169+0000","last_clean":"2024-09-21T02:13:26.882169+0000","last_became_active":"2024-09-21T02:13:24.898526+0000","last_became_peered":"2024-09-21T02:13:24.898526+0000","last_unstale":"2024-09-21T02:13:26.882169+0000","last_undegraded":"2024-09-21T02:13:26.882169+0000","last_fullsized":"2024-09-21T02:13:26.882169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:15.323641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964112+0000","last_change":"2024-09-21T02:13:25.906716+0000","last_active":"2024-09-21T02:13:26.964112+0000","last_peered":"2024-09-21T02:13:26.964112+0000","last_clean":"2024-09-21T02:13:26.964112+0000","last_became_active":"2024-09-21T02:13:25.905889+0000","last_became_peered":"2024-09-21T02:13:25.905889+0000","last_unstale":"2024-09-21T02:13:26.964112+0000","last_undegraded":"2024-09-21T02:13:26.964112+0000","last_fullsized":"2024-09-21T02:13:26.964112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:56.985078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:27.881748+0000","last_change":"2024-09-21T02:13:24.909178+0000","last_active":"2024-09-21T02:13:27.881748+0000","last_peered":"2024-09-21T02:13:27.881748+0000","last_clean":"2024-09-21T02:13:27.881748+0000","last_became_active":"2024-09-21T02:13:24.908471+0000","last_became_peered":"2024-09-21T02:13:24.908471+0000","last_unstale":"2024-09-21T02:13:27.881748+0000","last_undegraded":"2024-09-21T02:13:27.881748+0000","last_fullsized":"2024-09-21T02:13:27.881748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:31.067166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880621+0000","last_change":"2024-09-21T02:13:25.938245+0000","last_active":"2024-09-21T02:13:26.880621+0000","last_peered":"2024-09-21T02:13:26.880621+0000","last_clean":"2024-09-21T02:13:26.880621+0000","last_became_active":"2024-09-21T02:13:25.935157+0000","last_became_peered":"2024-09-21T02:13:25.935157+0000","last_unstale":"2024-09-21T02:13:26.880621+0000","last_undegraded":"2024-09-21T02:13:26.880621+0000","last_fullsized":"2024-09-21T02:13:26.880621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:31.810708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882102+0000","last_change":"2024-09-21T02:13:24.905877+0000","last_active":"2024-09-21T02:13:26.882102+0000","last_peered":"2024-09-21T02:13:26.882102+0000","last_clean":"2024-09-21T02:13:26.882102+0000","last_became_active":"2024-09-21T02:13:24.905486+0000","last_became_peered":"2024-09-21T02:13:24.905486+0000","last_unstale":"2024-09-21T02:13:26.882102+0000","last_undegraded":"2024-09-21T02:13:26.882102+0000","last_fullsized":"2024-09-21T02:13:26.882102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:54.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881249+0000","last_change":"2024-09-21T02:13:25.923192+0000","last_active":"2024-09-21T02:13:26.881249+0000","last_peered":"2024-09-21T02:13:26.881249+0000","last_clean":"2024-09-21T02:13:26.881249+0000","last_became_active":"2024-09-21T02:13:25.922551+0000","last_became_peered":"2024-09-21T02:13:25.922551+0000","last_unstale":"2024-09-21T02:13:26.881249+0000","last_undegraded":"2024-09-21T02:13:26.881249+0000","last_fullsized":"2024-09-21T02:13:26.881249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:00.144802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963169+0000","last_change":"2024-09-21T02:13:24.898279+0000","last_active":"2024-09-21T02:13:26.963169+0000","last_peered":"2024-09-21T02:13:26.963169+0000","last_clean":"2024-09-21T02:13:26.963169+0000","last_became_active":"2024-09-21T02:13:24.897369+0000","last_became_peered":"2024-09-21T02:13:24.897369+0000","last_unstale":"2024-09-21T02:13:26.963169+0000","last_undegraded":"2024-09-21T02:13:26.963169+0000","last_fullsized":"2024-09-21T02:13:26.963169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:56.842815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879396+0000","last_change":"2024-09-21T02:13:25.921611+0000","last_active":"2024-09-21T02:13:26.879396+0000","last_peered":"2024-09-21T02:13:26.879396+0000","last_clean":"2024-09-21T02:13:26.879396+0000","last_became_active":"2024-09-21T02:13:25.921165+0000","last_became_peered":"2024-09-21T02:13:25.921165+0000","last_unstale":"2024-09-21T02:13:26.879396+0000","last_undegraded":"2024-09-21T02:13:26.879396+0000","last_fullsized":"2024-09-21T02:13:26.879396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:14.240440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.881013+0000","last_change":"2024-09-21T02:13:24.912523+0000","last_active":"2024-09-21T02:13:26.881013+0000","last_peered":"2024-09-21T02:13:26.881013+0000","last_clean":"2024-09-21T02:13:26.881013+0000","last_became_active":"2024-09-21T02:13:24.909187+0000","last_became_peered":"2024-09-21T02:13:24.909187+0000","last_unstale":"2024-09-21T02:13:26.881013+0000","last_undegraded":"2024-09-21T02:13:26.881013+0000","last_fullsized":"2024-09-21T02:13:26.881013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.824838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879375+0000","last_change":"2024-09-21T02:13:25.938843+0000","last_active":"2024-09-21T02:13:26.879375+0000","last_peered":"2024-09-21T02:13:26.879375+0000","last_clean":"2024-09-21T02:13:26.879375+0000","last_became_active":"2024-09-21T02:13:25.938090+0000","last_became_peered":"2024-09-21T02:13:25.938090+0000","last_unstale":"2024-09-21T02:13:26.879375+0000","last_undegraded":"2024-09-21T02:13:26.879375+0000","last_fullsized":"2024-09-21T02:13:26.879375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:32.955964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.963689+0000","last_change":"2024-09-21T02:13:24.913917+0000","last_active":"2024-09-21T02:13:26.963689+0000","last_peered":"2024-09-21T02:13:26.963689+0000","last_clean":"2024-09-21T02:13:26.963689+0000","last_became_active":"2024-09-21T02:13:24.913151+0000","last_became_peered":"2024-09-21T02:13:24.913151+0000","last_unstale":"2024-09-21T02:13:26.963689+0000","last_undegraded":"2024-09-21T02:13:26.963689+0000","last_fullsized":"2024-09-21T02:13:26.963689+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:57.921560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965191+0000","last_change":"2024-09-21T02:13:25.937217+0000","last_active":"2024-09-21T02:13:26.965191+0000","last_peered":"2024-09-21T02:13:26.965191+0000","last_clean":"2024-09-21T02:13:26.965191+0000","last_became_active":"2024-09-21T02:13:25.936656+0000","last_became_peered":"2024-09-21T02:13:25.936656+0000","last_unstale":"2024-09-21T02:13:26.965191+0000","last_undegraded":"2024-09-21T02:13:26.965191+0000","last_fullsized":"2024-09-21T02:13:26.965191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:55.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.879890+0000","last_change":"2024-09-21T02:13:24.893809+0000","last_active":"2024-09-21T02:13:26.879890+0000","last_peered":"2024-09-21T02:13:26.879890+0000","last_clean":"2024-09-21T02:13:26.879890+0000","last_became_active":"2024-09-21T02:13:24.893342+0000","last_became_peered":"2024-09-21T02:13:24.893342+0000","last_unstale":"2024-09-21T02:13:26.879890+0000","last_undegraded":"2024-09-21T02:13:26.879890+0000","last_fullsized":"2024-09-21T02:13:26.879890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:55.664651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880297+0000","last_change":"2024-09-21T02:13:25.909794+0000","last_active":"2024-09-21T02:13:26.880297+0000","last_peered":"2024-09-21T02:13:26.880297+0000","last_clean":"2024-09-21T02:13:26.880297+0000","last_became_active":"2024-09-21T02:13:25.908961+0000","last_became_peered":"2024-09-21T02:13:25.908961+0000","last_unstale":"2024-09-21T02:13:26.880297+0000","last_undegraded":"2024-09-21T02:13:26.880297+0000","last_fullsized":"2024-09-21T02:13:26.880297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:42:37.003277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.880824+0000","last_change":"2024-09-21T02:13:25.915575+0000","last_active":"2024-09-21T02:13:26.880824+0000","last_peered":"2024-09-21T02:13:26.880824+0000","last_clean":"2024-09-21T02:13:26.880824+0000","last_became_active":"2024-09-21T02:13:25.915035+0000","last_became_peered":"2024-09-21T02:13:25.915035+0000","last_unstale":"2024-09-21T02:13:26.880824+0000","last_undegraded":"2024-09-21T02:13:26.880824+0000","last_fullsized":"2024-09-21T02:13:26.880824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:42.771535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.965874+0000","last_change":"2024-09-21T02:13:24.903167+0000","last_active":"2024-09-21T02:13:26.965874+0000","last_peered":"2024-09-21T02:13:26.965874+0000","last_clean":"2024-09-21T02:13:26.965874+0000","last_became_active":"2024-09-21T02:13:24.902698+0000","last_became_peered":"2024-09-21T02:13:24.902698+0000","last_unstale":"2024-09-21T02:13:26.965874+0000","last_undegraded":"2024-09-21T02:13:26.965874+0000","last_fullsized":"2024-09-21T02:13:26.965874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:52:10.234091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964514+0000","last_change":"2024-09-21T02:13:25.925357+0000","last_active":"2024-09-21T02:13:26.964514+0000","last_peered":"2024-09-21T02:13:26.964514+0000","last_clean":"2024-09-21T02:13:26.964514+0000","last_became_active":"2024-09-21T02:13:25.924899+0000","last_became_peered":"2024-09-21T02:13:25.924899+0000","last_unstale":"2024-09-21T02:13:26.964514+0000","last_undegraded":"2024-09-21T02:13:26.964514+0000","last_fullsized":"2024-09-21T02:13:26.964514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:35.680962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.964793+0000","last_change":"2024-09-21T02:13:24.898153+0000","last_active":"2024-09-21T02:13:26.964793+0000","last_peered":"2024-09-21T02:13:26.964793+0000","last_clean":"2024-09-21T02:13:26.964793+0000","last_became_active":"2024-09-21T02:13:24.897451+0000","last_became_peered":"2024-09-21T02:13:24.897451+0000","last_unstale":"2024-09-21T02:13:26.964793+0000","last_undegraded":"2024-09-21T02:13:26.964793+0000","last_fullsized":"2024-09-21T02:13:26.964793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:53.162379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883658+0000","last_change":"2024-09-21T02:13:25.911108+0000","last_active":"2024-09-21T02:13:26.883658+0000","last_peered":"2024-09-21T02:13:26.883658+0000","last_clean":"2024-09-21T02:13:26.883658+0000","last_became_active":"2024-09-21T02:13:25.910368+0000","last_became_peered":"2024-09-21T02:13:25.910368+0000","last_unstale":"2024-09-21T02:13:26.883658+0000","last_undegraded":"2024-09-21T02:13:26.883658+0000","last_fullsized":"2024-09-21T02:13:26.883658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:28.487414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883288+0000","last_change":"2024-09-21T02:13:24.905477+0000","last_active":"2024-09-21T02:13:26.883288+0000","last_peered":"2024-09-21T02:13:26.883288+0000","last_clean":"2024-09-21T02:13:26.883288+0000","last_became_active":"2024-09-21T02:13:24.904718+0000","last_became_peered":"2024-09-21T02:13:24.904718+0000","last_unstale":"2024-09-21T02:13:26.883288+0000","last_undegraded":"2024-09-21T02:13:26.883288+0000","last_fullsized":"2024-09-21T02:13:26.883288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:52.697461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.883454+0000","last_change":"2024-09-21T02:13:25.897102+0000","last_active":"2024-09-21T02:13:26.883454+0000","last_peered":"2024-09-21T02:13:26.883454+0000","last_clean":"2024-09-21T02:13:26.883454+0000","last_became_active":"2024-09-21T02:13:25.896342+0000","last_became_peered":"2024-09-21T02:13:25.896342+0000","last_unstale":"2024-09-21T02:13:26.883454+0000","last_undegraded":"2024-09-21T02:13:26.883454+0000","last_fullsized":"2024-09-21T02:13:26.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:24.855267+0000","last_clean_scrub_stamp":"2024-09-21T02:13:24.855267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:29.490211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T02:13:26.882146+0000","last_change":"2024-09-21T02:13:24.908318+0000","last_active":"2024-09-21T02:13:26.882146+0000","last_peered":"2024-09-21T02:13:26.882146+0000","last_clean":"2024-09-21T02:13:26.882146+0000","last_became_active":"2024-09-21T02:13:24.905193+0000","last_became_peered":"2024-09-21T02:13:24.905193+0000","last_unstale":"2024-09-21T02:13:26.882146+0000","last_undegraded":"2024-09-21T02:13:26.882146+0000","last_fullsized":"2024-09-21T02:13:26.882146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:23.844820+0000","last_clean_scrub_stamp":"2024-09-21T02:13:23.844820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:56.042929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":81753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":835584,"data_stored":672213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":237568,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36168,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335672,"statfs":{"total":96636764160,"available":96599728128,"internally_reserved":0,"allocated":831488,"data_stored":672191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":84037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:13:38.285 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:13:38.285 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T02:13:38.285 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T02:13:38.285 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T02:13:38.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 -- 172.21.15.170:0/33219822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50004d50 msgr2=0x7f6b50005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/33219822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50004d50 0x7f6b50005220 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b68071110 tx=0x7f6b6001caf0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6d132640 1 -- 172.21.15.170:0/33219822 reap_dead start 2024-09-21T02:13:38.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 -- 172.21.15.170:0/33219822 shutdown_connections 2024-09-21T02:13:38.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 -- 172.21.15.170:0/33219822 >> 172.21.15.170:0/33219822 conn(0x7f6b680b1c40 msgr2=0x7f6b680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:38.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 -- 172.21.15.170:0/33219822 shutdown_connections 2024-09-21T02:13:38.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 -- 172.21.15.170:0/33219822 wait complete. 2024-09-21T02:13:38.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 Processor -- start 2024-09-21T02:13:38.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.474+0000 7f6b6eb95640 1 -- start start 2024-09-21T02:13:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 0x7f6b6814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b6815a790 0x7f6b6814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b680ae770 0x7f6b680aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b680cb9f0 con 0x7f6b6809dbe0 2024-09-21T02:13:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b680cb870 con 0x7f6b6815a790 2024-09-21T02:13:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6b680cbb70 con 0x7f6b680ae770 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 0x7f6b6814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 0x7f6b6814b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54250/0 (socket says 172.21.15.170:54250) 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 -- 172.21.15.170:0/1657727932 learned_addr learned my addr 172.21.15.170:0/1657727932 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b67fff640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b6815a790 0x7f6b6814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6d132640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b680ae770 0x7f6b680aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 -- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b680ae770 msgr2=0x7f6b680aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b680ae770 0x7f6b680aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 -- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b6815a790 msgr2=0x7f6b6814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b6815a790 0x7f6b6814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 -- 172.21.15.170:0/1657727932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6809dff0 con 0x7f6b6809dbe0 2024-09-21T02:13:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6c931640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 0x7f6b6814b630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e7e0 tx=0x7f6b5c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c01c070 con 0x7f6b6809dbe0 2024-09-21T02:13:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b5c00a970 con 0x7f6b6809dbe0 2024-09-21T02:13:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00adc0 con 0x7f6b6809dbe0 2024-09-21T02:13:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b680af0c0 con 0x7f6b6809dbe0 2024-09-21T02:13:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6809ae40 con 0x7f6b6809dbe0 2024-09-21T02:13:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.478+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b680cbd50 con 0x7f6b6809dbe0 2024-09-21T02:13:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.482+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b5c01f070 con 0x7f6b6809dbe0 2024-09-21T02:13:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.482+0000 7f6b657fa640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b54073200 0x7f6b54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.482+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6b5c0959c0 con 0x7f6b6809dbe0 2024-09-21T02:13:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.482+0000 7f6b67fff640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b54073200 0x7f6b54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.482+0000 7f6b67fff640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b54073200 0x7f6b54075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009ba0 tx=0x7f6b580091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:38.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.486+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6b5c097e20 con 0x7f6b6809dbe0 2024-09-21T02:13:38.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.678+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6b6806de10 con 0x7f6b6809dbe0 2024-09-21T02:13:38.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.678+0000 7f6b657fa640 1 -- 172.21.15.170:0/1657727932 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6b680af0c0 con 0x7f6b6809dbe0 2024-09-21T02:13:38.679 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:38.679 INFO:teuthology.orchestra.run.smithi170.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T02:13:38.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b54073200 msgr2=0x7f6b54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b54073200 0x7f6b54075710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009ba0 tx=0x7f6b580091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 msgr2=0x7f6b6814b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 0x7f6b6814b630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e7e0 tx=0x7f6b5c00eca0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 shutdown_connections 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b54073200 0x7f6b54075710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b680ae770 0x7f6b680aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b6815a790 0x7f6b6814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 --2- 172.21.15.170:0/1657727932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6809dbe0 0x7f6b6814b630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 >> 172.21.15.170:0/1657727932 conn(0x7f6b680b1c40 msgr2=0x7f6b680acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 shutdown_connections 2024-09-21T02:13:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.682+0000 7f6b6eb95640 1 -- 172.21.15.170:0/1657727932 wait complete. 2024-09-21T02:13:38.700 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T02:13:38.700 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 -- 172.21.15.170:0/3304774636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80cadf0 msgr2=0x7f3bb80cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/3304774636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80cadf0 0x7f3bb80cb200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009c80 tx=0x7f3bb001cbc0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 -- 172.21.15.170:0/3304774636 shutdown_connections 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/3304774636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80cadf0 0x7f3bb80cb200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/3304774636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bb80ce4c0 0x7f3bb80ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 -- 172.21.15.170:0/3304774636 >> 172.21.15.170:0/3304774636 conn(0x7f3bb80b1c40 msgr2=0x7f3bb80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 -- 172.21.15.170:0/3304774636 shutdown_connections 2024-09-21T02:13:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 -- 172.21.15.170:0/3304774636 wait complete. 2024-09-21T02:13:38.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 Processor -- start 2024-09-21T02:13:38.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.846+0000 7f3bbd97f640 1 -- start start 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bb80cadf0 0x7f3bb8147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 0x7f3bb8165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bb815f990 0x7f3bb81473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8166160 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8165fe0 con 0x7f3bb80cadf0 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb81662e0 con 0x7f3bb815f990 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 0x7f3bb8165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 0x7f3bb8165820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54268/0 (socket says 172.21.15.170:54268) 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 -- 172.21.15.170:0/2898969124 learned_addr learned my addr 172.21.15.170:0/2898969124 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb77fe640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bb815f990 0x7f3bb81473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb6ffd640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bb80cadf0 0x7f3bb8147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 -- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bb815f990 msgr2=0x7f3bb81473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bb815f990 0x7f3bb81473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 -- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bb80cadf0 msgr2=0x7f3bb8147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bb80cadf0 0x7f3bb8147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 -- 172.21.15.170:0/2898969124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb809d250 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3baffff640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 0x7f3bb8165820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c9a0 tx=0x7f3ba000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0007bb0 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba0007e40 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb809d530 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000f6b0 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb8165d60 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb80cb200 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ba000f850 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb4ff9640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b98073200 0x7f3b98075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:38.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3ba0094a40 con 0x7f3bb80ce4c0 2024-09-21T02:13:38.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.850+0000 7f3bb6ffd640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b98073200 0x7f3b98075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:38.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.854+0000 7f3bb6ffd640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b98073200 0x7f3b98075710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800a750 tx=0x7f3ba8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:38.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:38.854+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3ba0014030 con 0x7f3bb80ce4c0 2024-09-21T02:13:39.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.050+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb80d0d90 con 0x7f3bb80ce4c0 2024-09-21T02:13:39.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bb4ff9640 1 -- 172.21.15.170:0/2898969124 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f3ba005fa70 con 0x7f3bb80ce4c0 2024-09-21T02:13:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:39.053 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:13:39.054 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:13:39.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b98073200 msgr2=0x7f3b98075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b98073200 0x7f3b98075710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800a750 tx=0x7f3ba8008040 comp rx=0 tx=0).stop 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 msgr2=0x7f3bb8165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 0x7f3bb8165820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c9a0 tx=0x7f3ba000ce60 comp rx=0 tx=0).stop 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 shutdown_connections 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b98073200 0x7f3b98075710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bb815f990 0x7f3bb81473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ce4c0 0x7f3bb8165820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 --2- 172.21.15.170:0/2898969124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bb80cadf0 0x7f3bb8147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 >> 172.21.15.170:0/2898969124 conn(0x7f3bb80b1c40 msgr2=0x7f3bb8166ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 shutdown_connections 2024-09-21T02:13:39.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.054+0000 7f3bbd97f640 1 -- 172.21.15.170:0/2898969124 wait complete. 2024-09-21T02:13:39.074 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 -- 172.21.15.170:0/1600462799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19b8004d50 msgr2=0x7f19b80051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 --2- 172.21.15.170:0/1600462799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19b8004d50 0x7f19b80051a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c610 tx=0x7f19c801d300 comp rx=0 tx=0).stop 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d5225640 1 -- 172.21.15.170:0/1600462799 reap_dead start 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 -- 172.21.15.170:0/1600462799 shutdown_connections 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 -- 172.21.15.170:0/1600462799 >> 172.21.15.170:0/1600462799 conn(0x7f19d00b1c40 msgr2=0x7f19d00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 -- 172.21.15.170:0/1600462799 shutdown_connections 2024-09-21T02:13:39.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 -- 172.21.15.170:0/1600462799 wait complete. 2024-09-21T02:13:39.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 Processor -- start 2024-09-21T02:13:39.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.222+0000 7f19d6c88640 1 -- start start 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19d009b900 0x7f19d014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19d009c500 0x7f19d009c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 0x7f19d006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19d00cd770 con 0x7f19d006eeb0 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f19d00cd5f0 con 0x7f19d009c500 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f19d0074e80 con 0x7f19d009b900 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 0x7f19d006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 0x7f19d006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54292/0 (socket says 172.21.15.170:54292) 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 -- 172.21.15.170:0/833973262 learned_addr learned my addr 172.21.15.170:0/833973262 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c7fff640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19d009c500 0x7f19d009c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d4a24640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19d009b900 0x7f19d014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 -- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19d009b900 msgr2=0x7f19d014d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19d009b900 0x7f19d014d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 -- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19d009c500 msgr2=0x7f19d009c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19d009c500 0x7f19d009c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 -- 172.21.15.170:0/833973262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d009ce50 con 0x7f19d006eeb0 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c7fff640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19d009c500 0x7f19d009c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d5225640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 0x7f19d006f2c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c610 tx=0x7f19c8001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80bd070 con 0x7f19d006eeb0 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19c80011d0 con 0x7f19d006eeb0 2024-09-21T02:13:39.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80062d0 con 0x7f19d006eeb0 2024-09-21T02:13:39.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d009d0e0 con 0x7f19d006eeb0 2024-09-21T02:13:39.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d009bd10 con 0x7f19d006eeb0 2024-09-21T02:13:39.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d0070670 con 0x7f19d006eeb0 2024-09-21T02:13:39.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19c80b5070 con 0x7f19d006eeb0 2024-09-21T02:13:39.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c57fa640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.226+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f19c812e470 con 0x7f19d006eeb0 2024-09-21T02:13:39.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.230+0000 7f19d4a24640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.230+0000 7f19d4a24640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19bc073200 0x7f19bc075710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f19d009c160 tx=0x7f19cc00c500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:39.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.230+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19c80170b0 con 0x7f19d006eeb0 2024-09-21T02:13:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.442+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19d00ca270 con 0x7f19d006eeb0 2024-09-21T02:13:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19c57fa640 1 -- 172.21.15.170:0/833973262 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f19c80f9690 con 0x7f19d006eeb0 2024-09-21T02:13:39.446 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:39.446 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:13:39.446 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:13:39.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19bc073200 msgr2=0x7f19bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19bc073200 0x7f19bc075710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f19d009c160 tx=0x7f19cc00c500 comp rx=0 tx=0).stop 2024-09-21T02:13:39.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 msgr2=0x7f19d006f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 0x7f19d006f2c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19c800c610 tx=0x7f19c8001d60 comp rx=0 tx=0).stop 2024-09-21T02:13:39.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 shutdown_connections 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19d006eeb0 0x7f19d006f2c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19d009c500 0x7f19d009c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 --2- 172.21.15.170:0/833973262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19d009b900 0x7f19d014d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 >> 172.21.15.170:0/833973262 conn(0x7f19d00b1c40 msgr2=0x7f19d00cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 shutdown_connections 2024-09-21T02:13:39.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.446+0000 7f19d6c88640 1 -- 172.21.15.170:0/833973262 wait complete. 2024-09-21T02:13:39.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 -- 172.21.15.170:0/575316531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 msgr2=0x7f66d40d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 --2- 172.21.15.170:0/575316531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d40d1260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66d408c5a0 tx=0x7f66c401c7f0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 -- 172.21.15.170:0/575316531 shutdown_connections 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 --2- 172.21.15.170:0/575316531 >> v2:172.21.15.191:3300/0 conn(0x7f66d40b1230 0x7f66d40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 --2- 172.21.15.170:0/575316531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d40d1260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 -- 172.21.15.170:0/575316531 >> 172.21.15.170:0/575316531 conn(0x7f66d408bf50 msgr2=0x7f66d409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 -- 172.21.15.170:0/575316531 shutdown_connections 2024-09-21T02:13:39.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.622+0000 7f66db4a3640 1 -- 172.21.15.170:0/575316531 wait complete. 2024-09-21T02:13:39.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 Processor -- start 2024-09-21T02:13:39.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 -- start start 2024-09-21T02:13:39.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66d40b1230 0x7f66d40b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d40b3fa0 0x7f66d4155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d41575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f66d416c070 con 0x7f66d40d0e50 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66d416bef0 con 0x7f66d40b3fa0 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f66d416c1f0 con 0x7f66d40b1230 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d41575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d41575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54306/0 (socket says 172.21.15.170:54306) 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 -- 172.21.15.170:0/2179220235 learned_addr learned my addr 172.21.15.170:0/2179220235 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d923f640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66d40b1230 0x7f66d40b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d8a3e640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d40b3fa0 0x7f66d4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 -- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66d40b1230 msgr2=0x7f66d40b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66d40b1230 0x7f66d40b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 -- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d40b3fa0 msgr2=0x7f66d4155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d40b3fa0 0x7f66d4155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 -- 172.21.15.170:0/2179220235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d40b7aa0 con 0x7f66d40d0e50 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d923f640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66d40b1230 0x7f66d40b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66d9a40640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d41575b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e940 tx=0x7f66cc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc004d30 con 0x7f66d40d0e50 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66cc00cac0 con 0x7f66d40d0e50 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d41f90f0 con 0x7f66d40d0e50 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc005020 con 0x7f66d40d0e50 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.626+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d41f8de0 con 0x7f66d40d0e50 2024-09-21T02:13:39.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.630+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d40b7810 con 0x7f66d40d0e50 2024-09-21T02:13:39.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.630+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f66cc0051c0 con 0x7f66d40d0e50 2024-09-21T02:13:39.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.630+0000 7f66c9ffb640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66b8073230 0x7f66b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.630+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f66cc094ec0 con 0x7f66d40d0e50 2024-09-21T02:13:39.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.630+0000 7f66d923f640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66b8073230 0x7f66b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.630+0000 7f66d923f640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66b8073230 0x7f66b8075740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66d000a850 tx=0x7f66d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:39.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.634+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f66cc016030 con 0x7f66d40d0e50 2024-09-21T02:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.794+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66d40b1640 con 0x7f66d40d0e50 2024-09-21T02:13:39.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.798+0000 7f66c9ffb640 1 -- 172.21.15.170:0/2179220235 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7f66cc05ff70 con 0x7f66d40d0e50 2024-09-21T02:13:39.799 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:39.799 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66b8073230 msgr2=0x7f66b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66b8073230 0x7f66b8075740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66d000a850 tx=0x7f66d0008040 comp rx=0 tx=0).stop 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 msgr2=0x7f66d41575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d41575b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e940 tx=0x7f66cc00ee00 comp rx=0 tx=0).stop 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 shutdown_connections 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66b8073230 0x7f66b8075740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d40d0e50 0x7f66d41575b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66d40b3fa0 0x7f66d4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 --2- 172.21.15.170:0/2179220235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66d40b1230 0x7f66d40b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 >> 172.21.15.170:0/2179220235 conn(0x7f66d408bf50 msgr2=0x7f66d40b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 shutdown_connections 2024-09-21T02:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.802+0000 7f66db4a3640 1 -- 172.21.15.170:0/2179220235 wait complete. 2024-09-21T02:13:39.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 -- 172.21.15.170:0/1618943665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940b5410 msgr2=0x7f0a9406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/1618943665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940b5410 0x7f0a9406fa30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a940b3a10 tx=0x7f0a8401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 -- 172.21.15.170:0/1618943665 shutdown_connections 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/1618943665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a940b2f40 0x7f0a940b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/1618943665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940b5410 0x7f0a9406fa30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 -- 172.21.15.170:0/1618943665 >> 172.21.15.170:0/1618943665 conn(0x7f0a940b1c40 msgr2=0x7f0a940b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 -- 172.21.15.170:0/1618943665 shutdown_connections 2024-09-21T02:13:39.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 -- 172.21.15.170:0/1618943665 wait complete. 2024-09-21T02:13:39.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 Processor -- start 2024-09-21T02:13:39.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.974+0000 7f0a9af4b640 1 -- start start 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a940b2f40 0x7f0a94072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a940b5410 0x7f0a94164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 0x7f0a9415f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9416a060 con 0x7f0a940ccb00 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0a940701b0 con 0x7f0a940b2f40 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9416a1e0 con 0x7f0a940b5410 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 0x7f0a9415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 0x7f0a9415f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54320/0 (socket says 172.21.15.170:54320) 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 -- 172.21.15.170:0/111410843 learned_addr learned my addr 172.21.15.170:0/111410843 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a98ce7640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a940b2f40 0x7f0a94072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a93fff640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a940b5410 0x7f0a94164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 -- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a940b5410 msgr2=0x7f0a94164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a940b5410 0x7f0a94164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 -- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a940b2f40 msgr2=0x7f0a94072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a940b2f40 0x7f0a94072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 -- 172.21.15.170:0/111410843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9415e3a0 con 0x7f0a940ccb00 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a994e8640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 0x7f0a9415f440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00b4c0 tx=0x7f0a8c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c015020 con 0x7f0a940ccb00 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a8c011920 con 0x7f0a940ccb00 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9415e680 con 0x7f0a940ccb00 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011d70 con 0x7f0a940ccb00 2024-09-21T02:13:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9415ec40 con 0x7f0a940ccb00 2024-09-21T02:13:39.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a940b3350 con 0x7f0a940ccb00 2024-09-21T02:13:39.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.978+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a8c018020 con 0x7f0a940ccb00 2024-09-21T02:13:39.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.982+0000 7f0a917fa640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a74073230 0x7f0a74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:39.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.982+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0a8c093e60 con 0x7f0a940ccb00 2024-09-21T02:13:39.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.982+0000 7f0a98ce7640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a74073230 0x7f0a74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:39.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.982+0000 7f0a98ce7640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a74073230 0x7f0a74075740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a840af000 tx=0x7f0a8401b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:39.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:39.986+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0a8c01f030 con 0x7f0a940ccb00 2024-09-21T02:13:40.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.178+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a940cef70 con 0x7f0a940ccb00 2024-09-21T02:13:40.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.178+0000 7f0a917fa640 1 -- 172.21.15.170:0/111410843 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f0a8c05ef10 con 0x7f0a940ccb00 2024-09-21T02:13:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:40.180 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:13:40.180 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a74073230 msgr2=0x7f0a74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a74073230 0x7f0a74075740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a840af000 tx=0x7f0a8401b120 comp rx=0 tx=0).stop 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 msgr2=0x7f0a9415f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 0x7f0a9415f440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00b4c0 tx=0x7f0a8c00b980 comp rx=0 tx=0).stop 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 shutdown_connections 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a74073230 0x7f0a74075740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a940b5410 0x7f0a94164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940ccb00 0x7f0a9415f440 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 --2- 172.21.15.170:0/111410843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a940b2f40 0x7f0a94072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 >> 172.21.15.170:0/111410843 conn(0x7f0a940b1c40 msgr2=0x7f0a940c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 shutdown_connections 2024-09-21T02:13:40.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.182+0000 7f0a9af4b640 1 -- 172.21.15.170:0/111410843 wait complete. 2024-09-21T02:13:40.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:13:25.292945+0000', 'modified': '2024-09-21T02:13:28.337813+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4381': {'gid': 4381, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/3675017549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6833', 'nonce': 3675017549}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 3675017549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4381, 'qdb_cluster': [4381]} 2024-09-21T02:13:40.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:13:40.603 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:13:40.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 -- 172.21.15.170:0/3395675981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b3fa0 msgr2=0x7f62240cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3395675981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b3fa0 0x7f62240cf970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6214009900 tx=0x7f621401c670 comp rx=0 tx=0).stop 2024-09-21T02:13:40.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 -- 172.21.15.170:0/3395675981 shutdown_connections 2024-09-21T02:13:40.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3395675981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240b1230 0x7f62240b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3395675981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b3fa0 0x7f62240cf970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 -- 172.21.15.170:0/3395675981 >> 172.21.15.170:0/3395675981 conn(0x7f622408bf50 msgr2=0x7f622409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 -- 172.21.15.170:0/3395675981 shutdown_connections 2024-09-21T02:13:40.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.770+0000 7f6229fa5640 1 -- 172.21.15.170:0/3395675981 wait complete. 2024-09-21T02:13:40.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 Processor -- start 2024-09-21T02:13:40.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 -- start start 2024-09-21T02:13:40.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 0x7f622415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240b3fa0 0x7f62240a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62240d0e50 0x7f62240cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f622416bfc0 con 0x7f62240b1230 2024-09-21T02:13:40.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f622416be40 con 0x7f62240d0e50 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f622416c140 con 0x7f62240b3fa0 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 0x7f622415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 0x7f622415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54342/0 (socket says 172.21.15.170:54342) 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 -- 172.21.15.170:0/3063652917 learned_addr learned my addr 172.21.15.170:0/3063652917 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6223fff640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62240d0e50 0x7f62240cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 -- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240b3fa0 msgr2=0x7f62240a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240b3fa0 0x7f62240a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 -- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62240d0e50 msgr2=0x7f62240cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62240d0e50 0x7f62240cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 -- 172.21.15.170:0/3063652917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62240cc500 con 0x7f62240b1230 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f62237fe640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 0x7f622415cc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6214009850 tx=0x7f62140a85f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f620bfff640 1 -- 172.21.15.170:0/3063652917 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140042c0 con 0x7f62240b1230 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f620bfff640 1 -- 172.21.15.170:0/3063652917 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6214003600 con 0x7f62240b1230 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62240cc220 con 0x7f62240b1230 2024-09-21T02:13:40.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f620bfff640 1 -- 172.21.15.170:0/3063652917 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214003a50 con 0x7f62240b1230 2024-09-21T02:13:40.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.774+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62240cd430 con 0x7f62240b1230 2024-09-21T02:13:40.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f620bfff640 1 -- 172.21.15.170:0/3063652917 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6214003bf0 con 0x7f62240b1230 2024-09-21T02:13:40.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f620bfff640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6210073200 0x7f6210075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f620bfff640 1 -- 172.21.15.170:0/3063652917 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f621412ec10 con 0x7f62240b1230 2024-09-21T02:13:40.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6222ffd640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6210073200 0x7f6210075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 Processor -- start 2024-09-21T02:13:40.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 -- start start 2024-09-21T02:13:40.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6222ffd640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6210073200 0x7f6210075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62240b4da0 tx=0x7f6218006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a3622256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a362225be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a362226120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a3620fa410 con 0x55a3620d4950 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a362037b40 con 0x55a361ce4460 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55a362026f70 con 0x55a3620bdca0 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a362225be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a362225be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54352/0 (socket says 172.21.15.170:54352) 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.778+0000 7f6222ffd640 1 -- 172.21.15.170:0/4051547190 learned_addr learned my addr 172.21.15.170:0/4051547190 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f62237fe640 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a3622256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6223fff640 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a362226120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6222ffd640 1 -- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 msgr2=0x55a362226120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6222ffd640 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a362226120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6222ffd640 1 -- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 msgr2=0x55a3622256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6222ffd640 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a3622256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6222ffd640 1 -- 172.21.15.170:0/4051547190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a361fee5b0 con 0x55a3620d4950 2024-09-21T02:13:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6222ffd640 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a362225be0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6218013830 tx=0x7f621800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b8d50d2a74f8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:40.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f62097fa640 1 -- 172.21.15.170:0/4051547190 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218012b10 con 0x55a3620d4950 2024-09-21T02:13:40.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f62097fa640 1 -- 172.21.15.170:0/4051547190 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6218022070 con 0x55a3620d4950 2024-09-21T02:13:40.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f62097fa640 1 -- 172.21.15.170:0/4051547190 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621801d3f0 con 0x55a3620d4950 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- 172.21.15.170:0/4051547190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 msgr2=0x55a362225be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a362225be0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6218013830 tx=0x7f621800e000 comp rx=0 tx=0).stop 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- 172.21.15.170:0/4051547190 shutdown_connections 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a362226120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a362225be0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- 172.21.15.170:0/4051547190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a3622256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- 172.21.15.170:0/4051547190 >> 172.21.15.170:0/4051547190 conn(0x55a3621aad90 msgr2=0x55a3621da690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- 172.21.15.170:0/4051547190 shutdown_connections 2024-09-21T02:13:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- 172.21.15.170:0/4051547190 wait complete. 2024-09-21T02:13:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- start start 2024-09-21T02:13:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 Processor -- start 2024-09-21T02:13:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36222c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a36222ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a36222d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a361fc15f0 con 0x55a3620d4950 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a36202ad80 con 0x55a361ce4460 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55a361fcd520 con 0x55a3620bdca0 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.782+0000 7f6231270000 10 client.0 _ll_register_callbacks cb 0x55a361fd1830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a36222d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a36222d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54368/0 (socket says 172.21.15.170:54368) 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 -- 172.21.15.170:0/3958862326 learned_addr learned my addr 172.21.15.170:0/3958862326 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6222ffd640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a36222ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62237fe640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36222c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 -- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 msgr2=0x55a36222ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a36222ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 -- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 msgr2=0x55a36222c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36222c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 -- 172.21.15.170:0/3958862326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a361fc5190 con 0x55a3620d4950 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6223fff640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a36222d3a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f621c00f660 tx=0x7f621c00fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9d01bd5347ddd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 1 -- 172.21.15.170:0/3958862326 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c003b80 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 1 -- 172.21.15.170:0/3958862326 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f621c003d20 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a361fbf110 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 1 -- 172.21.15.170:0/3958862326 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f621c00d8c0 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 10 client.4668 fetch_fsmap learned FSMap version 8 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a3620abf10 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 1 -- 172.21.15.170:0/3958862326 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a361fbf110 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 1 -- 172.21.15.170:0/3958862326 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f621c021050 con 0x55a3620d4950 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 10 client.4668 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 20 client.4668 populate_metadata read hostname 'smithi170' 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cc97d0 0x55a362236d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 4 client.4668 mds_command: new command op to 4381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:13:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 --> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a362039a90 con 0x55a361cc97d0 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62237fe640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cc97d0 0x55a362236d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 10 client.4668 ms_handle_connect on v2:172.21.15.170:6833/3675017549 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62237fe640 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cc97d0 0x55a362236d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60dc2e7d2b7112bb server_cookie=cbbb4b8f3bd165cd in_seq=0 out_seq=0 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 1 -- 172.21.15.170:0/3958862326 <== mds.0 v2:172.21.15.170:6833/3675017549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a362039a90 con 0x55a361cc97d0 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 10 client.4668 handle_command_reply: tid=0 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 client.4668 shutdown 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 msgr2=0x55a36222d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a36222d3a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f621c00f660 tx=0x7f621c00fb20 comp rx=0 tx=0).stop 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 shutdown_connections 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cc97d0 0x55a362236d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620bdca0 0x55a36222ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620d4950 0x55a36222d3a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f62027fc640 0 client.4668 ms_handle_reset on v2:172.21.15.170:6833/3675017549 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 --2- 172.21.15.170:0/3958862326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36222c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 >> 172.21.15.170:0/3958862326 conn(0x55a3621aad90 msgr2=0x55a3621e1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 shutdown_connections 2024-09-21T02:13:40.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 1 -- 172.21.15.170:0/3958862326 wait complete. 2024-09-21T02:13:40.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.786+0000 7f6231270000 20 client.4668 trim_cache size 0 max 16384 2024-09-21T02:13:40.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 Processor -- start 2024-09-21T02:13:40.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 -- start start 2024-09-21T02:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36229fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620d4950 0x55a3622a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f62242055c0 con 0x55a3620bdca0 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f62242046b0 con 0x55a361ce4460 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f622414bcc0 con 0x55a3620d4950 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54372/0 (socket says 172.21.15.170:54372) 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 -- 172.21.15.170:0/2878889489 learned_addr learned my addr 172.21.15.170:0/2878889489 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6223fff640 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620d4950 0x55a3622a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 -- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620d4950 msgr2=0x55a3622a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f62237fe640 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36229fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620d4950 0x55a3622a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 -- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 msgr2=0x55a36229fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36229fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 -- 172.21.15.170:0/2878889489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a362082db0 con 0x55a3620bdca0 2024-09-21T02:13:40.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.806+0000 7f6222ffd640 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a0370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f621801e000 tx=0x7f6218004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee11f0adff65dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:40.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6208ff9640 1 -- 172.21.15.170:0/2878889489 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621802b070 con 0x55a3620bdca0 2024-09-21T02:13:40.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6208ff9640 1 -- 172.21.15.170:0/2878889489 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6218022040 con 0x55a3620bdca0 2024-09-21T02:13:40.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6208ff9640 1 -- 172.21.15.170:0/2878889489 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218012b70 con 0x55a3620bdca0 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- 172.21.15.170:0/2878889489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 msgr2=0x55a3622a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a0370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f621801e000 tx=0x7f6218004700 comp rx=0 tx=0).stop 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- 172.21.15.170:0/2878889489 shutdown_connections 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3620d4950 0x55a3622a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a0370 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- 172.21.15.170:0/2878889489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a361ce4460 0x55a36229fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- 172.21.15.170:0/2878889489 >> 172.21.15.170:0/2878889489 conn(0x55a3621aad90 msgr2=0x55a362223620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- 172.21.15.170:0/2878889489 shutdown_connections 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- 172.21.15.170:0/2878889489 wait complete. 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- start start 2024-09-21T02:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 Processor -- start 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a361ce4460 0x55a3622a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3620d4950 0x55a3622a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a36202f3b0 con 0x55a3620bdca0 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a361ff2ee0 con 0x55a3620d4950 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55a362036910 con 0x55a361ce4460 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6231270000 10 client.0 _ll_register_callbacks cb 0x7f621c00a510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54380/0 (socket says 172.21.15.170:54380) 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 -- 172.21.15.170:0/1739578611 learned_addr learned my addr 172.21.15.170:0/1739578611 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6223fff640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3620d4950 0x55a3622a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f62237fe640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a361ce4460 0x55a3622a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 -- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a361ce4460 msgr2=0x55a3622a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a361ce4460 0x55a3622a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 -- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3620d4950 msgr2=0x55a3622a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3620d4950 0x55a3622a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 -- 172.21.15.170:0/1739578611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a361fedab0 con 0x55a3620bdca0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6223fff640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3620d4950 0x55a3622a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f6222ffd640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a43b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55a3620317e0 tx=0x7f6218013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5d162c7543ec5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.810+0000 7f62027fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 1 -- 172.21.15.170:0/1739578611 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218034070 con 0x55a3620bdca0 2024-09-21T02:13:40.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 1 -- 172.21.15.170:0/1739578611 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6218030660 con 0x55a3620bdca0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6218022040 con 0x55a3620bdca0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 1 -- 172.21.15.170:0/1739578611 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f621803f050 con 0x55a3620bdca0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 10 client.4674 fetch_fsmap learned FSMap version 8 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a361fcc070 con 0x55a3620bdca0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 1 -- 172.21.15.170:0/1739578611 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6218022040 con 0x55a3620bdca0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 1 -- 172.21.15.170:0/1739578611 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f6218042020 con 0x55a3620bdca0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 10 client.4674 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 20 client.4674 populate_metadata read hostname 'smithi170' 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cb7830 0x55a36225fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 4 client.4674 mds_command: new command op to 4381 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 --> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] -- command(tid 0: {"prefix": "status"}) -- 0x55a362035540 con 0x55a361cb7830 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62237fe640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cb7830 0x55a36225fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 10 client.4674 ms_handle_connect on v2:172.21.15.170:6833/3675017549 2024-09-21T02:13:40.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62237fe640 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cb7830 0x55a36225fe20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0ed9155f631cf3 server_cookie=29475205ed977560 in_seq=0 out_seq=0 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 1 -- 172.21.15.170:0/1739578611 <== mds.0 v2:172.21.15.170:6833/3675017549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55a362035540 con 0x55a361cb7830 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 10 client.4674 handle_command_reply: tid=0 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 client.4674 shutdown 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 msgr2=0x55a3622a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a43b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55a3620317e0 tx=0x7f6218013700 comp rx=0 tx=0).stop 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 shutdown_connections 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:6833/3675017549,v1:172.21.15.170:6835/3675017549] conn(0x55a361cb7830 0x55a36225fe20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3620d4950 0x55a3622a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a3620bdca0 0x55a3622a43b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f62027fc640 0 client.4674 ms_handle_reset on v2:172.21.15.170:6833/3675017549 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 --2- 172.21.15.170:0/1739578611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a361ce4460 0x55a3622a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 >> 172.21.15.170:0/1739578611 conn(0x55a3621aad90 msgr2=0x55a3622366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 shutdown_connections 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 1 -- 172.21.15.170:0/1739578611 wait complete. 2024-09-21T02:13:40.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6231270000 20 client.4674 trim_cache size 0 max 16384 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4381, 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 14.942590151999999, 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 8, 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 23, 2024-09-21T02:13:40.819 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 23, 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 20.998587067999999 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6210073200 msgr2=0x7f6210075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6210073200 0x7f6210075710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62240b4da0 tx=0x7f6218006bd0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 msgr2=0x7f622415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 0x7f622415cc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6214009850 tx=0x7f62140a85f0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 shutdown_connections 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6210073200 0x7f6210075710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240b3fa0 0x7f62240a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62240d0e50 0x7f62240cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 --2- 172.21.15.170:0/3063652917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62240b1230 0x7f622415cc90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 >> 172.21.15.170:0/3063652917 conn(0x7f622408bf50 msgr2=0x7f62240b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 shutdown_connections 2024-09-21T02:13:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:40.814+0000 7f6229fa5640 1 -- 172.21.15.170:0/3063652917 wait complete. 2024-09-21T02:13:40.834 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-21T02:13:40.857 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-21T02:13:40.857 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T02:13:40.857 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-21T02:13:40.857 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:13:40.857 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:40.857 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-21T02:13:40.857 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:13:40.858 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:40.858 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-21T02:13:40.858 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:13:40.858 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:40.858 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-21T02:13:40.858 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T02:13:40.858 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:40.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:40.858 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:13:40.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:40.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2024-09-21T02:13:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2024-09-21T02:13:40.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:13:40.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:40.921 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:13:40.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:40.927 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-09-21T02:13:40.992 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2024-09-21T02:13:40.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:13:40.994 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-21T02:13:40.994 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:13:40.994 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 -- 172.21.15.170:0/3533710690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 msgr2=0x7f0a3000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/3533710690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a3000b9d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400c660 tx=0x7f0a4401d2e0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 -- 172.21.15.170:0/3533710690 shutdown_connections 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/3533710690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a3000b9d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 -- 172.21.15.170:0/3533710690 >> 172.21.15.170:0/3533710690 conn(0x7f0a4c0b1c40 msgr2=0x7f0a4c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 -- 172.21.15.170:0/3533710690 shutdown_connections 2024-09-21T02:13:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 -- 172.21.15.170:0/3533710690 wait complete. 2024-09-21T02:13:41.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 Processor -- start 2024-09-21T02:13:41.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.154+0000 7f0a50c4b640 1 -- start start 2024-09-21T02:13:41.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a4c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a4c15bd50 0x7f0a4c1f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4c1f71a0 0x7f0a4c09be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c0cf760 con 0x7f0a30005260 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c0c3a10 con 0x7f0a4c15bd50 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c0cf8e0 con 0x7f0a4c1f71a0 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a4c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a4c1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54390/0 (socket says 172.21.15.170:54390) 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 -- 172.21.15.170:0/849930325 learned_addr learned my addr 172.21.15.170:0/849930325 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a49d9b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a4c15bd50 0x7f0a4c1f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 -- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4c1f71a0 msgr2=0x7f0a4c09be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4c1f71a0 0x7f0a4c09be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 -- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a4c15bd50 msgr2=0x7f0a4c1f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a4c15bd50 0x7f0a4c1f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 -- 172.21.15.170:0/849930325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c1f7b10 con 0x7f0a30005260 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a4a59c640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a4c1536b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c003df0 tx=0x7f0a3c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c015070 con 0x7f0a30005260 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a3c011860 con 0x7f0a30005260 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c001d50 con 0x7f0a30005260 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c1f75b0 con 0x7f0a30005260 2024-09-21T02:13:41.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c15e060 con 0x7f0a30005260 2024-09-21T02:13:41.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c0b3510 con 0x7f0a30005260 2024-09-21T02:13:41.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a3c018070 con 0x7f0a30005260 2024-09-21T02:13:41.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.158+0000 7f0a42ffd640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a34073200 0x7f0a34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:41.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.162+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0a3c094dc0 con 0x7f0a30005260 2024-09-21T02:13:41.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.162+0000 7f0a49d9b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a34073200 0x7f0a34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:41.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.162+0000 7f0a49d9b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a34073200 0x7f0a34075710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0b1bf0 tx=0x7f0a380091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:41.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.166+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0a4c1f75b0 con 0x7f0a30005260 2024-09-21T02:13:41.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a4c0cf410 con 0x7f0a30005260 2024-09-21T02:13:41.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a42ffd640 1 -- 172.21.15.170:0/849930325 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f0a3c05ff30 con 0x7f0a30005260 2024-09-21T02:13:41.366 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:13:41.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a34073200 msgr2=0x7f0a34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a34073200 0x7f0a34075710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0b1bf0 tx=0x7f0a380091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 msgr2=0x7f0a4c1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a4c1536b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c003df0 tx=0x7f0a3c001640 comp rx=0 tx=0).stop 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 shutdown_connections 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a34073200 0x7f0a34075710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a4c1f71a0 0x7f0a4c09be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a4c15bd50 0x7f0a4c1f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 --2- 172.21.15.170:0/849930325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a30005260 0x7f0a4c1536b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 >> 172.21.15.170:0/849930325 conn(0x7f0a4c0b1c40 msgr2=0x7f0a4c0c7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 shutdown_connections 2024-09-21T02:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41.366+0000 7f0a50c4b640 1 -- 172.21.15.170:0/849930325 wait complete. 2024-09-21T02:13:41.386 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:13:41.386 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:13:41.386 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:13:41.386 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:13:41.387 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:41.387 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:13:41.387 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:13:41.387 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:13:41.387 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:13:41.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:41.387 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:13:41.395 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42494sec preferred_lft 42494sec 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:13:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:41.397 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip addr flush dev ceph-brx 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set ceph-brx up 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:13:41.397 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:41.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:41.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:41.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:41.523 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:13:41.573 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:13:41.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:41.574 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:13:41.650 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:41.651 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:41.651 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:41.651 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:13:41.651 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:41.651 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:41.651 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:13:41.651 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:13:41.651 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:13:41.651 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:41.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:41.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:41.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:41.889 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:13:41.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:41.896 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list-id 2024-09-21T02:13:41.943 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:41.943 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:41.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:41.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T02:13:41.943 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:42.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:42.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:42.062 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T02:13:42.062 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:42.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:42.154 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:42.154 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:42.154 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:13:42.154 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:13:42.155 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:42.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:42.255 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:13:42.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:42.255 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:42.301 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:13:42.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:42.301 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:42.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:13:42.406 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:42.454 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:13:42.455 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:13:42.456 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:13:42.457 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:13:42.457 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:13:42.457 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:42.457 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:42.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:42.458 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:13:42.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:13:42.506 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:13:42.550 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:13:42.563 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:42.609 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:13:42.609 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:13:42.610 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:42.611 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:42.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:42.612 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:13:42.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:13:42.658+0000 7fb0c94b8580 -1 init, newargv = 0x7fb0b80046b0 newargc=15 2024-09-21T02:13:42.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[35155]: starting ceph client 2024-09-21T02:13:42.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[35155]: starting fuse 2024-09-21T02:13:43.762 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:13:43.778 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:43.826 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:13:43.827 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:13:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:43.829 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:43.829 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:43.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:43.829 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:13:43.877 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:13:43.878 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:13:43.878 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:13:43.879 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:13:44.002 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.35155.asok 2024-09-21T02:13:44.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:44.014 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35155.asok 2024-09-21T02:13:44.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:44.014 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35155.asok status 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "35155", 2024-09-21T02:13:44.191 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4686, 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 4686 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 585861539 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:44.192 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 585861539 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.4686 192.168.0.1:0/585861539", 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/585861539", 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 8, 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 23, 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:13:44.193 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:13:44.208 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:13:44.209 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:13:44.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 -- 172.21.15.170:0/2214823619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 msgr2=0x7f3af80ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:44.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 --2- 172.21.15.170:0/2214823619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af80ce5d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009bc0 tx=0x7f3af001cba0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 -- 172.21.15.170:0/2214823619 shutdown_connections 2024-09-21T02:13:44.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 --2- 172.21.15.170:0/2214823619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af80ce5d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 --2- 172.21.15.170:0/2214823619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3af80cf7e0 0x7f3af8070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 -- 172.21.15.170:0/2214823619 >> 172.21.15.170:0/2214823619 conn(0x7f3af80b1c40 msgr2=0x7f3af80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:44.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 -- 172.21.15.170:0/2214823619 shutdown_connections 2024-09-21T02:13:44.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.366+0000 7f3affb15640 1 -- 172.21.15.170:0/2214823619 wait complete. 2024-09-21T02:13:44.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 Processor -- start 2024-09-21T02:13:44.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 -- start start 2024-09-21T02:13:44.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3af80cb460 0x7f3af80ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:44.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af8152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:44.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3af80cf7e0 0x7f3af80b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:44.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3af81690d0 con 0x7f3af80ce1c0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3af8168f50 con 0x7f3af80cb460 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3af8169250 con 0x7f3af80cf7e0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af8152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af8152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54408/0 (socket says 172.21.15.170:54408) 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 -- 172.21.15.170:0/1419894734 learned_addr learned my addr 172.21.15.170:0/1419894734 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd8b1640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3af80cb460 0x7f3af80ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 -- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3af80cf7e0 msgr2=0x7f3af80b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3af80cf7e0 0x7f3af80b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 -- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3af80cb460 msgr2=0x7f3af80ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3af80cb460 0x7f3af80ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 -- 172.21.15.170:0/1419894734 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af80acfc0 con 0x7f3af80ce1c0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3afd0b0640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af8152480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c400 tx=0x7f3ae800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8011790 con 0x7f3af80ce1c0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ae8017070 con 0x7f3af80ce1c0 2024-09-21T02:13:44.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8011cc0 con 0x7f3af80ce1c0 2024-09-21T02:13:44.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af80ad2a0 con 0x7f3af80ce1c0 2024-09-21T02:13:44.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.370+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af80ad860 con 0x7f3af80ce1c0 2024-09-21T02:13:44.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.374+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af80aeb00 con 0x7f3af80ce1c0 2024-09-21T02:13:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.374+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ae801c460 con 0x7f3af80ce1c0 2024-09-21T02:13:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.374+0000 7f3aee7fc640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3adc073230 0x7f3adc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.374+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae8094090 con 0x7f3af80ce1c0 2024-09-21T02:13:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.374+0000 7f3afd8b1640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3adc073230 0x7f3adc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:44.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.374+0000 7f3afd8b1640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3adc073230 0x7f3adc075740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3af40058d0 tx=0x7f3af4005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:44.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.378+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3ae805f010 con 0x7f3af80ce1c0 2024-09-21T02:13:44.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.586+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3af80ce5d0 con 0x7f3af80ce1c0 2024-09-21T02:13:44.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.586+0000 7f3aee7fc640 1 -- 172.21.15.170:0/1419894734 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3ae805e910 con 0x7f3af80ce1c0 2024-09-21T02:13:44.587 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3adc073230 msgr2=0x7f3adc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3adc073230 0x7f3adc075740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3af40058d0 tx=0x7f3af4005860 comp rx=0 tx=0).stop 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 msgr2=0x7f3af8152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af8152480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c400 tx=0x7f3ae800c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 shutdown_connections 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3adc073230 0x7f3adc075740 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3af80cf7e0 0x7f3af80b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af80ce1c0 0x7f3af8152480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 --2- 172.21.15.170:0/1419894734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3af80cb460 0x7f3af80ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:44.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 >> 172.21.15.170:0/1419894734 conn(0x7f3af80b1c40 msgr2=0x7f3af8158830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:44.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 shutdown_connections 2024-09-21T02:13:44.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44.590+0000 7f3affb15640 1 -- 172.21.15.170:0/1419894734 wait complete. 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:13:44.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T02:13:44.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:44.609 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:13:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:44.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:44.618 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:13:44.665 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:13:44.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:44.666 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:44.717 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:44.718 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:13:44.718 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:44.718 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:44.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:13:44.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:13:44.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:13:44.718 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:44.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:44.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:44.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:44.839 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:13:44.885 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:13:44.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:44.886 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list-id 2024-09-21T02:13:44.932 INFO:teuthology.orchestra.run.smithi170.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T02:13:44.933 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:44.933 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:44.933 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:44.933 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T02:13:44.933 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:45.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:45.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:45.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:45.053 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T02:13:45.066 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 62:a2:c6:21:b3:1c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::60a2:c6ff:fe21:b31c/64 scope link 2024-09-21T02:13:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:45.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T02:13:45.121 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 62:a2:c6:21:b3:1c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::60a2:c6ff:fe21:b31c/64 scope link 2024-09-21T02:13:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:45.123 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T02:13:45.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:45.123 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:45.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T02:13:45.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T02:13:45.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T02:13:45.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T02:13:45.124 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T02:13:45.124 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:45.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:45.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:45.265 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:45.265 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:13:45.266 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.1 up 2024-09-21T02:13:45.266 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T02:13:45.266 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:13:45.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:45.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:45.323 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-21T02:13:45.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:45.323 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:45.368 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T02:13:45.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:45.369 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:45.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:13:45.471 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:13:45.518 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:13:45.519 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:45.520 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:45.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:45.521 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:13:45.569 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:13:45.569 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-21T02:13:45.570 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-21T02:13:45.614 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:13:45.630 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:13:45.682 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:13:45.683 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:45.684 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:45.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:45.685 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:13:45.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi170.stderr:2024-09-21T02:13:45.726+0000 7f9ad4a2a580 -1 init, newargv = 0x7f9ac4004bb0 newargc=15 2024-09-21T02:13:45.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi170.stderr:ceph-fuse[35300]: starting ceph client 2024-09-21T02:13:45.729 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:13:45.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi170.stderr:ceph-fuse[35300]: starting fuse 2024-09-21T02:13:46.831 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:13:46.847 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:46.894 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:46.895 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:13:46.896 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:13:46.897 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:46.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:46.898 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:13:46.949 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:13:46.950 INFO:teuthology.orchestra.run.smithi170.stdout:46 2024-09-21T02:13:46.950 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:13:46.950 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:13:46.951 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:13:46.952 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.2")) 2024-09-21T02:13:46.952 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:13:47.075 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.2.35300.asok 2024-09-21T02:13:47.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:47.087 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35300.asok 2024-09-21T02:13:47.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:47.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35300.asok status 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "2", 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-21T02:13:47.267 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "35300", 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4698, 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 4698 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:13:47.268 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 4063847516 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 4063847516 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.4698 192.168.0.2:0/4063847516", 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.2:0/4063847516", 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 8, 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 23, 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:13:47.269 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:13:47.270 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:13:47.270 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:13:47.283 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:13:47.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:13:47.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 -- 172.21.15.170:0/1012912179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cf7e0 msgr2=0x7fb48c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:47.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 --2- 172.21.15.170:0/1012912179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c070340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb48c070cc0 tx=0x7fb48001cd80 comp rx=0 tx=0).stop 2024-09-21T02:13:47.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 -- 172.21.15.170:0/1012912179 shutdown_connections 2024-09-21T02:13:47.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 --2- 172.21.15.170:0/1012912179 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb48c0cc8c0 0x7fb48c0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 --2- 172.21.15.170:0/1012912179 >> v2:172.21.15.191:3300/0 conn(0x7fb48c0cb460 0x7fb48c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 --2- 172.21.15.170:0/1012912179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c070340 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 -- 172.21.15.170:0/1012912179 >> 172.21.15.170:0/1012912179 conn(0x7fb48c0b1c40 msgr2=0x7fb48c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:47.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 -- 172.21.15.170:0/1012912179 shutdown_connections 2024-09-21T02:13:47.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.438+0000 7fb490de1640 1 -- 172.21.15.170:0/1012912179 wait complete. 2024-09-21T02:13:47.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 Processor -- start 2024-09-21T02:13:47.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 -- start start 2024-09-21T02:13:47.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb48c0cb460 0x7fb48c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:47.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 0x7fb48c1f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c0ced60 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c0cebe0 con 0x7fb48c0cf7e0 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb490de1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb48c0ceee0 con 0x7fb48c0cb460 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb489d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 0x7fb48c1f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb489d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 0x7fb48c1f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54912/0 (socket says 172.21.15.170:54912) 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.442+0000 7fb489d9b640 1 -- 172.21.15.170:0/2221546791 learned_addr learned my addr 172.21.15.170:0/2221546791 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb48ad9d640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb48a59c640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb48c0cb460 0x7fb48c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb489d9b640 1 -- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb48c0cb460 msgr2=0x7fb48c06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb489d9b640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb48c0cb460 0x7fb48c06f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb489d9b640 1 -- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c0cf7e0 msgr2=0x7fb48c14ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb489d9b640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c14ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb489d9b640 1 -- 172.21.15.170:0/2221546791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c1fa000 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb48ad9d640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c14ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb489d9b640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 0x7fb48c1f9290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0058d0 tx=0x7fb47c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c017070 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47c009d20 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c0c5460 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0106c0 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c14b9b0 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c1f9d70 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb47c01a070 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb47affd640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb46c073230 0x7fb46c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb47c0942f0 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.446+0000 7fb48a59c640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb46c073230 0x7fb46c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:47.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.450+0000 7fb48a59c640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb46c073230 0x7fb46c075740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4800af000 tx=0x7fb48001b5c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:47.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.450+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb47c05f3a0 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.662+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb48c0cccd0 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.662+0000 7fb47affd640 1 -- 172.21.15.170:0/2221546791 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb47c05eca0 con 0x7fb48c0cc8c0 2024-09-21T02:13:47.663 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:13:47.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb46c073230 msgr2=0x7fb46c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:47.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb46c073230 0x7fb46c075740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4800af000 tx=0x7fb48001b5c0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 msgr2=0x7fb48c1f9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:47.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 0x7fb48c1f9290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0058d0 tx=0x7fb47c00dde0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 shutdown_connections 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb46c073230 0x7fb46c075740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb48c0cf7e0 0x7fb48c14ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb48c0cc8c0 0x7fb48c1f9290 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 --2- 172.21.15.170:0/2221546791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb48c0cb460 0x7fb48c06f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 >> 172.21.15.170:0/2221546791 conn(0x7fb48c0b1c40 msgr2=0x7fb48c0b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 shutdown_connections 2024-09-21T02:13:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:47.666+0000 7fb490de1640 1 -- 172.21.15.170:0/2221546791 wait complete. 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:13:47.686 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:13:47.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:47.687 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:47.694 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:13:47.695 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42481sec preferred_lft 42481sec 2024-09-21T02:13:47.696 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:13:47.696 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:47.696 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:13:47.696 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:47.696 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:13:47.696 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:47.696 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T02:13:47.696 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:47.696 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:47.696 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add name ceph-brx type bridge 2024-09-21T02:13:47.696 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr flush dev ceph-brx 2024-09-21T02:13:47.697 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set ceph-brx up 2024-09-21T02:13:47.697 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T02:13:47.697 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:47.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:47.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:47.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:47.822 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:13:47.831 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:13:47.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:47.832 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:47.904 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:13:47.905 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:47.905 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:47.905 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:13:47.905 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:13:47.905 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:13:47.905 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:47.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:48.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:48.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:48.023 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:13:48.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:48.072 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2024-09-21T02:13:48.119 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:48.120 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:48.120 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:48.120 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T02:13:48.120 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:48.210 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:48.231 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:48.238 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T02:13:48.238 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:48.278 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:48.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:48.338 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:48.338 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:48.338 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:13:48.338 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T02:13:48.338 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:48.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:48.436 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:13:48.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:48.436 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:48.483 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:13:48.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:48.484 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:48.531 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:13:48.589 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:13:48.637 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:48.638 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:48.639 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:48.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:48.640 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:13:48.688 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:13:48.688 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:13:48.733 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:13:48.747 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:13:48.797 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:48.798 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:48.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:48.800 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:13:48.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:13:48.842+0000 7fe2fefcf580 -1 init, newargv = 0x7fe2e80046b0 newargc=15 2024-09-21T02:13:48.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[33832]: starting ceph client 2024-09-21T02:13:48.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[33832]: starting fuse 2024-09-21T02:13:49.944 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:13:49.962 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:13:50.010 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:13:50.011 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:50.012 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:50.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:50.013 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:13:50.060 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:13:50.060 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:13:50.060 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:50.061 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:13:50.062 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:13:50.185 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.33832.asok 2024-09-21T02:13:50.189 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:50.198 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33832.asok 2024-09-21T02:13:50.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:50.198 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33832.asok status 2024-09-21T02:13:50.381 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:13:50.381 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:13:50.381 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:13:50.381 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "33832", 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 4529, 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:13:50.382 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 4529 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 399902490 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:13:50.383 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 399902490 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.4529 192.168.0.1:0/399902490", 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/399902490", 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 8, 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 23, 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:13:50.384 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:13:50.397 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T02:13:50.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:13:50.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 -- 172.21.15.170:0/210289349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb460 msgr2=0x7f8a940cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 --2- 172.21.15.170:0/210289349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb460 0x7f8a940cb870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a940710f0 tx=0x7f8a8801c800 comp rx=0 tx=0).stop 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 -- 172.21.15.170:0/210289349 shutdown_connections 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 --2- 172.21.15.170:0/210289349 >> v2:172.21.15.191:3300/0 conn(0x7f8a940ce1c0 0x7f8a940ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 --2- 172.21.15.170:0/210289349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb460 0x7f8a940cb870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 -- 172.21.15.170:0/210289349 >> 172.21.15.170:0/210289349 conn(0x7f8a940b1c40 msgr2=0x7f8a940b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 -- 172.21.15.170:0/210289349 shutdown_connections 2024-09-21T02:13:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.557+0000 7f8a99d88640 1 -- 172.21.15.170:0/210289349 wait complete. 2024-09-21T02:13:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 Processor -- start 2024-09-21T02:13:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- start start 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a940cb460 0x7f8a9414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 0x7f8a940ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a940cf7e0 0x7f8a940add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94168f50 con 0x7f8a940ce1c0 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8a940cf140 con 0x7f8a940cf7e0 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8a941690d0 con 0x7f8a940cb460 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 0x7f8a940ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 0x7f8a940ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54932/0 (socket says 172.21.15.170:54932) 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 -- 172.21.15.170:0/1084223457 learned_addr learned my addr 172.21.15.170:0/1084223457 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 -- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a940cb460 msgr2=0x7f8a9414fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:50.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a940cb460 0x7f8a9414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 -- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a940cf7e0 msgr2=0x7f8a940add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a93fff640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a940cf7e0 0x7f8a940add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a940cf7e0 0x7f8a940add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 -- 172.21.15.170:0/1084223457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a94150b10 con 0x7f8a940ce1c0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a92ffd640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 0x7f8a940ad830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a8801cce0 tx=0x7f8a880a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880b2040 con 0x7f8a940ce1c0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a880042b0 con 0x7f8a940ce1c0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9409ba60 con 0x7f8a940ce1c0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88003600 con 0x7f8a940ce1c0 2024-09-21T02:13:50.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9409c020 con 0x7f8a940ce1c0 2024-09-21T02:13:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.561+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a940ce5d0 con 0x7f8a940ce1c0 2024-09-21T02:13:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.565+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a880bd030 con 0x7f8a940ce1c0 2024-09-21T02:13:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.565+0000 7f8a7bfff640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a80073200 0x7f8a80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.565+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8a8812e140 con 0x7f8a940ce1c0 2024-09-21T02:13:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.565+0000 7f8a937fe640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a80073200 0x7f8a80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.565+0000 7f8a937fe640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a80073200 0x7f8a80075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:50.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.569+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8a880c4020 con 0x7f8a940ce1c0 2024-09-21T02:13:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.781+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8a940b54e0 con 0x7f8a940ce1c0 2024-09-21T02:13:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.781+0000 7f8a7bfff640 1 -- 172.21.15.170:0/1084223457 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8a880163f0 con 0x7f8a940ce1c0 2024-09-21T02:13:50.783 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:13:50.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a80073200 msgr2=0x7f8a80075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:50.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a80073200 0x7f8a80075710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84007940 comp rx=0 tx=0).stop 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 msgr2=0x7f8a940ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 0x7f8a940ad830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a8801cce0 tx=0x7f8a880a5820 comp rx=0 tx=0).stop 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 shutdown_connections 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a80073200 0x7f8a80075710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a940cf7e0 0x7f8a940add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940ce1c0 0x7f8a940ad830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 --2- 172.21.15.170:0/1084223457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a940cb460 0x7f8a9414fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 >> 172.21.15.170:0/1084223457 conn(0x7f8a940b1c40 msgr2=0x7f8a940acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 shutdown_connections 2024-09-21T02:13:50.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:50.785+0000 7f8a99d88640 1 -- 172.21.15.170:0/1084223457 wait complete. 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:13:50.805 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T02:13:50.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:50.806 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:13:50.812 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:13:50.813 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42478sec preferred_lft 42478sec 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.814 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:13:50.815 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.815 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:13:50.815 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:50.815 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:13:50.815 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:50.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:50.815 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:13:50.869 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:13:50.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:50.870 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:50.921 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:50.922 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:13:50.922 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:13:50.922 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:50.922 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:50.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:13:50.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:13:50.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:13:50.923 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:51.009 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:51.039 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:51.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:51.046 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:13:51.052 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:13:51.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:51.052 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2024-09-21T02:13:51.104 INFO:teuthology.orchestra.run.smithi191.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T02:13:51.104 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:51.104 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:51.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:51.105 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T02:13:51.105 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:51.193 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:51.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:51.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:51.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.234 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:13:51.235 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether f2:c9:c6:8c:c4:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:13:51.235 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:13:51.235 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.235 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::f0c9:c6ff:fe8c:c4fb/64 scope link 2024-09-21T02:13:51.235 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:51.236 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether f2:c9:c6:8c:c4:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T02:13:51.290 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T02:13:51.291 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.291 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::f0c9:c6ff:fe8c:c4fb/64 scope link 2024-09-21T02:13:51.291 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:13:51.291 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T02:13:51.292 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:51.380 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:51.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:51.436 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:13:51.436 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:13:51.436 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.1 up 2024-09-21T02:13:51.436 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T02:13:51.436 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:13:51.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:13:51.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:51.496 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-21T02:13:51.496 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:51.496 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:51.543 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T02:13:51.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:13:51.543 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:51.591 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:13:51.641 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:51.688 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:13:51.689 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:51.690 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:51.691 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:51.691 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:51.691 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:51.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:51.692 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:13:51.739 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:13:51.739 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-21T02:13:51.739 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-21T02:13:51.785 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:13:51.799 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:13:51.848 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:13:51.849 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:51.850 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:51.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:51.851 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:13:51.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi191.stderr:2024-09-21T02:13:51.890+0000 7f50a830f580 -1 init, newargv = 0x7f508c001090 newargc=15 2024-09-21T02:13:51.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi191.stderr:ceph-fuse[33951]: starting ceph client 2024-09-21T02:13:51.896 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:13:51.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi191.stderr:ceph-fuse[33951]: starting fuse 2024-09-21T02:13:52.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:13:53.014 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:13:53.061 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:13:53.062 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:13:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:53.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.064 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:13:53.112 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:13:53.112 INFO:teuthology.orchestra.run.smithi191.stdout:46 2024-09-21T02:13:53.112 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:13:53.112 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:13:53.113 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:13:53.114 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:13:53.114 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.3")) 2024-09-21T02:13:53.114 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:13:53.236 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.3.33951.asok 2024-09-21T02:13:53.240 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:13:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:13:53.247 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33951.asok 2024-09-21T02:13:53.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.248 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33951.asok status 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "3", 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-21T02:13:53.429 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "33951", 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 4618, 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 4618 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:13:53.430 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1736805339 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.2:0", 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1736805339 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.4618 192.168.0.2:0/1736805339", 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.2:0/1736805339", 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 8, 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 23, 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:13:53.431 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:13:53.432 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:13:53.432 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:13:53.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.445 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:13:53.445 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:53.452 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:13:53.452 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:53.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.452 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:53.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.509 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:13:53.509 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:53.560 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:13:53.560 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:53.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.560 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T02:13:53.569 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:53.603 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:13:53.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.619 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:13:53.619 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:53.696 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:13:53.696 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:53.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.696 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:13:53.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.753 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:13:53.753 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:53.801 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:13:53.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:53.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:53.801 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T02:13:53.856 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T02:13:55.323 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-21T02:13:55.323 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:13:55.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 -- 172.21.15.170:0/2859037323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 msgr2=0x7f87e406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 --2- 172.21.15.170:0/2859037323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e406fa30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f87e4071d80 tx=0x7f87e001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 -- 172.21.15.170:0/2859037323 shutdown_connections 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 --2- 172.21.15.170:0/2859037323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e40cd210 0x7f87e40b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 --2- 172.21.15.170:0/2859037323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e406fa30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 -- 172.21.15.170:0/2859037323 >> 172.21.15.170:0/2859037323 conn(0x7f87e40b1c40 msgr2=0x7f87e40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 -- 172.21.15.170:0/2859037323 shutdown_connections 2024-09-21T02:13:55.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 -- 172.21.15.170:0/2859037323 wait complete. 2024-09-21T02:13:55.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 Processor -- start 2024-09-21T02:13:55.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.489+0000 7f87ec02b640 1 -- start start 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e40b2f40 0x7f87e409d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e4156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e4156fd0 0x7f87e40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4169fd0 con 0x7f87e40b5410 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4169e50 con 0x7f87e40b2f40 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f87e416a150 con 0x7f87e4156fd0 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e4156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e4156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45580/0 (socket says 172.21.15.170:45580) 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 -- 172.21.15.170:0/1220195288 learned_addr learned my addr 172.21.15.170:0/1220195288 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:55.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ea5c8640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e4156fd0 0x7f87e40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e9dc7640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e40b2f40 0x7f87e409d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 -- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e4156fd0 msgr2=0x7f87e40b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e4156fd0 0x7f87e40b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 -- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e40b2f40 msgr2=0x7f87e409d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e40b2f40 0x7f87e409d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 -- 172.21.15.170:0/1220195288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e41f6c90 con 0x7f87e40b5410 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ea5c8640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e4156fd0 0x7f87e40b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87e95c6640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e4156a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f87d400b440 tx=0x7f87d400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4015020 con 0x7f87e40b5410 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87d4011980 con 0x7f87e40b5410 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e41f6f70 con 0x7f87e40b5410 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4011dd0 con 0x7f87e40b5410 2024-09-21T02:13:55.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41f7530 con 0x7f87e40b5410 2024-09-21T02:13:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e41f7910 con 0x7f87e40b5410 2024-09-21T02:13:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.493+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f87d4018020 con 0x7f87e40b5410 2024-09-21T02:13:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.497+0000 7f87da7fc640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073230 0x7f87c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.497+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f87d4093f40 con 0x7f87e40b5410 2024-09-21T02:13:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.497+0000 7f87e9dc7640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073230 0x7f87c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.497+0000 7f87e9dc7640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073230 0x7f87c8075740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87e00af000 tx=0x7f87e001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.501+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f87e41f7910 con 0x7f87e40b5410 2024-09-21T02:13:55.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.673+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87e406fa30 con 0x7f87e40b5410 2024-09-21T02:13:55.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.673+0000 7f87da7fc640 1 -- 172.21.15.170:0/1220195288 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7f87d401f030 con 0x7f87e40b5410 2024-09-21T02:13:55.676 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:13:55.677 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073230 msgr2=0x7f87c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073230 0x7f87c8075740 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87e00af000 tx=0x7f87e001b100 comp rx=0 tx=0).stop 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 msgr2=0x7f87e4156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e4156a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f87d400b440 tx=0x7f87d400b900 comp rx=0 tx=0).stop 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 shutdown_connections 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073230 0x7f87c8075740 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e4156fd0 0x7f87e40b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87e40b5410 0x7f87e4156a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 --2- 172.21.15.170:0/1220195288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e40b2f40 0x7f87e409d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 >> 172.21.15.170:0/1220195288 conn(0x7f87e40b1c40 msgr2=0x7f87e40c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:55.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 shutdown_connections 2024-09-21T02:13:55.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.677+0000 7f87ec02b640 1 -- 172.21.15.170:0/1220195288 wait complete. 2024-09-21T02:13:55.696 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:13:23.204915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:13:24.242738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:13:55.697 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:13:55.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3350111172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40d3020 msgr2=0x7ff7e40cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3350111172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40d3020 0x7ff7e40cf020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40b53c0 tx=0x7ff7d401c770 comp rx=0 tx=0).stop 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3350111172 shutdown_connections 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3350111172 >> v2:172.21.15.191:3300/0 conn(0x7ff7e40cf970 0x7ff7e40d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3350111172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40d3020 0x7ff7e40cf020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3350111172 >> 172.21.15.170:0/3350111172 conn(0x7ff7e409d8b0 msgr2=0x7ff7e40d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3350111172 shutdown_connections 2024-09-21T02:13:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.849+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3350111172 wait complete. 2024-09-21T02:13:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 Processor -- start 2024-09-21T02:13:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 -- start start 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e40cf560 0x7ff7e409e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 0x7ff7e4166620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7e40d3020 0x7ff7e4160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e416bd90 con 0x7ff7e40cf970 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e416bc10 con 0x7ff7e40cf560 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e416bf10 con 0x7ff7e40d3020 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 0x7ff7e4166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 0x7ff7e4166620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45612/0 (socket says 172.21.15.170:45612) 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 -- 172.21.15.170:0/3094473505 learned_addr learned my addr 172.21.15.170:0/3094473505 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e37fe640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e40cf560 0x7ff7e409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e3fff640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7e40d3020 0x7ff7e4160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 -- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7e40d3020 msgr2=0x7ff7e4160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7e40d3020 0x7ff7e4160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 -- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e40cf560 msgr2=0x7ff7e409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e40cf560 0x7ff7e409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 -- 172.21.15.170:0/3094473505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e40a1520 con 0x7ff7e40cf970 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e37fe640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e40cf560 0x7ff7e409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7daffd640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 0x7ff7e4166620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c3f0 tx=0x7ff7cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011790 con 0x7ff7e40cf970 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7cc017070 con 0x7ff7e40cf970 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e40a1800 con 0x7ff7e40cf970 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011cc0 con 0x7ff7e40cf970 2024-09-21T02:13:55.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.853+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e4161190 con 0x7ff7e40cf970 2024-09-21T02:13:55.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.857+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e40cf020 con 0x7ff7e40cf970 2024-09-21T02:13:55.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.857+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7cc01c460 con 0x7ff7e40cf970 2024-09-21T02:13:55.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.857+0000 7ff7e0ff9640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7c4073200 0x7ff7c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:55.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.857+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff7cc094ad0 con 0x7ff7e40cf970 2024-09-21T02:13:55.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.857+0000 7ff7e37fe640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7c4073200 0x7ff7c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:55.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.857+0000 7ff7e37fe640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7c4073200 0x7ff7c4075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40af000 tx=0x7ff7d401afb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:55.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:55.861+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff7cc05fb80 con 0x7ff7e40cf970 2024-09-21T02:13:56.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.037+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7e40b1030 con 0x7ff7e40cf970 2024-09-21T02:13:56.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.037+0000 7ff7e0ff9640 1 -- 172.21.15.170:0/3094473505 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7ff7cc01ae40 con 0x7ff7e40cf970 2024-09-21T02:13:56.039 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:13:56.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7c4073200 msgr2=0x7ff7c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7c4073200 0x7ff7c4075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40af000 tx=0x7ff7d401afb0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 msgr2=0x7ff7e4166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 0x7ff7e4166620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c3f0 tx=0x7ff7cc00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 shutdown_connections 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7c4073200 0x7ff7c4075710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7e40d3020 0x7ff7e4160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7e40cf970 0x7ff7e4166620 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 --2- 172.21.15.170:0/3094473505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7e40cf560 0x7ff7e409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 >> 172.21.15.170:0/3094473505 conn(0x7ff7e409d8b0 msgr2=0x7ff7e406ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 shutdown_connections 2024-09-21T02:13:56.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.041+0000 7ff7e9b57640 1 -- 172.21.15.170:0/3094473505 wait complete. 2024-09-21T02:13:56.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T02:13:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/3485519833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9415f470 msgr2=0x7f7a9415f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/3485519833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9415f470 0x7f7a9415f880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a94074d80 tx=0x7f7a8c01caf0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/3485519833 shutdown_connections 2024-09-21T02:13:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/3485519833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9415f470 0x7f7a9415f880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/3485519833 >> v2:172.21.15.191:3300/0 conn(0x7f7a940cc8a0 0x7f7a940cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/3485519833 >> 172.21.15.170:0/3485519833 conn(0x7f7a9409a9c0 msgr2=0x7f7a940d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/3485519833 shutdown_connections 2024-09-21T02:13:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.217+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/3485519833 wait complete. 2024-09-21T02:13:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 Processor -- start 2024-09-21T02:13:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- start start 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a940b04e0 0x7f7a94157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 0x7f7a94151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a9415f470 0x7f7a9415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7a94169350 con 0x7f7a940cc8a0 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a941691d0 con 0x7f7a940b04e0 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7a941694d0 con 0x7f7a9415f470 2024-09-21T02:13:56.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 0x7f7a94151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 0x7f7a94151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45624/0 (socket says 172.21.15.170:45624) 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 -- 172.21.15.170:0/256591796 learned_addr learned my addr 172.21.15.170:0/256591796 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9ab5d640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a9415f470 0x7f7a9415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9a35c640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a940b04e0 0x7f7a94157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 -- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a9415f470 msgr2=0x7f7a9415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a9415f470 0x7f7a9415de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 -- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a940b04e0 msgr2=0x7f7a94157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a940b04e0 0x7f7a94157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 -- 172.21.15.170:0/256591796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a94159410 con 0x7f7a940cc8a0 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9a35c640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a940b04e0 0x7f7a94157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a99b5b640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 0x7f7a94151230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000d950 tx=0x7f7a9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90017070 con 0x7f7a940cc8a0 2024-09-21T02:13:56.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a90009d20 con 0x7f7a940cc8a0 2024-09-21T02:13:56.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a941581e0 con 0x7f7a940cc8a0 2024-09-21T02:13:56.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a900106c0 con 0x7f7a940cc8a0 2024-09-21T02:13:56.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a94159130 con 0x7f7a940cc8a0 2024-09-21T02:13:56.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.221+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a940cccb0 con 0x7f7a940cc8a0 2024-09-21T02:13:56.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.225+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a9001a070 con 0x7f7a940cc8a0 2024-09-21T02:13:56.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.225+0000 7f7a8affd640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7a78073200 0x7f7a78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.225+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7a90094770 con 0x7f7a940cc8a0 2024-09-21T02:13:56.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.225+0000 7f7a9a35c640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7a78073200 0x7f7a78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.225+0000 7f7a9a35c640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7a78073200 0x7f7a78075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007fd0 tx=0x7f7a84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:56.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.229+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a9005f900 con 0x7f7a940cc8a0 2024-09-21T02:13:56.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.401+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a9415fb30 con 0x7f7a940cc8a0 2024-09-21T02:13:56.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.401+0000 7f7a8affd640 1 -- 172.21.15.170:0/256591796 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f7a9005f200 con 0x7f7a940cc8a0 2024-09-21T02:13:56.405 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2024-09-21T02:13:56.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7a78073200 msgr2=0x7f7a78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7a78073200 0x7f7a78075710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007fd0 tx=0x7f7a84009040 comp rx=0 tx=0).stop 2024-09-21T02:13:56.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 msgr2=0x7f7a94151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 0x7f7a94151230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000d950 tx=0x7f7a9000de10 comp rx=0 tx=0).stop 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 shutdown_connections 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7a78073200 0x7f7a78075710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a9415f470 0x7f7a9415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a940cc8a0 0x7f7a94151230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 --2- 172.21.15.170:0/256591796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a940b04e0 0x7f7a94157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 >> 172.21.15.170:0/256591796 conn(0x7f7a9409a9c0 msgr2=0x7f7a941558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 shutdown_connections 2024-09-21T02:13:56.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.405+0000 7f7a9c5c0640 1 -- 172.21.15.170:0/256591796 wait complete. 2024-09-21T02:13:56.426 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T02:13:56.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8f15b640 1 -- 172.21.15.170:0/1224359869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a78004d50 msgr2=0x7f9a78005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/1224359869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a78004d50 0x7f9a78005160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009eb0 tx=0x7f9a8001d5d0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8d6f8640 1 -- 172.21.15.170:0/1224359869 reap_dead start 2024-09-21T02:13:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8f15b640 1 -- 172.21.15.170:0/1224359869 shutdown_connections 2024-09-21T02:13:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8f15b640 1 -- 172.21.15.170:0/1224359869 >> 172.21.15.170:0/1224359869 conn(0x7f9a880d0170 msgr2=0x7f9a880d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8f15b640 1 -- 172.21.15.170:0/1224359869 shutdown_connections 2024-09-21T02:13:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.581+0000 7f9a8f15b640 1 -- 172.21.15.170:0/1224359869 wait complete. 2024-09-21T02:13:56.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 Processor -- start 2024-09-21T02:13:56.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- start start 2024-09-21T02:13:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a78004d50 0x7f9a8809d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 0x7f9a881f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a8815b410 0x7f9a881f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a880ce780 con 0x7f9a8815b000 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9a880b5bb0 con 0x7f9a8815b410 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9a880ce900 con 0x7f9a78004d50 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 0x7f9a881f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 0x7f9a881f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45646/0 (socket says 172.21.15.170:45646) 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 -- 172.21.15.170:0/3741690586 learned_addr learned my addr 172.21.15.170:0/3741690586 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8d6f8640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a8815b410 0x7f9a881f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8cef7640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a78004d50 0x7f9a8809d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 -- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a78004d50 msgr2=0x7f9a8809d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a78004d50 0x7f9a8809d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 -- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a8815b410 msgr2=0x7f9a881f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a8815b410 0x7f9a881f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 -- 172.21.15.170:0/3741690586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a880c9ac0 con 0x7f9a8815b000 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a87fff640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 0x7f9a881f6970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400d820 tx=0x7f9a7400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74018070 con 0x7f9a8815b000 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a740052e0 con 0x7f9a8815b000 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7400b8c0 con 0x7f9a8815b000 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8809ad90 con 0x7f9a8815b000 2024-09-21T02:13:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8809b350 con 0x7f9a8815b000 2024-09-21T02:13:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.585+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a88073b60 con 0x7f9a8815b000 2024-09-21T02:13:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.589+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a7401b070 con 0x7f9a8815b000 2024-09-21T02:13:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.589+0000 7f9a857fa640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9a6c073230 0x7f9a6c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.589+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9a74093850 con 0x7f9a8815b000 2024-09-21T02:13:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.589+0000 7f9a8cef7640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9a6c073230 0x7f9a6c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.589+0000 7f9a8cef7640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9a6c073230 0x7f9a6c075740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c005d10 tx=0x7f9a7c005c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:56.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.593+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9a74001d10 con 0x7f9a8815b000 2024-09-21T02:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.753+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9a880af6c0 con 0x7f9a8815b000 2024-09-21T02:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.753+0000 7f9a857fa640 1 -- 172.21.15.170:0/3741690586 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9a7405e340 con 0x7f9a8815b000 2024-09-21T02:13:56.757 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2024-09-21T02:13:56.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9a6c073230 msgr2=0x7f9a6c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9a6c073230 0x7f9a6c075740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c005d10 tx=0x7f9a7c005c60 comp rx=0 tx=0).stop 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 msgr2=0x7f9a881f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 0x7f9a881f6970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400d820 tx=0x7f9a7400dce0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 shutdown_connections 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9a6c073230 0x7f9a6c075740 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a8815b410 0x7f9a881f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a8815b000 0x7f9a881f6970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 --2- 172.21.15.170:0/3741690586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a78004d50 0x7f9a8809d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 >> 172.21.15.170:0/3741690586 conn(0x7f9a880d0170 msgr2=0x7f9a880b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 shutdown_connections 2024-09-21T02:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.757+0000 7f9a8f15b640 1 -- 172.21.15.170:0/3741690586 wait complete. 2024-09-21T02:13:56.777 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-21T02:13:56.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 -- 172.21.15.170:0/1005490383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febe0004810 msgr2=0x7febe0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 --2- 172.21.15.170:0/1005490383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febe0004810 0x7febe0004c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7febe4009b30 tx=0x7febe401d270 comp rx=0 tx=0).stop 2024-09-21T02:13:56.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febf9ce6640 1 -- 172.21.15.170:0/1005490383 reap_dead start 2024-09-21T02:13:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 -- 172.21.15.170:0/1005490383 shutdown_connections 2024-09-21T02:13:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 -- 172.21.15.170:0/1005490383 >> 172.21.15.170:0/1005490383 conn(0x7febf409a9c0 msgr2=0x7febf40b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 -- 172.21.15.170:0/1005490383 shutdown_connections 2024-09-21T02:13:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 -- 172.21.15.170:0/1005490383 wait complete. 2024-09-21T02:13:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 Processor -- start 2024-09-21T02:13:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.929+0000 7febfb749640 1 -- start start 2024-09-21T02:13:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febe0004810 0x7febf415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 0x7febf406ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febf409e880 0x7febf406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7febf40b0e10 con 0x7febf409e470 2024-09-21T02:13:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7febf40b0c90 con 0x7febf409e880 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7febf40ce300 con 0x7febe0004810 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 0x7febf406ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 0x7febf406ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45666/0 (socket says 172.21.15.170:45666) 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 -- 172.21.15.170:0/3219215047 learned_addr learned my addr 172.21.15.170:0/3219215047 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf9ce6640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febf409e880 0x7febf406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf94e5640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febe0004810 0x7febf415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 -- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febe0004810 msgr2=0x7febf415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febe0004810 0x7febf415cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 -- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febf409e880 msgr2=0x7febf406d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febf409e880 0x7febf406d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 -- 172.21.15.170:0/3219215047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf415d1d0 con 0x7febf409e470 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febf8ce4640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 0x7febf406ce30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febe4009a80 tx=0x7febe40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40b1040 con 0x7febf409e470 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febe40ac7b0 con 0x7febf409e470 2024-09-21T02:13:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf41f7e00 con 0x7febf409e470 2024-09-21T02:13:56.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf41f7320 con 0x7febf409e470 2024-09-21T02:13:56.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40acc00 con 0x7febf409e470 2024-09-21T02:13:56.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd40052a0 con 0x7febf409e470 2024-09-21T02:13:56.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.933+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7febe40be020 con 0x7febf409e470 2024-09-21T02:13:56.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.937+0000 7febe9ffb640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febd8073200 0x7febd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:56.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.937+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7febe412ef80 con 0x7febf409e470 2024-09-21T02:13:56.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.937+0000 7febf94e5640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febd8073200 0x7febd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:56.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.937+0000 7febf94e5640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febd8073200 0x7febd8075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febf0009a10 tx=0x7febf00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:56.941+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7febe40c5020 con 0x7febf409e470 2024-09-21T02:13:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.173+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7febd4005590 con 0x7febf409e470 2024-09-21T02:13:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.945+0000 7febe9ffb640 1 -- 172.21.15.170:0/3219215047 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v52) ==== 161+0+0 (secure 0 0 0) 0x7febe4016f80 con 0x7febf409e470 2024-09-21T02:13:57.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febd8073200 msgr2=0x7febd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:57.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febd8073200 0x7febd8075710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febf0009a10 tx=0x7febf00091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:57.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 msgr2=0x7febf406ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:57.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 0x7febf406ce30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febe4009a80 tx=0x7febe40056a0 comp rx=0 tx=0).stop 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 shutdown_connections 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febd8073200 0x7febd8075710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febf409e880 0x7febf406d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febf409e470 0x7febf406ce30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 --2- 172.21.15.170:0/3219215047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febe0004810 0x7febf415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 >> 172.21.15.170:0/3219215047 conn(0x7febf409a9c0 msgr2=0x7febf4070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 shutdown_connections 2024-09-21T02:13:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:57.949+0000 7febfb749640 1 -- 172.21.15.170:0/3219215047 wait complete. 2024-09-21T02:13:57.977 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:13:58.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 -- 172.21.15.170:0/4002240594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b5410 msgr2=0x7f28ec06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/4002240594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b5410 0x7f28ec06fa30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0c7150 tx=0x7f28dc01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 -- 172.21.15.170:0/4002240594 shutdown_connections 2024-09-21T02:13:58.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/4002240594 >> v2:172.21.15.191:3300/0 conn(0x7f28ec0cd210 0x7f28ec0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/4002240594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b5410 0x7f28ec06fa30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 -- 172.21.15.170:0/4002240594 >> 172.21.15.170:0/4002240594 conn(0x7f28ec0b1c40 msgr2=0x7f28ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:58.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 -- 172.21.15.170:0/4002240594 shutdown_connections 2024-09-21T02:13:58.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 -- 172.21.15.170:0/4002240594 wait complete. 2024-09-21T02:13:58.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 Processor -- start 2024-09-21T02:13:58.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.133+0000 7f28f0f0c640 1 -- start start 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 0x7f28ec15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28ec0b5410 0x7f28ec146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28ec158940 0x7f28ec1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec169fd0 con 0x7f28ec0b2f40 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec169e50 con 0x7f28ec0b5410 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec16a150 con 0x7f28ec158940 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 0x7f28ec15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 0x7f28ec15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45690/0 (socket says 172.21.15.170:45690) 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 -- 172.21.15.170:0/423765818 learned_addr learned my addr 172.21.15.170:0/423765818 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28e1d9b640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28ec0b5410 0x7f28ec146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ead9d640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28ec158940 0x7f28ec1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 -- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28ec158940 msgr2=0x7f28ec1f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28ec158940 0x7f28ec1f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 -- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28ec0b5410 msgr2=0x7f28ec146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28ec0b5410 0x7f28ec146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 -- 172.21.15.170:0/423765818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec159b80 con 0x7f28ec0b2f40 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28e1d9b640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28ec0b5410 0x7f28ec146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28ea59c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 0x7f28ec15c7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0b0040 tx=0x7f28dc0a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0b2040 con 0x7f28ec0b2f40 2024-09-21T02:13:58.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28dc0a6cc0 con 0x7f28ec0b2f40 2024-09-21T02:13:58.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec15bfd0 con 0x7f28ec0b2f40 2024-09-21T02:13:58.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc003860 con 0x7f28ec0b2f40 2024-09-21T02:13:58.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1f73b0 con 0x7f28ec0b2f40 2024-09-21T02:13:58.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.137+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ec1f7610 con 0x7f28ec0b2f40 2024-09-21T02:13:58.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.141+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f28dc0bf060 con 0x7f28ec0b2f40 2024-09-21T02:13:58.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.141+0000 7f28e37fe640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28cc073200 0x7f28cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.141+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f28dc12e730 con 0x7f28ec0b2f40 2024-09-21T02:13:58.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.141+0000 7f28e1d9b640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28cc073200 0x7f28cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.141+0000 7f28e1d9b640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28cc073200 0x7f28cc075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28ec158e70 tx=0x7f28d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:58.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.145+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f28dc0164b0 con 0x7f28ec0b2f40 2024-09-21T02:13:58.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f28ec06fa30 con 0x7f28ec0b2f40 2024-09-21T02:13:58.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28e37fe640 1 -- 172.21.15.170:0/423765818 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f28dc0f9870 con 0x7f28ec0b2f40 2024-09-21T02:13:58.318 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:13:58.318 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:13:58.319 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:13:58.320 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:13:58.321 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:13:58.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28cc073200 msgr2=0x7f28cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28cc073200 0x7f28cc075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28ec158e70 tx=0x7f28d4008040 comp rx=0 tx=0).stop 2024-09-21T02:13:58.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 msgr2=0x7f28ec15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 0x7f28ec15c7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0b0040 tx=0x7f28dc0a56d0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 shutdown_connections 2024-09-21T02:13:58.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28cc073200 0x7f28cc075710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28ec158940 0x7f28ec1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28ec0b5410 0x7f28ec146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 --2- 172.21.15.170:0/423765818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ec0b2f40 0x7f28ec15c7d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 >> 172.21.15.170:0/423765818 conn(0x7f28ec0b1c40 msgr2=0x7f28ec0ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:58.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.317+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 shutdown_connections 2024-09-21T02:13:58.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.321+0000 7f28f0f0c640 1 -- 172.21.15.170:0/423765818 wait complete. 2024-09-21T02:13:58.340 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/891215274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4070660 msgr2=0x7fa4c40cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/891215274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4070660 0x7fa4c40cee90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009cb0 tx=0x7fa4bc01cda0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/891215274 shutdown_connections 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/891215274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4070660 0x7fa4c40cee90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/891215274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c40b2fc0 0x7fa4c40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/891215274 >> 172.21.15.170:0/891215274 conn(0x7fa4c40d0170 msgr2=0x7fa4c40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/891215274 shutdown_connections 2024-09-21T02:13:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.501+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/891215274 wait complete. 2024-09-21T02:13:58.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 Processor -- start 2024-09-21T02:13:58.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- start start 2024-09-21T02:13:58.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c4070660 0x7fa4c41508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 0x7fa4c415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cea80 0x7fa4c40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4169fe0 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40b3640 con 0x7fa4c40cea80 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c416a160 con 0x7fa4c4070660 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 0x7fa4c415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 0x7fa4c415c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45702/0 (socket says 172.21.15.170:45702) 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 -- 172.21.15.170:0/4223386447 learned_addr learned my addr 172.21.15.170:0/4223386447 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c259c640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c4070660 0x7fa4c41508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 -- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c4070660 msgr2=0x7fa4c41508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c4070660 0x7fa4c41508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c2d9d640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cea80 0x7fa4c40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 -- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cea80 msgr2=0x7fa4c40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cea80 0x7fa4c40b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 -- 172.21.15.170:0/4223386447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c4158450 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c1d9b640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 0x7fa4c415c250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c390 tx=0x7fa4b800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8017070 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4b80053c0 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c4158730 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8003840 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c40b1a90 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.505+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c41581c0 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.509+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4b801a070 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.509+0000 7fa4b2ffd640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa4a4073230 0x7fa4a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.509+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa4b8093820 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.509+0000 7fa4c259c640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa4a4073230 0x7fa4a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.509+0000 7fa4c259c640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa4a4073230 0x7fa4a4075740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009ad0 tx=0x7fa4b40091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:58.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.513+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa4b8021020 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.681+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa4c40cee90 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4b2ffd640 1 -- 172.21.15.170:0/4223386447 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fa4b805e9b0 con 0x7fa4c40b2fc0 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa4a4073230 msgr2=0x7fa4a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa4a4073230 0x7fa4a4075740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009ad0 tx=0x7fa4b40091e0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 msgr2=0x7fa4c415c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 0x7fa4c415c250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c390 tx=0x7fa4b800c850 comp rx=0 tx=0).stop 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 shutdown_connections 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa4a4073230 0x7fa4a4075740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c40cea80 0x7fa4c40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c40b2fc0 0x7fa4c415c250 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 --2- 172.21.15.170:0/4223386447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c4070660 0x7fa4c41508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:58.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.689+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 >> 172.21.15.170:0/4223386447 conn(0x7fa4c40d0170 msgr2=0x7fa4c41532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:58.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.693+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 shutdown_connections 2024-09-21T02:13:58.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:58.693+0000 7fa4c8ea5640 1 -- 172.21.15.170:0/4223386447 wait complete. 2024-09-21T02:13:58.711 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:13:58.711 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:13:58.791 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:13:58.849 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:13:58.849 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:58.849 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:13:58.855 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:58.855 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:13:58.902 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:13:59.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 -- 172.21.15.170:0/1578391216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b6690 msgr2=0x7f77c80b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/1578391216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b6690 0x7f77c80b7d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009840 tx=0x7f77b001c820 comp rx=0 tx=0).stop 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 -- 172.21.15.170:0/1578391216 shutdown_connections 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/1578391216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c80b82a0 0x7f77c80b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/1578391216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b6690 0x7f77c80b7d60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 -- 172.21.15.170:0/1578391216 >> 172.21.15.170:0/1578391216 conn(0x7f77c809d870 msgr2=0x7f77c808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 -- 172.21.15.170:0/1578391216 shutdown_connections 2024-09-21T02:13:59.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 -- 172.21.15.170:0/1578391216 wait complete. 2024-09-21T02:13:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 Processor -- start 2024-09-21T02:13:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.105+0000 7f77ce0a5640 1 -- start start 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c806f430 0x7f77c8150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 0x7f77c815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b82a0 0x7f77c815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f77c816c220 con 0x7f77c80b6690 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77c816c0a0 con 0x7f77c80b82a0 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f77c816c3a0 con 0x7f77c806f430 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 0x7f77c815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 0x7f77c815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45716/0 (socket says 172.21.15.170:45716) 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 -- 172.21.15.170:0/2107562911 learned_addr learned my addr 172.21.15.170:0/2107562911 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c77fe640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c806f430 0x7f77c8150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c7fff640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b82a0 0x7f77c815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 -- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c806f430 msgr2=0x7f77c8150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c806f430 0x7f77c8150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 -- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b82a0 msgr2=0x7f77c815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b82a0 0x7f77c815df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 -- 172.21.15.170:0/2107562911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c80cae60 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77c6ffd640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 0x7f77c815d9e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009790 tx=0x7f77b00a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0004340 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b00bd070 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0003860 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c8151230 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c80c8e30 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c80b7d60 con 0x7f77c80b6690 2024-09-21T02:13:59.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.109+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77b0003a00 con 0x7f77c80b6690 2024-09-21T02:13:59.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.113+0000 7f77bffff640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f77a8073200 0x7f77a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.113+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f77b012f260 con 0x7f77c80b6690 2024-09-21T02:13:59.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.113+0000 7f77c77fe640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f77a8073200 0x7f77a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.113+0000 7f77c77fe640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f77a8073200 0x7f77a8075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77c80ceb00 tx=0x7f77b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:59.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.117+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f77b0016410 con 0x7f77c80b6690 2024-09-21T02:13:59.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.293+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f77c80cabf0 con 0x7f77c80b6690 2024-09-21T02:13:59.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.293+0000 7f77bffff640 1 -- 172.21.15.170:0/2107562911 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f77b00fa310 con 0x7f77c80b6690 2024-09-21T02:13:59.297 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f77a8073200 msgr2=0x7f77a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f77a8073200 0x7f77a8075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77c80ceb00 tx=0x7f77b8009040 comp rx=0 tx=0).stop 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 msgr2=0x7f77c815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 0x7f77c815d9e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009790 tx=0x7f77b00a6760 comp rx=0 tx=0).stop 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 shutdown_connections 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f77a8073200 0x7f77a8075710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c80b82a0 0x7f77c815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77c80b6690 0x7f77c815d9e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 --2- 172.21.15.170:0/2107562911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c806f430 0x7f77c8150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 >> 172.21.15.170:0/2107562911 conn(0x7f77c809d870 msgr2=0x7f77c80b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:59.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 shutdown_connections 2024-09-21T02:13:59.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.297+0000 7f77ce0a5640 1 -- 172.21.15.170:0/2107562911 wait complete. 2024-09-21T02:13:59.318 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:13:59.318 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:13:59.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 -- 172.21.15.170:0/1660499344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f475c005260 msgr2=0x7f475c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 --2- 172.21.15.170:0/1660499344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f475c005260 0x7f475c00b9d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f476800c610 tx=0x7f476801d2c0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 -- 172.21.15.170:0/1660499344 shutdown_connections 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 --2- 172.21.15.170:0/1660499344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f475c005260 0x7f475c00b9d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 -- 172.21.15.170:0/1660499344 >> 172.21.15.170:0/1660499344 conn(0x7f47700b1c40 msgr2=0x7f47700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 -- 172.21.15.170:0/1660499344 shutdown_connections 2024-09-21T02:13:59.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.513+0000 7f4776115640 1 -- 172.21.15.170:0/1660499344 wait complete. 2024-09-21T02:13:59.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 Processor -- start 2024-09-21T02:13:59.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 -- start start 2024-09-21T02:13:59.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f475c005260 0x7f47701504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477015b970 0x7f47700ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 0x7f47700ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47700cf760 con 0x7f477015eef0 2024-09-21T02:13:59.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47700c3a10 con 0x7f477015b970 2024-09-21T02:13:59.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f47700cf8e0 con 0x7f475c005260 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 0x7f47700ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 0x7f47700ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45732/0 (socket says 172.21.15.170:45732) 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 -- 172.21.15.170:0/4080061897 learned_addr learned my addr 172.21.15.170:0/4080061897 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476f7fe640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f475c005260 0x7f47701504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4766ffd640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477015b970 0x7f47700ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 -- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f475c005260 msgr2=0x7f47701504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f475c005260 0x7f47701504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 -- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477015b970 msgr2=0x7f47700ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477015b970 0x7f47700ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 -- 172.21.15.170:0/4080061897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477015e1a0 con 0x7f477015eef0 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476ffff640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 0x7f47700ce990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f476801db60 tx=0x7f4768002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768003d90 con 0x7f477015eef0 2024-09-21T02:13:59.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4768006070 con 0x7f477015eef0 2024-09-21T02:13:59.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477015e410 con 0x7f477015eef0 2024-09-21T02:13:59.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680064a0 con 0x7f477015eef0 2024-09-21T02:13:59.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.517+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477015e9d0 con 0x7f477015eef0 2024-09-21T02:13:59.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.521+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477015fe00 con 0x7f477015eef0 2024-09-21T02:13:59.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.521+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4768006640 con 0x7f477015eef0 2024-09-21T02:13:59.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.521+0000 7f476cff9640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4750073200 0x7f4750075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:13:59.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.521+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f476812e4a0 con 0x7f477015eef0 2024-09-21T02:13:59.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.521+0000 7f476f7fe640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4750073200 0x7f4750075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:13:59.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.521+0000 7f476f7fe640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4750073200 0x7f4750075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4760000f70 tx=0x7f4760006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:13:59.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.525+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4768017070 con 0x7f477015eef0 2024-09-21T02:13:59.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.709+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47700b3510 con 0x7f477015eef0 2024-09-21T02:13:59.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.709+0000 7f476cff9640 1 -- 172.21.15.170:0/4080061897 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f47680f9440 con 0x7f477015eef0 2024-09-21T02:13:59.712 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:13:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4750073200 msgr2=0x7f4750075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4750073200 0x7f4750075710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4760000f70 tx=0x7f4760006b70 comp rx=0 tx=0).stop 2024-09-21T02:13:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 msgr2=0x7f47700ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:13:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 0x7f47700ce990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f476801db60 tx=0x7f4768002150 comp rx=0 tx=0).stop 2024-09-21T02:13:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 shutdown_connections 2024-09-21T02:13:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4750073200 0x7f4750075710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f477015eef0 0x7f47700ce990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f477015b970 0x7f47700ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 --2- 172.21.15.170:0/4080061897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f475c005260 0x7f47701504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:13:59.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 >> 172.21.15.170:0/4080061897 conn(0x7f47700b1c40 msgr2=0x7f47700c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:13:59.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 shutdown_connections 2024-09-21T02:13:59.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:13:59.713+0000 7f4776115640 1 -- 172.21.15.170:0/4080061897 wait complete. 2024-09-21T02:13:59.735 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:13:59.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:59.735 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:13:59.760 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:13:59.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:13:59.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:13:59.760 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:13:59.813 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:13:59.813 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:13:59.820 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:13:59.820 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:13:59.868 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 -- 172.21.15.170:0/1372615860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0b1230 msgr2=0x7fc7bc0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/1372615860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc0b1640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009c80 tx=0x7fc7b401cd10 comp rx=0 tx=0).stop 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 -- 172.21.15.170:0/1372615860 shutdown_connections 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/1372615860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc0b1640 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/1372615860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7bc0d0e50 0x7fc7bc0d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 -- 172.21.15.170:0/1372615860 >> 172.21.15.170:0/1372615860 conn(0x7fc7bc08bf50 msgr2=0x7fc7bc09d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 -- 172.21.15.170:0/1372615860 shutdown_connections 2024-09-21T02:14:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.021+0000 7fc7c33b0640 1 -- 172.21.15.170:0/1372615860 wait complete. 2024-09-21T02:14:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 Processor -- start 2024-09-21T02:14:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 -- start start 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7bc0b3fa0 0x7fc7bc147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 0x7fc7bc0c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc16bff0 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc16be70 con 0x7fc7bc0b1230 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc16c170 con 0x7fc7bc0b3fa0 2024-09-21T02:14:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 0x7fc7bc0c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 0x7fc7bc0c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45756/0 (socket says 172.21.15.170:45756) 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 -- 172.21.15.170:0/3461057924 learned_addr learned my addr 172.21.15.170:0/3461057924 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c114c640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 -- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7bc0b3fa0 msgr2=0x7fc7bc147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c094b640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7bc0b3fa0 0x7fc7bc147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7bc0b3fa0 0x7fc7bc147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 -- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7bc0b1230 msgr2=0x7fc7bc1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc1472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 -- 172.21.15.170:0/3461057924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc157200 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c114c640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c194d640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 0x7fc7bc0c9dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009bd0 tx=0x7fc7b40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40bc070 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b4004350 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc156160 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.025+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40b2040 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc156720 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc155ed0 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7b40bf070 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7b1ffb640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc7a0073200 0x7fc7a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc7b412df90 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7c114c640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc7a0073200 0x7fc7a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.029+0000 7fc7c114c640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc7a0073200 0x7fc7a0075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800a830 tx=0x7fc7b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:00.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.033+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc7b4016990 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7bc0b1640 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7b1ffb640 1 -- 172.21.15.170:0/3461057924 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fc7b40f90c0 con 0x7fc7bc0d0e50 2024-09-21T02:14:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc7a0073200 msgr2=0x7fc7a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc7a0073200 0x7fc7a0075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800a830 tx=0x7fc7b8008040 comp rx=0 tx=0).stop 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 msgr2=0x7fc7bc0c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 0x7fc7bc0c9dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009bd0 tx=0x7fc7b40092c0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 shutdown_connections 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc7a0073200 0x7fc7a0075710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7bc0b3fa0 0x7fc7bc147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7bc0d0e50 0x7fc7bc0c9dd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 --2- 172.21.15.170:0/3461057924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7bc0b1230 0x7fc7bc1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 >> 172.21.15.170:0/3461057924 conn(0x7fc7bc08bf50 msgr2=0x7fc7bc075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.209+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 shutdown_connections 2024-09-21T02:14:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.213+0000 7fc7c33b0640 1 -- 172.21.15.170:0/3461057924 wait complete. 2024-09-21T02:14:00.231 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:00.231 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:00.237 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:00.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.385+0000 7f8bd224b640 1 -- 172.21.15.170:0/2279714346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b2f40 msgr2=0x7f8bcc0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.385+0000 7f8bd224b640 1 --2- 172.21.15.170:0/2279714346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b2f40 0x7f8bcc0b3350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc071d80 tx=0x7f8bc001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- 172.21.15.170:0/2279714346 shutdown_connections 2024-09-21T02:14:00.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 --2- 172.21.15.170:0/2279714346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bcc0ccb00 0x7f8bcc0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 --2- 172.21.15.170:0/2279714346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b2f40 0x7f8bcc0b3350 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 --2- 172.21.15.170:0/2279714346 >> v2:172.21.15.191:3300/0 conn(0x7f8bcc0b5410 0x7f8bcc06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- 172.21.15.170:0/2279714346 >> 172.21.15.170:0/2279714346 conn(0x7f8bcc0b1c40 msgr2=0x7f8bcc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- 172.21.15.170:0/2279714346 shutdown_connections 2024-09-21T02:14:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- 172.21.15.170:0/2279714346 wait complete. 2024-09-21T02:14:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 Processor -- start 2024-09-21T02:14:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- start start 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bcc0b2f40 0x7f8bcc157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 0x7f8bcc157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bcc0ccb00 0x7f8bcc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0700f0 con 0x7f8bcc0b5410 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc06ff70 con 0x7f8bcc0b2f40 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bd224b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0ce800 con 0x7f8bcc0ccb00 2024-09-21T02:14:00.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 0x7f8bcc157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 0x7f8bcc157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45776/0 (socket says 172.21.15.170:45776) 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 -- 172.21.15.170:0/785799140 learned_addr learned my addr 172.21.15.170:0/785799140 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcb7fe640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bcc0b2f40 0x7f8bcc157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcbfff640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bcc0ccb00 0x7f8bcc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 -- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bcc0ccb00 msgr2=0x7f8bcc158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bcc0ccb00 0x7f8bcc158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 -- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bcc0b2f40 msgr2=0x7f8bcc157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bcc0b2f40 0x7f8bcc157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 -- 172.21.15.170:0/785799140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc09ca60 con 0x7f8bcc0b5410 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.389+0000 7f8bcaffd640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 0x7f8bcc157840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8bc001af70 tx=0x7f8bc00a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00b2040 con 0x7f8bcc0b5410 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bc00043b0 con 0x7f8bcc0b5410 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc0b0de0 con 0x7f8bcc0b5410 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0005290 con 0x7f8bcc0b5410 2024-09-21T02:14:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc09cfe0 con 0x7f8bcc0b5410 2024-09-21T02:14:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc0ced10 con 0x7f8bcc0b5410 2024-09-21T02:14:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8bc00be060 con 0x7f8bcc0b5410 2024-09-21T02:14:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bb3fff640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bb8073200 0x7f8bb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8bc012d7b0 con 0x7f8bcc0b5410 2024-09-21T02:14:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bcb7fe640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bb8073200 0x7f8bb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.393+0000 7f8bcb7fe640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bb8073200 0x7f8bb8075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005fd0 tx=0x7f8bbc005d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:00.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.397+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8bc00164b0 con 0x7f8bcc0b5410 2024-09-21T02:14:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.573+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8bcc06de10 con 0x7f8bcc0b5410 2024-09-21T02:14:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.573+0000 7f8bb3fff640 1 -- 172.21.15.170:0/785799140 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f8bc00f8860 con 0x7f8bcc0b5410 2024-09-21T02:14:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:00.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bb8073200 msgr2=0x7f8bb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bb8073200 0x7f8bb8075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005fd0 tx=0x7f8bbc005d80 comp rx=0 tx=0).stop 2024-09-21T02:14:00.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 msgr2=0x7f8bcc157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 0x7f8bcc157840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8bc001af70 tx=0x7f8bc00a5630 comp rx=0 tx=0).stop 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 shutdown_connections 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bb8073200 0x7f8bb8075710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bcc0ccb00 0x7f8bcc158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bcc0b5410 0x7f8bcc157840 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 --2- 172.21.15.170:0/785799140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bcc0b2f40 0x7f8bcc157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 >> 172.21.15.170:0/785799140 conn(0x7f8bcc0b1c40 msgr2=0x7f8bcc0c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 shutdown_connections 2024-09-21T02:14:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.577+0000 7f8bd224b640 1 -- 172.21.15.170:0/785799140 wait complete. 2024-09-21T02:14:00.598 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:14:00.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:00.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:00.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:00.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:14:00.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:00.624 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:14:00.683 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:00.683 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:00.689 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:00.690 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:00.737 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 -- 172.21.15.170:0/505215321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0b5410 msgr2=0x7f668c06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 --2- 172.21.15.170:0/505215321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0b5410 0x7f668c06fa30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b3a10 tx=0x7f667c01c770 comp rx=0 tx=0).stop 2024-09-21T02:14:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 -- 172.21.15.170:0/505215321 shutdown_connections 2024-09-21T02:14:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 --2- 172.21.15.170:0/505215321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 0x7f668c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 --2- 172.21.15.170:0/505215321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0b5410 0x7f668c06fa30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 -- 172.21.15.170:0/505215321 >> 172.21.15.170:0/505215321 conn(0x7f668c0b1c40 msgr2=0x7f668c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:00.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 -- 172.21.15.170:0/505215321 shutdown_connections 2024-09-21T02:14:00.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.937+0000 7f66929cf640 1 -- 172.21.15.170:0/505215321 wait complete. 2024-09-21T02:14:00.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 Processor -- start 2024-09-21T02:14:00.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 -- start start 2024-09-21T02:14:00.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 0x7f668c15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c0b5410 0x7f668c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 0x7f668c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f668c06ff70 con 0x7f668c0ccb00 2024-09-21T02:14:00.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0cf970 con 0x7f668c0b5410 2024-09-21T02:14:00.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f668c0700f0 con 0x7f668c0b2f40 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 0x7f668c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 0x7f668c14b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45788/0 (socket says 172.21.15.170:45788) 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 -- 172.21.15.170:0/877743063 learned_addr learned my addr 172.21.15.170:0/877743063 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f668bfff640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 0x7f668c15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f668b7fe640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c0b5410 0x7f668c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 -- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 msgr2=0x7f668c15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 0x7f668c15fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 -- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c0b5410 msgr2=0x7f668c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c0b5410 0x7f668c15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 -- 172.21.15.170:0/877743063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c1f6f20 con 0x7f668c0ccb00 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f668bfff640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 0x7f668c15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6690f6c640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 0x7f668c14b110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6684004670 tx=0x7f668400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668401a070 con 0x7f668c0ccb00 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66840069e0 con 0x7f668c0ccb00 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c1f7200 con 0x7f668c0ccb00 2024-09-21T02:14:00.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684006e50 con 0x7f668c0ccb00 2024-09-21T02:14:00.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.941+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1f77c0 con 0x7f668c0ccb00 2024-09-21T02:14:00.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.945+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c0b3350 con 0x7f668c0ccb00 2024-09-21T02:14:00.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.945+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f668401d070 con 0x7f668c0ccb00 2024-09-21T02:14:00.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.945+0000 7f6688ff9640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66780733d0 0x7f66780758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.945+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6684096e50 con 0x7f668c0ccb00 2024-09-21T02:14:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.945+0000 7f668bfff640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66780733d0 0x7f66780758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:00.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.945+0000 7f668bfff640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66780733d0 0x7f66780758e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f667c0af000 tx=0x7f667c01afb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:00.949+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6684061e70 con 0x7f668c0ccb00 2024-09-21T02:14:01.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.133+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f668c160380 con 0x7f668c0ccb00 2024-09-21T02:14:01.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.133+0000 7f6688ff9640 1 -- 172.21.15.170:0/877743063 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f6684061770 con 0x7f668c0ccb00 2024-09-21T02:14:01.135 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66780733d0 msgr2=0x7f66780758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66780733d0 0x7f66780758e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f667c0af000 tx=0x7f667c01afb0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 msgr2=0x7f668c14b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 0x7f668c14b110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6684004670 tx=0x7f668400d430 comp rx=0 tx=0).stop 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 shutdown_connections 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f66780733d0 0x7f66780758e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f668c0ccb00 0x7f668c14b110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f668c0b5410 0x7f668c15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 --2- 172.21.15.170:0/877743063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f668c0b2f40 0x7f668c15fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 >> 172.21.15.170:0/877743063 conn(0x7f668c0b1c40 msgr2=0x7f668c0c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 shutdown_connections 2024-09-21T02:14:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.137+0000 7f66929cf640 1 -- 172.21.15.170:0/877743063 wait complete. 2024-09-21T02:14:01.157 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:01.157 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:01.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:01.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 -- 172.21.15.170:0/1776850682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80d0e50 msgr2=0x7fa7d80d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 --2- 172.21.15.170:0/1776850682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80d0e50 0x7fa7d80d1260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa7d808c5a0 tx=0x7fa7cc01c7f0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 -- 172.21.15.170:0/1776850682 shutdown_connections 2024-09-21T02:14:01.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 --2- 172.21.15.170:0/1776850682 >> v2:172.21.15.191:3300/0 conn(0x7fa7d80b1230 0x7fa7d80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 --2- 172.21.15.170:0/1776850682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80d0e50 0x7fa7d80d1260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 -- 172.21.15.170:0/1776850682 >> 172.21.15.170:0/1776850682 conn(0x7fa7d808bf50 msgr2=0x7fa7d809d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:01.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 -- 172.21.15.170:0/1776850682 shutdown_connections 2024-09-21T02:14:01.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.357+0000 7fa7de11d640 1 -- 172.21.15.170:0/1776850682 wait complete. 2024-09-21T02:14:01.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 Processor -- start 2024-09-21T02:14:01.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 -- start start 2024-09-21T02:14:01.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 0x7fa7d80b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b3fa0 0x7fa7d8155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80d0e50 0x7fa7d81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d816c070 con 0x7fa7d80b1230 2024-09-21T02:14:01.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d816bef0 con 0x7fa7d80b3fa0 2024-09-21T02:14:01.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d816c1f0 con 0x7fa7d80d0e50 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 0x7fa7d80b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 0x7fa7d80b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45792/0 (socket says 172.21.15.170:45792) 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 -- 172.21.15.170:0/3765432164 learned_addr learned my addr 172.21.15.170:0/3765432164 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d6ffd640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b3fa0 0x7fa7d8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d7fff640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80d0e50 0x7fa7d81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 -- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80d0e50 msgr2=0x7fa7d81575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80d0e50 0x7fa7d81575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 -- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b3fa0 msgr2=0x7fa7d8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b3fa0 0x7fa7d8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 -- 172.21.15.170:0/3765432164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d80b7aa0 con 0x7fa7d80b1230 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7d77fe640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 0x7fa7d80b9200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800b410 tx=0x7fa7c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8015030 con 0x7fa7d80b1230 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7c8004020 con 0x7fa7d80b1230 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d81f90f0 con 0x7fa7d80b1230 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8004470 con 0x7fa7d80b1230 2024-09-21T02:14:01.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.361+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81f8de0 con 0x7fa7d80b1230 2024-09-21T02:14:01.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.365+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d80b1640 con 0x7fa7d80b1230 2024-09-21T02:14:01.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.365+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7c8018070 con 0x7fa7d80b1230 2024-09-21T02:14:01.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.365+0000 7fa7bffff640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c4073230 0x7fa7c4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.365+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa7c8094280 con 0x7fa7d80b1230 2024-09-21T02:14:01.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.365+0000 7fa7d6ffd640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c4073230 0x7fa7c4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.365+0000 7fa7d6ffd640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c4073230 0x7fa7c4075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0af000 tx=0x7fa7cc01b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.369+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa7c805f410 con 0x7fa7d80b1230 2024-09-21T02:14:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.545+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa7d8075d70 con 0x7fa7d80b1230 2024-09-21T02:14:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.549+0000 7fa7bffff640 1 -- 172.21.15.170:0/3765432164 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa7c805ed10 con 0x7fa7d80b1230 2024-09-21T02:14:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c4073230 msgr2=0x7fa7c4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c4073230 0x7fa7c4075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0af000 tx=0x7fa7cc01b030 comp rx=0 tx=0).stop 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 msgr2=0x7fa7d80b9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 0x7fa7d80b9200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800b410 tx=0x7fa7c800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 shutdown_connections 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c4073230 0x7fa7c4075740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7d80d0e50 0x7fa7d81575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7d80b3fa0 0x7fa7d8155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 --2- 172.21.15.170:0/3765432164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d80b1230 0x7fa7d80b9200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 >> 172.21.15.170:0/3765432164 conn(0x7fa7d808bf50 msgr2=0x7fa7d80b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:01.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 shutdown_connections 2024-09-21T02:14:01.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.553+0000 7fa7de11d640 1 -- 172.21.15.170:0/3765432164 wait complete. 2024-09-21T02:14:01.573 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:14:01.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:01.573 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-21T02:14:01.585 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:01.585 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:14:01.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:01.585 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T02:14:01.634 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:01.634 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:01.641 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:01.641 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:01.689 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:01.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 -- 172.21.15.170:0/1224402395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 msgr2=0x7f17840d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 --2- 172.21.15.170:0/1224402395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17840d0eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f177c009bc0 tx=0x7f177c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 -- 172.21.15.170:0/1224402395 shutdown_connections 2024-09-21T02:14:01.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 --2- 172.21.15.170:0/1224402395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17840d0eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 --2- 172.21.15.170:0/1224402395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b4540 0x7f178415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 -- 172.21.15.170:0/1224402395 >> 172.21.15.170:0/1224402395 conn(0x7f17840cee30 msgr2=0x7f17840cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:01.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 -- 172.21.15.170:0/1224402395 shutdown_connections 2024-09-21T02:14:01.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.845+0000 7f178b856640 1 -- 172.21.15.170:0/1224402395 wait complete. 2024-09-21T02:14:01.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 Processor -- start 2024-09-21T02:14:01.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 -- start start 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b7ab0 0x7f1784156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17841fa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1784143230 0x7f1784143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f178416a560 con 0x7f17840d0aa0 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f178416a3e0 con 0x7f1784143230 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f178416a6e0 con 0x7f17840b7ab0 2024-09-21T02:14:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17841fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17841fa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45816/0 (socket says 172.21.15.170:45816) 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 -- 172.21.15.170:0/3005315584 learned_addr learned my addr 172.21.15.170:0/3005315584 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f17895f2640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b7ab0 0x7f1784156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 -- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b7ab0 msgr2=0x7f1784156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1789df3640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1784143230 0x7f1784143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b7ab0 0x7f1784156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 -- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1784143230 msgr2=0x7f1784143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1784143230 0x7f1784143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 -- 172.21.15.170:0/3005315584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1784157140 con 0x7f17840d0aa0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f17895f2640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b7ab0 0x7f1784156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1789df3640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1784143230 0x7f1784143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1788df1640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17841fa610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f177400b410 tx=0x7f177400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774015020 con 0x7f17840d0aa0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1774011920 con 0x7f17840d0aa0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841525d0 con 0x7f17840d0aa0 2024-09-21T02:14:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774011d70 con 0x7f17840d0aa0 2024-09-21T02:14:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.849+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841528b0 con 0x7f17840d0aa0 2024-09-21T02:14:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.853+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178415c850 con 0x7f17840d0aa0 2024-09-21T02:14:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.853+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1774018020 con 0x7f17840d0aa0 2024-09-21T02:14:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.853+0000 7f1779ffb640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1768073200 0x7f1768075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.853+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f17740940e0 con 0x7f17840d0aa0 2024-09-21T02:14:01.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.853+0000 7f17895f2640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1768073200 0x7f1768075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:01.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.853+0000 7f17895f2640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1768073200 0x7f1768075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1780000f70 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:01.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:01.857+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f177401f020 con 0x7f17840d0aa0 2024-09-21T02:14:02.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.037+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1784143b80 con 0x7f17840d0aa0 2024-09-21T02:14:02.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.037+0000 7f1779ffb640 1 -- 172.21.15.170:0/3005315584 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f177405f220 con 0x7f17840d0aa0 2024-09-21T02:14:02.039 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1768073200 msgr2=0x7f1768075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1768073200 0x7f1768075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1780000f70 tx=0x7f1780009040 comp rx=0 tx=0).stop 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 msgr2=0x7f17841fa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17841fa610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f177400b410 tx=0x7f177400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 shutdown_connections 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1768073200 0x7f1768075710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1784143230 0x7f1784143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17840d0aa0 0x7f17841fa610 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 --2- 172.21.15.170:0/3005315584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17840b7ab0 0x7f1784156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 >> 172.21.15.170:0/3005315584 conn(0x7f17840cee30 msgr2=0x7f17840c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 shutdown_connections 2024-09-21T02:14:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.041+0000 7f178b856640 1 -- 172.21.15.170:0/3005315584 wait complete. 2024-09-21T02:14:02.061 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:02.061 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:02.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:02.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/1198519804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb415f990 msgr2=0x7f4cb4070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/1198519804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb415f990 0x7f4cb4070030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40cc8b0 tx=0x7f4cb001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/1198519804 shutdown_connections 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/1198519804 >> v2:172.21.15.191:3300/0 conn(0x7f4cb4070570 0x7f4cb40ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/1198519804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb415f990 0x7f4cb4070030 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/1198519804 >> 172.21.15.170:0/1198519804 conn(0x7f4cb40b1c40 msgr2=0x7f4cb40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/1198519804 shutdown_connections 2024-09-21T02:14:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.213+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/1198519804 wait complete. 2024-09-21T02:14:02.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 Processor -- start 2024-09-21T02:14:02.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 -- start start 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cb4070570 0x7f4cb41f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 0x7f4cb409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cb415f990 0x7f4cb41fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4169100 con 0x7f4cb40cea00 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb40cb710 con 0x7f4cb4070570 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cbb4e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb4169280 con 0x7f4cb415f990 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 0x7f4cb409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 0x7f4cb409a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45832/0 (socket says 172.21.15.170:45832) 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 -- 172.21.15.170:0/4038389099 learned_addr learned my addr 172.21.15.170:0/4038389099 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb9284640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cb4070570 0x7f4cb41f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb9a85640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cb415f990 0x7f4cb41fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 -- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cb415f990 msgr2=0x7f4cb41fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cb415f990 0x7f4cb41fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 -- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cb4070570 msgr2=0x7f4cb41f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cb4070570 0x7f4cb41f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 -- 172.21.15.170:0/4038389099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb409c900 con 0x7f4cb40cea00 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb9a85640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cb415f990 0x7f4cb41fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4cb8a83640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 0x7f4cb409a9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b440 tx=0x7f4ca400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4015020 con 0x7f4cb40cea00 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ca40119a0 con 0x7f4cb40cea00 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.217+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011df0 con 0x7f4cb40cea00 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb40c73d0 con 0x7f4cb40cea00 2024-09-21T02:14:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb409cb70 con 0x7f4cb40cea00 2024-09-21T02:14:02.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb41fb590 con 0x7f4cb40cea00 2024-09-21T02:14:02.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4ca4018020 con 0x7f4cb40cea00 2024-09-21T02:14:02.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4ca9ffb640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c98073230 0x7f4c98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4ca4094150 con 0x7f4cb40cea00 2024-09-21T02:14:02.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4cb9284640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c98073230 0x7f4c98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.221+0000 7f4cb9284640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c98073230 0x7f4c98075740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00af000 tx=0x7f4cb001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:02.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.225+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4ca400d040 con 0x7f4cb40cea00 2024-09-21T02:14:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.405+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4cb40cd8a0 con 0x7f4cb40cea00 2024-09-21T02:14:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.405+0000 7f4ca9ffb640 1 -- 172.21.15.170:0/4038389099 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f4ca405f170 con 0x7f4cb40cea00 2024-09-21T02:14:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:02.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c98073230 msgr2=0x7f4c98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c98073230 0x7f4c98075740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00af000 tx=0x7f4cb001b100 comp rx=0 tx=0).stop 2024-09-21T02:14:02.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 msgr2=0x7f4cb409a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 0x7f4cb409a9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b440 tx=0x7f4ca400b900 comp rx=0 tx=0).stop 2024-09-21T02:14:02.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 shutdown_connections 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4c98073230 0x7f4c98075740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cb415f990 0x7f4cb41fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cb40cea00 0x7f4cb409a9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 --2- 172.21.15.170:0/4038389099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cb4070570 0x7f4cb41f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 >> 172.21.15.170:0/4038389099 conn(0x7f4cb40b1c40 msgr2=0x7f4cb40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 shutdown_connections 2024-09-21T02:14:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.409+0000 7f4cbb4e8640 1 -- 172.21.15.170:0/4038389099 wait complete. 2024-09-21T02:14:02.428 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:14:02.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:02.428 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-21T02:14:02.442 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:02.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:14:02.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:02.442 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T02:14:02.488 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:14:02.488 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:02.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1308002595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 msgr2=0x7f5f280d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1308002595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f280d0eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009bc0 tx=0x7f5f2001cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1308002595 shutdown_connections 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1308002595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f280d0eb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1308002595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b7ab0 0x7f5f2815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1308002595 >> 172.21.15.170:0/1308002595 conn(0x7f5f280cee30 msgr2=0x7f5f280cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1308002595 shutdown_connections 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.645+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1308002595 wait complete. 2024-09-21T02:14:02.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 Processor -- start 2024-09-21T02:14:02.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 -- start start 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b3e30 0x7f5f2815b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280b7ab0 0x7f5f280ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f28159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2816a560 con 0x7f5f280d0aa0 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2816a3e0 con 0x7f5f280b7ab0 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2816a6e0 con 0x7f5f280b3e30 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f28159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f28159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45838/0 (socket says 172.21.15.170:45838) 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 -- 172.21.15.170:0/1392514403 learned_addr learned my addr 172.21.15.170:0/1392514403 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f277fe640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b3e30 0x7f5f2815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 -- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b3e30 msgr2=0x7f5f2815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f26ffd640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280b7ab0 0x7f5f280ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b3e30 0x7f5f2815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 -- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280b7ab0 msgr2=0x7f5f280ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280b7ab0 0x7f5f280ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 -- 172.21.15.170:0/1392514403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2806e0b0 con 0x7f5f280d0aa0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f277fe640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b3e30 0x7f5f2815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f27fff640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f28159920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f200b0040 tx=0x7f5f20009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200b2040 con 0x7f5f280d0aa0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f20004130 con 0x7f5f280d0aa0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2806e390 con 0x7f5f280d0aa0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004560 con 0x7f5f280d0aa0 2024-09-21T02:14:02.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.649+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2806e950 con 0x7f5f280d0aa0 2024-09-21T02:14:02.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.653+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2806eb90 con 0x7f5f280d0aa0 2024-09-21T02:14:02.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.653+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5f200bd030 con 0x7f5f280d0aa0 2024-09-21T02:14:02.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.653+0000 7f5f0ffff640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5f14073260 0x7f5f14075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:02.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.653+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5f2012e2c0 con 0x7f5f280d0aa0 2024-09-21T02:14:02.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.653+0000 7f5f277fe640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5f14073260 0x7f5f14075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:02.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.653+0000 7f5f277fe640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5f14073260 0x7f5f14075770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009aa0 tx=0x7f5f1c005d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:02.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.657+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5f200c4060 con 0x7f5f280d0aa0 2024-09-21T02:14:02.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.869+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f2815c850 con 0x7f5f280d0aa0 2024-09-21T02:14:02.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f0ffff640 1 -- 172.21.15.170:0/1392514403 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f5f20016790 con 0x7f5f280d0aa0 2024-09-21T02:14:02.873 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:02.873 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:14:02.873 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:14:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5f14073260 msgr2=0x7f5f14075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5f14073260 0x7f5f14075770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009aa0 tx=0x7f5f1c005d30 comp rx=0 tx=0).stop 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 msgr2=0x7f5f28159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f28159920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f200b0040 tx=0x7f5f20009200 comp rx=0 tx=0).stop 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 shutdown_connections 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5f14073260 0x7f5f14075770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f280d0aa0 0x7f5f28159920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280b7ab0 0x7f5f280ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 --2- 172.21.15.170:0/1392514403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280b3e30 0x7f5f2815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 >> 172.21.15.170:0/1392514403 conn(0x7f5f280cee30 msgr2=0x7f5f2815ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 shutdown_connections 2024-09-21T02:14:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:02.873+0000 7f5f2db1a640 1 -- 172.21.15.170:0/1392514403 wait complete. 2024-09-21T02:14:02.896 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:03.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 -- 172.21.15.170:0/3784754844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 msgr2=0x7f93981620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 --2- 172.21.15.170:0/3784754844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981620c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f93980d3180 tx=0x7f938801c820 comp rx=0 tx=0).stop 2024-09-21T02:14:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 -- 172.21.15.170:0/3784754844 shutdown_connections 2024-09-21T02:14:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 --2- 172.21.15.170:0/3784754844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981620c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 -- 172.21.15.170:0/3784754844 >> 172.21.15.170:0/3784754844 conn(0x7f939809d850 msgr2=0x7f939808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 -- 172.21.15.170:0/3784754844 shutdown_connections 2024-09-21T02:14:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.045+0000 7f939d72b640 1 -- 172.21.15.170:0/3784754844 wait complete. 2024-09-21T02:14:03.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 Processor -- start 2024-09-21T02:14:03.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- start start 2024-09-21T02:14:03.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93980cfb50 0x7f939814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93980cff60 0x7f93980b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f939816d080 con 0x7f93980b9670 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f939816cf00 con 0x7f93980cff60 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f939816d200 con 0x7f93980cfb50 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981f8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45854/0 (socket says 172.21.15.170:45854) 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 -- 172.21.15.170:0/2663196628 learned_addr learned my addr 172.21.15.170:0/2663196628 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 -- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93980cfb50 msgr2=0x7f939814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93980cfb50 0x7f939814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 -- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93980cff60 msgr2=0x7f93980b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93980cff60 0x7f93980b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 -- 172.21.15.170:0/2663196628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93981f9050 con 0x7f93980b9670 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f9396ffd640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981f8b10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f938801cd00 tx=0x7f93880a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880b2040 con 0x7f93980b9670 2024-09-21T02:14:03.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93880041b0 con 0x7f93980b9670 2024-09-21T02:14:03.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388003600 con 0x7f93980b9670 2024-09-21T02:14:03.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93981f9560 con 0x7f93980b9670 2024-09-21T02:14:03.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93980b1250 con 0x7f93980b9670 2024-09-21T02:14:03.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.049+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93981620c0 con 0x7f93980b9670 2024-09-21T02:14:03.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.053+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f93880bd030 con 0x7f93980b9670 2024-09-21T02:14:03.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.053+0000 7f937f7fe640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9384073230 0x7f9384075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.053+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f938812e6c0 con 0x7f93980b9670 2024-09-21T02:14:03.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.053+0000 7f93967fc640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9384073230 0x7f9384075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.053+0000 7f93967fc640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9384073230 0x7f9384075740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f938c007fd0 tx=0x7f938c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.057+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f93880c4060 con 0x7f93980b9670 2024-09-21T02:14:03.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.249+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93980d2b70 con 0x7f93980b9670 2024-09-21T02:14:03.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.249+0000 7f937f7fe640 1 -- 172.21.15.170:0/2663196628 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f9388016410 con 0x7f93980b9670 2024-09-21T02:14:03.251 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:03.251 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:14:03.251 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:14:03.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9384073230 msgr2=0x7f9384075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9384073230 0x7f9384075740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f938c007fd0 tx=0x7f938c007d40 comp rx=0 tx=0).stop 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 msgr2=0x7f93981f8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981f8b10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f938801cd00 tx=0x7f93880a5690 comp rx=0 tx=0).stop 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 shutdown_connections 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9384073230 0x7f9384075740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93980cfb50 0x7f939814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93980cff60 0x7f93980b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 --2- 172.21.15.170:0/2663196628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93980b9670 0x7f93981f8b10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 >> 172.21.15.170:0/2663196628 conn(0x7f939809d850 msgr2=0x7f939806dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 shutdown_connections 2024-09-21T02:14:03.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.253+0000 7f939d72b640 1 -- 172.21.15.170:0/2663196628 wait complete. 2024-09-21T02:14:03.272 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:03.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a4326640 1 -- 172.21.15.170:0/2855688845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0688003a40 msgr2=0x7f0688003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a4326640 1 --2- 172.21.15.170:0/2855688845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0688003a40 0x7f0688003eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f069400c310 tx=0x7f069401cf60 comp rx=0 tx=0).stop 2024-09-21T02:14:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a28c3640 1 -- 172.21.15.170:0/2855688845 reap_dead start 2024-09-21T02:14:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a4326640 1 -- 172.21.15.170:0/2855688845 shutdown_connections 2024-09-21T02:14:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a4326640 1 -- 172.21.15.170:0/2855688845 >> 172.21.15.170:0/2855688845 conn(0x7f069c09a9c0 msgr2=0x7f069c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a4326640 1 -- 172.21.15.170:0/2855688845 shutdown_connections 2024-09-21T02:14:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.413+0000 7f06a4326640 1 -- 172.21.15.170:0/2855688845 wait complete. 2024-09-21T02:14:03.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 Processor -- start 2024-09-21T02:14:03.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 -- start start 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0688003a40 0x7f069c14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 0x7f069c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c14c4d0 0x7f069c06d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f069c167f10 con 0x7f069c15e790 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f069c167d90 con 0x7f069c14c4d0 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f069c168090 con 0x7f0688003a40 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 0x7f069c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 0x7f069c06ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45872/0 (socket says 172.21.15.170:45872) 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 -- 172.21.15.170:0/556056707 learned_addr learned my addr 172.21.15.170:0/556056707 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:03.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a20c2640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0688003a40 0x7f069c14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a28c3640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c14c4d0 0x7f069c06d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 -- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0688003a40 msgr2=0x7f069c14bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0688003a40 0x7f069c14bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 -- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c14c4d0 msgr2=0x7f069c06d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c14c4d0 0x7f069c06d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 -- 172.21.15.170:0/556056707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c14c290 con 0x7f069c15e790 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a28c3640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c14c4d0 0x7f069c06d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a18c1640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 0x7f069c06ce00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f06900018a0 tx=0x7f0690001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690015070 con 0x7f069c15e790 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f069000ea00 con 0x7f069c15e790 2024-09-21T02:14:03.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c15eee0 con 0x7f069c15e790 2024-09-21T02:14:03.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069000ee30 con 0x7f069c15e790 2024-09-21T02:14:03.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.417+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c14cc30 con 0x7f069c15e790 2024-09-21T02:14:03.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.421+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c14ca20 con 0x7f069c15e790 2024-09-21T02:14:03.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.421+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0690018070 con 0x7f069c15e790 2024-09-21T02:14:03.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.421+0000 7f068effd640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0680073200 0x7f0680075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.421+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0690094ee0 con 0x7f069c15e790 2024-09-21T02:14:03.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.421+0000 7f06a20c2640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0680073200 0x7f0680075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.421+0000 7f06a20c2640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0680073200 0x7f0680075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06940a9000 tx=0x7f0694019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.425+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f069005ff00 con 0x7f069c15e790 2024-09-21T02:14:03.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.585+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f069c0cc060 con 0x7f069c15e790 2024-09-21T02:14:03.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.589+0000 7f068effd640 1 -- 172.21.15.170:0/556056707 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7f069005f800 con 0x7f069c15e790 2024-09-21T02:14:03.591 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:03.591 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:03.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0680073200 msgr2=0x7f0680075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0680073200 0x7f0680075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f06940a9000 tx=0x7f0694019da0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 msgr2=0x7f069c06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 0x7f069c06ce00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f06900018a0 tx=0x7f0690001d60 comp rx=0 tx=0).stop 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 shutdown_connections 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0680073200 0x7f0680075710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f069c14c4d0 0x7f069c06d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f069c15e790 0x7f069c06ce00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 --2- 172.21.15.170:0/556056707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0688003a40 0x7f069c14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 >> 172.21.15.170:0/556056707 conn(0x7f069c09a9c0 msgr2=0x7f069c070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 shutdown_connections 2024-09-21T02:14:03.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.593+0000 7f06a4326640 1 -- 172.21.15.170:0/556056707 wait complete. 2024-09-21T02:14:03.612 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:14:03.612 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:03.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 -- 172.21.15.170:0/312182384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840ccb00 msgr2=0x7f69840ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 --2- 172.21.15.170:0/312182384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840ccb00 0x7f69840ccf10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6984071d80 tx=0x7f697c01cd80 comp rx=0 tx=0).stop 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 -- 172.21.15.170:0/312182384 shutdown_connections 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 --2- 172.21.15.170:0/312182384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840ccb00 0x7f69840ccf10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 --2- 172.21.15.170:0/312182384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69840b2f40 0x7f69840b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 --2- 172.21.15.170:0/312182384 >> v2:172.21.15.191:3300/0 conn(0x7f69840b5410 0x7f698406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 -- 172.21.15.170:0/312182384 >> 172.21.15.170:0/312182384 conn(0x7f69840b1c40 msgr2=0x7f69840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 -- 172.21.15.170:0/312182384 shutdown_connections 2024-09-21T02:14:03.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 -- 172.21.15.170:0/312182384 wait complete. 2024-09-21T02:14:03.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 Processor -- start 2024-09-21T02:14:03.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.761+0000 7f6989052640 1 -- start start 2024-09-21T02:14:03.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69840b2f40 0x7f698414a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 0x7f6984164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69840ccb00 0x7f698414c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69840700f0 con 0x7f69840b5410 2024-09-21T02:14:03.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f698406ff70 con 0x7f69840b2f40 2024-09-21T02:14:03.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f69840ce800 con 0x7f69840ccb00 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 0x7f6984164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 0x7f6984164ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45886/0 (socket says 172.21.15.170:45886) 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 -- 172.21.15.170:0/574363992 learned_addr learned my addr 172.21.15.170:0/574363992 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 -- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69840ccb00 msgr2=0x7f698414c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69840ccb00 0x7f698414c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 -- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69840b2f40 msgr2=0x7f698414a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69840b2f40 0x7f698414a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 -- 172.21.15.170:0/574363992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6984164ff0 con 0x7f69840b5410 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f698259c640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 0x7f6984164ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f697400d610 tx=0x7f697400dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974017070 con 0x7f69840b5410 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6974005330 con 0x7f69840b5410 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698415d3d0 con 0x7f69840b5410 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740047b0 con 0x7f69840b5410 2024-09-21T02:14:03.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698415f820 con 0x7f69840b5410 2024-09-21T02:14:03.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69840ced60 con 0x7f69840b5410 2024-09-21T02:14:03.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6974004950 con 0x7f69840b5410 2024-09-21T02:14:03.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f697b7fe640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6964073200 0x7f6964075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:03.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6974094570 con 0x7f69840b5410 2024-09-21T02:14:03.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.765+0000 7f6982d9d640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6964073200 0x7f6964075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:03.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.769+0000 7f6982d9d640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6964073200 0x7f6964075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f696c0058d0 tx=0x7f696c005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:03.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.769+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f697405f700 con 0x7f69840b5410 2024-09-21T02:14:03.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.961+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69840b3610 con 0x7f69840b5410 2024-09-21T02:14:03.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f697b7fe640 1 -- 172.21.15.170:0/574363992 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f697405f000 con 0x7f69840b5410 2024-09-21T02:14:03.966 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:03.966 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:14:03.966 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:14:03.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6964073200 msgr2=0x7f6964075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6964073200 0x7f6964075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f696c0058d0 tx=0x7f696c005860 comp rx=0 tx=0).stop 2024-09-21T02:14:03.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 msgr2=0x7f6984164ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:03.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 0x7f6984164ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f697400d610 tx=0x7f697400dad0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 shutdown_connections 2024-09-21T02:14:03.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6964073200 0x7f6964075710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69840ccb00 0x7f698414c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69840b5410 0x7f6984164ab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 --2- 172.21.15.170:0/574363992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69840b2f40 0x7f698414a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:03.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 >> 172.21.15.170:0/574363992 conn(0x7f69840b1c40 msgr2=0x7f69840c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:03.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 shutdown_connections 2024-09-21T02:14:03.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:03.965+0000 7f6989052640 1 -- 172.21.15.170:0/574363992 wait complete. 2024-09-21T02:14:03.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 -- 172.21.15.170:0/1526274010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 msgr2=0x7f79600ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 --2- 172.21.15.170:0/1526274010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 0x7f79600ce5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7950009840 tx=0x7f795001c820 comp rx=0 tx=0).stop 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 -- 172.21.15.170:0/1526274010 shutdown_connections 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 --2- 172.21.15.170:0/1526274010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 0x7f79600ce5d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 --2- 172.21.15.170:0/1526274010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79600cb460 0x7f79600cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 -- 172.21.15.170:0/1526274010 >> 172.21.15.170:0/1526274010 conn(0x7f79600b1c40 msgr2=0x7f79600b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 -- 172.21.15.170:0/1526274010 shutdown_connections 2024-09-21T02:14:04.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.141+0000 7f796762e640 1 -- 172.21.15.170:0/1526274010 wait complete. 2024-09-21T02:14:04.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 Processor -- start 2024-09-21T02:14:04.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- start start 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79600cb460 0x7f796009c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 0x7f79601484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 0x7f796006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79601690d0 con 0x7f79600cf7e0 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7960168f50 con 0x7f79600ce1c0 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7960169250 con 0x7f79600cb460 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 0x7f796006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 0x7f796006eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45900/0 (socket says 172.21.15.170:45900) 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 -- 172.21.15.170:0/1658494809 learned_addr learned my addr 172.21.15.170:0/1658494809 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f79653ca640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79600cb460 0x7f796009c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7964bc9640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 0x7f79601484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 -- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79600cb460 msgr2=0x7f796009c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79600cb460 0x7f796009c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 -- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 msgr2=0x7f79601484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 0x7f79601484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 -- 172.21.15.170:0/1658494809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7960148a20 con 0x7f79600cf7e0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f7965bcb640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 0x7f796006eeb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7950009790 tx=0x7f79500a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500bb070 con 0x7f79600cf7e0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7950005030 con 0x7f79600cf7e0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500b2040 con 0x7f79600cf7e0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796009d900 con 0x7f79600cf7e0 2024-09-21T02:14:04.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796009d620 con 0x7f79600cf7e0 2024-09-21T02:14:04.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.145+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796009d0b0 con 0x7f79600cf7e0 2024-09-21T02:14:04.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.149+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f79500be070 con 0x7f79600cf7e0 2024-09-21T02:14:04.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.149+0000 7f795dffb640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f794c073230 0x7f794c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.149+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f795012dba0 con 0x7f79600cf7e0 2024-09-21T02:14:04.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.149+0000 7f79653ca640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f794c073230 0x7f794c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.149+0000 7f79653ca640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f794c073230 0x7f794c075740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7958001650 tx=0x7f7958009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.153+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7950016410 con 0x7f79600cf7e0 2024-09-21T02:14:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79600cb870 con 0x7f79600cf7e0 2024-09-21T02:14:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f795dffb640 1 -- 172.21.15.170:0/1658494809 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5226 (secure 0 0 0) 0x7f79500f8d30 con 0x7f79600cf7e0 2024-09-21T02:14:04.358 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:04.358 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"btime":"2024-09-21T02:13:28:950597+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1603907766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1603907766},{"type":"v1","addr":"172.21.15.191:6835","nonce":1603907766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4349,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2971360955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2971360955},{"type":"v1","addr":"172.21.15.191:6837","nonce":2971360955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/969065835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":969065835},{"type":"v1","addr":"172.21.15.170:6834","nonce":969065835}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:13:25.292945+0000","modified":"2024-09-21T02:13:28.337813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4381":{"gid":4381,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/3675017549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3675017549},{"type":"v1","addr":"172.21.15.170:6835","nonce":3675017549}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4381,"qdb_cluster":[4381]},"id":1}]} 2024-09-21T02:14:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2024-09-21T02:14:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f794c073230 msgr2=0x7f794c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f794c073230 0x7f794c075740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7958001650 tx=0x7f7958009040 comp rx=0 tx=0).stop 2024-09-21T02:14:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 msgr2=0x7f796006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 0x7f796006eeb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7950009790 tx=0x7f79500a5790 comp rx=0 tx=0).stop 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 shutdown_connections 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f794c073230 0x7f794c075740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79600cf7e0 0x7f796006eeb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79600ce1c0 0x7f79601484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 --2- 172.21.15.170:0/1658494809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79600cb460 0x7f796009c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 >> 172.21.15.170:0/1658494809 conn(0x7f79600b1c40 msgr2=0x7f79600b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 shutdown_connections 2024-09-21T02:14:04.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.357+0000 7f796762e640 1 -- 172.21.15.170:0/1658494809 wait complete. 2024-09-21T02:14:04.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:04.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 -- 172.21.15.170:0/936021796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 msgr2=0x7f22dc0ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 --2- 172.21.15.170:0/936021796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc0ccf10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f22dc070e90 tx=0x7f22d401cb80 comp rx=0 tx=0).stop 2024-09-21T02:14:04.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 -- 172.21.15.170:0/936021796 shutdown_connections 2024-09-21T02:14:04.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 --2- 172.21.15.170:0/936021796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc0ccf10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 --2- 172.21.15.170:0/936021796 >> v2:172.21.15.191:3300/0 conn(0x7f22dc0b2f40 0x7f22dc0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 -- 172.21.15.170:0/936021796 >> 172.21.15.170:0/936021796 conn(0x7f22dc0b1c40 msgr2=0x7f22dc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 -- 172.21.15.170:0/936021796 shutdown_connections 2024-09-21T02:14:04.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.525+0000 7f22e4174640 1 -- 172.21.15.170:0/936021796 wait complete. 2024-09-21T02:14:04.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 Processor -- start 2024-09-21T02:14:04.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- start start 2024-09-21T02:14:04.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc0b2f40 0x7f22dc154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22dc0b5410 0x7f22dc09cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc1f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc169e50 con 0x7f22dc0ccb00 2024-09-21T02:14:04.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0cf970 con 0x7f22dc0b2f40 2024-09-21T02:14:04.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc169fd0 con 0x7f22dc0b5410 2024-09-21T02:14:04.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc1f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc1f9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45920/0 (socket says 172.21.15.170:45920) 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 -- 172.21.15.170:0/460681921 learned_addr learned my addr 172.21.15.170:0/460681921 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 -- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22dc0b5410 msgr2=0x7f22dc09cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e170f640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22dc0b5410 0x7f22dc09cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22dc0b5410 0x7f22dc09cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e1f10640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc0b2f40 0x7f22dc154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 -- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc0b2f40 msgr2=0x7f22dc154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc0b2f40 0x7f22dc154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 -- 172.21.15.170:0/460681921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc151910 con 0x7f22dc0ccb00 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e2711640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc1f9fb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22d401c5f0 tx=0x7f22d4005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b2040 con 0x7f22dc0ccb00 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22d40bb070 con 0x7f22dc0ccb00 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc151bf0 con 0x7f22dc0ccb00 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4004530 con 0x7f22dc0ccb00 2024-09-21T02:14:04.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc155300 con 0x7f22dc0ccb00 2024-09-21T02:14:04.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.529+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc151e30 con 0x7f22dc0ccb00 2024-09-21T02:14:04.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.533+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f22d40046d0 con 0x7f22dc0ccb00 2024-09-21T02:14:04.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.533+0000 7f22d27fc640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22c0073200 0x7f22c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.533+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f22d412eba0 con 0x7f22dc0ccb00 2024-09-21T02:14:04.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.533+0000 7f22e1f10640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22c0073200 0x7f22c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.533+0000 7f22e1f10640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22c0073200 0x7f22c0075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22d80015f0 tx=0x7f22d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.537+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f22dc151e30 con 0x7f22dc0ccb00 2024-09-21T02:14:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.697+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22dc151670 con 0x7f22dc0ccb00 2024-09-21T02:14:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22d27fc640 1 -- 172.21.15.170:0/460681921 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16717 (secure 0 0 0) 0x7f22d4016770 con 0x7f22dc0ccb00 2024-09-21T02:14:04.702 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:04.702 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:13:26.871235+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:13:23.204915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:13:24.242738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22c0073200 msgr2=0x7f22c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22c0073200 0x7f22c0075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22d80015f0 tx=0x7f22d8009040 comp rx=0 tx=0).stop 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 msgr2=0x7f22dc1f9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc1f9fb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22d401c5f0 tx=0x7f22d4005a60 comp rx=0 tx=0).stop 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 shutdown_connections 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22c0073200 0x7f22c0075710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22dc0b5410 0x7f22dc09cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22dc0ccb00 0x7f22dc1f9fb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 --2- 172.21.15.170:0/460681921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22dc0b2f40 0x7f22dc154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 >> 172.21.15.170:0/460681921 conn(0x7f22dc0b1c40 msgr2=0x7f22dc0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 shutdown_connections 2024-09-21T02:14:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.701+0000 7f22e4174640 1 -- 172.21.15.170:0/460681921 wait complete. 2024-09-21T02:14:04.722 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:14:04.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 -- 172.21.15.170:0/3089728128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 msgr2=0x7fa73c0ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 --2- 172.21.15.170:0/3089728128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c0ce4c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0cc8b0 tx=0x7fa73001c8a0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 -- 172.21.15.170:0/3089728128 shutdown_connections 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 --2- 172.21.15.170:0/3089728128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c0ce4c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 --2- 172.21.15.170:0/3089728128 >> v2:172.21.15.191:3300/0 conn(0x7fa73c15f990 0x7fa73c070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 -- 172.21.15.170:0/3089728128 >> 172.21.15.170:0/3089728128 conn(0x7fa73c0b1c40 msgr2=0x7fa73c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 -- 172.21.15.170:0/3089728128 shutdown_connections 2024-09-21T02:14:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.861+0000 7fa74114d640 1 -- 172.21.15.170:0/3089728128 wait complete. 2024-09-21T02:14:04.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 Processor -- start 2024-09-21T02:14:04.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- start start 2024-09-21T02:14:04.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c1f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa73c0cea00 0x7fa73c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa73c15f990 0x7fa73c09a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c168800 con 0x7fa73c070570 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c0cb760 con 0x7fa73c15f990 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa73c168980 con 0x7fa73c0cea00 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c1f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c1f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45942/0 (socket says 172.21.15.170:45942) 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 -- 172.21.15.170:0/158961802 learned_addr learned my addr 172.21.15.170:0/158961802 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73a59c640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa73c0cea00 0x7fa73c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 -- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa73c0cea00 msgr2=0x7fa73c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73b59e640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa73c15f990 0x7fa73c09a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa73c0cea00 0x7fa73c156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 -- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa73c15f990 msgr2=0x7fa73c09a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa73c15f990 0x7fa73c09a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 -- 172.21.15.170:0/158961802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c1fa540 con 0x7fa73c070570 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa73ad9d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c1f6f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c940 tx=0x7fa72c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c004220 con 0x7fa73c070570 2024-09-21T02:14:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa72c0079b0 con 0x7fa73c070570 2024-09-21T02:14:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73c1f77a0 con 0x7fa73c070570 2024-09-21T02:14:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0044b0 con 0x7fa73c070570 2024-09-21T02:14:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73c1f7d60 con 0x7fa73c070570 2024-09-21T02:14:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.865+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73c0cd8a0 con 0x7fa73c070570 2024-09-21T02:14:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.869+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa72c005020 con 0x7fa73c070570 2024-09-21T02:14:04.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.869+0000 7fa7237fe640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa728073230 0x7fa728075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:04.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.869+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa72c094630 con 0x7fa73c070570 2024-09-21T02:14:04.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.869+0000 7fa73a59c640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa728073230 0x7fa728075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:04.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.869+0000 7fa73a59c640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa728073230 0x7fa728075740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7300af000 tx=0x7fa73001b0e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:04.873+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa72c014030 con 0x7fa73c070570 2024-09-21T02:14:05.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.077+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa73c0cf2a0 con 0x7fa73c070570 2024-09-21T02:14:05.078 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:14:05.077+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:14:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa7237fe640 1 -- 172.21.15.170:0/158961802 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fa72c05f7c0 con 0x7fa73c070570 2024-09-21T02:14:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:14:05.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa728073230 msgr2=0x7fa728075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa728073230 0x7fa728075740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa7300af000 tx=0x7fa73001b0e0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 msgr2=0x7fa73c1f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c1f6f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c940 tx=0x7fa72c00ce00 comp rx=0 tx=0).stop 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 shutdown_connections 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa728073230 0x7fa728075740 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa73c0cea00 0x7fa73c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa73c15f990 0x7fa73c09a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 --2- 172.21.15.170:0/158961802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa73c070570 0x7fa73c1f6f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 >> 172.21.15.170:0/158961802 conn(0x7fa73c0b1c40 msgr2=0x7fa73c0c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 shutdown_connections 2024-09-21T02:14:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.081+0000 7fa74114d640 1 -- 172.21.15.170:0/158961802 wait complete. 2024-09-21T02:14:05.093 INFO:tasks.ceph.mds.c.smithi170.stderr: -2712> 2024-09-21T02:13:29.834+0000 7f212ad5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:14:05.104 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:14:05.156 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:14:05.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- 172.21.15.170:0/3774625294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0744b0 msgr2=0x7ff28c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 --2- 172.21.15.170:0/3774625294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0744b0 0x7ff28c0ce780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0726b0 tx=0x7ff28401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- 172.21.15.170:0/3774625294 shutdown_connections 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 --2- 172.21.15.170:0/3774625294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0744b0 0x7ff28c0ce780 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 --2- 172.21.15.170:0/3774625294 >> v2:172.21.15.191:3300/0 conn(0x7ff28c0cf7d0 0x7ff28c073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- 172.21.15.170:0/3774625294 >> 172.21.15.170:0/3774625294 conn(0x7ff28c0d0170 msgr2=0x7ff28c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- 172.21.15.170:0/3774625294 shutdown_connections 2024-09-21T02:14:05.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- 172.21.15.170:0/3774625294 wait complete. 2024-09-21T02:14:05.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 Processor -- start 2024-09-21T02:14:05.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- start start 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28c0740a0 0x7ff28c0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28c0744b0 0x7ff28c0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 0x7ff28c14c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0cee40 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c0cecc0 con 0x7ff28c0744b0 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff291f94640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff28c166ca0 con 0x7ff28c0740a0 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 0x7ff28c14c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 0x7ff28c14c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45976/0 (socket says 172.21.15.170:45976) 2024-09-21T02:14:05.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.261+0000 7ff28bfff640 1 -- 172.21.15.170:0/368456771 learned_addr learned my addr 172.21.15.170:0/368456771 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28bfff640 1 -- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28c0740a0 msgr2=0x7ff28c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28bfff640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28c0740a0 0x7ff28c0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28affd640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28c0744b0 0x7ff28c0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28bfff640 1 -- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28c0744b0 msgr2=0x7ff28c0ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28bfff640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28c0744b0 0x7ff28c0ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28bfff640 1 -- 172.21.15.170:0/368456771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c1fa740 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28bfff640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 0x7ff28c14c350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff284009850 tx=0x7ff2840a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840b2040 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2840043d0 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c0ca510 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284003860 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c14da00 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c14bf90 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff2840c0060 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff273fff640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff278073230 0x7ff278075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:05.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7ff28412df00 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28b7fe640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff278073230 0x7ff278075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:05.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.265+0000 7ff28b7fe640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff278073230 0x7ff278075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2800058d0 tx=0x7ff280005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:05.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.269+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff2840164b0 con 0x7ff28c0cf7d0 2024-09-21T02:14:05.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:05.477+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff28c167220 con 0x7ff28c0cf7d0 2024-09-21T02:14:06.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.081+0000 7ff273fff640 1 -- 172.21.15.170:0/368456771 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7ff2840f8ec0 con 0x7ff28c0cf7d0 2024-09-21T02:14:06.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff278073230 msgr2=0x7ff278075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff278073230 0x7ff278075740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2800058d0 tx=0x7ff280005860 comp rx=0 tx=0).stop 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 msgr2=0x7ff28c14c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 0x7ff28c14c350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff284009850 tx=0x7ff2840a56b0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 shutdown_connections 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff278073230 0x7ff278075740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28c0cf7d0 0x7ff28c14c350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff28c0744b0 0x7ff28c0ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 --2- 172.21.15.170:0/368456771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28c0740a0 0x7ff28c0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 >> 172.21.15.170:0/368456771 conn(0x7ff28c0d0170 msgr2=0x7ff28c06ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 shutdown_connections 2024-09-21T02:14:06.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.085+0000 7ff291f94640 1 -- 172.21.15.170:0/368456771 wait complete. 2024-09-21T02:14:06.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 -- 172.21.15.170:0/2076365754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 msgr2=0x7f7210003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 --2- 172.21.15.170:0/2076365754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f7210003e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7218009b60 tx=0x7f721801d2b0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722ceda640 1 -- 172.21.15.170:0/2076365754 reap_dead start 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 -- 172.21.15.170:0/2076365754 shutdown_connections 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 -- 172.21.15.170:0/2076365754 >> 172.21.15.170:0/2076365754 conn(0x7f72280ced50 msgr2=0x7f72280cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 -- 172.21.15.170:0/2076365754 shutdown_connections 2024-09-21T02:14:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 -- 172.21.15.170:0/2076365754 wait complete. 2024-09-21T02:14:06.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 Processor -- start 2024-09-21T02:14:06.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.277+0000 7f722e93d640 1 -- start start 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f722814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72281544c0 0x7f72281569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7228156ef0 0x7f7228157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7228167e10 con 0x7f7210003a10 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7228167c90 con 0x7f7228156ef0 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7228167f90 con 0x7f72281544c0 2024-09-21T02:14:06.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f722814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f722814f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54028/0 (socket says 172.21.15.170:54028) 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 -- 172.21.15.170:0/4225020733 learned_addr learned my addr 172.21.15.170:0/4225020733 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f72277fe640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72281544c0 0x7f72281569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722ceda640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7228156ef0 0x7f7228157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 -- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72281544c0 msgr2=0x7f72281569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72281544c0 0x7f72281569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 -- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7228156ef0 msgr2=0x7f7228157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7228156ef0 0x7f7228157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 -- 172.21.15.170:0/4225020733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7228157840 con 0x7f7210003a10 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7227fff640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f722814f8a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f721801d0e0 tx=0x7f72180055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72180b1040 con 0x7f7210003a10 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7218004030 con 0x7f7210003a10 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72280b5b10 con 0x7f7210003a10 2024-09-21T02:14:06.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218004440 con 0x7f7210003a10 2024-09-21T02:14:06.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72280b6260 con 0x7f7210003a10 2024-09-21T02:14:06.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722815a170 con 0x7f7210003a10 2024-09-21T02:14:06.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.281+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f72180bd020 con 0x7f7210003a10 2024-09-21T02:14:06.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.285+0000 7f7224ff9640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7214073200 0x7f7214075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:06.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.285+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7f721812f430 con 0x7f7210003a10 2024-09-21T02:14:06.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.285+0000 7f72277fe640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7214073200 0x7f7214075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:06.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.285+0000 7f72277fe640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7214073200 0x7f7214075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7228153fd0 tx=0x7f721c00f530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:06.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.289+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f72180c4020 con 0x7f7210003a10 2024-09-21T02:14:06.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:06.453+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f72280cd0f0 con 0x7f7210003a10 2024-09-21T02:14:07.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f7224ff9640 1 -- 172.21.15.170:0/4225020733 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f7218016fc0 con 0x7f7210003a10 2024-09-21T02:14:07.093 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7214073200 msgr2=0x7f7214075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7214073200 0x7f7214075710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7228153fd0 tx=0x7f721c00f530 comp rx=0 tx=0).stop 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 msgr2=0x7f722814f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f722814f8a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f721801d0e0 tx=0x7f72180055d0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 shutdown_connections 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7214073200 0x7f7214075710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72281544c0 0x7f72281569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7228156ef0 0x7f7228157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 --2- 172.21.15.170:0/4225020733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7210003a10 0x7f722814f8a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 >> 172.21.15.170:0/4225020733 conn(0x7f72280ced50 msgr2=0x7f7228162460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.093+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 shutdown_connections 2024-09-21T02:14:07.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.097+0000 7f722e93d640 1 -- 172.21.15.170:0/4225020733 wait complete. 2024-09-21T02:14:07.126 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:14:07.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- 172.21.15.170:0/2521613077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57840051b0 msgr2=0x7f578400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 --2- 172.21.15.170:0/2521613077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57840051b0 0x7f578400b9d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5798072e00 tx=0x7f579001cae0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f579ed8b640 1 -- 172.21.15.170:0/2521613077 reap_dead start 2024-09-21T02:14:07.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- 172.21.15.170:0/2521613077 shutdown_connections 2024-09-21T02:14:07.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- 172.21.15.170:0/2521613077 >> 172.21.15.170:0/2521613077 conn(0x7f57980b1c40 msgr2=0x7f57980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:07.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- 172.21.15.170:0/2521613077 shutdown_connections 2024-09-21T02:14:07.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- 172.21.15.170:0/2521613077 wait complete. 2024-09-21T02:14:07.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 Processor -- start 2024-09-21T02:14:07.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- start start 2024-09-21T02:14:07.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57840051b0 0x7f5798156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57981535d0 0x7f57980c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 0x7f579815f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f57980af5c0 con 0x7f579815ef10 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f57980af440 con 0x7f57840051b0 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f57a07ee640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f57980af740 con 0x7f57981535d0 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f579ed8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 0x7f579815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f579ed8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 0x7f579815f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54054/0 (socket says 172.21.15.170:54054) 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f579ed8b640 1 -- 172.21.15.170:0/496021831 learned_addr learned my addr 172.21.15.170:0/496021831 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f579dd89640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57981535d0 0x7f57980c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.293+0000 7f579e58a640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57840051b0 0x7f5798156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579ed8b640 1 -- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57981535d0 msgr2=0x7f57980c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579ed8b640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57981535d0 0x7f57980c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579ed8b640 1 -- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57840051b0 msgr2=0x7f5798156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579ed8b640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57840051b0 0x7f5798156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579ed8b640 1 -- 172.21.15.170:0/496021831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579815f860 con 0x7f579815ef10 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579ed8b640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 0x7f579815f320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f579001c910 tx=0x7f5790003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790006cb0 con 0x7f579815ef10 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57900a5be0 con 0x7f579815ef10 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579809d0a0 con 0x7f579815ef10 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790001990 con 0x7f579815ef10 2024-09-21T02:14:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579809c5c0 con 0x7f579815ef10 2024-09-21T02:14:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57980b3a50 con 0x7f579815ef10 2024-09-21T02:14:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57900ae430 con 0x7f579815ef10 2024-09-21T02:14:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f578effd640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f577c073230 0x7f577c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5386+0+0 (secure 0 0 0) 0x7f579012de30 con 0x7f579815ef10 2024-09-21T02:14:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579e58a640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f577c073230 0x7f577c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.297+0000 7f579e58a640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f577c073230 0x7f577c075740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5788001110 tx=0x7f5788000f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:07.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.301+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f579815f860 con 0x7f579815ef10 2024-09-21T02:14:07.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:07.477+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f579806ef40 con 0x7f579815ef10 2024-09-21T02:14:08.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.093+0000 7f578effd640 1 -- 172.21.15.170:0/496021831 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f57900167f0 con 0x7f579815ef10 2024-09-21T02:14:08.096 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f577c073230 msgr2=0x7f577c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f577c073230 0x7f577c075740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5788001110 tx=0x7f5788000f40 comp rx=0 tx=0).stop 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 msgr2=0x7f579815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 0x7f579815f320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f579001c910 tx=0x7f5790003f40 comp rx=0 tx=0).stop 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 shutdown_connections 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f577c073230 0x7f577c075740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.097+0000 7f57a07ee640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57981535d0 0x7f57980c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.101+0000 7f57a07ee640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579815ef10 0x7f579815f320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.101+0000 7f57a07ee640 1 --2- 172.21.15.170:0/496021831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57840051b0 0x7f5798156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.101+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 >> 172.21.15.170:0/496021831 conn(0x7f57980b1c40 msgr2=0x7f57980b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:08.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.101+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 shutdown_connections 2024-09-21T02:14:08.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.101+0000 7f57a07ee640 1 -- 172.21.15.170:0/496021831 wait complete. 2024-09-21T02:14:08.131 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:14:08.131 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:14:08.131 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:14:08.132 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:08.132 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:14:08.132 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:14:08.132 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:14:08.132 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:08.133 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:14:08.133 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:14:08.133 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:14:08.133 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:08.134 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:14:08.134 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:14:08.134 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:14:08.134 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:08.145 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:14:08.145 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:14:08.148 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:14:08.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:14:08.149 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:14:08.149 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:14:08.161 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:14:08.161 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:14:08.190 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:14:08.190 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:08.191 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:14:08.191 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:08.204 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:14:08.204 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:08.205 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:14:08.206 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:08.206 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:14:08.284 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:14:08.285 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:14:08.302 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:14:08.304 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 -- 172.21.15.170:0/2884185718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b1230 msgr2=0x7f38580b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 --2- 172.21.15.170:0/2884185718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b1230 0x7f38580b1640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3850009c80 tx=0x7f385001cd10 comp rx=0 tx=0).stop 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 -- 172.21.15.170:0/2884185718 shutdown_connections 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 --2- 172.21.15.170:0/2884185718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b1230 0x7f38580b1640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 --2- 172.21.15.170:0/2884185718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580d0e50 0x7f38580d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 -- 172.21.15.170:0/2884185718 >> 172.21.15.170:0/2884185718 conn(0x7f385808bf50 msgr2=0x7f385809d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 -- 172.21.15.170:0/2884185718 shutdown_connections 2024-09-21T02:14:08.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 -- 172.21.15.170:0/2884185718 wait complete. 2024-09-21T02:14:08.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 Processor -- start 2024-09-21T02:14:08.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.357+0000 7f385ff7b640 1 -- start start 2024-09-21T02:14:08.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580b1230 0x7f3858157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:08.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 0x7f385815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:08.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38580d0e50 0x7f38580c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:08.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f385816c090 con 0x7f38580b3fa0 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f385816bf10 con 0x7f38580d0e50 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f385816c210 con 0x7f38580b1230 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 0x7f385815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 0x7f385815ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54100/0 (socket says 172.21.15.170:54100) 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 -- 172.21.15.170:0/399585112 learned_addr learned my addr 172.21.15.170:0/399585112 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385e518640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38580d0e50 0x7f38580c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385dd17640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580b1230 0x7f3858157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 -- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580b1230 msgr2=0x7f3858157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580b1230 0x7f3858157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 -- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38580d0e50 msgr2=0x7f38580c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38580d0e50 0x7f38580c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 -- 172.21.15.170:0/399585112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38581fd490 con 0x7f38580b3fa0 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385dd17640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580b1230 0x7f3858157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385d516640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 0x7f385815ec50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f385400e970 tx=0x7f385400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:08.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385400cd20 con 0x7f38580b3fa0 2024-09-21T02:14:08.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3854004020 con 0x7f38580b3fa0 2024-09-21T02:14:08.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385815e170 con 0x7f38580b3fa0 2024-09-21T02:14:08.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854004470 con 0x7f38580b3fa0 2024-09-21T02:14:08.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38580c8810 con 0x7f38580b3fa0 2024-09-21T02:14:08.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.361+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38580b1640 con 0x7f38580b3fa0 2024-09-21T02:14:08.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.365+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3854003600 con 0x7f38580b3fa0 2024-09-21T02:14:08.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.365+0000 7f384e7fc640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f383c073230 0x7f383c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:08.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.365+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7f3854094670 con 0x7f38580b3fa0 2024-09-21T02:14:08.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.365+0000 7f385dd17640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f383c073230 0x7f383c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:08.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.365+0000 7f385dd17640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f383c073230 0x7f383c075740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38480016b0 tx=0x7f3848009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:08.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.369+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f385405fab0 con 0x7f38580b3fa0 2024-09-21T02:14:08.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:08.553+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f38580cee70 con 0x7f38580b3fa0 2024-09-21T02:14:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.117+0000 7f384e7fc640 1 -- 172.21.15.170:0/399585112 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7f385405f3b0 con 0x7f38580b3fa0 2024-09-21T02:14:09.118 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f383c073230 msgr2=0x7f383c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f383c073230 0x7f383c075740 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38480016b0 tx=0x7f3848009040 comp rx=0 tx=0).stop 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 msgr2=0x7f385815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 0x7f385815ec50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f385400e970 tx=0x7f385400ee30 comp rx=0 tx=0).stop 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 shutdown_connections 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f383c073230 0x7f383c075740 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38580d0e50 0x7f38580c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38580b3fa0 0x7f385815ec50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 --2- 172.21.15.170:0/399585112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f38580b1230 0x7f3858157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 >> 172.21.15.170:0/399585112 conn(0x7f385808bf50 msgr2=0x7f38580b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 shutdown_connections 2024-09-21T02:14:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.121+0000 7f385ff7b640 1 -- 172.21.15.170:0/399585112 wait complete. 2024-09-21T02:14:09.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:14:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 -- 172.21.15.170:0/655751654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80cd210 msgr2=0x7f33c80b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/655751654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80cd210 0x7f33c80b3350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f33c8071d80 tx=0x7f33c001c800 comp rx=0 tx=0).stop 2024-09-21T02:14:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 -- 172.21.15.170:0/655751654 shutdown_connections 2024-09-21T02:14:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/655751654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80cd210 0x7f33c80b3350 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/655751654 >> v2:172.21.15.191:3300/0 conn(0x7f33c80b5410 0x7f33c806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 -- 172.21.15.170:0/655751654 >> 172.21.15.170:0/655751654 conn(0x7f33c80b1c40 msgr2=0x7f33c80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 -- 172.21.15.170:0/655751654 shutdown_connections 2024-09-21T02:14:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 -- 172.21.15.170:0/655751654 wait complete. 2024-09-21T02:14:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 Processor -- start 2024-09-21T02:14:09.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.305+0000 7f33cf3ab640 1 -- start start 2024-09-21T02:14:09.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 0x7f33c8150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b5410 0x7f33c81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c8154910 0x7f33c80c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8169fd0 con 0x7f33c80b2f40 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8169e50 con 0x7f33c80b5410 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f33c816a150 con 0x7f33c8154910 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 0x7f33c8150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 0x7f33c8150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54104/0 (socket says 172.21.15.170:54104) 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 -- 172.21.15.170:0/525151519 learned_addr learned my addr 172.21.15.170:0/525151519 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cc946640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b5410 0x7f33c81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 -- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c8154910 msgr2=0x7f33c80c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd948640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c8154910 0x7f33c80c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c8154910 0x7f33c80c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 -- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b5410 msgr2=0x7f33c81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b5410 0x7f33c81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 -- 172.21.15.170:0/525151519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c80af840 con 0x7f33c80b2f40 2024-09-21T02:14:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cc946640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b5410 0x7f33c81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cd147640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 0x7f33c8150480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f33b800ccb0 tx=0x7f33b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8004b30 con 0x7f33c80b2f40 2024-09-21T02:14:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33b8004dc0 con 0x7f33c80b2f40 2024-09-21T02:14:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c81550e0 con 0x7f33c80b2f40 2024-09-21T02:14:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8005280 con 0x7f33c80b2f40 2024-09-21T02:14:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c80afb00 con 0x7f33c80b2f40 2024-09-21T02:14:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c806fa30 con 0x7f33c80b2f40 2024-09-21T02:14:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33b8005420 con 0x7f33c80b2f40 2024-09-21T02:14:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33c5ffb640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33b4073200 0x7f33b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.309+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f33b8094990 con 0x7f33c80b2f40 2024-09-21T02:14:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.313+0000 7f33cc946640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33b4073200 0x7f33b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.313+0000 7f33cc946640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33b4073200 0x7f33b4075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33c80af390 tx=0x7f33bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.317+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33b8014030 con 0x7f33c80b2f40 2024-09-21T02:14:09.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.549+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f33c80ce8f0 con 0x7f33c80b2f40 2024-09-21T02:14:09.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33c5ffb640 1 -- 172.21.15.170:0/525151519 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+6645 (secure 0 0 0) 0x7f33b805fd90 con 0x7f33c80b2f40 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:14:09.555 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:09.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.557 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.558 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.559 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:14:09.560 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.561 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:09.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:14:09.563 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:09.564 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-mds", 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCrK+5mYQm5ORAAVX2y/AoRLCLEV5w1pciMwA==", 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCrK+5mYhO5ORAAkhiybx/3ARWQKa9sdE1AFQ==", 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.565 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-osd", 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCrK+5mhRu5ORAAPRoHTY1ph1O26JQVGFxpLQ==", 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCrK+5maiO5ORAABRJbPxDZutcQIBPoVS/fNw==", 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCrK+5m7S65ORAA2/Xn5FZWowECrFJRu5NS/A==", 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCrK+5mHzu5ORAA8ChYybUmDIEfPpcC7XquyQ==", 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:09.568 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:14:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:14:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:09.569 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:14:09.569 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33b4073200 msgr2=0x7f33b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33b4073200 0x7f33b4075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33c80af390 tx=0x7f33bc009040 comp rx=0 tx=0).stop 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 msgr2=0x7f33c8150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 0x7f33c8150480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f33b800ccb0 tx=0x7f33b800a620 comp rx=0 tx=0).stop 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 shutdown_connections 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33b4073200 0x7f33b4075710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33c8154910 0x7f33c80c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c80b5410 0x7f33c81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 --2- 172.21.15.170:0/525151519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33c80b2f40 0x7f33c8150480 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 >> 172.21.15.170:0/525151519 conn(0x7f33c80b1c40 msgr2=0x7f33c815c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 shutdown_connections 2024-09-21T02:14:09.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.553+0000 7f33cf3ab640 1 -- 172.21.15.170:0/525151519 wait complete. 2024-09-21T02:14:09.575 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T02:14:09.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 -- 172.21.15.170:0/2037893875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780cfbf0 msgr2=0x7fc0780cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/2037893875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780cfbf0 0x7fc0780cf3a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc06800aa10 tx=0x7fc06801dc30 comp rx=0 tx=0).stop 2024-09-21T02:14:09.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 -- 172.21.15.170:0/2037893875 shutdown_connections 2024-09-21T02:14:09.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/2037893875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 0x7fc0780caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/2037893875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780cfbf0 0x7fc0780cf3a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 -- 172.21.15.170:0/2037893875 >> 172.21.15.170:0/2037893875 conn(0x7fc0780b1c40 msgr2=0x7fc0780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 -- 172.21.15.170:0/2037893875 shutdown_connections 2024-09-21T02:14:09.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 -- 172.21.15.170:0/2037893875 wait complete. 2024-09-21T02:14:09.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 Processor -- start 2024-09-21T02:14:09.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.733+0000 7fc07f1ad640 1 -- start start 2024-09-21T02:14:09.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 0x7fc07815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 0x7fc0780ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0780cfbf0 0x7fc0780b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc078168460 con 0x7fc0780705b0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781682e0 con 0x7fc0780cfbf0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc0781685e0 con 0x7fc0780701a0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 0x7fc0780ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 0x7fc0780ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54120/0 (socket says 172.21.15.170:54120) 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 -- 172.21.15.170:0/3762055929 learned_addr learned my addr 172.21.15.170:0/3762055929 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07d74a640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0780cfbf0 0x7fc0780b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07cf49640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 0x7fc07815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 -- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 msgr2=0x7fc07815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 0x7fc07815d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 -- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0780cfbf0 msgr2=0x7fc0780b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0780cfbf0 0x7fc0780b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 -- 172.21.15.170:0/3762055929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07815f210 con 0x7fc0780705b0 2024-09-21T02:14:09.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07cf49640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 0x7fc07815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc077fff640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 0x7fc0780ad4c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00d650 tx=0x7fc06c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c017070 con 0x7fc0780705b0 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc06c005310 con 0x7fc0780705b0 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0781f9ec0 con 0x7fc0780705b0 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c009820 con 0x7fc0780705b0 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07815ebc0 con 0x7fc0780705b0 2024-09-21T02:14:09.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0780caa70 con 0x7fc0780705b0 2024-09-21T02:14:09.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc06c0099c0 con 0x7fc0780705b0 2024-09-21T02:14:09.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc0757fa640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc064073200 0x7fc064075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:09.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fc06c093fd0 con 0x7fc0780705b0 2024-09-21T02:14:09.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.737+0000 7fc07cf49640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc064073200 0x7fc064075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:09.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.741+0000 7fc07cf49640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc064073200 0x7fc064075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b0000 tx=0x7fc06801c470 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:09.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.745+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc06c05f340 con 0x7fc0780705b0 2024-09-21T02:14:09.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.981+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc078160580 con 0x7fc0780705b0 2024-09-21T02:14:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.985+0000 7fc0757fa640 1 -- 172.21.15.170:0/3762055929 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v13) ==== 90+0+0 (secure 0 0 0) 0x7fc06c05ec40 con 0x7fc0780705b0 2024-09-21T02:14:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc064073200 msgr2=0x7fc064075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc064073200 0x7fc064075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b0000 tx=0x7fc06801c470 comp rx=0 tx=0).stop 2024-09-21T02:14:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 msgr2=0x7fc0780ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 0x7fc0780ad4c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00d650 tx=0x7fc06c00db10 comp rx=0 tx=0).stop 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 shutdown_connections 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc064073200 0x7fc064075710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0780cfbf0 0x7fc0780b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0780705b0 0x7fc0780ad4c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 --2- 172.21.15.170:0/3762055929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0780701a0 0x7fc07815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 >> 172.21.15.170:0/3762055929 conn(0x7fc0780b1c40 msgr2=0x7fc0780b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:09.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 shutdown_connections 2024-09-21T02:14:09.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:09.989+0000 7fc07f1ad640 1 -- 172.21.15.170:0/3762055929 wait complete. 2024-09-21T02:14:10.008 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T02:14:10.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.161+0000 7f1423350640 1 -- 172.21.15.170:0/3605052096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c0cadf0 msgr2=0x7f141c0cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.161+0000 7f1423350640 1 --2- 172.21.15.170:0/3605052096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c0cadf0 0x7f141c0cb200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f141c0cc8b0 tx=0x7f141401deb0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.161+0000 7f1423350640 1 -- 172.21.15.170:0/3605052096 shutdown_connections 2024-09-21T02:14:10.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.161+0000 7f1423350640 1 --2- 172.21.15.170:0/3605052096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c0cadf0 0x7f141c0cb200 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.161+0000 7f1423350640 1 --2- 172.21.15.170:0/3605052096 >> v2:172.21.15.191:3300/0 conn(0x7f141c0ce4c0 0x7f141c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.161+0000 7f1423350640 1 -- 172.21.15.170:0/3605052096 >> 172.21.15.170:0/3605052096 conn(0x7f141c0b1c40 msgr2=0x7f141c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 -- 172.21.15.170:0/3605052096 shutdown_connections 2024-09-21T02:14:10.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 -- 172.21.15.170:0/3605052096 wait complete. 2024-09-21T02:14:10.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 Processor -- start 2024-09-21T02:14:10.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 -- start start 2024-09-21T02:14:10.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f141c0cadf0 0x7f141c0af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f141c0ce4c0 0x7f141c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 0x7f141c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f141c168170 con 0x7f141c15f990 2024-09-21T02:14:10.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f141c167ff0 con 0x7f141c0cadf0 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1423350640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f141c1682f0 con 0x7f141c0ce4c0 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 0x7f141c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 0x7f141c0d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54142/0 (socket says 172.21.15.170:54142) 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 -- 172.21.15.170:0/3088671415 learned_addr learned my addr 172.21.15.170:0/3088671415 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14208eb640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f141c0ce4c0 0x7f141c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14210ec640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f141c0cadf0 0x7f141c0af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 -- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f141c0ce4c0 msgr2=0x7f141c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f141c0ce4c0 0x7f141c155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 -- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f141c0cadf0 msgr2=0x7f141c0af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f141c0cadf0 0x7f141c0af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 -- 172.21.15.170:0/3088671415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1fd4a0 con 0x7f141c15f990 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f14218ed640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 0x7f141c0d02e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f141400ac20 tx=0x7f14140a6c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140bd070 con 0x7f141c15f990 2024-09-21T02:14:10.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1414004e00 con 0x7f141c15f990 2024-09-21T02:14:10.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.165+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140b3040 con 0x7f141c15f990 2024-09-21T02:14:10.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c0ad0a0 con 0x7f141c15f990 2024-09-21T02:14:10.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1547b0 con 0x7f141c15f990 2024-09-21T02:14:10.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c0adc40 con 0x7f141c15f990 2024-09-21T02:14:10.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1414004320 con 0x7f141c15f990 2024-09-21T02:14:10.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f1411ffb640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1400073230 0x7f1400075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f141412dff0 con 0x7f141c15f990 2024-09-21T02:14:10.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f14210ec640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1400073230 0x7f1400075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.169+0000 7f14210ec640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1400073230 0x7f1400075740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1418001570 tx=0x7f1418009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.173+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1414017aa0 con 0x7f141c15f990 2024-09-21T02:14:10.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.393+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f141c0cb200 con 0x7f141c15f990 2024-09-21T02:14:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.397+0000 7f1411ffb640 1 -- 172.21.15.170:0/3088671415 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f14140f9360 con 0x7f141c15f990 2024-09-21T02:14:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1400073230 msgr2=0x7f1400075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1400073230 0x7f1400075740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1418001570 tx=0x7f1418009040 comp rx=0 tx=0).stop 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 msgr2=0x7f141c0d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 0x7f141c0d02e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f141400ac20 tx=0x7f14140a6c70 comp rx=0 tx=0).stop 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 shutdown_connections 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1400073230 0x7f1400075740 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f141c0ce4c0 0x7f141c155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c15f990 0x7f141c0d02e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 --2- 172.21.15.170:0/3088671415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f141c0cadf0 0x7f141c0af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 >> 172.21.15.170:0/3088671415 conn(0x7f141c0b1c40 msgr2=0x7f141c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 shutdown_connections 2024-09-21T02:14:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.401+0000 7f1423350640 1 -- 172.21.15.170:0/3088671415 wait complete. 2024-09-21T02:14:10.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- 172.21.15.170:0/709435092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8990004d50 msgr2=0x7f8990005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 --2- 172.21.15.170:0/709435092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8990004d50 0x7f8990005220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f89a8071110 tx=0x7f89a001caf0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a77fe640 1 -- 172.21.15.170:0/709435092 reap_dead start 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- 172.21.15.170:0/709435092 shutdown_connections 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- 172.21.15.170:0/709435092 >> 172.21.15.170:0/709435092 conn(0x7f89a80b1c40 msgr2=0x7f89a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- 172.21.15.170:0/709435092 shutdown_connections 2024-09-21T02:14:10.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- 172.21.15.170:0/709435092 wait complete. 2024-09-21T02:14:10.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 Processor -- start 2024-09-21T02:14:10.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- start start 2024-09-21T02:14:10.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 0x7f89a81f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89a806eeb0 0x7f89a806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89a814b0a0 0x7f89a814b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89a80cb9f0 con 0x7f89a8152ca0 2024-09-21T02:14:10.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f89a80cb870 con 0x7f89a806eeb0 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89ad938640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f89a80cbb70 con 0x7f89a814b0a0 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 0x7f89a81f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 0x7f89a81f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54156/0 (socket says 172.21.15.170:54156) 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3780464023 learned_addr learned my addr 172.21.15.170:0/3780464023 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a67fc640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89a806eeb0 0x7f89a806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a77fe640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89a814b0a0 0x7f89a814b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89a814b0a0 msgr2=0x7f89a814b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89a814b0a0 0x7f89a814b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89a806eeb0 msgr2=0x7f89a806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89a806eeb0 0x7f89a806f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.565+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3780464023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a806f800 con 0x7f89a8152ca0 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f89a6ffd640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 0x7f89a81f7020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f899800d850 tx=0x7f899800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998018070 con 0x7f89a8152ca0 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89980052e0 con 0x7f89a8152ca0 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a81f7970 con 0x7f89a8152ca0 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980036e0 con 0x7f89a8152ca0 2024-09-21T02:14:10.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a81f7f30 con 0x7f89a8152ca0 2024-09-21T02:14:10.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a80cf010 con 0x7f89a8152ca0 2024-09-21T02:14:10.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f899801b070 con 0x7f89a8152ca0 2024-09-21T02:14:10.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f898ffff640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8994073230 0x7f8994075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f8998093d30 con 0x7f89a8152ca0 2024-09-21T02:14:10.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.569+0000 7f89a67fc640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8994073230 0x7f8994075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.573+0000 7f89a67fc640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8994073230 0x7f8994075740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89a80cd130 tx=0x7f899c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.573+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f89a806f800 con 0x7f89a8152ca0 2024-09-21T02:14:10.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.789+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f89a80b3710 con 0x7f89a8152ca0 2024-09-21T02:14:10.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f898ffff640 1 -- 172.21.15.170:0/3780464023 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v15) ==== 90+0+0 (secure 0 0 0) 0x7f899805ebc0 con 0x7f89a8152ca0 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8994073230 msgr2=0x7f8994075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8994073230 0x7f8994075740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89a80cd130 tx=0x7f899c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 msgr2=0x7f89a81f7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 0x7f89a81f7020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f899800d850 tx=0x7f899800dd10 comp rx=0 tx=0).stop 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 shutdown_connections 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8994073230 0x7f8994075740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89a814b0a0 0x7f89a814b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89a806eeb0 0x7f89a806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 --2- 172.21.15.170:0/3780464023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a8152ca0 0x7f89a81f7020 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 >> 172.21.15.170:0/3780464023 conn(0x7f89a80b1c40 msgr2=0x7f89a809d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 shutdown_connections 2024-09-21T02:14:10.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.797+0000 7f89ad938640 1 -- 172.21.15.170:0/3780464023 wait complete. 2024-09-21T02:14:10.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T02:14:10.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 -- 172.21.15.170:0/1052690429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b5410 msgr2=0x7f804806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 --2- 172.21.15.170:0/1052690429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b5410 0x7f804806fa30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8038009840 tx=0x7f803801c820 comp rx=0 tx=0).stop 2024-09-21T02:14:10.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 -- 172.21.15.170:0/1052690429 shutdown_connections 2024-09-21T02:14:10.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 --2- 172.21.15.170:0/1052690429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b5410 0x7f804806fa30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 -- 172.21.15.170:0/1052690429 >> 172.21.15.170:0/1052690429 conn(0x7f80480b1c40 msgr2=0x7f80480b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:10.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 -- 172.21.15.170:0/1052690429 shutdown_connections 2024-09-21T02:14:10.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.969+0000 7f804d703640 1 -- 172.21.15.170:0/1052690429 wait complete. 2024-09-21T02:14:10.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 Processor -- start 2024-09-21T02:14:10.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 -- start start 2024-09-21T02:14:10.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 0x7f8048157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80480b5410 0x7f80480cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8048158670 0x7f80480ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f804816b570 con 0x7f80480b2f40 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8048070180 con 0x7f8048158670 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f804816b6f0 con 0x7f80480b5410 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 0x7f8048157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 0x7f8048157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54176/0 (socket says 172.21.15.170:54176) 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 -- 172.21.15.170:0/424553745 learned_addr learned my addr 172.21.15.170:0/424553745 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f803ffff640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80480b5410 0x7f80480cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f80477fe640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8048158670 0x7f80480ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 -- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80480b5410 msgr2=0x7f80480cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80480b5410 0x7f80480cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 -- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8048158670 msgr2=0x7f80480ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8048158670 0x7f80480ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 -- 172.21.15.170:0/424553745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8048158120 con 0x7f80480b2f40 2024-09-21T02:14:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f8046ffd640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 0x7f8048157a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8038009790 tx=0x7f80380a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80380be020 con 0x7f80480b2f40 2024-09-21T02:14:10.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8038004230 con 0x7f80480b2f40 2024-09-21T02:14:10.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804809d8c0 con 0x7f80480b2f40 2024-09-21T02:14:10.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038003600 con 0x7f80480b2f40 2024-09-21T02:14:10.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.973+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804809de80 con 0x7f80480b2f40 2024-09-21T02:14:10.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.977+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804806fa30 con 0x7f80480b2f40 2024-09-21T02:14:10.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.977+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f80380037a0 con 0x7f80480b2f40 2024-09-21T02:14:10.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.977+0000 7f803f7fe640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8028073200 0x7f8028075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:10.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.977+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f803812e6b0 con 0x7f80480b2f40 2024-09-21T02:14:10.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.977+0000 7f803ffff640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8028073200 0x7f8028075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.977+0000 7f803ffff640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8028073200 0x7f8028075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8048070cc0 tx=0x7f8030009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:10.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:10.981+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8038016410 con 0x7f80480b2f40 2024-09-21T02:14:11.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.209+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f8048168a00 con 0x7f80480b2f40 2024-09-21T02:14:11.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f803f7fe640 1 -- 172.21.15.170:0/424553745 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v16) ==== 90+0+0 (secure 0 0 0) 0x7f80380f9b20 con 0x7f80480b2f40 2024-09-21T02:14:11.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8028073200 msgr2=0x7f8028075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8028073200 0x7f8028075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8048070cc0 tx=0x7f8030009040 comp rx=0 tx=0).stop 2024-09-21T02:14:11.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 msgr2=0x7f8048157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 0x7f8048157a20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8038009790 tx=0x7f80380a6760 comp rx=0 tx=0).stop 2024-09-21T02:14:11.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 shutdown_connections 2024-09-21T02:14:11.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8028073200 0x7f8028075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80480b5410 0x7f80480cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8048158670 0x7f80480ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 --2- 172.21.15.170:0/424553745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80480b2f40 0x7f8048157a20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.213+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 >> 172.21.15.170:0/424553745 conn(0x7f80480b1c40 msgr2=0x7f80480cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.217+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 shutdown_connections 2024-09-21T02:14:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.217+0000 7f804d703640 1 -- 172.21.15.170:0/424553745 wait complete. 2024-09-21T02:14:11.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 -- 172.21.15.170:0/3989338154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300cd210 msgr2=0x7fca300b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 --2- 172.21.15.170:0/3989338154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300cd210 0x7fca300b3350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca300b3a10 tx=0x7fca2801cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 -- 172.21.15.170:0/3989338154 shutdown_connections 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 --2- 172.21.15.170:0/3989338154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300cd210 0x7fca300b3350 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 --2- 172.21.15.170:0/3989338154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300b5410 0x7fca3006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 -- 172.21.15.170:0/3989338154 >> 172.21.15.170:0/3989338154 conn(0x7fca300b1c40 msgr2=0x7fca300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 -- 172.21.15.170:0/3989338154 shutdown_connections 2024-09-21T02:14:11.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.385+0000 7fca385d4640 1 -- 172.21.15.170:0/3989338154 wait complete. 2024-09-21T02:14:11.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 Processor -- start 2024-09-21T02:14:11.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 -- start start 2024-09-21T02:14:11.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300b2f40 0x7fca301545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 0x7fca300c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca30157450 0x7fca300c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fca30169fd0 con 0x7fca300b5410 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca30169e50 con 0x7fca30157450 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fca3016a150 con 0x7fca300b2f40 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 0x7fca300c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 0x7fca300c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54196/0 (socket says 172.21.15.170:54196) 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 -- 172.21.15.170:0/3326673986 learned_addr learned my addr 172.21.15.170:0/3326673986 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca36370640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300b2f40 0x7fca301545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 -- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300b2f40 msgr2=0x7fca301545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300b2f40 0x7fca301545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 -- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca30157450 msgr2=0x7fca300c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca30157450 0x7fca300c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 -- 172.21.15.170:0/3326673986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3015b020 con 0x7fca300b5410 2024-09-21T02:14:11.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca35b6f640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 0x7fca300c9350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c3c0 tx=0x7fca2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:11.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c011790 con 0x7fca300b5410 2024-09-21T02:14:11.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca2c017070 con 0x7fca300b5410 2024-09-21T02:14:11.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3015b2e0 con 0x7fca300b5410 2024-09-21T02:14:11.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c011cc0 con 0x7fca300b5410 2024-09-21T02:14:11.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.389+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3015b8a0 con 0x7fca300b5410 2024-09-21T02:14:11.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.393+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3006fa30 con 0x7fca300b5410 2024-09-21T02:14:11.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.393+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fca2c005020 con 0x7fca300b5410 2024-09-21T02:14:11.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.393+0000 7fca26ffd640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca14073200 0x7fca14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.393+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fca2c0942f0 con 0x7fca300b5410 2024-09-21T02:14:11.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.393+0000 7fca36370640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca14073200 0x7fca14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.393+0000 7fca36370640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca14073200 0x7fca14075710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca200058d0 tx=0x7fca20005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:11.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.397+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fca2c05f740 con 0x7fca300b5410 2024-09-21T02:14:11.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.617+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fca300b3350 con 0x7fca300b5410 2024-09-21T02:14:11.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.621+0000 7fca26ffd640 1 -- 172.21.15.170:0/3326673986 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v17) ==== 97+0+0 (secure 0 0 0) 0x7fca2c05f040 con 0x7fca300b5410 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca14073200 msgr2=0x7fca14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca14073200 0x7fca14075710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca200058d0 tx=0x7fca20005860 comp rx=0 tx=0).stop 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 msgr2=0x7fca300c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 0x7fca300c9350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c3c0 tx=0x7fca2c00c880 comp rx=0 tx=0).stop 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 shutdown_connections 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca14073200 0x7fca14075710 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca30157450 0x7fca300c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca300b5410 0x7fca300c9350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 --2- 172.21.15.170:0/3326673986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300b2f40 0x7fca301545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 >> 172.21.15.170:0/3326673986 conn(0x7fca300b1c40 msgr2=0x7fca300b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:11.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 shutdown_connections 2024-09-21T02:14:11.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.625+0000 7fca385d4640 1 -- 172.21.15.170:0/3326673986 wait complete. 2024-09-21T02:14:11.645 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T02:14:11.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 -- 172.21.15.170:0/3948876316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d18004d50 msgr2=0x7f2d18005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 --2- 172.21.15.170:0/3948876316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d18004d50 0x7f2d18005160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c09b010 tx=0x7f2d2401d2a0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d3206b640 1 -- 172.21.15.170:0/3948876316 reap_dead start 2024-09-21T02:14:11.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 -- 172.21.15.170:0/3948876316 shutdown_connections 2024-09-21T02:14:11.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 -- 172.21.15.170:0/3948876316 >> 172.21.15.170:0/3948876316 conn(0x7f2d2c09a9c0 msgr2=0x7f2d2c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:11.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 -- 172.21.15.170:0/3948876316 shutdown_connections 2024-09-21T02:14:11.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 -- 172.21.15.170:0/3948876316 wait complete. 2024-09-21T02:14:11.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 Processor -- start 2024-09-21T02:14:11.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.801+0000 7f2d33ace640 1 -- start start 2024-09-21T02:14:11.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0cad70 0x7f2d2c1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 0x7f2d2c0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c0c8b70 0x7f2d2c1f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0b18b0 con 0x7f2d2c0c8220 2024-09-21T02:14:11.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0b1730 con 0x7f2d2c0cad70 2024-09-21T02:14:11.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0b1a30 con 0x7f2d2c0c8b70 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 0x7f2d2c0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 0x7f2d2c0c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54214/0 (socket says 172.21.15.170:54214) 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 -- 172.21.15.170:0/2227143599 learned_addr learned my addr 172.21.15.170:0/2227143599 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d3186a640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0cad70 0x7f2d2c1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d3206b640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c0c8b70 0x7f2d2c1f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 -- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c0c8b70 msgr2=0x7f2d2c1f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c0c8b70 0x7f2d2c1f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 -- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0cad70 msgr2=0x7f2d2c1526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0cad70 0x7f2d2c1526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:11.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 -- 172.21.15.170:0/2227143599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1601b0 con 0x7f2d2c0c8220 2024-09-21T02:14:11.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d31069640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 0x7f2d2c0c8630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00e9a0 tx=0x7f2d1c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:11.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00ccc0 con 0x7f2d2c0c8220 2024-09-21T02:14:11.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d1c00ce60 con 0x7f2d2c0c8220 2024-09-21T02:14:11.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c0cb630 con 0x7f2d2c0c8220 2024-09-21T02:14:11.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c005280 con 0x7f2d2c0c8220 2024-09-21T02:14:11.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c0c9640 con 0x7f2d2c0c8220 2024-09-21T02:14:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c06ce00 con 0x7f2d2c0c8220 2024-09-21T02:14:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2d1c005420 con 0x7f2d2c0c8220 2024-09-21T02:14:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d227fc640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2d10073200 0x7f2d10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.805+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f2d1c094870 con 0x7f2d2c0c8220 2024-09-21T02:14:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.809+0000 7f2d3186a640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2d10073200 0x7f2d10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.809+0000 7f2d3186a640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2d10073200 0x7f2d10075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c06d1e0 tx=0x7f2d28005f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:11.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:11.813+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2d1c05fd00 con 0x7f2d2c0c8220 2024-09-21T02:14:12.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.037+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2d2c0af030 con 0x7f2d2c0c8220 2024-09-21T02:14:12.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.041+0000 7f2d227fc640 1 -- 172.21.15.170:0/2227143599 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v18) ==== 90+0+0 (secure 0 0 0) 0x7f2d1c05f600 con 0x7f2d2c0c8220 2024-09-21T02:14:12.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2d10073200 msgr2=0x7f2d10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2d10073200 0x7f2d10075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c06d1e0 tx=0x7f2d28005f40 comp rx=0 tx=0).stop 2024-09-21T02:14:12.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 msgr2=0x7f2d2c0c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 0x7f2d2c0c8630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00e9a0 tx=0x7f2d1c00ee60 comp rx=0 tx=0).stop 2024-09-21T02:14:12.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 shutdown_connections 2024-09-21T02:14:12.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2d10073200 0x7f2d10075710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c0c8b70 0x7f2d2c1f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c0c8220 0x7f2d2c0c8630 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 --2- 172.21.15.170:0/2227143599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0cad70 0x7f2d2c1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 >> 172.21.15.170:0/2227143599 conn(0x7f2d2c09a9c0 msgr2=0x7f2d2c0adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 shutdown_connections 2024-09-21T02:14:12.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.045+0000 7f2d33ace640 1 -- 172.21.15.170:0/2227143599 wait complete. 2024-09-21T02:14:12.066 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:14:12.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 -- 172.21.15.170:0/3253938223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59015f990 msgr2=0x7fe590070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 --2- 172.21.15.170:0/3253938223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59015f990 0x7fe590070030 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009840 tx=0x7fe58c01c820 comp rx=0 tx=0).stop 2024-09-21T02:14:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 -- 172.21.15.170:0/3253938223 shutdown_connections 2024-09-21T02:14:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 --2- 172.21.15.170:0/3253938223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59015f990 0x7fe590070030 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 -- 172.21.15.170:0/3253938223 >> 172.21.15.170:0/3253938223 conn(0x7fe5900b1c40 msgr2=0x7fe5900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 -- 172.21.15.170:0/3253938223 shutdown_connections 2024-09-21T02:14:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.205+0000 7fe597b4e640 1 -- 172.21.15.170:0/3253938223 wait complete. 2024-09-21T02:14:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 Processor -- start 2024-09-21T02:14:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- start start 2024-09-21T02:14:12.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe590070570 0x7fe5901fb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 0x7fe590155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59015f990 0x7fe59009bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe590169ba0 con 0x7fe5900cadf0 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe590169a20 con 0x7fe59015f990 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe590169d20 con 0x7fe590070570 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 0x7fe590155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 0x7fe590155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54218/0 (socket says 172.21.15.170:54218) 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 -- 172.21.15.170:0/222074783 learned_addr learned my addr 172.21.15.170:0/222074783 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5958ea640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe590070570 0x7fe5901fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5960eb640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59015f990 0x7fe59009bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 -- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe590070570 msgr2=0x7fe5901fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe590070570 0x7fe5901fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 -- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59015f990 msgr2=0x7fe59009bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59015f990 0x7fe59009bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 -- 172.21.15.170:0/222074783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900ca3a0 con 0x7fe5900cadf0 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5950e9640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 0x7fe590155a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe58000b410 tx=0x7fe58000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580015030 con 0x7fe5900cadf0 2024-09-21T02:14:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe580004020 con 0x7fe5900cadf0 2024-09-21T02:14:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901fa890 con 0x7fe5900cadf0 2024-09-21T02:14:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5800044b0 con 0x7fe5900cadf0 2024-09-21T02:14:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901fab30 con 0x7fe5900cadf0 2024-09-21T02:14:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590070030 con 0x7fe5900cadf0 2024-09-21T02:14:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.209+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe580018070 con 0x7fe5900cadf0 2024-09-21T02:14:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.213+0000 7fe5867fc640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe574073230 0x7fe574075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.213+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fe5800939c0 con 0x7fe5900cadf0 2024-09-21T02:14:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.213+0000 7fe5958ea640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe574073230 0x7fe574075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.213+0000 7fe5958ea640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe574073230 0x7fe574075740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0af000 tx=0x7fe58c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.217+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe580095e20 con 0x7fe5900cadf0 2024-09-21T02:14:12.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe5900ce2a0 con 0x7fe5900cadf0 2024-09-21T02:14:12.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe5867fc640 1 -- 172.21.15.170:0/222074783 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7fe58005ee50 con 0x7fe5900cadf0 2024-09-21T02:14:12.382 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:12.382 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:14:12.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:12.382 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:14:12.382 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:14:12.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:12.383 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:14:12.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe574073230 msgr2=0x7fe574075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe574073230 0x7fe574075740 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0af000 tx=0x7fe58c01b060 comp rx=0 tx=0).stop 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 msgr2=0x7fe590155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 0x7fe590155a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe58000b410 tx=0x7fe58000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 shutdown_connections 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe574073230 0x7fe574075740 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59015f990 0x7fe59009bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5900cadf0 0x7fe590155a00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 --2- 172.21.15.170:0/222074783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe590070570 0x7fe5901fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 >> 172.21.15.170:0/222074783 conn(0x7fe5900b1c40 msgr2=0x7fe5900b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 shutdown_connections 2024-09-21T02:14:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.381+0000 7fe597b4e640 1 -- 172.21.15.170:0/222074783 wait complete. 2024-09-21T02:14:12.403 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:14:12.403 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 -- 172.21.15.170:0/957710899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0b4d80 msgr2=0x7feefc0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 --2- 172.21.15.170:0/957710899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0b4d80 0x7feefc0ccae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feee4009bc0 tx=0x7feee401cb10 comp rx=0 tx=0).stop 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 -- 172.21.15.170:0/957710899 shutdown_connections 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 --2- 172.21.15.170:0/957710899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0b4d80 0x7feefc0ccae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 --2- 172.21.15.170:0/957710899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feefc0b52d0 0x7feefc0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 -- 172.21.15.170:0/957710899 >> 172.21.15.170:0/957710899 conn(0x7feefc0b1c40 msgr2=0x7feefc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 -- 172.21.15.170:0/957710899 shutdown_connections 2024-09-21T02:14:12.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 -- 172.21.15.170:0/957710899 wait complete. 2024-09-21T02:14:12.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 Processor -- start 2024-09-21T02:14:12.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.553+0000 7fef01287640 1 -- start start 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feefc0b4d80 0x7feefc156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feefc0b52d0 0x7feefc1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 0x7feefc0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feefc169dd0 con 0x7feefc0cd020 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feefc169c50 con 0x7feefc0b4d80 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7feefc169f50 con 0x7feefc0b52d0 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 0x7feefc0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 0x7feefc0b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54236/0 (socket says 172.21.15.170:54236) 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 -- 172.21.15.170:0/3549757414 learned_addr learned my addr 172.21.15.170:0/3549757414 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefaffd640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feefc0b4d80 0x7feefc156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefa7fc640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feefc0b52d0 0x7feefc1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 -- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feefc0b52d0 msgr2=0x7feefc1519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feefc0b52d0 0x7feefc1519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 -- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feefc0b4d80 msgr2=0x7feefc156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feefc0b4d80 0x7feefc156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 -- 172.21.15.170:0/3549757414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc1556e0 con 0x7feefc0cd020 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefb7fe640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 0x7feefc0b0ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c340 tx=0x7feeec00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec011790 con 0x7feefc0cd020 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeec017070 con 0x7feefc0cd020 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc155980 con 0x7feefc0cd020 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec011d20 con 0x7feefc0cd020 2024-09-21T02:14:12.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc155f40 con 0x7feefc0cd020 2024-09-21T02:14:12.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc0ccae0 con 0x7feefc0cd020 2024-09-21T02:14:12.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feeec01c460 con 0x7feefc0cd020 2024-09-21T02:14:12.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feef37fe640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feedc073200 0x7feedc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:12.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7feeec093b10 con 0x7feefc0cd020 2024-09-21T02:14:12.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.557+0000 7feefaffd640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feedc073200 0x7feedc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:12.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.561+0000 7feefaffd640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feedc073200 0x7feedc075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feef400bb00 tx=0x7feef4005dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:12.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.565+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feeec05efa0 con 0x7feefc0cd020 2024-09-21T02:14:12.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:12.725+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feefc0c9f90 con 0x7feefc0cd020 2024-09-21T02:14:13.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.129+0000 7feef37fe640 1 -- 172.21.15.170:0/3549757414 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7feeec05e8a0 con 0x7feefc0cd020 2024-09-21T02:14:13.133 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:14:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feedc073200 msgr2=0x7feedc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feedc073200 0x7feedc075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feef400bb00 tx=0x7feef4005dd0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 msgr2=0x7feefc0b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:13.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 0x7feefc0b0ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c340 tx=0x7feeec00c800 comp rx=0 tx=0).stop 2024-09-21T02:14:13.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 shutdown_connections 2024-09-21T02:14:13.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feedc073200 0x7feedc075710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.133+0000 7fef01287640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feefc0b52d0 0x7feefc1519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.137+0000 7fef01287640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feefc0cd020 0x7feefc0b0ba0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.137+0000 7fef01287640 1 --2- 172.21.15.170:0/3549757414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feefc0b4d80 0x7feefc156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.137+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 >> 172.21.15.170:0/3549757414 conn(0x7feefc0b1c40 msgr2=0x7feefc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:13.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.137+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 shutdown_connections 2024-09-21T02:14:13.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.137+0000 7fef01287640 1 -- 172.21.15.170:0/3549757414 wait complete. 2024-09-21T02:14:13.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:14:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 -- 172.21.15.170:0/1194074378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 msgr2=0x7f7a00004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:13.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/1194074378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a00004c20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00c260 tx=0x7f79fc01cf00 comp rx=0 tx=0).stop 2024-09-21T02:14:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 -- 172.21.15.170:0/1194074378 shutdown_connections 2024-09-21T02:14:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/1194074378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a00004c20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 -- 172.21.15.170:0/1194074378 >> 172.21.15.170:0/1194074378 conn(0x7f7a1409a9c0 msgr2=0x7f7a140d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 -- 172.21.15.170:0/1194074378 shutdown_connections 2024-09-21T02:14:13.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.329+0000 7f7a1a5af640 1 -- 172.21.15.170:0/1194074378 wait complete. 2024-09-21T02:14:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 Processor -- start 2024-09-21T02:14:13.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- start start 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a14153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1415de10 0x7f7a1415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a140c9a40 0x7f7a140c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1415f800 con 0x7f7a00004810 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1415f680 con 0x7f7a140c9a40 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1415f980 con 0x7f7a1415de10 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a14153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a14153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54254/0 (socket says 172.21.15.170:54254) 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 -- 172.21.15.170:0/3869476387 learned_addr learned my addr 172.21.15.170:0/3869476387 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a0bfff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1415de10 0x7f7a1415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a18b4c640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a140c9a40 0x7f7a140c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 -- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1415de10 msgr2=0x7f7a1415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1415de10 0x7f7a1415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 -- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a140c9a40 msgr2=0x7f7a140c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a140c9a40 0x7f7a140c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 -- 172.21.15.170:0/3869476387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a140ca390 con 0x7f7a00004810 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a0bfff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1415de10 0x7f7a1415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a13fff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a14153850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c510 tx=0x7f7a0400c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040052d0 con 0x7f7a00004810 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a04005490 con 0x7f7a00004810 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a140c96d0 con 0x7f7a00004810 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040038b0 con 0x7f7a00004810 2024-09-21T02:14:13.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a140c8910 con 0x7f7a00004810 2024-09-21T02:14:13.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.333+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1415fee0 con 0x7f7a00004810 2024-09-21T02:14:13.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.337+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a04003a50 con 0x7f7a00004810 2024-09-21T02:14:13.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.337+0000 7f7a117fa640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f79f4073200 0x7f79f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:13.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.337+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f7a04094760 con 0x7f7a00004810 2024-09-21T02:14:13.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.337+0000 7f7a0bfff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f79f4073200 0x7f79f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:13.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.337+0000 7f7a0bfff640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f79f4073200 0x7f79f4075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0ab000 tx=0x7f79fc019d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:13.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.341+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7a140ca390 con 0x7f7a00004810 2024-09-21T02:14:13.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:13.513+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7a140cb6d0 con 0x7f7a00004810 2024-09-21T02:14:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.145+0000 7f7a117fa640 1 -- 172.21.15.170:0/3869476387 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f7a0405fa40 con 0x7f7a00004810 2024-09-21T02:14:14.146 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:14:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.145+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f79f4073200 msgr2=0x7f79f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.145+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f79f4073200 0x7f79f4075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0ab000 tx=0x7f79fc019d40 comp rx=0 tx=0).stop 2024-09-21T02:14:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.145+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 msgr2=0x7f7a14153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.145+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a14153850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c510 tx=0x7f7a0400c9d0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 shutdown_connections 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f79f4073200 0x7f79f4075710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1415de10 0x7f7a1415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a140c9a40 0x7f7a140c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 --2- 172.21.15.170:0/3869476387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a00004810 0x7f7a14153850 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 >> 172.21.15.170:0/3869476387 conn(0x7f7a1409a9c0 msgr2=0x7f7a14070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 shutdown_connections 2024-09-21T02:14:14.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.149+0000 7f7a1a5af640 1 -- 172.21.15.170:0/3869476387 wait complete. 2024-09-21T02:14:14.196 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:14:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 -- 172.21.15.170:0/930277571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d080cb790 msgr2=0x7f1d080ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/930277571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d080cb790 0x7f1d080ce5b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1d080ca750 tx=0x7f1cf001db40 comp rx=0 tx=0).stop 2024-09-21T02:14:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 -- 172.21.15.170:0/930277571 shutdown_connections 2024-09-21T02:14:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/930277571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d080cb790 0x7f1d080ce5b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/930277571 >> v2:172.21.15.191:3300/0 conn(0x7f1d080cb380 0x7f1d080ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 -- 172.21.15.170:0/930277571 >> 172.21.15.170:0/930277571 conn(0x7f1d080d0170 msgr2=0x7f1d080d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 -- 172.21.15.170:0/930277571 shutdown_connections 2024-09-21T02:14:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 -- 172.21.15.170:0/930277571 wait complete. 2024-09-21T02:14:14.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 Processor -- start 2024-09-21T02:14:14.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 -- start start 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 0x7f1d080c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.352+0000 7f1d0cdef640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d080cb380 0x7f1d08146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d080cb790 0x7f1d081f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1d08074650 con 0x7f1d0806f9a0 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1d080744d0 con 0x7f1d080cb790 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1d080747d0 con 0x7f1d080cb380 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 0x7f1d080c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 0x7f1d080c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54266/0 (socket says 172.21.15.170:54266) 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 -- 172.21.15.170:0/3425172275 learned_addr learned my addr 172.21.15.170:0/3425172275 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfffff640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d080cb380 0x7f1d08146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d06d9d640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d080cb790 0x7f1d081f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 -- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d080cb380 msgr2=0x7f1d08146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d080cb380 0x7f1d08146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 -- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d080cb790 msgr2=0x7f1d081f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d080cb790 0x7f1d081f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 -- 172.21.15.170:0/3425172275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d080c8ec0 con 0x7f1d0806f9a0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfffff640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d080cb380 0x7f1d08146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d06d9d640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d080cb790 0x7f1d081f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0659c640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 0x7f1d080c4cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c940 tx=0x7f1cf800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8004220 con 0x7f1d0806f9a0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cf80079b0 con 0x7f1d0806f9a0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80044b0 con 0x7f1d0806f9a0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d081f7c60 con 0x7f1d0806f9a0 2024-09-21T02:14:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d080c9130 con 0x7f1d0806f9a0 2024-09-21T02:14:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0806fdb0 con 0x7f1d0806f9a0 2024-09-21T02:14:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1cf8003600 con 0x7f1d0806f9a0 2024-09-21T02:14:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfeffd640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ce8073200 0x7f1ce8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7f1cf8094310 con 0x7f1d0806f9a0 2024-09-21T02:14:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.356+0000 7f1cfffff640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ce8073200 0x7f1ce8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:14.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.360+0000 7f1cfffff640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ce8073200 0x7f1ce8075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00b0000 tx=0x7f1cf001c380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:14.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.364+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1d081f7c60 con 0x7f1d0806f9a0 2024-09-21T02:14:14.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:14.572+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1d080ad020 con 0x7f1d0806f9a0 2024-09-21T02:14:15.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.168+0000 7f1cfeffd640 1 -- 172.21.15.170:0/3425172275 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f1cf8014030 con 0x7f1d0806f9a0 2024-09-21T02:14:15.177 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 4 and data pool 5 2024-09-21T02:14:15.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ce8073200 msgr2=0x7f1ce8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ce8073200 0x7f1ce8075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00b0000 tx=0x7f1cf001c380 comp rx=0 tx=0).stop 2024-09-21T02:14:15.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 msgr2=0x7f1d080c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 0x7f1d080c4cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c940 tx=0x7f1cf800ce00 comp rx=0 tx=0).stop 2024-09-21T02:14:15.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 shutdown_connections 2024-09-21T02:14:15.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ce8073200 0x7f1ce8075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d080cb380 0x7f1d08146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d080cb790 0x7f1d081f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 --2- 172.21.15.170:0/3425172275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0806f9a0 0x7f1d080c4cc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.184+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 >> 172.21.15.170:0/3425172275 conn(0x7f1d080d0170 msgr2=0x7f1d080ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:15.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.188+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 shutdown_connections 2024-09-21T02:14:15.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.188+0000 7f1d0cdef640 1 -- 172.21.15.170:0/3425172275 wait complete. 2024-09-21T02:14:15.222 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:15.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 -- 172.21.15.170:0/2847149770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 msgr2=0x7fa79006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 --2- 172.21.15.170:0/2847149770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa79006fa30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa7900c7150 tx=0x7fa78401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 -- 172.21.15.170:0/2847149770 shutdown_connections 2024-09-21T02:14:15.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 --2- 172.21.15.170:0/2847149770 >> v2:172.21.15.191:3300/0 conn(0x7fa7900cd210 0x7fa7900b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 --2- 172.21.15.170:0/2847149770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa79006fa30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 -- 172.21.15.170:0/2847149770 >> 172.21.15.170:0/2847149770 conn(0x7fa7900b1c40 msgr2=0x7fa7900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:15.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.368+0000 7fa796d0d640 1 -- 172.21.15.170:0/2847149770 shutdown_connections 2024-09-21T02:14:15.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 -- 172.21.15.170:0/2847149770 wait complete. 2024-09-21T02:14:15.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 Processor -- start 2024-09-21T02:14:15.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 -- start start 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7900b2f40 0x7fa7901fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa7900c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa79009da00 0x7fa7900c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa790169fd0 con 0x7fa7900b5410 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa790169e50 con 0x7fa79009da00 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.372+0000 7fa796d0d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa79016a150 con 0x7fa7900b2f40 2024-09-21T02:14:15.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa7900c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa7900c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54280/0 (socket says 172.21.15.170:54280) 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 -- 172.21.15.170:0/4218269744 learned_addr learned my addr 172.21.15.170:0/4218269744 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa794aa9640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7900b2f40 0x7fa7901fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa7952aa640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa79009da00 0x7fa7900c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 -- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7900b2f40 msgr2=0x7fa7901fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7900b2f40 0x7fa7901fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 -- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa79009da00 msgr2=0x7fa7900c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa79009da00 0x7fa7900c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 -- 172.21.15.170:0/4218269744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7901faa40 con 0x7fa7900b5410 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa7952aa640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa79009da00 0x7fa7900c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78ffff640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa7900c7de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa78000b410 tx=0x7fa78000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780015020 con 0x7fa7900b5410 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa780011920 con 0x7fa7900b5410 2024-09-21T02:14:15.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa790155800 con 0x7fa7900b5410 2024-09-21T02:14:15.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780011d70 con 0x7fa7900b5410 2024-09-21T02:14:15.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa790155dc0 con 0x7fa7900b5410 2024-09-21T02:14:15.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79006fa30 con 0x7fa7900b5410 2024-09-21T02:14:15.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa780018020 con 0x7fa7900b5410 2024-09-21T02:14:15.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78d7fa640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa77c073200 0x7fa77c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa7800942c0 con 0x7fa7900b5410 2024-09-21T02:14:15.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.376+0000 7fa794aa9640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa77c073200 0x7fa77c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.380+0000 7fa794aa9640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa77c073200 0x7fa77c075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7840af000 tx=0x7fa78401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:15.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.384+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa78001f020 con 0x7fa7900b5410 2024-09-21T02:14:15.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.556+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7900c4950 con 0x7fa7900b5410 2024-09-21T02:14:15.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa78d7fa640 1 -- 172.21.15.170:0/4218269744 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16730 (secure 0 0 0) 0x7fa78005f260 con 0x7fa7900b5410 2024-09-21T02:14:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:15.562 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":34,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:14.576815+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:12.737491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:13.529023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:15.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa77c073200 msgr2=0x7fa77c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa77c073200 0x7fa77c075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7840af000 tx=0x7fa78401b100 comp rx=0 tx=0).stop 2024-09-21T02:14:15.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 msgr2=0x7fa7900c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa7900c7de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa78000b410 tx=0x7fa78000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 shutdown_connections 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa77c073200 0x7fa77c075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa79009da00 0x7fa7900c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7900b5410 0x7fa7900c7de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 --2- 172.21.15.170:0/4218269744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7900b2f40 0x7fa7901fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 >> 172.21.15.170:0/4218269744 conn(0x7fa7900b1c40 msgr2=0x7fa7900cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 shutdown_connections 2024-09-21T02:14:15.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.560+0000 7fa796d0d640 1 -- 172.21.15.170:0/4218269744 wait complete. 2024-09-21T02:14:15.584 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3635955255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 msgr2=0x7fe6b40ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3635955255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b40ce8d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40cdc00 tx=0x7fe6a401c6e0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3635955255 shutdown_connections 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3635955255 >> v2:172.21.15.191:3300/0 conn(0x7fe6b40cadf0 0x7fe6b40cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3635955255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b40ce8d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3635955255 >> 172.21.15.170:0/3635955255 conn(0x7fe6b40b1c40 msgr2=0x7fe6b40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3635955255 shutdown_connections 2024-09-21T02:14:15.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.740+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3635955255 wait complete. 2024-09-21T02:14:15.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 Processor -- start 2024-09-21T02:14:15.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- start start 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b40cadf0 0x7fe6b41505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b409b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b415f990 0x7fe6b40acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4169100 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4074f00 con 0x7fe6b415f990 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b4169280 con 0x7fe6b40cadf0 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b409b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b409b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47840/0 (socket says 172.21.15.170:47840) 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 -- 172.21.15.170:0/3498982415 learned_addr learned my addr 172.21.15.170:0/3498982415 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b2d9d640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b415f990 0x7fe6b40acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b259c640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b40cadf0 0x7fe6b41505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 -- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b40cadf0 msgr2=0x7fe6b41505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b40cadf0 0x7fe6b41505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 -- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b415f990 msgr2=0x7fe6b40acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b415f990 0x7fe6b40acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 -- 172.21.15.170:0/3498982415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b40adc10 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b1d9b640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b409b5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe6a401ad90 tx=0x7fe6a40a8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004380 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6a4003600 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4003a50 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b409aaf0 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b4152590 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.744+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b40ad980 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.748+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6a4003bf0 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.748+0000 7fe6a2ffd640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe694073230 0x7fe694075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:15.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.748+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe6a412ed20 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.748+0000 7fe6b259c640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe694073230 0x7fe694075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:15.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.748+0000 7fe6b259c640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe694073230 0x7fe694075740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800a750 tx=0x7fe6a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:15.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.752+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe6a40bf060 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.924+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6b40cb200 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6a2ffd640 1 -- 172.21.15.170:0/3498982415 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16730 (secure 0 0 0) 0x7fe6a40ac030 con 0x7fe6b40ce4c0 2024-09-21T02:14:15.930 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:15.931 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":34,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:14.576815+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:12.737491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:13.529023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:15.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe694073230 msgr2=0x7fe694075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe694073230 0x7fe694075740 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800a750 tx=0x7fe6a8008040 comp rx=0 tx=0).stop 2024-09-21T02:14:15.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 msgr2=0x7fe6b409b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:15.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b409b5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe6a401ad90 tx=0x7fe6a40a8620 comp rx=0 tx=0).stop 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 shutdown_connections 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe694073230 0x7fe694075740 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6b415f990 0x7fe6b40acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b40ce4c0 0x7fe6b409b5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 --2- 172.21.15.170:0/3498982415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6b40cadf0 0x7fe6b41505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 >> 172.21.15.170:0/3498982415 conn(0x7fe6b40b1c40 msgr2=0x7fe6b40b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 shutdown_connections 2024-09-21T02:14:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:15.928+0000 7fe6b8e37640 1 -- 172.21.15.170:0/3498982415 wait complete. 2024-09-21T02:14:15.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:14:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 -- 172.21.15.170:0/1203657474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d1bd0 msgr2=0x7fa4880d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 --2- 172.21.15.170:0/1203657474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d1bd0 0x7fa4880d4510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009840 tx=0x7fa47c01c820 comp rx=0 tx=0).stop 2024-09-21T02:14:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 -- 172.21.15.170:0/1203657474 shutdown_connections 2024-09-21T02:14:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 --2- 172.21.15.170:0/1203657474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d1bd0 0x7fa4880d4510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 --2- 172.21.15.170:0/1203657474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4880d17c0 0x7fa4880b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 -- 172.21.15.170:0/1203657474 >> 172.21.15.170:0/1203657474 conn(0x7fa48808bf50 msgr2=0x7fa48809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 -- 172.21.15.170:0/1203657474 shutdown_connections 2024-09-21T02:14:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.112+0000 7fa48e147640 1 -- 172.21.15.170:0/1203657474 wait complete. 2024-09-21T02:14:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 Processor -- start 2024-09-21T02:14:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 -- start start 2024-09-21T02:14:16.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d17c0 0x7fa4881f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 0x7fa488150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4880d4a50 0x7fa48814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa48816bf40 con 0x7fa4880d1bd0 2024-09-21T02:14:16.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa48816bdc0 con 0x7fa4880d17c0 2024-09-21T02:14:16.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa48816c0c0 con 0x7fa4880d4a50 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 0x7fa488150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 0x7fa488150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47846/0 (socket says 172.21.15.170:47846) 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 -- 172.21.15.170:0/2025118282 learned_addr learned my addr 172.21.15.170:0/2025118282 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa4877fe640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d17c0 0x7fa4881f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa487fff640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4880d4a50 0x7fa48814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 -- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4880d4a50 msgr2=0x7fa48814e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4880d4a50 0x7fa48814e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 -- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d17c0 msgr2=0x7fa4881f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d17c0 0x7fa4881f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 -- 172.21.15.170:0/2025118282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa488151930 con 0x7fa4880d1bd0 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa4877fe640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d17c0 0x7fa4881f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa486ffd640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 0x7fa488150dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009790 tx=0x7fa47c005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c0b2040 con 0x7fa4880d1bd0 2024-09-21T02:14:16.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa488151c10 con 0x7fa4880d1bd0 2024-09-21T02:14:16.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47c004330 con 0x7fa4880d1bd0 2024-09-21T02:14:16.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.116+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003820 con 0x7fa4880d1bd0 2024-09-21T02:14:16.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48806e9c0 con 0x7fa4880d1bd0 2024-09-21T02:14:16.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4880b3a10 con 0x7fa4880d1bd0 2024-09-21T02:14:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa47c0bd060 con 0x7fa4880d1bd0 2024-09-21T02:14:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa46ffff640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa474073230 0x7fa474075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa47c12d820 con 0x7fa4880d1bd0 2024-09-21T02:14:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa4877fe640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa474073230 0x7fa474075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.120+0000 7fa4877fe640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa474073230 0x7fa474075740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa478009c10 tx=0x7fa4780091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.124+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa47c016410 con 0x7fa4880d1bd0 2024-09-21T02:14:16.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:16.336+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa4881516c0 con 0x7fa4880d1bd0 2024-09-21T02:14:17.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.188+0000 7fa46ffff640 1 -- 172.21.15.170:0/2025118282 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa47c0f88d0 con 0x7fa4880d1bd0 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa474073230 msgr2=0x7fa474075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa474073230 0x7fa474075740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa478009c10 tx=0x7fa4780091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 msgr2=0x7fa488150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 0x7fa488150dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009790 tx=0x7fa47c005ca0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 shutdown_connections 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa474073230 0x7fa474075740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4880d4a50 0x7fa48814e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4880d1bd0 0x7fa488150dc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 --2- 172.21.15.170:0/2025118282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4880d17c0 0x7fa4881f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 >> 172.21.15.170:0/2025118282 conn(0x7fa48808bf50 msgr2=0x7fa4880d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:17.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 shutdown_connections 2024-09-21T02:14:17.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.192+0000 7fa48e147640 1 -- 172.21.15.170:0/2025118282 wait complete. 2024-09-21T02:14:17.217 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:14:17.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 -- 172.21.15.170:0/2143164688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa194003a40 msgr2=0x7fa194003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2143164688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa194003a40 0x7fa194003eb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00016f0 tx=0x7fa1a001c7b0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1a6d9d640 1 -- 172.21.15.170:0/2143164688 reap_dead start 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 -- 172.21.15.170:0/2143164688 shutdown_connections 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 -- 172.21.15.170:0/2143164688 >> 172.21.15.170:0/2143164688 conn(0x7fa1a80d2a30 msgr2=0x7fa1a80cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 -- 172.21.15.170:0/2143164688 shutdown_connections 2024-09-21T02:14:17.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 -- 172.21.15.170:0/2143164688 wait complete. 2024-09-21T02:14:17.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 Processor -- start 2024-09-21T02:14:17.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.380+0000 7fa1aca67640 1 -- start start 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a814b230 0x7fa1a814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 0x7fa1a8153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1a8158b80 0x7fa1a8158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8168e30 con 0x7fa1a81573e0 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8168cb0 con 0x7fa1a814b230 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a8168fb0 con 0x7fa1a8158b80 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 0x7fa1a8153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 0x7fa1a8153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47866/0 (socket says 172.21.15.170:47866) 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 -- 172.21.15.170:0/2848512288 learned_addr learned my addr 172.21.15.170:0/2848512288 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a6d9d640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1a8158b80 0x7fa1a8158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a659c640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a814b230 0x7fa1a814ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 -- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1a8158b80 msgr2=0x7fa1a8158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1a8158b80 0x7fa1a8158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 -- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a814b230 msgr2=0x7fa1a814ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a814b230 0x7fa1a814ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 -- 172.21.15.170:0/2848512288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a814bad0 con 0x7fa1a81573e0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a659c640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a814b230 0x7fa1a814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1a5d9b640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 0x7fa1a8153500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa190000c10 tx=0x7fa1900049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190015020 con 0x7fa1a81573e0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa190010980 con 0x7fa1a81573e0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a8157ae0 con 0x7fa1a81573e0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190010dd0 con 0x7fa1a81573e0 2024-09-21T02:14:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a8158380 con 0x7fa1a81573e0 2024-09-21T02:14:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8157ed0 con 0x7fa1a81573e0 2024-09-21T02:14:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa190018020 con 0x7fa1a81573e0 2024-09-21T02:14:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa19effd640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa188073200 0x7fa188075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.384+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa190094d00 con 0x7fa1a81573e0 2024-09-21T02:14:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.388+0000 7fa1a659c640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa188073200 0x7fa188075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.388+0000 7fa1a659c640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa188073200 0x7fa188075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00016c0 tx=0x7fa1a00195a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:17.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.392+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa19001f020 con 0x7fa1a81573e0 2024-09-21T02:14:17.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:17.604+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa1a80d0700 con 0x7fa1a81573e0 2024-09-21T02:14:18.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.188+0000 7fa19effd640 1 -- 172.21.15.170:0/2848512288 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fa1a814bad0 con 0x7fa1a81573e0 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa188073200 msgr2=0x7fa188075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa188073200 0x7fa188075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00016c0 tx=0x7fa1a00195a0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 msgr2=0x7fa1a8153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 0x7fa1a8153500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa190000c10 tx=0x7fa1900049e0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 shutdown_connections 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa188073200 0x7fa188075710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1a8158b80 0x7fa1a8158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1a81573e0 0x7fa1a8153500 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 --2- 172.21.15.170:0/2848512288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a814b230 0x7fa1a814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.192+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 >> 172.21.15.170:0/2848512288 conn(0x7fa1a80d2a30 msgr2=0x7fa1a815b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.196+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 shutdown_connections 2024-09-21T02:14:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.196+0000 7fa1aca67640 1 -- 172.21.15.170:0/2848512288 wait complete. 2024-09-21T02:14:18.225 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:14:18.225 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 -- 172.21.15.170:0/4182746584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f171815f990 msgr2=0x7f1718070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 --2- 172.21.15.170:0/4182746584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f171815f990 0x7f1718070030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1700009840 tx=0x7f170001c7f0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 -- 172.21.15.170:0/4182746584 shutdown_connections 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 --2- 172.21.15.170:0/4182746584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17180cadf0 0x7f17180cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 --2- 172.21.15.170:0/4182746584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f171815f990 0x7f1718070030 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 -- 172.21.15.170:0/4182746584 >> 172.21.15.170:0/4182746584 conn(0x7f17180b1c40 msgr2=0x7f17180b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 -- 172.21.15.170:0/4182746584 shutdown_connections 2024-09-21T02:14:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.380+0000 7f171e0cc640 1 -- 172.21.15.170:0/4182746584 wait complete. 2024-09-21T02:14:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 Processor -- start 2024-09-21T02:14:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 -- start start 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1718070570 0x7f1718153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 0x7f17181641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171815f990 0x7f17181646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f17180ccb70 con 0x7f17180cadf0 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f17180cccf0 con 0x7f1718070570 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f17180cce70 con 0x7f171815f990 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 0x7f17181641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 0x7f17181641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47884/0 (socket says 172.21.15.170:47884) 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 -- 172.21.15.170:0/2520736630 learned_addr learned my addr 172.21.15.170:0/2520736630 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1717fff640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171815f990 0x7f17181646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f17177fe640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1718070570 0x7f1718153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 -- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171815f990 msgr2=0x7f17181646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171815f990 0x7f17181646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 -- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1718070570 msgr2=0x7f1718153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1718070570 0x7f1718153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 -- 172.21.15.170:0/2520736630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17180b2f40 con 0x7f17180cadf0 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f1716ffd640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 0x7f17181641b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f171000cbc0 tx=0x7f171000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710016020 con 0x7f17180cadf0 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f171000ab40 con 0x7f17180cadf0 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1718153bc0 con 0x7f17180cadf0 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171000ce40 con 0x7f17180cadf0 2024-09-21T02:14:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.384+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17180b4de0 con 0x7f17180cadf0 2024-09-21T02:14:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.388+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17180b4890 con 0x7f17180cadf0 2024-09-21T02:14:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.388+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1710019020 con 0x7f17180cadf0 2024-09-21T02:14:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.388+0000 7f170ffff640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16fc073200 0x7f16fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.388+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1710094130 con 0x7f17180cadf0 2024-09-21T02:14:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.388+0000 7f17177fe640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16fc073200 0x7f16fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.388+0000 7f17177fe640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16fc073200 0x7f16fc075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17000af000 tx=0x7f170001b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:18.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.392+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1710020030 con 0x7f17180cadf0 2024-09-21T02:14:18.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.592+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17180cb200 con 0x7f17180cadf0 2024-09-21T02:14:18.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f170ffff640 1 -- 172.21.15.170:0/2520736630 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f17180cb740 con 0x7f17180cadf0 2024-09-21T02:14:18.598 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:18.598 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"btime":"2024-09-21T02:14:18:191863+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:14.576721+0000","modified":"2024-09-21T02:14:17.607956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4914":{"gid":4914,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2504310121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2504310121},{"type":"v1","addr":"172.21.15.170:6835","nonce":2504310121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4914,"qdb_cluster":[4914]},"id":2}]} 2024-09-21T02:14:18.598 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 17 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16fc073200 msgr2=0x7f16fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16fc073200 0x7f16fc075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17000af000 tx=0x7f170001b030 comp rx=0 tx=0).stop 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 msgr2=0x7f17181641b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 0x7f17181641b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f171000cbc0 tx=0x7f171000cbf0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 shutdown_connections 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16fc073200 0x7f16fc075710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f171815f990 0x7f17181646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17180cadf0 0x7f17181641b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 --2- 172.21.15.170:0/2520736630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1718070570 0x7f1718153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 >> 172.21.15.170:0/2520736630 conn(0x7f17180b1c40 msgr2=0x7f171815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:18.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 shutdown_connections 2024-09-21T02:14:18.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.596+0000 7f171e0cc640 1 -- 172.21.15.170:0/2520736630 wait complete. 2024-09-21T02:14:18.618 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 -- 172.21.15.170:0/1257147291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f329815f990 msgr2=0x7f3298070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 --2- 172.21.15.170:0/1257147291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f329815f990 0x7f3298070030 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32980b2d00 tx=0x7f329001c880 comp rx=0 tx=0).stop 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 -- 172.21.15.170:0/1257147291 shutdown_connections 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 --2- 172.21.15.170:0/1257147291 >> v2:172.21.15.191:3300/0 conn(0x7f3298070570 0x7f32980ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 --2- 172.21.15.170:0/1257147291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f329815f990 0x7f3298070030 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 -- 172.21.15.170:0/1257147291 >> 172.21.15.170:0/1257147291 conn(0x7f32980b1c40 msgr2=0x7f32980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 -- 172.21.15.170:0/1257147291 shutdown_connections 2024-09-21T02:14:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 -- 172.21.15.170:0/1257147291 wait complete. 2024-09-21T02:14:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 Processor -- start 2024-09-21T02:14:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.756+0000 7f329e850640 1 -- start start 2024-09-21T02:14:18.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3298070570 0x7f32980c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 0x7f329815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329815f990 0x7f32981618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3298169100 con 0x7f32980cea00 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f32980cb710 con 0x7f329815f990 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3298169280 con 0x7f3298070570 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 0x7f329815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 0x7f329815fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47912/0 (socket says 172.21.15.170:47912) 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 -- 172.21.15.170:0/2902232997 learned_addr learned my addr 172.21.15.170:0/2902232997 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3297fff640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3298070570 0x7f32980c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329cded640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329815f990 0x7f32981618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 -- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3298070570 msgr2=0x7f32980c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3298070570 0x7f32980c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 -- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329815f990 msgr2=0x7f32981618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329815f990 0x7f32981618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 -- 172.21.15.170:0/2902232997 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3298160ad0 con 0x7f32980cea00 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3297fff640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3298070570 0x7f32980c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329cded640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329815f990 0x7f32981618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f32977fe640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 0x7f329815fda0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f328000d720 tx=0x7f328000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280017070 con 0x7f32980cea00 2024-09-21T02:14:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3280009d20 con 0x7f32980cea00 2024-09-21T02:14:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280005280 con 0x7f32980cea00 2024-09-21T02:14:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3298160db0 con 0x7f32980cea00 2024-09-21T02:14:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329815e710 con 0x7f32980cea00 2024-09-21T02:14:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32980cd8a0 con 0x7f32980cea00 2024-09-21T02:14:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3280005420 con 0x7f32980cea00 2024-09-21T02:14:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3294ff9640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3278073200 0x7f3278075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f32800942c0 con 0x7f32980cea00 2024-09-21T02:14:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.760+0000 7f3297fff640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3278073200 0x7f3278075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.764+0000 7f3297fff640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3278073200 0x7f3278075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f32900af000 tx=0x7f329001b0c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:18.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.768+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f328005f370 con 0x7f32980cea00 2024-09-21T02:14:18.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.924+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3298160860 con 0x7f32980cea00 2024-09-21T02:14:18.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.928+0000 7f3294ff9640 1 -- 172.21.15.170:0/2902232997 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f328005ec70 con 0x7f32980cea00 2024-09-21T02:14:18.932 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:18.932 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":35,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:16.149601+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:12.737491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:13.529023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:18.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3278073200 msgr2=0x7f3278075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3278073200 0x7f3278075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f32900af000 tx=0x7f329001b0c0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 msgr2=0x7f329815fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:18.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 0x7f329815fda0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f328000d720 tx=0x7f328000dbe0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 shutdown_connections 2024-09-21T02:14:18.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3278073200 0x7f3278075710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329815f990 0x7f32981618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32980cea00 0x7f329815fda0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 --2- 172.21.15.170:0/2902232997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3298070570 0x7f32980c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:18.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 >> 172.21.15.170:0/2902232997 conn(0x7f32980b1c40 msgr2=0x7f329806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:18.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 shutdown_connections 2024-09-21T02:14:18.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:18.932+0000 7f329e850640 1 -- 172.21.15.170:0/2902232997 wait complete. 2024-09-21T02:14:18.953 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:14:18.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369e888640 1 -- 172.21.15.170:0/2261154143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3680003a40 msgr2=0x7f3680003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369e888640 1 --2- 172.21.15.170:0/2261154143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3680003a40 0x7f3680003eb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f36880016c0 tx=0x7f368801c790 comp rx=0 tx=0).stop 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369ce25640 1 -- 172.21.15.170:0/2261154143 reap_dead start 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369e888640 1 -- 172.21.15.170:0/2261154143 shutdown_connections 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369e888640 1 -- 172.21.15.170:0/2261154143 >> 172.21.15.170:0/2261154143 conn(0x7f36980b1c40 msgr2=0x7f36980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369e888640 1 -- 172.21.15.170:0/2261154143 shutdown_connections 2024-09-21T02:14:19.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.104+0000 7f369e888640 1 -- 172.21.15.170:0/2261154143 wait complete. 2024-09-21T02:14:19.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 Processor -- start 2024-09-21T02:14:19.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 -- start start 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3680003a40 0x7f369815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36981fb5c0 0x7f369815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 0x7f36981fad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3698167200 con 0x7f36981fa970 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3698074f00 con 0x7f3680003a40 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3698167380 con 0x7f36981fb5c0 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 0x7f36981fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 0x7f36981fad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47920/0 (socket says 172.21.15.170:47920) 2024-09-21T02:14:19.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 -- 172.21.15.170:0/2524742252 learned_addr learned my addr 172.21.15.170:0/2524742252 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f3697fff640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3680003a40 0x7f369815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f36977fe640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36981fb5c0 0x7f369815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 -- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36981fb5c0 msgr2=0x7f369815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36981fb5c0 0x7f369815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 -- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3680003a40 msgr2=0x7f369815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3680003a40 0x7f369815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 -- 172.21.15.170:0/2524742252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981fb2c0 con 0x7f36981fa970 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f36977fe640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36981fb5c0 0x7f369815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f3697fff640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3680003a40 0x7f369815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369ce25640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 0x7f36981fad80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3698072030 tx=0x7f369000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000b8a0 con 0x7f36981fa970 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3690018070 con 0x7f36981fa970 2024-09-21T02:14:19.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000bd00 con 0x7f36981fa970 2024-09-21T02:14:19.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369815f090 con 0x7f36981fa970 2024-09-21T02:14:19.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.108+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36980c9e70 con 0x7f36981fa970 2024-09-21T02:14:19.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.112+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36980af850 con 0x7f36981fa970 2024-09-21T02:14:19.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.112+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f369001c410 con 0x7f36981fa970 2024-09-21T02:14:19.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.112+0000 7f3694ff9640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3684073230 0x7f3684075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.112+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3690093b10 con 0x7f36981fa970 2024-09-21T02:14:19.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.112+0000 7f3697fff640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3684073230 0x7f3684075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.112+0000 7f3697fff640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3684073230 0x7f3684075740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3698074030 tx=0x7f3680005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:19.116 INFO:tasks.ceph.mds.c.smithi170.stderr:2024-09-21T02:14:19.112+0000 7fb02af69640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:14:19.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.116+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3690021020 con 0x7f36981fa970 2024-09-21T02:14:19.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.292+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36980c7210 con 0x7f3684073230 2024-09-21T02:14:19.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.304+0000 7f3694ff9640 1 -- 172.21.15.170:0/2524742252 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242138 (secure 0 0 0) 0x7f36980c7210 con 0x7f3684073230 2024-09-21T02:14:19.308 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:19.310 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:19.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3684073230 msgr2=0x7f3684075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3684073230 0x7f3684075740 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3698074030 tx=0x7f3680005840 comp rx=0 tx=0).stop 2024-09-21T02:14:19.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 msgr2=0x7f36981fad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 0x7f36981fad80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3698072030 tx=0x7f369000dd50 comp rx=0 tx=0).stop 2024-09-21T02:14:19.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 shutdown_connections 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3684073230 0x7f3684075740 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36981fb5c0 0x7f369815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36981fa970 0x7f36981fad80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 --2- 172.21.15.170:0/2524742252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3680003a40 0x7f369815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 >> 172.21.15.170:0/2524742252 conn(0x7f36980b1c40 msgr2=0x7f369806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 shutdown_connections 2024-09-21T02:14:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.308+0000 7f369e888640 1 -- 172.21.15.170:0/2524742252 wait complete. 2024-09-21T02:14:19.330 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-21T02:14:18.883339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245136,"kb_used_data":3952,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729584,"statfs":{"total":773094113280,"available":772843094016,"internally_reserved":0,"allocated":4046848,"data_stored":2248644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.748363"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180103+0000","last_change":"2024-09-21T02:14:15.221703+0000","last_active":"2024-09-21T02:14:17.180103+0000","last_peered":"2024-09-21T02:14:17.180103+0000","last_clean":"2024-09-21T02:14:17.180103+0000","last_became_active":"2024-09-21T02:14:15.219448+0000","last_became_peered":"2024-09-21T02:14:15.219448+0000","last_unstale":"2024-09-21T02:14:17.180103+0000","last_undegraded":"2024-09-21T02:14:17.180103+0000","last_fullsized":"2024-09-21T02:14:17.180103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:27.282489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182169+0000","last_change":"2024-09-21T02:14:15.218508+0000","last_active":"2024-09-21T02:14:17.182169+0000","last_peered":"2024-09-21T02:14:17.182169+0000","last_clean":"2024-09-21T02:14:17.182169+0000","last_became_active":"2024-09-21T02:14:15.217625+0000","last_became_peered":"2024-09-21T02:14:15.217625+0000","last_unstale":"2024-09-21T02:14:17.182169+0000","last_undegraded":"2024-09-21T02:14:17.182169+0000","last_fullsized":"2024-09-21T02:14:17.182169+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.766624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179430+0000","last_change":"2024-09-21T02:14:15.203118+0000","last_active":"2024-09-21T02:14:17.179430+0000","last_peered":"2024-09-21T02:14:17.179430+0000","last_clean":"2024-09-21T02:14:17.179430+0000","last_became_active":"2024-09-21T02:14:15.202380+0000","last_became_peered":"2024-09-21T02:14:15.202380+0000","last_unstale":"2024-09-21T02:14:17.179430+0000","last_undegraded":"2024-09-21T02:14:17.179430+0000","last_fullsized":"2024-09-21T02:14:17.179430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:30.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180846+0000","last_change":"2024-09-21T02:14:15.192842+0000","last_active":"2024-09-21T02:14:17.180846+0000","last_peered":"2024-09-21T02:14:17.180846+0000","last_clean":"2024-09-21T02:14:17.180846+0000","last_became_active":"2024-09-21T02:14:15.190820+0000","last_became_peered":"2024-09-21T02:14:15.190820+0000","last_unstale":"2024-09-21T02:14:17.180846+0000","last_undegraded":"2024-09-21T02:14:17.180846+0000","last_fullsized":"2024-09-21T02:14:17.180846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:07.544915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182194+0000","last_change":"2024-09-21T02:14:15.212128+0000","last_active":"2024-09-21T02:14:17.182194+0000","last_peered":"2024-09-21T02:14:17.182194+0000","last_clean":"2024-09-21T02:14:17.182194+0000","last_became_active":"2024-09-21T02:14:15.211547+0000","last_became_peered":"2024-09-21T02:14:15.211547+0000","last_unstale":"2024-09-21T02:14:17.182194+0000","last_undegraded":"2024-09-21T02:14:17.182194+0000","last_fullsized":"2024-09-21T02:14:17.182194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:20.509138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.152085+0000","last_change":"2024-09-21T02:14:14.152085+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.152085+0000","last_undegraded":"2024-09-21T02:14:14.152085+0000","last_fullsized":"2024-09-21T02:14:14.152085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182364+0000","last_change":"2024-09-21T02:14:14.180675+0000","last_active":"2024-09-21T02:14:17.182364+0000","last_peered":"2024-09-21T02:14:17.182364+0000","last_clean":"2024-09-21T02:14:17.182364+0000","last_became_active":"2024-09-21T02:14:14.179791+0000","last_became_peered":"2024-09-21T02:14:14.179791+0000","last_unstale":"2024-09-21T02:14:17.182364+0000","last_undegraded":"2024-09-21T02:14:17.182364+0000","last_fullsized":"2024-09-21T02:14:17.182364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:47.635791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145869+0000","last_change":"2024-09-21T02:14:13.145869+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145869+0000","last_undegraded":"2024-09-21T02:14:13.145869+0000","last_fullsized":"2024-09-21T02:14:13.145869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181257+0000","last_change":"2024-09-21T02:14:15.201013+0000","last_active":"2024-09-21T02:14:17.181257+0000","last_peered":"2024-09-21T02:14:17.181257+0000","last_clean":"2024-09-21T02:14:17.181257+0000","last_became_active":"2024-09-21T02:14:15.199797+0000","last_became_peered":"2024-09-21T02:14:15.199797+0000","last_unstale":"2024-09-21T02:14:17.181257+0000","last_undegraded":"2024-09-21T02:14:17.181257+0000","last_fullsized":"2024-09-21T02:14:17.181257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:33.202747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180405+0000","last_change":"2024-09-21T02:14:14.195211+0000","last_active":"2024-09-21T02:14:17.180405+0000","last_peered":"2024-09-21T02:14:17.180405+0000","last_clean":"2024-09-21T02:14:17.180405+0000","last_became_active":"2024-09-21T02:14:14.194442+0000","last_became_peered":"2024-09-21T02:14:14.194442+0000","last_unstale":"2024-09-21T02:14:17.180405+0000","last_undegraded":"2024-09-21T02:14:17.180405+0000","last_fullsized":"2024-09-21T02:14:17.180405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:22.063757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.271582+0000","last_change":"2024-09-21T02:14:13.271582+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.271582+0000","last_undegraded":"2024-09-21T02:14:13.271582+0000","last_fullsized":"2024-09-21T02:14:13.271582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.271002+0000","last_change":"2024-09-21T02:14:13.271002+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.271002+0000","last_undegraded":"2024-09-21T02:14:13.271002+0000","last_fullsized":"2024-09-21T02:14:13.271002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145287+0000","last_change":"2024-09-21T02:14:13.145287+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145287+0000","last_undegraded":"2024-09-21T02:14:13.145287+0000","last_fullsized":"2024-09-21T02:14:13.145287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179304+0000","last_change":"2024-09-21T02:14:15.188344+0000","last_active":"2024-09-21T02:14:17.179304+0000","last_peered":"2024-09-21T02:14:17.179304+0000","last_clean":"2024-09-21T02:14:17.179304+0000","last_became_active":"2024-09-21T02:14:15.187467+0000","last_became_peered":"2024-09-21T02:14:15.187467+0000","last_unstale":"2024-09-21T02:14:17.179304+0000","last_undegraded":"2024-09-21T02:14:17.179304+0000","last_fullsized":"2024-09-21T02:14:17.179304+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:22.010859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270238+0000","last_change":"2024-09-21T02:14:13.270238+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270238+0000","last_undegraded":"2024-09-21T02:14:13.270238+0000","last_fullsized":"2024-09-21T02:14:13.270238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139263+0000","last_change":"2024-09-21T02:14:13.139263+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139263+0000","last_undegraded":"2024-09-21T02:14:13.139263+0000","last_fullsized":"2024-09-21T02:14:13.139263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181442+0000","last_change":"2024-09-21T02:14:15.186681+0000","last_active":"2024-09-21T02:14:17.181442+0000","last_peered":"2024-09-21T02:14:17.181442+0000","last_clean":"2024-09-21T02:14:17.181442+0000","last_became_active":"2024-09-21T02:14:15.185804+0000","last_became_peered":"2024-09-21T02:14:15.185804+0000","last_unstale":"2024-09-21T02:14:17.181442+0000","last_undegraded":"2024-09-21T02:14:17.181442+0000","last_fullsized":"2024-09-21T02:14:17.181442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:12.770185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.140792+0000","last_change":"2024-09-21T02:14:13.140792+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.140792+0000","last_undegraded":"2024-09-21T02:14:13.140792+0000","last_fullsized":"2024-09-21T02:14:13.140792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180186+0000","last_change":"2024-09-21T02:14:15.222984+0000","last_active":"2024-09-21T02:14:17.180186+0000","last_peered":"2024-09-21T02:14:17.180186+0000","last_clean":"2024-09-21T02:14:17.180186+0000","last_became_active":"2024-09-21T02:14:15.220840+0000","last_became_peered":"2024-09-21T02:14:15.220840+0000","last_unstale":"2024-09-21T02:14:17.180186+0000","last_undegraded":"2024-09-21T02:14:17.180186+0000","last_fullsized":"2024-09-21T02:14:17.180186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.417152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151179+0000","last_change":"2024-09-21T02:14:14.151179+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151179+0000","last_undegraded":"2024-09-21T02:14:14.151179+0000","last_fullsized":"2024-09-21T02:14:14.151179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.153737+0000","last_change":"2024-09-21T02:14:13.153737+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.153737+0000","last_undegraded":"2024-09-21T02:14:13.153737+0000","last_fullsized":"2024-09-21T02:14:13.153737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270175+0000","last_change":"2024-09-21T02:14:13.270175+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270175+0000","last_undegraded":"2024-09-21T02:14:13.270175+0000","last_fullsized":"2024-09-21T02:14:13.270175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180319+0000","last_change":"2024-09-21T02:14:15.206105+0000","last_active":"2024-09-21T02:14:17.180319+0000","last_peered":"2024-09-21T02:14:17.180319+0000","last_clean":"2024-09-21T02:14:17.180319+0000","last_became_active":"2024-09-21T02:14:15.205311+0000","last_became_peered":"2024-09-21T02:14:15.205311+0000","last_unstale":"2024-09-21T02:14:17.180319+0000","last_undegraded":"2024-09-21T02:14:17.180319+0000","last_fullsized":"2024-09-21T02:14:17.180319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:57.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179485+0000","last_change":"2024-09-21T02:14:14.180926+0000","last_active":"2024-09-21T02:14:17.179485+0000","last_peered":"2024-09-21T02:14:17.179485+0000","last_clean":"2024-09-21T02:14:17.179485+0000","last_became_active":"2024-09-21T02:14:14.179759+0000","last_became_peered":"2024-09-21T02:14:14.179759+0000","last_unstale":"2024-09-21T02:14:17.179485+0000","last_undegraded":"2024-09-21T02:14:17.179485+0000","last_fullsized":"2024-09-21T02:14:17.179485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:34.042299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179525+0000","last_change":"2024-09-21T02:14:14.182658+0000","last_active":"2024-09-21T02:14:17.179525+0000","last_peered":"2024-09-21T02:14:17.179525+0000","last_clean":"2024-09-21T02:14:17.179525+0000","last_became_active":"2024-09-21T02:14:14.181767+0000","last_became_peered":"2024-09-21T02:14:14.181767+0000","last_unstale":"2024-09-21T02:14:17.179525+0000","last_undegraded":"2024-09-21T02:14:17.179525+0000","last_fullsized":"2024-09-21T02:14:17.179525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:32.022100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145319+0000","last_change":"2024-09-21T02:14:13.145319+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145319+0000","last_undegraded":"2024-09-21T02:14:13.145319+0000","last_fullsized":"2024-09-21T02:14:13.145319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.146177+0000","last_change":"2024-09-21T02:14:13.146177+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.146177+0000","last_undegraded":"2024-09-21T02:14:13.146177+0000","last_fullsized":"2024-09-21T02:14:13.146177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270610+0000","last_change":"2024-09-21T02:14:13.270610+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270610+0000","last_undegraded":"2024-09-21T02:14:13.270610+0000","last_fullsized":"2024-09-21T02:14:13.270610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151615+0000","last_change":"2024-09-21T02:14:14.151615+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151615+0000","last_undegraded":"2024-09-21T02:14:14.151615+0000","last_fullsized":"2024-09-21T02:14:14.151615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.142074+0000","last_change":"2024-09-21T02:14:13.142074+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.142074+0000","last_undegraded":"2024-09-21T02:14:13.142074+0000","last_fullsized":"2024-09-21T02:14:13.142074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139080+0000","last_change":"2024-09-21T02:14:13.139080+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139080+0000","last_undegraded":"2024-09-21T02:14:13.139080+0000","last_fullsized":"2024-09-21T02:14:13.139080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270180+0000","last_change":"2024-09-21T02:14:13.270180+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270180+0000","last_undegraded":"2024-09-21T02:14:13.270180+0000","last_fullsized":"2024-09-21T02:14:13.270180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.138751+0000","last_change":"2024-09-21T02:14:13.138751+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.138751+0000","last_undegraded":"2024-09-21T02:14:13.138751+0000","last_fullsized":"2024-09-21T02:14:13.138751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151331+0000","last_change":"2024-09-21T02:14:14.151331+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151331+0000","last_undegraded":"2024-09-21T02:14:14.151331+0000","last_fullsized":"2024-09-21T02:14:14.151331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152162+0000","last_change":"2024-09-21T02:14:13.152162+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152162+0000","last_undegraded":"2024-09-21T02:14:13.152162+0000","last_fullsized":"2024-09-21T02:14:13.152162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139561+0000","last_change":"2024-09-21T02:14:13.139561+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139561+0000","last_undegraded":"2024-09-21T02:14:13.139561+0000","last_fullsized":"2024-09-21T02:14:13.139561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152035+0000","last_change":"2024-09-21T02:14:13.152035+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152035+0000","last_undegraded":"2024-09-21T02:14:13.152035+0000","last_fullsized":"2024-09-21T02:14:13.152035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.153292+0000","last_change":"2024-09-21T02:14:13.153292+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.153292+0000","last_undegraded":"2024-09-21T02:14:13.153292+0000","last_fullsized":"2024-09-21T02:14:13.153292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180922+0000","last_change":"2024-09-21T02:14:15.199669+0000","last_active":"2024-09-21T02:14:17.180922+0000","last_peered":"2024-09-21T02:14:17.180922+0000","last_clean":"2024-09-21T02:14:17.180922+0000","last_became_active":"2024-09-21T02:14:15.198854+0000","last_became_peered":"2024-09-21T02:14:15.198854+0000","last_unstale":"2024-09-21T02:14:17.180922+0000","last_undegraded":"2024-09-21T02:14:17.180922+0000","last_fullsized":"2024-09-21T02:14:17.180922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:19.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152253+0000","last_change":"2024-09-21T02:14:13.152253+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152253+0000","last_undegraded":"2024-09-21T02:14:13.152253+0000","last_fullsized":"2024-09-21T02:14:13.152253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179768+0000","last_change":"2024-09-21T02:14:15.200391+0000","last_active":"2024-09-21T02:14:17.179768+0000","last_peered":"2024-09-21T02:14:17.179768+0000","last_clean":"2024-09-21T02:14:17.179768+0000","last_became_active":"2024-09-21T02:14:15.196024+0000","last_became_peered":"2024-09-21T02:14:15.196024+0000","last_unstale":"2024-09-21T02:14:17.179768+0000","last_undegraded":"2024-09-21T02:14:17.179768+0000","last_fullsized":"2024-09-21T02:14:17.179768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:16.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270179+0000","last_change":"2024-09-21T02:14:13.270179+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270179+0000","last_undegraded":"2024-09-21T02:14:13.270179+0000","last_fullsized":"2024-09-21T02:14:13.270179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179042+0000","last_change":"2024-09-21T02:14:14.180582+0000","last_active":"2024-09-21T02:14:17.179042+0000","last_peered":"2024-09-21T02:14:17.179042+0000","last_clean":"2024-09-21T02:14:17.179042+0000","last_became_active":"2024-09-21T02:14:14.179876+0000","last_became_peered":"2024-09-21T02:14:14.179876+0000","last_unstale":"2024-09-21T02:14:17.179042+0000","last_undegraded":"2024-09-21T02:14:17.179042+0000","last_fullsized":"2024-09-21T02:14:17.179042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:29.774228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.152172+0000","last_change":"2024-09-21T02:14:14.152172+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.152172+0000","last_undegraded":"2024-09-21T02:14:14.152172+0000","last_fullsized":"2024-09-21T02:14:14.152172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":126,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T02:14:13.139218+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:13.139218+0000","last_peered":"2024-09-21T02:14:13.139218+0000","last_clean":"2024-09-21T02:14:13.139218+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:13.139218+0000","last_undegraded":"2024-09-21T02:14:13.139218+0000","last_fullsized":"2024-09-21T02:14:13.139218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":722021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36272,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335568,"statfs":{"total":96636764160,"available":96599621632,"internally_reserved":0,"allocated":946176,"data_stored":722021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":146337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:19.334 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 -- 172.21.15.170:0/3169453027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200cadf0 msgr2=0x7f8c200cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 --2- 172.21.15.170:0/3169453027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200cadf0 0x7f8c200cb200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8c200cc8b0 tx=0x7f8c1801cb80 comp rx=0 tx=0).stop 2024-09-21T02:14:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 -- 172.21.15.170:0/3169453027 shutdown_connections 2024-09-21T02:14:19.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 --2- 172.21.15.170:0/3169453027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200cadf0 0x7f8c200cb200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 --2- 172.21.15.170:0/3169453027 >> v2:172.21.15.191:3300/0 conn(0x7f8c200ce4c0 0x7f8c200ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 -- 172.21.15.170:0/3169453027 >> 172.21.15.170:0/3169453027 conn(0x7f8c200b1c40 msgr2=0x7f8c200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 -- 172.21.15.170:0/3169453027 shutdown_connections 2024-09-21T02:14:19.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 -- 172.21.15.170:0/3169453027 wait complete. 2024-09-21T02:14:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 Processor -- start 2024-09-21T02:14:19.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.524+0000 7f8c24b66640 1 -- start start 2024-09-21T02:14:19.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200cadf0 0x7f8c201fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 0x7f8c200b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c2015f990 0x7f8c200cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c20168170 con 0x7f8c200ce4c0 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8c20167ff0 con 0x7f8c200cadf0 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8c201682f0 con 0x7f8c2015f990 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 0x7f8c200b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 0x7f8c200b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47940/0 (socket says 172.21.15.170:47940) 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 -- 172.21.15.170:0/2297558043 learned_addr learned my addr 172.21.15.170:0/2297558043 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1e59c640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200cadf0 0x7f8c201fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 -- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c2015f990 msgr2=0x7f8c200cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1ed9d640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c2015f990 0x7f8c200cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c2015f990 0x7f8c200cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 -- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200cadf0 msgr2=0x7f8c201fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200cadf0 0x7f8c201fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 -- 172.21.15.170:0/2297558043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2009c4e0 con 0x7f8c200ce4c0 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1e59c640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200cadf0 0x7f8c201fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c1dd9b640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 0x7f8c200b4a60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8c140058d0 tx=0x7f8c1400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14017070 con 0x7f8c200ce4c0 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c14009d20 con 0x7f8c200ce4c0 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c201fca40 con 0x7f8c200ce4c0 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140106c0 con 0x7f8c200ce4c0 2024-09-21T02:14:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c201fd000 con 0x7f8c200ce4c0 2024-09-21T02:14:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c201fc790 con 0x7f8c200ce4c0 2024-09-21T02:14:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.528+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c1401a070 con 0x7f8c200ce4c0 2024-09-21T02:14:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.532+0000 7f8c0effd640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8c00073230 0x7f8c00075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.532+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8c140943d0 con 0x7f8c200ce4c0 2024-09-21T02:14:19.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.532+0000 7f8c1e59c640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8c00073230 0x7f8c00075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.532+0000 7f8c1e59c640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8c00073230 0x7f8c00075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c10001400 tx=0x7f8c10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:19.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.536+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8c1405f3f0 con 0x7f8c200ce4c0 2024-09-21T02:14:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.708+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c200b4fa0 con 0x7f8c00073230 2024-09-21T02:14:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.720+0000 7f8c0effd640 1 -- 172.21.15.170:0/2297558043 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242138 (secure 0 0 0) 0x7f8c200b4fa0 con 0x7f8c00073230 2024-09-21T02:14:19.725 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:19.726 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8c00073230 msgr2=0x7f8c00075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8c00073230 0x7f8c00075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c10001400 tx=0x7f8c10009040 comp rx=0 tx=0).stop 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 msgr2=0x7f8c200b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 0x7f8c200b4a60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8c140058d0 tx=0x7f8c1400dde0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 shutdown_connections 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8c00073230 0x7f8c00075740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c2015f990 0x7f8c200cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c200ce4c0 0x7f8c200b4a60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 --2- 172.21.15.170:0/2297558043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c200cadf0 0x7f8c201fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 >> 172.21.15.170:0/2297558043 conn(0x7f8c200b1c40 msgr2=0x7f8c200cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:19.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 shutdown_connections 2024-09-21T02:14:19.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.724+0000 7f8c24b66640 1 -- 172.21.15.170:0/2297558043 wait complete. 2024-09-21T02:14:19.746 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-21T02:14:18.883339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245136,"kb_used_data":3952,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729584,"statfs":{"total":773094113280,"available":772843094016,"internally_reserved":0,"allocated":4046848,"data_stored":2248644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.748363"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180103+0000","last_change":"2024-09-21T02:14:15.221703+0000","last_active":"2024-09-21T02:14:17.180103+0000","last_peered":"2024-09-21T02:14:17.180103+0000","last_clean":"2024-09-21T02:14:17.180103+0000","last_became_active":"2024-09-21T02:14:15.219448+0000","last_became_peered":"2024-09-21T02:14:15.219448+0000","last_unstale":"2024-09-21T02:14:17.180103+0000","last_undegraded":"2024-09-21T02:14:17.180103+0000","last_fullsized":"2024-09-21T02:14:17.180103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:27.282489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182169+0000","last_change":"2024-09-21T02:14:15.218508+0000","last_active":"2024-09-21T02:14:17.182169+0000","last_peered":"2024-09-21T02:14:17.182169+0000","last_clean":"2024-09-21T02:14:17.182169+0000","last_became_active":"2024-09-21T02:14:15.217625+0000","last_became_peered":"2024-09-21T02:14:15.217625+0000","last_unstale":"2024-09-21T02:14:17.182169+0000","last_undegraded":"2024-09-21T02:14:17.182169+0000","last_fullsized":"2024-09-21T02:14:17.182169+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.766624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179430+0000","last_change":"2024-09-21T02:14:15.203118+0000","last_active":"2024-09-21T02:14:17.179430+0000","last_peered":"2024-09-21T02:14:17.179430+0000","last_clean":"2024-09-21T02:14:17.179430+0000","last_became_active":"2024-09-21T02:14:15.202380+0000","last_became_peered":"2024-09-21T02:14:15.202380+0000","last_unstale":"2024-09-21T02:14:17.179430+0000","last_undegraded":"2024-09-21T02:14:17.179430+0000","last_fullsized":"2024-09-21T02:14:17.179430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:30.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180846+0000","last_change":"2024-09-21T02:14:15.192842+0000","last_active":"2024-09-21T02:14:17.180846+0000","last_peered":"2024-09-21T02:14:17.180846+0000","last_clean":"2024-09-21T02:14:17.180846+0000","last_became_active":"2024-09-21T02:14:15.190820+0000","last_became_peered":"2024-09-21T02:14:15.190820+0000","last_unstale":"2024-09-21T02:14:17.180846+0000","last_undegraded":"2024-09-21T02:14:17.180846+0000","last_fullsized":"2024-09-21T02:14:17.180846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:07.544915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182194+0000","last_change":"2024-09-21T02:14:15.212128+0000","last_active":"2024-09-21T02:14:17.182194+0000","last_peered":"2024-09-21T02:14:17.182194+0000","last_clean":"2024-09-21T02:14:17.182194+0000","last_became_active":"2024-09-21T02:14:15.211547+0000","last_became_peered":"2024-09-21T02:14:15.211547+0000","last_unstale":"2024-09-21T02:14:17.182194+0000","last_undegraded":"2024-09-21T02:14:17.182194+0000","last_fullsized":"2024-09-21T02:14:17.182194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:20.509138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.152085+0000","last_change":"2024-09-21T02:14:14.152085+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.152085+0000","last_undegraded":"2024-09-21T02:14:14.152085+0000","last_fullsized":"2024-09-21T02:14:14.152085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182364+0000","last_change":"2024-09-21T02:14:14.180675+0000","last_active":"2024-09-21T02:14:17.182364+0000","last_peered":"2024-09-21T02:14:17.182364+0000","last_clean":"2024-09-21T02:14:17.182364+0000","last_became_active":"2024-09-21T02:14:14.179791+0000","last_became_peered":"2024-09-21T02:14:14.179791+0000","last_unstale":"2024-09-21T02:14:17.182364+0000","last_undegraded":"2024-09-21T02:14:17.182364+0000","last_fullsized":"2024-09-21T02:14:17.182364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:47.635791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145869+0000","last_change":"2024-09-21T02:14:13.145869+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145869+0000","last_undegraded":"2024-09-21T02:14:13.145869+0000","last_fullsized":"2024-09-21T02:14:13.145869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181257+0000","last_change":"2024-09-21T02:14:15.201013+0000","last_active":"2024-09-21T02:14:17.181257+0000","last_peered":"2024-09-21T02:14:17.181257+0000","last_clean":"2024-09-21T02:14:17.181257+0000","last_became_active":"2024-09-21T02:14:15.199797+0000","last_became_peered":"2024-09-21T02:14:15.199797+0000","last_unstale":"2024-09-21T02:14:17.181257+0000","last_undegraded":"2024-09-21T02:14:17.181257+0000","last_fullsized":"2024-09-21T02:14:17.181257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:33.202747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180405+0000","last_change":"2024-09-21T02:14:14.195211+0000","last_active":"2024-09-21T02:14:17.180405+0000","last_peered":"2024-09-21T02:14:17.180405+0000","last_clean":"2024-09-21T02:14:17.180405+0000","last_became_active":"2024-09-21T02:14:14.194442+0000","last_became_peered":"2024-09-21T02:14:14.194442+0000","last_unstale":"2024-09-21T02:14:17.180405+0000","last_undegraded":"2024-09-21T02:14:17.180405+0000","last_fullsized":"2024-09-21T02:14:17.180405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:22.063757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.271582+0000","last_change":"2024-09-21T02:14:13.271582+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.271582+0000","last_undegraded":"2024-09-21T02:14:13.271582+0000","last_fullsized":"2024-09-21T02:14:13.271582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.271002+0000","last_change":"2024-09-21T02:14:13.271002+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.271002+0000","last_undegraded":"2024-09-21T02:14:13.271002+0000","last_fullsized":"2024-09-21T02:14:13.271002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145287+0000","last_change":"2024-09-21T02:14:13.145287+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145287+0000","last_undegraded":"2024-09-21T02:14:13.145287+0000","last_fullsized":"2024-09-21T02:14:13.145287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179304+0000","last_change":"2024-09-21T02:14:15.188344+0000","last_active":"2024-09-21T02:14:17.179304+0000","last_peered":"2024-09-21T02:14:17.179304+0000","last_clean":"2024-09-21T02:14:17.179304+0000","last_became_active":"2024-09-21T02:14:15.187467+0000","last_became_peered":"2024-09-21T02:14:15.187467+0000","last_unstale":"2024-09-21T02:14:17.179304+0000","last_undegraded":"2024-09-21T02:14:17.179304+0000","last_fullsized":"2024-09-21T02:14:17.179304+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:22.010859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270238+0000","last_change":"2024-09-21T02:14:13.270238+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270238+0000","last_undegraded":"2024-09-21T02:14:13.270238+0000","last_fullsized":"2024-09-21T02:14:13.270238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139263+0000","last_change":"2024-09-21T02:14:13.139263+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139263+0000","last_undegraded":"2024-09-21T02:14:13.139263+0000","last_fullsized":"2024-09-21T02:14:13.139263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181442+0000","last_change":"2024-09-21T02:14:15.186681+0000","last_active":"2024-09-21T02:14:17.181442+0000","last_peered":"2024-09-21T02:14:17.181442+0000","last_clean":"2024-09-21T02:14:17.181442+0000","last_became_active":"2024-09-21T02:14:15.185804+0000","last_became_peered":"2024-09-21T02:14:15.185804+0000","last_unstale":"2024-09-21T02:14:17.181442+0000","last_undegraded":"2024-09-21T02:14:17.181442+0000","last_fullsized":"2024-09-21T02:14:17.181442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:12.770185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.140792+0000","last_change":"2024-09-21T02:14:13.140792+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.140792+0000","last_undegraded":"2024-09-21T02:14:13.140792+0000","last_fullsized":"2024-09-21T02:14:13.140792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180186+0000","last_change":"2024-09-21T02:14:15.222984+0000","last_active":"2024-09-21T02:14:17.180186+0000","last_peered":"2024-09-21T02:14:17.180186+0000","last_clean":"2024-09-21T02:14:17.180186+0000","last_became_active":"2024-09-21T02:14:15.220840+0000","last_became_peered":"2024-09-21T02:14:15.220840+0000","last_unstale":"2024-09-21T02:14:17.180186+0000","last_undegraded":"2024-09-21T02:14:17.180186+0000","last_fullsized":"2024-09-21T02:14:17.180186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.417152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151179+0000","last_change":"2024-09-21T02:14:14.151179+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151179+0000","last_undegraded":"2024-09-21T02:14:14.151179+0000","last_fullsized":"2024-09-21T02:14:14.151179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.153737+0000","last_change":"2024-09-21T02:14:13.153737+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.153737+0000","last_undegraded":"2024-09-21T02:14:13.153737+0000","last_fullsized":"2024-09-21T02:14:13.153737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270175+0000","last_change":"2024-09-21T02:14:13.270175+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270175+0000","last_undegraded":"2024-09-21T02:14:13.270175+0000","last_fullsized":"2024-09-21T02:14:13.270175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180319+0000","last_change":"2024-09-21T02:14:15.206105+0000","last_active":"2024-09-21T02:14:17.180319+0000","last_peered":"2024-09-21T02:14:17.180319+0000","last_clean":"2024-09-21T02:14:17.180319+0000","last_became_active":"2024-09-21T02:14:15.205311+0000","last_became_peered":"2024-09-21T02:14:15.205311+0000","last_unstale":"2024-09-21T02:14:17.180319+0000","last_undegraded":"2024-09-21T02:14:17.180319+0000","last_fullsized":"2024-09-21T02:14:17.180319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:57.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179485+0000","last_change":"2024-09-21T02:14:14.180926+0000","last_active":"2024-09-21T02:14:17.179485+0000","last_peered":"2024-09-21T02:14:17.179485+0000","last_clean":"2024-09-21T02:14:17.179485+0000","last_became_active":"2024-09-21T02:14:14.179759+0000","last_became_peered":"2024-09-21T02:14:14.179759+0000","last_unstale":"2024-09-21T02:14:17.179485+0000","last_undegraded":"2024-09-21T02:14:17.179485+0000","last_fullsized":"2024-09-21T02:14:17.179485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:34.042299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179525+0000","last_change":"2024-09-21T02:14:14.182658+0000","last_active":"2024-09-21T02:14:17.179525+0000","last_peered":"2024-09-21T02:14:17.179525+0000","last_clean":"2024-09-21T02:14:17.179525+0000","last_became_active":"2024-09-21T02:14:14.181767+0000","last_became_peered":"2024-09-21T02:14:14.181767+0000","last_unstale":"2024-09-21T02:14:17.179525+0000","last_undegraded":"2024-09-21T02:14:17.179525+0000","last_fullsized":"2024-09-21T02:14:17.179525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:32.022100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145319+0000","last_change":"2024-09-21T02:14:13.145319+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145319+0000","last_undegraded":"2024-09-21T02:14:13.145319+0000","last_fullsized":"2024-09-21T02:14:13.145319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.146177+0000","last_change":"2024-09-21T02:14:13.146177+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.146177+0000","last_undegraded":"2024-09-21T02:14:13.146177+0000","last_fullsized":"2024-09-21T02:14:13.146177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270610+0000","last_change":"2024-09-21T02:14:13.270610+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270610+0000","last_undegraded":"2024-09-21T02:14:13.270610+0000","last_fullsized":"2024-09-21T02:14:13.270610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151615+0000","last_change":"2024-09-21T02:14:14.151615+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151615+0000","last_undegraded":"2024-09-21T02:14:14.151615+0000","last_fullsized":"2024-09-21T02:14:14.151615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.142074+0000","last_change":"2024-09-21T02:14:13.142074+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.142074+0000","last_undegraded":"2024-09-21T02:14:13.142074+0000","last_fullsized":"2024-09-21T02:14:13.142074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139080+0000","last_change":"2024-09-21T02:14:13.139080+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139080+0000","last_undegraded":"2024-09-21T02:14:13.139080+0000","last_fullsized":"2024-09-21T02:14:13.139080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270180+0000","last_change":"2024-09-21T02:14:13.270180+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270180+0000","last_undegraded":"2024-09-21T02:14:13.270180+0000","last_fullsized":"2024-09-21T02:14:13.270180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.138751+0000","last_change":"2024-09-21T02:14:13.138751+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.138751+0000","last_undegraded":"2024-09-21T02:14:13.138751+0000","last_fullsized":"2024-09-21T02:14:13.138751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151331+0000","last_change":"2024-09-21T02:14:14.151331+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151331+0000","last_undegraded":"2024-09-21T02:14:14.151331+0000","last_fullsized":"2024-09-21T02:14:14.151331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152162+0000","last_change":"2024-09-21T02:14:13.152162+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152162+0000","last_undegraded":"2024-09-21T02:14:13.152162+0000","last_fullsized":"2024-09-21T02:14:13.152162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139561+0000","last_change":"2024-09-21T02:14:13.139561+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139561+0000","last_undegraded":"2024-09-21T02:14:13.139561+0000","last_fullsized":"2024-09-21T02:14:13.139561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152035+0000","last_change":"2024-09-21T02:14:13.152035+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152035+0000","last_undegraded":"2024-09-21T02:14:13.152035+0000","last_fullsized":"2024-09-21T02:14:13.152035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.153292+0000","last_change":"2024-09-21T02:14:13.153292+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.153292+0000","last_undegraded":"2024-09-21T02:14:13.153292+0000","last_fullsized":"2024-09-21T02:14:13.153292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180922+0000","last_change":"2024-09-21T02:14:15.199669+0000","last_active":"2024-09-21T02:14:17.180922+0000","last_peered":"2024-09-21T02:14:17.180922+0000","last_clean":"2024-09-21T02:14:17.180922+0000","last_became_active":"2024-09-21T02:14:15.198854+0000","last_became_peered":"2024-09-21T02:14:15.198854+0000","last_unstale":"2024-09-21T02:14:17.180922+0000","last_undegraded":"2024-09-21T02:14:17.180922+0000","last_fullsized":"2024-09-21T02:14:17.180922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:19.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152253+0000","last_change":"2024-09-21T02:14:13.152253+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152253+0000","last_undegraded":"2024-09-21T02:14:13.152253+0000","last_fullsized":"2024-09-21T02:14:13.152253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179768+0000","last_change":"2024-09-21T02:14:15.200391+0000","last_active":"2024-09-21T02:14:17.179768+0000","last_peered":"2024-09-21T02:14:17.179768+0000","last_clean":"2024-09-21T02:14:17.179768+0000","last_became_active":"2024-09-21T02:14:15.196024+0000","last_became_peered":"2024-09-21T02:14:15.196024+0000","last_unstale":"2024-09-21T02:14:17.179768+0000","last_undegraded":"2024-09-21T02:14:17.179768+0000","last_fullsized":"2024-09-21T02:14:17.179768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:16.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270179+0000","last_change":"2024-09-21T02:14:13.270179+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270179+0000","last_undegraded":"2024-09-21T02:14:13.270179+0000","last_fullsized":"2024-09-21T02:14:13.270179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179042+0000","last_change":"2024-09-21T02:14:14.180582+0000","last_active":"2024-09-21T02:14:17.179042+0000","last_peered":"2024-09-21T02:14:17.179042+0000","last_clean":"2024-09-21T02:14:17.179042+0000","last_became_active":"2024-09-21T02:14:14.179876+0000","last_became_peered":"2024-09-21T02:14:14.179876+0000","last_unstale":"2024-09-21T02:14:17.179042+0000","last_undegraded":"2024-09-21T02:14:17.179042+0000","last_fullsized":"2024-09-21T02:14:17.179042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:29.774228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.152172+0000","last_change":"2024-09-21T02:14:14.152172+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.152172+0000","last_undegraded":"2024-09-21T02:14:14.152172+0000","last_fullsized":"2024-09-21T02:14:14.152172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":126,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T02:14:13.139218+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:13.139218+0000","last_peered":"2024-09-21T02:14:13.139218+0000","last_clean":"2024-09-21T02:14:13.139218+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:13.139218+0000","last_undegraded":"2024-09-21T02:14:13.139218+0000","last_fullsized":"2024-09-21T02:14:13.139218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":722021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36272,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335568,"statfs":{"total":96636764160,"available":96599621632,"internally_reserved":0,"allocated":946176,"data_stored":722021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":146337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:19.750 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3779052742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 msgr2=0x7f8a640cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3779052742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a640cb870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009840 tx=0x7f8a4c01c820 comp rx=0 tx=0).stop 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3779052742 shutdown_connections 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3779052742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a640cb870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3779052742 >> 172.21.15.170:0/3779052742 conn(0x7f8a640b1c40 msgr2=0x7f8a640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3779052742 shutdown_connections 2024-09-21T02:14:19.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3779052742 wait complete. 2024-09-21T02:14:19.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 Processor -- start 2024-09-21T02:14:19.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.944+0000 7f8a68fd5640 1 -- start start 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a6414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a640ce1c0 0x7f8a6415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a640cf7e0 0x7f8a6415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6416a2f0 con 0x7f8a640cb460 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6416a170 con 0x7f8a640ce1c0 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6416a470 con 0x7f8a640cf7e0 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a6414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a6414deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47958/0 (socket says 172.21.15.170:47958) 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 -- 172.21.15.170:0/3481604908 learned_addr learned my addr 172.21.15.170:0/3481604908 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:19.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a61d9b640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a640ce1c0 0x7f8a6415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 -- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a640cf7e0 msgr2=0x7f8a6415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a640cf7e0 0x7f8a6415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 -- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a640ce1c0 msgr2=0x7f8a6415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a640ce1c0 0x7f8a6415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 -- 172.21.15.170:0/3481604908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6415e4c0 con 0x7f8a640cb460 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a6259c640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a6414deb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00cb70 tx=0x7f8a5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c016020 con 0x7f8a640cb460 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a5c00ab60 con 0x7f8a640cb460 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641f90d0 con 0x7f8a640cb460 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c012420 con 0x7f8a640cb460 2024-09-21T02:14:19.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641f9690 con 0x7f8a640cb460 2024-09-21T02:14:19.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a640ce5d0 con 0x7f8a640cb460 2024-09-21T02:14:19.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a5c019020 con 0x7f8a640cb460 2024-09-21T02:14:19.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a5affd640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a44073200 0x7f8a44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:19.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.948+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8a5c0944a0 con 0x7f8a640cb460 2024-09-21T02:14:19.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.952+0000 7f8a61d9b640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a44073200 0x7f8a44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:19.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.952+0000 7f8a61d9b640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a44073200 0x7f8a44075710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0af000 tx=0x7f8a4c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:19.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:19.956+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8a5c020020 con 0x7f8a640cb460 2024-09-21T02:14:20.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.120+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a640ad9d0 con 0x7f8a44073200 2024-09-21T02:14:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.132+0000 7f8a5affd640 1 -- 172.21.15.170:0/3481604908 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242138 (secure 0 0 0) 0x7f8a640ad9d0 con 0x7f8a44073200 2024-09-21T02:14:20.137 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:20.138 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a44073200 msgr2=0x7f8a44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a44073200 0x7f8a44075710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0af000 tx=0x7f8a4c01b060 comp rx=0 tx=0).stop 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 msgr2=0x7f8a6414deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a6414deb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00cb70 tx=0x7f8a5c00cc60 comp rx=0 tx=0).stop 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 shutdown_connections 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8a44073200 0x7f8a44075710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a640cf7e0 0x7f8a6415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a640ce1c0 0x7f8a6415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 --2- 172.21.15.170:0/3481604908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a640cb460 0x7f8a6414deb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 >> 172.21.15.170:0/3481604908 conn(0x7f8a640b1c40 msgr2=0x7f8a640adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 shutdown_connections 2024-09-21T02:14:20.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.136+0000 7f8a68fd5640 1 -- 172.21.15.170:0/3481604908 wait complete. 2024-09-21T02:14:20.156 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-21T02:14:18.883339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245136,"kb_used_data":3952,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729584,"statfs":{"total":773094113280,"available":772843094016,"internally_reserved":0,"allocated":4046848,"data_stored":2248644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.748363"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180103+0000","last_change":"2024-09-21T02:14:15.221703+0000","last_active":"2024-09-21T02:14:17.180103+0000","last_peered":"2024-09-21T02:14:17.180103+0000","last_clean":"2024-09-21T02:14:17.180103+0000","last_became_active":"2024-09-21T02:14:15.219448+0000","last_became_peered":"2024-09-21T02:14:15.219448+0000","last_unstale":"2024-09-21T02:14:17.180103+0000","last_undegraded":"2024-09-21T02:14:17.180103+0000","last_fullsized":"2024-09-21T02:14:17.180103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:27.282489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182169+0000","last_change":"2024-09-21T02:14:15.218508+0000","last_active":"2024-09-21T02:14:17.182169+0000","last_peered":"2024-09-21T02:14:17.182169+0000","last_clean":"2024-09-21T02:14:17.182169+0000","last_became_active":"2024-09-21T02:14:15.217625+0000","last_became_peered":"2024-09-21T02:14:15.217625+0000","last_unstale":"2024-09-21T02:14:17.182169+0000","last_undegraded":"2024-09-21T02:14:17.182169+0000","last_fullsized":"2024-09-21T02:14:17.182169+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.766624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179430+0000","last_change":"2024-09-21T02:14:15.203118+0000","last_active":"2024-09-21T02:14:17.179430+0000","last_peered":"2024-09-21T02:14:17.179430+0000","last_clean":"2024-09-21T02:14:17.179430+0000","last_became_active":"2024-09-21T02:14:15.202380+0000","last_became_peered":"2024-09-21T02:14:15.202380+0000","last_unstale":"2024-09-21T02:14:17.179430+0000","last_undegraded":"2024-09-21T02:14:17.179430+0000","last_fullsized":"2024-09-21T02:14:17.179430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:30.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180846+0000","last_change":"2024-09-21T02:14:15.192842+0000","last_active":"2024-09-21T02:14:17.180846+0000","last_peered":"2024-09-21T02:14:17.180846+0000","last_clean":"2024-09-21T02:14:17.180846+0000","last_became_active":"2024-09-21T02:14:15.190820+0000","last_became_peered":"2024-09-21T02:14:15.190820+0000","last_unstale":"2024-09-21T02:14:17.180846+0000","last_undegraded":"2024-09-21T02:14:17.180846+0000","last_fullsized":"2024-09-21T02:14:17.180846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:07.544915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182194+0000","last_change":"2024-09-21T02:14:15.212128+0000","last_active":"2024-09-21T02:14:17.182194+0000","last_peered":"2024-09-21T02:14:17.182194+0000","last_clean":"2024-09-21T02:14:17.182194+0000","last_became_active":"2024-09-21T02:14:15.211547+0000","last_became_peered":"2024-09-21T02:14:15.211547+0000","last_unstale":"2024-09-21T02:14:17.182194+0000","last_undegraded":"2024-09-21T02:14:17.182194+0000","last_fullsized":"2024-09-21T02:14:17.182194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:20.509138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.152085+0000","last_change":"2024-09-21T02:14:14.152085+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.152085+0000","last_undegraded":"2024-09-21T02:14:14.152085+0000","last_fullsized":"2024-09-21T02:14:14.152085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182364+0000","last_change":"2024-09-21T02:14:14.180675+0000","last_active":"2024-09-21T02:14:17.182364+0000","last_peered":"2024-09-21T02:14:17.182364+0000","last_clean":"2024-09-21T02:14:17.182364+0000","last_became_active":"2024-09-21T02:14:14.179791+0000","last_became_peered":"2024-09-21T02:14:14.179791+0000","last_unstale":"2024-09-21T02:14:17.182364+0000","last_undegraded":"2024-09-21T02:14:17.182364+0000","last_fullsized":"2024-09-21T02:14:17.182364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:47.635791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145869+0000","last_change":"2024-09-21T02:14:13.145869+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145869+0000","last_undegraded":"2024-09-21T02:14:13.145869+0000","last_fullsized":"2024-09-21T02:14:13.145869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181257+0000","last_change":"2024-09-21T02:14:15.201013+0000","last_active":"2024-09-21T02:14:17.181257+0000","last_peered":"2024-09-21T02:14:17.181257+0000","last_clean":"2024-09-21T02:14:17.181257+0000","last_became_active":"2024-09-21T02:14:15.199797+0000","last_became_peered":"2024-09-21T02:14:15.199797+0000","last_unstale":"2024-09-21T02:14:17.181257+0000","last_undegraded":"2024-09-21T02:14:17.181257+0000","last_fullsized":"2024-09-21T02:14:17.181257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:33.202747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180405+0000","last_change":"2024-09-21T02:14:14.195211+0000","last_active":"2024-09-21T02:14:17.180405+0000","last_peered":"2024-09-21T02:14:17.180405+0000","last_clean":"2024-09-21T02:14:17.180405+0000","last_became_active":"2024-09-21T02:14:14.194442+0000","last_became_peered":"2024-09-21T02:14:14.194442+0000","last_unstale":"2024-09-21T02:14:17.180405+0000","last_undegraded":"2024-09-21T02:14:17.180405+0000","last_fullsized":"2024-09-21T02:14:17.180405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:22.063757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.271582+0000","last_change":"2024-09-21T02:14:13.271582+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.271582+0000","last_undegraded":"2024-09-21T02:14:13.271582+0000","last_fullsized":"2024-09-21T02:14:13.271582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.271002+0000","last_change":"2024-09-21T02:14:13.271002+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.271002+0000","last_undegraded":"2024-09-21T02:14:13.271002+0000","last_fullsized":"2024-09-21T02:14:13.271002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145287+0000","last_change":"2024-09-21T02:14:13.145287+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145287+0000","last_undegraded":"2024-09-21T02:14:13.145287+0000","last_fullsized":"2024-09-21T02:14:13.145287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179304+0000","last_change":"2024-09-21T02:14:15.188344+0000","last_active":"2024-09-21T02:14:17.179304+0000","last_peered":"2024-09-21T02:14:17.179304+0000","last_clean":"2024-09-21T02:14:17.179304+0000","last_became_active":"2024-09-21T02:14:15.187467+0000","last_became_peered":"2024-09-21T02:14:15.187467+0000","last_unstale":"2024-09-21T02:14:17.179304+0000","last_undegraded":"2024-09-21T02:14:17.179304+0000","last_fullsized":"2024-09-21T02:14:17.179304+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:22.010859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270238+0000","last_change":"2024-09-21T02:14:13.270238+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270238+0000","last_undegraded":"2024-09-21T02:14:13.270238+0000","last_fullsized":"2024-09-21T02:14:13.270238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139263+0000","last_change":"2024-09-21T02:14:13.139263+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139263+0000","last_undegraded":"2024-09-21T02:14:13.139263+0000","last_fullsized":"2024-09-21T02:14:13.139263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181442+0000","last_change":"2024-09-21T02:14:15.186681+0000","last_active":"2024-09-21T02:14:17.181442+0000","last_peered":"2024-09-21T02:14:17.181442+0000","last_clean":"2024-09-21T02:14:17.181442+0000","last_became_active":"2024-09-21T02:14:15.185804+0000","last_became_peered":"2024-09-21T02:14:15.185804+0000","last_unstale":"2024-09-21T02:14:17.181442+0000","last_undegraded":"2024-09-21T02:14:17.181442+0000","last_fullsized":"2024-09-21T02:14:17.181442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:12.770185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.140792+0000","last_change":"2024-09-21T02:14:13.140792+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.140792+0000","last_undegraded":"2024-09-21T02:14:13.140792+0000","last_fullsized":"2024-09-21T02:14:13.140792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180186+0000","last_change":"2024-09-21T02:14:15.222984+0000","last_active":"2024-09-21T02:14:17.180186+0000","last_peered":"2024-09-21T02:14:17.180186+0000","last_clean":"2024-09-21T02:14:17.180186+0000","last_became_active":"2024-09-21T02:14:15.220840+0000","last_became_peered":"2024-09-21T02:14:15.220840+0000","last_unstale":"2024-09-21T02:14:17.180186+0000","last_undegraded":"2024-09-21T02:14:17.180186+0000","last_fullsized":"2024-09-21T02:14:17.180186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.417152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151179+0000","last_change":"2024-09-21T02:14:14.151179+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151179+0000","last_undegraded":"2024-09-21T02:14:14.151179+0000","last_fullsized":"2024-09-21T02:14:14.151179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.153737+0000","last_change":"2024-09-21T02:14:13.153737+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.153737+0000","last_undegraded":"2024-09-21T02:14:13.153737+0000","last_fullsized":"2024-09-21T02:14:13.153737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270175+0000","last_change":"2024-09-21T02:14:13.270175+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270175+0000","last_undegraded":"2024-09-21T02:14:13.270175+0000","last_fullsized":"2024-09-21T02:14:13.270175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180319+0000","last_change":"2024-09-21T02:14:15.206105+0000","last_active":"2024-09-21T02:14:17.180319+0000","last_peered":"2024-09-21T02:14:17.180319+0000","last_clean":"2024-09-21T02:14:17.180319+0000","last_became_active":"2024-09-21T02:14:15.205311+0000","last_became_peered":"2024-09-21T02:14:15.205311+0000","last_unstale":"2024-09-21T02:14:17.180319+0000","last_undegraded":"2024-09-21T02:14:17.180319+0000","last_fullsized":"2024-09-21T02:14:17.180319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:57.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179485+0000","last_change":"2024-09-21T02:14:14.180926+0000","last_active":"2024-09-21T02:14:17.179485+0000","last_peered":"2024-09-21T02:14:17.179485+0000","last_clean":"2024-09-21T02:14:17.179485+0000","last_became_active":"2024-09-21T02:14:14.179759+0000","last_became_peered":"2024-09-21T02:14:14.179759+0000","last_unstale":"2024-09-21T02:14:17.179485+0000","last_undegraded":"2024-09-21T02:14:17.179485+0000","last_fullsized":"2024-09-21T02:14:17.179485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:34.042299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179525+0000","last_change":"2024-09-21T02:14:14.182658+0000","last_active":"2024-09-21T02:14:17.179525+0000","last_peered":"2024-09-21T02:14:17.179525+0000","last_clean":"2024-09-21T02:14:17.179525+0000","last_became_active":"2024-09-21T02:14:14.181767+0000","last_became_peered":"2024-09-21T02:14:14.181767+0000","last_unstale":"2024-09-21T02:14:17.179525+0000","last_undegraded":"2024-09-21T02:14:17.179525+0000","last_fullsized":"2024-09-21T02:14:17.179525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:32.022100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.145319+0000","last_change":"2024-09-21T02:14:13.145319+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.145319+0000","last_undegraded":"2024-09-21T02:14:13.145319+0000","last_fullsized":"2024-09-21T02:14:13.145319+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.146177+0000","last_change":"2024-09-21T02:14:13.146177+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.146177+0000","last_undegraded":"2024-09-21T02:14:13.146177+0000","last_fullsized":"2024-09-21T02:14:13.146177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270610+0000","last_change":"2024-09-21T02:14:13.270610+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270610+0000","last_undegraded":"2024-09-21T02:14:13.270610+0000","last_fullsized":"2024-09-21T02:14:13.270610+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151615+0000","last_change":"2024-09-21T02:14:14.151615+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151615+0000","last_undegraded":"2024-09-21T02:14:14.151615+0000","last_fullsized":"2024-09-21T02:14:14.151615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.142074+0000","last_change":"2024-09-21T02:14:13.142074+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.142074+0000","last_undegraded":"2024-09-21T02:14:13.142074+0000","last_fullsized":"2024-09-21T02:14:13.142074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139080+0000","last_change":"2024-09-21T02:14:13.139080+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139080+0000","last_undegraded":"2024-09-21T02:14:13.139080+0000","last_fullsized":"2024-09-21T02:14:13.139080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270180+0000","last_change":"2024-09-21T02:14:13.270180+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270180+0000","last_undegraded":"2024-09-21T02:14:13.270180+0000","last_fullsized":"2024-09-21T02:14:13.270180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.138751+0000","last_change":"2024-09-21T02:14:13.138751+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.138751+0000","last_undegraded":"2024-09-21T02:14:13.138751+0000","last_fullsized":"2024-09-21T02:14:13.138751+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.151331+0000","last_change":"2024-09-21T02:14:14.151331+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.151331+0000","last_undegraded":"2024-09-21T02:14:14.151331+0000","last_fullsized":"2024-09-21T02:14:14.151331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152162+0000","last_change":"2024-09-21T02:14:13.152162+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152162+0000","last_undegraded":"2024-09-21T02:14:13.152162+0000","last_fullsized":"2024-09-21T02:14:13.152162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.139561+0000","last_change":"2024-09-21T02:14:13.139561+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.139561+0000","last_undegraded":"2024-09-21T02:14:13.139561+0000","last_fullsized":"2024-09-21T02:14:13.139561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152035+0000","last_change":"2024-09-21T02:14:13.152035+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152035+0000","last_undegraded":"2024-09-21T02:14:13.152035+0000","last_fullsized":"2024-09-21T02:14:13.152035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.153292+0000","last_change":"2024-09-21T02:14:13.153292+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.153292+0000","last_undegraded":"2024-09-21T02:14:13.153292+0000","last_fullsized":"2024-09-21T02:14:13.153292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180922+0000","last_change":"2024-09-21T02:14:15.199669+0000","last_active":"2024-09-21T02:14:17.180922+0000","last_peered":"2024-09-21T02:14:17.180922+0000","last_clean":"2024-09-21T02:14:17.180922+0000","last_became_active":"2024-09-21T02:14:15.198854+0000","last_became_peered":"2024-09-21T02:14:15.198854+0000","last_unstale":"2024-09-21T02:14:17.180922+0000","last_undegraded":"2024-09-21T02:14:17.180922+0000","last_fullsized":"2024-09-21T02:14:17.180922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:19.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.152253+0000","last_change":"2024-09-21T02:14:13.152253+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.152253+0000","last_undegraded":"2024-09-21T02:14:13.152253+0000","last_fullsized":"2024-09-21T02:14:13.152253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179768+0000","last_change":"2024-09-21T02:14:15.200391+0000","last_active":"2024-09-21T02:14:17.179768+0000","last_peered":"2024-09-21T02:14:17.179768+0000","last_clean":"2024-09-21T02:14:17.179768+0000","last_became_active":"2024-09-21T02:14:15.196024+0000","last_became_peered":"2024-09-21T02:14:15.196024+0000","last_unstale":"2024-09-21T02:14:17.179768+0000","last_undegraded":"2024-09-21T02:14:17.179768+0000","last_fullsized":"2024-09-21T02:14:17.179768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:16.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T02:14:13.270179+0000","last_change":"2024-09-21T02:14:13.270179+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:13.270179+0000","last_undegraded":"2024-09-21T02:14:13.270179+0000","last_fullsized":"2024-09-21T02:14:13.270179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179042+0000","last_change":"2024-09-21T02:14:14.180582+0000","last_active":"2024-09-21T02:14:17.179042+0000","last_peered":"2024-09-21T02:14:17.179042+0000","last_clean":"2024-09-21T02:14:17.179042+0000","last_became_active":"2024-09-21T02:14:14.179876+0000","last_became_peered":"2024-09-21T02:14:14.179876+0000","last_unstale":"2024-09-21T02:14:17.179042+0000","last_undegraded":"2024-09-21T02:14:17.179042+0000","last_fullsized":"2024-09-21T02:14:17.179042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:29.774228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:14.134329+0000","last_change":"2024-09-21T02:14:14.134329+0000","last_active":"2024-09-21T02:14:14.134329+0000","last_peered":"2024-09-21T02:14:14.134329+0000","last_clean":"2024-09-21T02:14:14.134329+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.134329+0000","last_undegraded":"2024-09-21T02:14:14.134329+0000","last_fullsized":"2024-09-21T02:14:14.134329+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":3,"reported_epoch":33,"state":"creating+activating","last_fresh":"2024-09-21T02:14:14.152172+0000","last_change":"2024-09-21T02:14:14.152172+0000","last_active":"2024-09-21T02:14:13.123434+0000","last_peered":"2024-09-21T02:14:13.123434+0000","last_clean":"2024-09-21T02:14:13.123434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:14.152172+0000","last_undegraded":"2024-09-21T02:14:14.152172+0000","last_fullsized":"2024-09-21T02:14:14.152172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":126,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T02:14:13.139218+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:13.139218+0000","last_peered":"2024-09-21T02:14:13.139218+0000","last_clean":"2024-09-21T02:14:13.139218+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:13.139218+0000","last_undegraded":"2024-09-21T02:14:13.139218+0000","last_fullsized":"2024-09-21T02:14:13.139218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":722021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36272,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335568,"statfs":{"total":96636764160,"available":96599621632,"internally_reserved":0,"allocated":946176,"data_stored":722021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":146337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:20.159 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-21T02:14:20.159 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.160 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.161 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.162 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.163 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1f is not active+clean 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 33, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:14:14.152085+0000', 'last_change': '2024-09-21T02:14:14.152085+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.152085+0000', 'last_undegraded': '2024-09-21T02:14:14.152085+0000', 'last_fullsized': '2024-09-21T02:14:14.152085+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': ['3', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.164 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1d is not active+clean 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.145869+0000', 'last_change': '2024-09-21T02:14:13.145869+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.145869+0000', 'last_undegraded': '2024-09-21T02:14:13.145869+0000', 'last_fullsized': '2024-09-21T02:14:13.145869+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1b is not active+clean 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.271582+0000', 'last_change': '2024-09-21T02:14:13.271582+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.271582+0000', 'last_undegraded': '2024-09-21T02:14:13.271582+0000', 'last_fullsized': '2024-09-21T02:14:13.271582+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1a is not active+clean 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.271002+0000', 'last_change': '2024-09-21T02:14:13.271002+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.271002+0000', 'last_undegraded': '2024-09-21T02:14:13.271002+0000', 'last_fullsized': '2024-09-21T02:14:13.271002+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.165 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:PG 4.19 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.145287+0000', 'last_change': '2024-09-21T02:14:13.145287+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.145287+0000', 'last_undegraded': '2024-09-21T02:14:13.145287+0000', 'last_fullsized': '2024-09-21T02:14:13.145287+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:PG 4.18 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.270238+0000', 'last_change': '2024-09-21T02:14:13.270238+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.270238+0000', 'last_undegraded': '2024-09-21T02:14:13.270238+0000', 'last_fullsized': '2024-09-21T02:14:13.270238+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:PG 4.17 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.139263+0000', 'last_change': '2024-09-21T02:14:13.139263+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.139263+0000', 'last_undegraded': '2024-09-21T02:14:13.139263+0000', 'last_fullsized': '2024-09-21T02:14:13.139263+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:PG 4.16 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.140792+0000', 'last_change': '2024-09-21T02:14:13.140792+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.140792+0000', 'last_undegraded': '2024-09-21T02:14:13.140792+0000', 'last_fullsized': '2024-09-21T02:14:13.140792+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:PG 4.15 is not active+clean 2024-09-21T02:14:20.166 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 33, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:14:14.151179+0000', 'last_change': '2024-09-21T02:14:14.151179+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.151179+0000', 'last_undegraded': '2024-09-21T02:14:14.151179+0000', 'last_fullsized': '2024-09-21T02:14:14.151179+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': ['3', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 4.14 is not active+clean 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.153737+0000', 'last_change': '2024-09-21T02:14:13.153737+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.153737+0000', 'last_undegraded': '2024-09-21T02:14:13.153737+0000', 'last_fullsized': '2024-09-21T02:14:13.153737+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 4.13 is not active+clean 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.270175+0000', 'last_change': '2024-09-21T02:14:13.270175+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.270175+0000', 'last_undegraded': '2024-09-21T02:14:13.270175+0000', 'last_fullsized': '2024-09-21T02:14:13.270175+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.167 INFO:tasks.ceph.ceph_manager.ceph:PG 4.10 is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.145319+0000', 'last_change': '2024-09-21T02:14:13.145319+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.145319+0000', 'last_undegraded': '2024-09-21T02:14:13.145319+0000', 'last_fullsized': '2024-09-21T02:14:13.145319+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 4.f is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.146177+0000', 'last_change': '2024-09-21T02:14:13.146177+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.146177+0000', 'last_undegraded': '2024-09-21T02:14:13.146177+0000', 'last_fullsized': '2024-09-21T02:14:13.146177+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 4.e is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.270610+0000', 'last_change': '2024-09-21T02:14:13.270610+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.270610+0000', 'last_undegraded': '2024-09-21T02:14:13.270610+0000', 'last_fullsized': '2024-09-21T02:14:13.270610+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 4.d is not active+clean 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 33, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:14:14.151615+0000', 'last_change': '2024-09-21T02:14:14.151615+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.151615+0000', 'last_undegraded': '2024-09-21T02:14:14.151615+0000', 'last_fullsized': '2024-09-21T02:14:14.151615+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:14:20.168 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:PG 4.c is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.142074+0000', 'last_change': '2024-09-21T02:14:13.142074+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.142074+0000', 'last_undegraded': '2024-09-21T02:14:13.142074+0000', 'last_fullsized': '2024-09-21T02:14:13.142074+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:PG 4.b is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.139080+0000', 'last_change': '2024-09-21T02:14:13.139080+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.139080+0000', 'last_undegraded': '2024-09-21T02:14:13.139080+0000', 'last_fullsized': '2024-09-21T02:14:13.139080+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:PG 4.a is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.270180+0000', 'last_change': '2024-09-21T02:14:13.270180+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.270180+0000', 'last_undegraded': '2024-09-21T02:14:13.270180+0000', 'last_fullsized': '2024-09-21T02:14:13.270180+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-21T02:14:20.169 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:PG 4.9 is not active+clean 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.138751+0000', 'last_change': '2024-09-21T02:14:13.138751+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.138751+0000', 'last_undegraded': '2024-09-21T02:14:13.138751+0000', 'last_fullsized': '2024-09-21T02:14:13.138751+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:PG 4.8 is not active+clean 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 33, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:14:14.151331+0000', 'last_change': '2024-09-21T02:14:14.151331+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.151331+0000', 'last_undegraded': '2024-09-21T02:14:14.151331+0000', 'last_fullsized': '2024-09-21T02:14:14.151331+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:PG 4.7 is not active+clean 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.152162+0000', 'last_change': '2024-09-21T02:14:13.152162+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.152162+0000', 'last_undegraded': '2024-09-21T02:14:13.152162+0000', 'last_fullsized': '2024-09-21T02:14:13.152162+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-21T02:14:20.170 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:PG 4.6 is not active+clean 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.139561+0000', 'last_change': '2024-09-21T02:14:13.139561+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.139561+0000', 'last_undegraded': '2024-09-21T02:14:13.139561+0000', 'last_fullsized': '2024-09-21T02:14:13.139561+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:PG 4.4 is not active+clean 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.152035+0000', 'last_change': '2024-09-21T02:14:13.152035+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.152035+0000', 'last_undegraded': '2024-09-21T02:14:13.152035+0000', 'last_fullsized': '2024-09-21T02:14:13.152035+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:PG 4.3 is not active+clean 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.153292+0000', 'last_change': '2024-09-21T02:14:13.153292+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.153292+0000', 'last_undegraded': '2024-09-21T02:14:13.153292+0000', 'last_fullsized': '2024-09-21T02:14:13.153292+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:PG 4.2 is not active+clean 2024-09-21T02:14:20.171 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.152253+0000', 'last_change': '2024-09-21T02:14:13.152253+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.152253+0000', 'last_undegraded': '2024-09-21T02:14:13.152253+0000', 'last_fullsized': '2024-09-21T02:14:13.152253+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1 is not active+clean 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:13.270179+0000', 'last_change': '2024-09-21T02:14:13.270179+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:13.270179+0000', 'last_undegraded': '2024-09-21T02:14:13.270179+0000', 'last_fullsized': '2024-09-21T02:14:13.270179+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:14.134329+0000', 'last_change': '2024-09-21T02:14:14.134329+0000', 'last_active': '2024-09-21T02:14:14.134329+0000', 'last_peered': '2024-09-21T02:14:14.134329+0000', 'last_clean': '2024-09-21T02:14:14.134329+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.134329+0000', 'last_undegraded': '2024-09-21T02:14:14.134329+0000', 'last_fullsized': '2024-09-21T02:14:14.134329+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:14.134329+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:PG 4.5 is not active+clean 2024-09-21T02:14:20.172 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 33, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:14:14.152172+0000', 'last_change': '2024-09-21T02:14:14.152172+0000', 'last_active': '2024-09-21T02:14:13.123434+0000', 'last_peered': '2024-09-21T02:14:13.123434+0000', 'last_clean': '2024-09-21T02:14:13.123434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:14.152172+0000', 'last_undegraded': '2024-09-21T02:14:14.152172+0000', 'last_fullsized': '2024-09-21T02:14:14.152172+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:13.123434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': ['3', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:14:20.173 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 -- 172.21.15.170:0/605235204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70200b5410 msgr2=0x7f702006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 --2- 172.21.15.170:0/605235204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70200b5410 0x7f702006fa30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f70200c7150 tx=0x7f701001c800 comp rx=0 tx=0).stop 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 -- 172.21.15.170:0/605235204 shutdown_connections 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 --2- 172.21.15.170:0/605235204 >> v2:172.21.15.191:3300/0 conn(0x7f70200cd210 0x7f70200b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 --2- 172.21.15.170:0/605235204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70200b5410 0x7f702006fa30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 -- 172.21.15.170:0/605235204 >> 172.21.15.170:0/605235204 conn(0x7f70200b1c40 msgr2=0x7f70200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 -- 172.21.15.170:0/605235204 shutdown_connections 2024-09-21T02:14:20.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.336+0000 7f7024b51640 1 -- 172.21.15.170:0/605235204 wait complete. 2024-09-21T02:14:20.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 Processor -- start 2024-09-21T02:14:20.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 -- start start 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70200b2f40 0x7f70201fa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70200b5410 0x7f7020154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 0x7f702015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7020169fd0 con 0x7f70201566c0 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7020169e50 con 0x7f70200b5410 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f702016a150 con 0x7f70200b2f40 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 0x7f702015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 0x7f702015c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47962/0 (socket says 172.21.15.170:47962) 2024-09-21T02:14:20.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 -- 172.21.15.170:0/2493076790 learned_addr learned my addr 172.21.15.170:0/2493076790 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701e59c640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70200b2f40 0x7f70201fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701dd9b640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70200b5410 0x7f7020154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 -- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70200b2f40 msgr2=0x7f70201fa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70200b2f40 0x7f70201fa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 -- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70200b5410 msgr2=0x7f7020154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70200b5410 0x7f7020154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 -- 172.21.15.170:0/2493076790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702009d890 con 0x7f70201566c0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701dd9b640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70200b5410 0x7f7020154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f701ed9d640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 0x7f702015c1d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f701800ca60 tx=0x7f701800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018007d90 con 0x7f70201566c0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7018004c20 con 0x7f70201566c0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018005280 con 0x7f70201566c0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702009db70 con 0x7f70201566c0 2024-09-21T02:14:20.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.340+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201f9dd0 con 0x7f70201566c0 2024-09-21T02:14:20.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.344+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702006fa30 con 0x7f70201566c0 2024-09-21T02:14:20.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.344+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7018005420 con 0x7f70201566c0 2024-09-21T02:14:20.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.344+0000 7f700effd640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7000073200 0x7f7000075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.344+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7018094a90 con 0x7f70201566c0 2024-09-21T02:14:20.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.344+0000 7f701e59c640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7000073200 0x7f7000075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.344+0000 7f701e59c640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7000073200 0x7f7000075710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70100af000 tx=0x7f701001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.348+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7018014030 con 0x7f70201566c0 2024-09-21T02:14:20.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.580+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f702009d5f0 con 0x7f70201566c0 2024-09-21T02:14:20.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f700effd640 1 -- 172.21.15.170:0/2493076790 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1807 (secure 0 0 0) 0x7f701805fa10 con 0x7f70201566c0 2024-09-21T02:14:20.586 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:20.586 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1726884793,"num_in_osds":8,"osd_in_since":1726884783,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":52},{"state_name":"creating+peering","count":22},{"state_name":"active+clean","count":18},{"state_name":"creating+activating","count":5}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":251019264,"bytes_avail":772843094016,"bytes_total":773094113280,"unknown_pgs_ratio":0.53608244657516479,"inactive_pgs_ratio":0.27835050225257874,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":17,"btime":"2024-09-21T02:14:18:191863+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4914}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-21T02:14:14.882101+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7000073200 msgr2=0x7f7000075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7000073200 0x7f7000075710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70100af000 tx=0x7f701001b040 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 msgr2=0x7f702015c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 0x7f702015c1d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f701800ca60 tx=0x7f701800cf20 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 shutdown_connections 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7000073200 0x7f7000075710 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70201566c0 0x7f702015c1d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70200b5410 0x7f7020154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 --2- 172.21.15.170:0/2493076790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70200b2f40 0x7f70201fa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 >> 172.21.15.170:0/2493076790 conn(0x7f70200b1c40 msgr2=0x7f7020159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 shutdown_connections 2024-09-21T02:14:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.584+0000 7f7024b51640 1 -- 172.21.15.170:0/2493076790 wait complete. 2024-09-21T02:14:20.606 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T02:14:20.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a443af640 1 -- 172.21.15.170:0/4062034904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a28004810 msgr2=0x7f4a28004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a443af640 1 --2- 172.21.15.170:0/4062034904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a28004810 0x7f4a28004c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c260 tx=0x7f4a2c01f6c0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a4294c640 1 -- 172.21.15.170:0/4062034904 reap_dead start 2024-09-21T02:14:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a443af640 1 -- 172.21.15.170:0/4062034904 shutdown_connections 2024-09-21T02:14:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a443af640 1 -- 172.21.15.170:0/4062034904 >> 172.21.15.170:0/4062034904 conn(0x7f4a3c0cee30 msgr2=0x7f4a3c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a443af640 1 -- 172.21.15.170:0/4062034904 shutdown_connections 2024-09-21T02:14:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.756+0000 7f4a443af640 1 -- 172.21.15.170:0/4062034904 wait complete. 2024-09-21T02:14:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 Processor -- start 2024-09-21T02:14:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 -- start start 2024-09-21T02:14:20.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a28004810 0x7f4a3c1f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 0x7f4a3c15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c0b0a80 0x7f4a3c0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c0d1620 con 0x7f4a3c06f220 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c0d14a0 con 0x7f4a3c0b0a80 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c0d17a0 con 0x7f4a28004810 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 0x7f4a3c15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 0x7f4a3c15e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47994/0 (socket says 172.21.15.170:47994) 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 -- 172.21.15.170:0/2442923497 learned_addr learned my addr 172.21.15.170:0/2442923497 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4294c640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c0b0a80 0x7f4a3c0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4214b640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a28004810 0x7f4a3c1f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 -- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a28004810 msgr2=0x7f4a3c1f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a28004810 0x7f4a3c1f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 -- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c0b0a80 msgr2=0x7f4a3c0b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c0b0a80 0x7f4a3c0b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 -- 172.21.15.170:0/2442923497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c15e7f0 con 0x7f4a3c06f220 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a4194a640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 0x7f4a3c15e2b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c01ddc0 tx=0x7f4a2c001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0bf070 con 0x7f4a3c06f220 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a2c006530 con 0x7f4a3c06f220 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c0b13d0 con 0x7f4a3c06f220 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0b3070 con 0x7f4a3c06f220 2024-09-21T02:14:20.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.760+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c156980 con 0x7f4a3c06f220 2024-09-21T02:14:20.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.764+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c0d1d00 con 0x7f4a3c06f220 2024-09-21T02:14:20.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.764+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a2c0c2070 con 0x7f4a3c06f220 2024-09-21T02:14:20.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.764+0000 7f4a32ffd640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a20073200 0x7f4a20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:20.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.764+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4a2c131ef0 con 0x7f4a3c06f220 2024-09-21T02:14:20.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.764+0000 7f4a4214b640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a20073200 0x7f4a20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:20.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.764+0000 7f4a4214b640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a20073200 0x7f4a20075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a340061f0 tx=0x7f4a34006dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:20.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.768+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a2c0c9070 con 0x7f4a3c06f220 2024-09-21T02:14:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.936+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a3c1f6840 con 0x7f4a20073200 2024-09-21T02:14:20.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.952+0000 7f4a32ffd640 1 -- 172.21.15.170:0/2442923497 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251956 (secure 0 0 0) 0x7f4a3c1f6840 con 0x7f4a20073200 2024-09-21T02:14:20.957 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:20.958 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a20073200 msgr2=0x7f4a20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a20073200 0x7f4a20075710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a340061f0 tx=0x7f4a34006dd0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 msgr2=0x7f4a3c15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 0x7f4a3c15e2b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c01ddc0 tx=0x7f4a2c001d00 comp rx=0 tx=0).stop 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 shutdown_connections 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a20073200 0x7f4a20075710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c0b0a80 0x7f4a3c0b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c06f220 0x7f4a3c15e2b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 --2- 172.21.15.170:0/2442923497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a28004810 0x7f4a3c1f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 >> 172.21.15.170:0/2442923497 conn(0x7f4a3c0cee30 msgr2=0x7f4a3c0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 shutdown_connections 2024-09-21T02:14:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:20.956+0000 7f4a443af640 1 -- 172.21.15.170:0/2442923497 wait complete. 2024-09-21T02:14:20.979 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-21T02:14:20.884067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245452,"kb_used_data":4252,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729268,"statfs":{"total":773094113280,"available":772842770432,"internally_reserved":0,"allocated":4354048,"data_stored":2387280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.735761"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182931+0000","last_change":"2024-09-21T02:14:15.183292+0000","last_active":"2024-09-21T02:14:17.182931+0000","last_peered":"2024-09-21T02:14:17.182931+0000","last_clean":"2024-09-21T02:14:17.182931+0000","last_became_active":"2024-09-21T02:14:15.182677+0000","last_became_peered":"2024-09-21T02:14:15.182677+0000","last_unstale":"2024-09-21T02:14:17.182931+0000","last_undegraded":"2024-09-21T02:14:17.182931+0000","last_fullsized":"2024-09-21T02:14:17.182931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:01.672457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167597+0000","last_change":"2024-09-21T02:14:15.214151+0000","last_active":"2024-09-21T02:14:16.167597+0000","last_peered":"2024-09-21T02:14:16.167597+0000","last_clean":"2024-09-21T02:14:16.167597+0000","last_became_active":"2024-09-21T02:14:15.213317+0000","last_became_peered":"2024-09-21T02:14:15.213317+0000","last_unstale":"2024-09-21T02:14:16.167597+0000","last_undegraded":"2024-09-21T02:14:16.167597+0000","last_fullsized":"2024-09-21T02:14:16.167597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:48.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180231+0000","last_change":"2024-09-21T02:14:15.183780+0000","last_active":"2024-09-21T02:14:17.180231+0000","last_peered":"2024-09-21T02:14:17.180231+0000","last_clean":"2024-09-21T02:14:17.180231+0000","last_became_active":"2024-09-21T02:14:15.183335+0000","last_became_peered":"2024-09-21T02:14:15.183335+0000","last_unstale":"2024-09-21T02:14:17.180231+0000","last_undegraded":"2024-09-21T02:14:17.180231+0000","last_fullsized":"2024-09-21T02:14:17.180231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:46.068943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180103+0000","last_change":"2024-09-21T02:14:15.221703+0000","last_active":"2024-09-21T02:14:17.180103+0000","last_peered":"2024-09-21T02:14:17.180103+0000","last_clean":"2024-09-21T02:14:17.180103+0000","last_became_active":"2024-09-21T02:14:15.219448+0000","last_became_peered":"2024-09-21T02:14:15.219448+0000","last_unstale":"2024-09-21T02:14:17.180103+0000","last_undegraded":"2024-09-21T02:14:17.180103+0000","last_fullsized":"2024-09-21T02:14:17.180103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:27.282489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167414+0000","last_change":"2024-09-21T02:14:15.204321+0000","last_active":"2024-09-21T02:14:16.167414+0000","last_peered":"2024-09-21T02:14:16.167414+0000","last_clean":"2024-09-21T02:14:16.167414+0000","last_became_active":"2024-09-21T02:14:15.203516+0000","last_became_peered":"2024-09-21T02:14:15.203516+0000","last_unstale":"2024-09-21T02:14:16.167414+0000","last_undegraded":"2024-09-21T02:14:16.167414+0000","last_fullsized":"2024-09-21T02:14:16.167414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:03.903502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168814+0000","last_change":"2024-09-21T02:14:15.210136+0000","last_active":"2024-09-21T02:14:16.168814+0000","last_peered":"2024-09-21T02:14:16.168814+0000","last_clean":"2024-09-21T02:14:16.168814+0000","last_became_active":"2024-09-21T02:14:15.209438+0000","last_became_peered":"2024-09-21T02:14:15.209438+0000","last_unstale":"2024-09-21T02:14:16.168814+0000","last_undegraded":"2024-09-21T02:14:16.168814+0000","last_fullsized":"2024-09-21T02:14:16.168814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:00.474590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181828+0000","last_change":"2024-09-21T02:14:15.194359+0000","last_active":"2024-09-21T02:14:17.181828+0000","last_peered":"2024-09-21T02:14:17.181828+0000","last_clean":"2024-09-21T02:14:17.181828+0000","last_became_active":"2024-09-21T02:14:15.193926+0000","last_became_peered":"2024-09-21T02:14:15.193926+0000","last_unstale":"2024-09-21T02:14:17.181828+0000","last_undegraded":"2024-09-21T02:14:17.181828+0000","last_fullsized":"2024-09-21T02:14:17.181828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:26.736916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181780+0000","last_change":"2024-09-21T02:14:15.198361+0000","last_active":"2024-09-21T02:14:17.181780+0000","last_peered":"2024-09-21T02:14:17.181780+0000","last_clean":"2024-09-21T02:14:17.181780+0000","last_became_active":"2024-09-21T02:14:15.198032+0000","last_became_peered":"2024-09-21T02:14:15.198032+0000","last_unstale":"2024-09-21T02:14:17.181780+0000","last_undegraded":"2024-09-21T02:14:17.181780+0000","last_fullsized":"2024-09-21T02:14:17.181780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:25.771631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181300+0000","last_change":"2024-09-21T02:14:15.191098+0000","last_active":"2024-09-21T02:14:17.181300+0000","last_peered":"2024-09-21T02:14:17.181300+0000","last_clean":"2024-09-21T02:14:17.181300+0000","last_became_active":"2024-09-21T02:14:15.190688+0000","last_became_peered":"2024-09-21T02:14:15.190688+0000","last_unstale":"2024-09-21T02:14:17.181300+0000","last_undegraded":"2024-09-21T02:14:17.181300+0000","last_fullsized":"2024-09-21T02:14:17.181300+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:12.511846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182169+0000","last_change":"2024-09-21T02:14:15.218508+0000","last_active":"2024-09-21T02:14:17.182169+0000","last_peered":"2024-09-21T02:14:17.182169+0000","last_clean":"2024-09-21T02:14:17.182169+0000","last_became_active":"2024-09-21T02:14:15.217625+0000","last_became_peered":"2024-09-21T02:14:15.217625+0000","last_unstale":"2024-09-21T02:14:17.182169+0000","last_undegraded":"2024-09-21T02:14:17.182169+0000","last_fullsized":"2024-09-21T02:14:17.182169+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.766624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162156+0000","last_change":"2024-09-21T02:14:15.220694+0000","last_active":"2024-09-21T02:14:16.162156+0000","last_peered":"2024-09-21T02:14:16.162156+0000","last_clean":"2024-09-21T02:14:16.162156+0000","last_became_active":"2024-09-21T02:14:15.220312+0000","last_became_peered":"2024-09-21T02:14:15.220312+0000","last_unstale":"2024-09-21T02:14:16.162156+0000","last_undegraded":"2024-09-21T02:14:16.162156+0000","last_fullsized":"2024-09-21T02:14:16.162156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:42.304704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181094+0000","last_change":"2024-09-21T02:14:15.221471+0000","last_active":"2024-09-21T02:14:17.181094+0000","last_peered":"2024-09-21T02:14:17.181094+0000","last_clean":"2024-09-21T02:14:17.181094+0000","last_became_active":"2024-09-21T02:14:15.221130+0000","last_became_peered":"2024-09-21T02:14:15.221130+0000","last_unstale":"2024-09-21T02:14:17.181094+0000","last_undegraded":"2024-09-21T02:14:17.181094+0000","last_fullsized":"2024-09-21T02:14:17.181094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:38.704450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180801+0000","last_change":"2024-09-21T02:14:15.218389+0000","last_active":"2024-09-21T02:14:17.180801+0000","last_peered":"2024-09-21T02:14:17.180801+0000","last_clean":"2024-09-21T02:14:17.180801+0000","last_became_active":"2024-09-21T02:14:15.217790+0000","last_became_peered":"2024-09-21T02:14:15.217790+0000","last_unstale":"2024-09-21T02:14:17.180801+0000","last_undegraded":"2024-09-21T02:14:17.180801+0000","last_fullsized":"2024-09-21T02:14:17.180801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:13.765471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179430+0000","last_change":"2024-09-21T02:14:15.203118+0000","last_active":"2024-09-21T02:14:17.179430+0000","last_peered":"2024-09-21T02:14:17.179430+0000","last_clean":"2024-09-21T02:14:17.179430+0000","last_became_active":"2024-09-21T02:14:15.202380+0000","last_became_peered":"2024-09-21T02:14:15.202380+0000","last_unstale":"2024-09-21T02:14:17.179430+0000","last_undegraded":"2024-09-21T02:14:17.179430+0000","last_fullsized":"2024-09-21T02:14:17.179430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:30.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178955+0000","last_change":"2024-09-21T02:14:15.183321+0000","last_active":"2024-09-21T02:14:17.178955+0000","last_peered":"2024-09-21T02:14:17.178955+0000","last_clean":"2024-09-21T02:14:17.178955+0000","last_became_active":"2024-09-21T02:14:15.182939+0000","last_became_peered":"2024-09-21T02:14:15.182939+0000","last_unstale":"2024-09-21T02:14:17.178955+0000","last_undegraded":"2024-09-21T02:14:17.178955+0000","last_fullsized":"2024-09-21T02:14:17.178955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:01.672457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178558+0000","last_change":"2024-09-21T02:14:15.196638+0000","last_active":"2024-09-21T02:14:17.178558+0000","last_peered":"2024-09-21T02:14:17.178558+0000","last_clean":"2024-09-21T02:14:17.178558+0000","last_became_active":"2024-09-21T02:14:15.195665+0000","last_became_peered":"2024-09-21T02:14:15.195665+0000","last_unstale":"2024-09-21T02:14:17.178558+0000","last_undegraded":"2024-09-21T02:14:17.178558+0000","last_fullsized":"2024-09-21T02:14:17.178558+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:03.893645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162078+0000","last_change":"2024-09-21T02:14:15.190824+0000","last_active":"2024-09-21T02:14:16.162078+0000","last_peered":"2024-09-21T02:14:16.162078+0000","last_clean":"2024-09-21T02:14:16.162078+0000","last_became_active":"2024-09-21T02:14:15.190212+0000","last_became_peered":"2024-09-21T02:14:15.190212+0000","last_unstale":"2024-09-21T02:14:16.162078+0000","last_undegraded":"2024-09-21T02:14:16.162078+0000","last_fullsized":"2024-09-21T02:14:16.162078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:17.761345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168548+0000","last_change":"2024-09-21T02:14:15.222180+0000","last_active":"2024-09-21T02:14:16.168548+0000","last_peered":"2024-09-21T02:14:16.168548+0000","last_clean":"2024-09-21T02:14:16.168548+0000","last_became_active":"2024-09-21T02:14:15.221412+0000","last_became_peered":"2024-09-21T02:14:15.221412+0000","last_unstale":"2024-09-21T02:14:16.168548+0000","last_undegraded":"2024-09-21T02:14:16.168548+0000","last_fullsized":"2024-09-21T02:14:16.168548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:29.103971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.169541+0000","last_change":"2024-09-21T02:14:15.211355+0000","last_active":"2024-09-21T02:14:16.169541+0000","last_peered":"2024-09-21T02:14:16.169541+0000","last_clean":"2024-09-21T02:14:16.169541+0000","last_became_active":"2024-09-21T02:14:15.210408+0000","last_became_peered":"2024-09-21T02:14:15.210408+0000","last_unstale":"2024-09-21T02:14:16.169541+0000","last_undegraded":"2024-09-21T02:14:16.169541+0000","last_fullsized":"2024-09-21T02:14:16.169541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:11.338884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181159+0000","last_change":"2024-09-21T02:14:15.226510+0000","last_active":"2024-09-21T02:14:17.181159+0000","last_peered":"2024-09-21T02:14:17.181159+0000","last_clean":"2024-09-21T02:14:17.181159+0000","last_became_active":"2024-09-21T02:14:15.225677+0000","last_became_peered":"2024-09-21T02:14:15.225677+0000","last_unstale":"2024-09-21T02:14:17.181159+0000","last_undegraded":"2024-09-21T02:14:17.181159+0000","last_fullsized":"2024-09-21T02:14:17.181159+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:49.578216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179469+0000","last_change":"2024-09-21T02:14:15.198346+0000","last_active":"2024-09-21T02:14:17.179469+0000","last_peered":"2024-09-21T02:14:17.179469+0000","last_clean":"2024-09-21T02:14:17.179469+0000","last_became_active":"2024-09-21T02:14:15.197977+0000","last_became_peered":"2024-09-21T02:14:15.197977+0000","last_unstale":"2024-09-21T02:14:17.179469+0000","last_undegraded":"2024-09-21T02:14:17.179469+0000","last_fullsized":"2024-09-21T02:14:17.179469+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:00.074804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.183680+0000","last_change":"2024-09-21T02:14:15.213671+0000","last_active":"2024-09-21T02:14:17.183680+0000","last_peered":"2024-09-21T02:14:17.183680+0000","last_clean":"2024-09-21T02:14:17.183680+0000","last_became_active":"2024-09-21T02:14:15.213347+0000","last_became_peered":"2024-09-21T02:14:15.213347+0000","last_unstale":"2024-09-21T02:14:17.183680+0000","last_undegraded":"2024-09-21T02:14:17.183680+0000","last_fullsized":"2024-09-21T02:14:17.183680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:03.903502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178417+0000","last_change":"2024-09-21T02:14:15.194252+0000","last_active":"2024-09-21T02:14:17.178417+0000","last_peered":"2024-09-21T02:14:17.178417+0000","last_clean":"2024-09-21T02:14:17.178417+0000","last_became_active":"2024-09-21T02:14:15.191411+0000","last_became_peered":"2024-09-21T02:14:15.191411+0000","last_unstale":"2024-09-21T02:14:17.178417+0000","last_undegraded":"2024-09-21T02:14:17.178417+0000","last_fullsized":"2024-09-21T02:14:17.178417+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:55.467722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181995+0000","last_change":"2024-09-21T02:14:15.219221+0000","last_active":"2024-09-21T02:14:17.181995+0000","last_peered":"2024-09-21T02:14:17.181995+0000","last_clean":"2024-09-21T02:14:17.181995+0000","last_became_active":"2024-09-21T02:14:15.218863+0000","last_became_peered":"2024-09-21T02:14:15.218863+0000","last_unstale":"2024-09-21T02:14:17.181995+0000","last_undegraded":"2024-09-21T02:14:17.181995+0000","last_fullsized":"2024-09-21T02:14:17.181995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:46.551654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180846+0000","last_change":"2024-09-21T02:14:15.192842+0000","last_active":"2024-09-21T02:14:17.180846+0000","last_peered":"2024-09-21T02:14:17.180846+0000","last_clean":"2024-09-21T02:14:17.180846+0000","last_became_active":"2024-09-21T02:14:15.190820+0000","last_became_peered":"2024-09-21T02:14:15.190820+0000","last_unstale":"2024-09-21T02:14:17.180846+0000","last_undegraded":"2024-09-21T02:14:17.180846+0000","last_fullsized":"2024-09-21T02:14:17.180846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:07.544915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182194+0000","last_change":"2024-09-21T02:14:15.212128+0000","last_active":"2024-09-21T02:14:17.182194+0000","last_peered":"2024-09-21T02:14:17.182194+0000","last_clean":"2024-09-21T02:14:17.182194+0000","last_became_active":"2024-09-21T02:14:15.211547+0000","last_became_peered":"2024-09-21T02:14:15.211547+0000","last_unstale":"2024-09-21T02:14:17.182194+0000","last_undegraded":"2024-09-21T02:14:17.182194+0000","last_fullsized":"2024-09-21T02:14:17.182194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:20.509138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167938+0000","last_change":"2024-09-21T02:14:15.172726+0000","last_active":"2024-09-21T02:14:16.167938+0000","last_peered":"2024-09-21T02:14:16.167938+0000","last_clean":"2024-09-21T02:14:16.167938+0000","last_became_active":"2024-09-21T02:14:15.171830+0000","last_became_peered":"2024-09-21T02:14:15.171830+0000","last_unstale":"2024-09-21T02:14:16.167938+0000","last_undegraded":"2024-09-21T02:14:16.167938+0000","last_fullsized":"2024-09-21T02:14:16.167938+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:11.129900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.185631+0000","last_change":"2024-09-21T02:14:15.227725+0000","last_active":"2024-09-21T02:14:17.185631+0000","last_peered":"2024-09-21T02:14:17.185631+0000","last_clean":"2024-09-21T02:14:17.185631+0000","last_became_active":"2024-09-21T02:14:15.227445+0000","last_became_peered":"2024-09-21T02:14:15.227445+0000","last_unstale":"2024-09-21T02:14:17.185631+0000","last_undegraded":"2024-09-21T02:14:17.185631+0000","last_fullsized":"2024-09-21T02:14:17.185631+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:11.129900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179092+0000","last_change":"2024-09-21T02:14:15.189074+0000","last_active":"2024-09-21T02:14:17.179092+0000","last_peered":"2024-09-21T02:14:17.179092+0000","last_clean":"2024-09-21T02:14:17.179092+0000","last_became_active":"2024-09-21T02:14:15.188662+0000","last_became_peered":"2024-09-21T02:14:15.188662+0000","last_unstale":"2024-09-21T02:14:17.179092+0000","last_undegraded":"2024-09-21T02:14:17.179092+0000","last_fullsized":"2024-09-21T02:14:17.179092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:32.044091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179152+0000","last_change":"2024-09-21T02:14:15.171330+0000","last_active":"2024-09-21T02:14:17.179152+0000","last_peered":"2024-09-21T02:14:17.179152+0000","last_clean":"2024-09-21T02:14:17.179152+0000","last_became_active":"2024-09-21T02:14:15.170440+0000","last_became_peered":"2024-09-21T02:14:15.170440+0000","last_unstale":"2024-09-21T02:14:17.179152+0000","last_undegraded":"2024-09-21T02:14:17.179152+0000","last_fullsized":"2024-09-21T02:14:17.179152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:03.903502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.183445+0000","last_change":"2024-09-21T02:14:15.210382+0000","last_active":"2024-09-21T02:14:17.183445+0000","last_peered":"2024-09-21T02:14:17.183445+0000","last_clean":"2024-09-21T02:14:17.183445+0000","last_became_active":"2024-09-21T02:14:15.210082+0000","last_became_peered":"2024-09-21T02:14:15.210082+0000","last_unstale":"2024-09-21T02:14:17.183445+0000","last_undegraded":"2024-09-21T02:14:17.183445+0000","last_fullsized":"2024-09-21T02:14:17.183445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:03.893645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161967+0000","last_change":"2024-09-21T02:14:15.185248+0000","last_active":"2024-09-21T02:14:16.161967+0000","last_peered":"2024-09-21T02:14:16.161967+0000","last_clean":"2024-09-21T02:14:16.161967+0000","last_became_active":"2024-09-21T02:14:15.184881+0000","last_became_peered":"2024-09-21T02:14:15.184881+0000","last_unstale":"2024-09-21T02:14:16.161967+0000","last_undegraded":"2024-09-21T02:14:16.161967+0000","last_fullsized":"2024-09-21T02:14:16.161967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:25.642713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181083+0000","last_change":"2024-09-21T02:14:15.197089+0000","last_active":"2024-09-21T02:14:17.181083+0000","last_peered":"2024-09-21T02:14:17.181083+0000","last_clean":"2024-09-21T02:14:17.181083+0000","last_became_active":"2024-09-21T02:14:15.196068+0000","last_became_peered":"2024-09-21T02:14:15.196068+0000","last_unstale":"2024-09-21T02:14:17.181083+0000","last_undegraded":"2024-09-21T02:14:17.181083+0000","last_fullsized":"2024-09-21T02:14:17.181083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:46.961964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162311+0000","last_change":"2024-09-21T02:14:14.162293+0000","last_active":"2024-09-21T02:14:16.162311+0000","last_peered":"2024-09-21T02:14:16.162311+0000","last_clean":"2024-09-21T02:14:16.162311+0000","last_became_active":"2024-09-21T02:14:14.161294+0000","last_became_peered":"2024-09-21T02:14:14.161294+0000","last_unstale":"2024-09-21T02:14:16.162311+0000","last_undegraded":"2024-09-21T02:14:16.162311+0000","last_fullsized":"2024-09-21T02:14:16.162311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:31.033196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.183081+0000","last_change":"2024-09-21T02:14:15.199491+0000","last_active":"2024-09-21T02:14:17.183081+0000","last_peered":"2024-09-21T02:14:17.183081+0000","last_clean":"2024-09-21T02:14:17.183081+0000","last_became_active":"2024-09-21T02:14:15.199000+0000","last_became_peered":"2024-09-21T02:14:15.199000+0000","last_unstale":"2024-09-21T02:14:17.183081+0000","last_undegraded":"2024-09-21T02:14:17.183081+0000","last_fullsized":"2024-09-21T02:14:17.183081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:32.044091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182364+0000","last_change":"2024-09-21T02:14:14.180675+0000","last_active":"2024-09-21T02:14:17.182364+0000","last_peered":"2024-09-21T02:14:17.182364+0000","last_clean":"2024-09-21T02:14:17.182364+0000","last_became_active":"2024-09-21T02:14:14.179791+0000","last_became_peered":"2024-09-21T02:14:14.179791+0000","last_unstale":"2024-09-21T02:14:17.182364+0000","last_undegraded":"2024-09-21T02:14:17.182364+0000","last_fullsized":"2024-09-21T02:14:17.182364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:47.635791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182358+0000","last_change":"2024-09-21T02:14:15.222639+0000","last_active":"2024-09-21T02:14:17.182358+0000","last_peered":"2024-09-21T02:14:17.182358+0000","last_clean":"2024-09-21T02:14:17.182358+0000","last_became_active":"2024-09-21T02:14:15.222297+0000","last_became_peered":"2024-09-21T02:14:15.222297+0000","last_unstale":"2024-09-21T02:14:17.182358+0000","last_undegraded":"2024-09-21T02:14:17.182358+0000","last_fullsized":"2024-09-21T02:14:17.182358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:25.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.170956+0000","last_change":"2024-09-21T02:14:14.195324+0000","last_active":"2024-09-21T02:14:16.170956+0000","last_peered":"2024-09-21T02:14:16.170956+0000","last_clean":"2024-09-21T02:14:16.170956+0000","last_became_active":"2024-09-21T02:14:14.194573+0000","last_became_peered":"2024-09-21T02:14:14.194573+0000","last_unstale":"2024-09-21T02:14:16.170956+0000","last_undegraded":"2024-09-21T02:14:16.170956+0000","last_fullsized":"2024-09-21T02:14:16.170956+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:00.785055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181257+0000","last_change":"2024-09-21T02:14:15.201013+0000","last_active":"2024-09-21T02:14:17.181257+0000","last_peered":"2024-09-21T02:14:17.181257+0000","last_clean":"2024-09-21T02:14:17.181257+0000","last_became_active":"2024-09-21T02:14:15.199797+0000","last_became_peered":"2024-09-21T02:14:15.199797+0000","last_unstale":"2024-09-21T02:14:17.181257+0000","last_undegraded":"2024-09-21T02:14:17.181257+0000","last_fullsized":"2024-09-21T02:14:17.181257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:33.202747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180405+0000","last_change":"2024-09-21T02:14:14.195211+0000","last_active":"2024-09-21T02:14:17.180405+0000","last_peered":"2024-09-21T02:14:17.180405+0000","last_clean":"2024-09-21T02:14:17.180405+0000","last_became_active":"2024-09-21T02:14:14.194442+0000","last_became_peered":"2024-09-21T02:14:14.194442+0000","last_unstale":"2024-09-21T02:14:17.180405+0000","last_undegraded":"2024-09-21T02:14:17.180405+0000","last_fullsized":"2024-09-21T02:14:17.180405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:22.063757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179287+0000","last_change":"2024-09-21T02:14:15.175120+0000","last_active":"2024-09-21T02:14:17.179287+0000","last_peered":"2024-09-21T02:14:17.179287+0000","last_clean":"2024-09-21T02:14:17.179287+0000","last_became_active":"2024-09-21T02:14:15.174114+0000","last_became_peered":"2024-09-21T02:14:15.174114+0000","last_unstale":"2024-09-21T02:14:17.179287+0000","last_undegraded":"2024-09-21T02:14:17.179287+0000","last_fullsized":"2024-09-21T02:14:17.179287+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:57.464186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182141+0000","last_change":"2024-09-21T02:14:14.185340+0000","last_active":"2024-09-21T02:14:17.182141+0000","last_peered":"2024-09-21T02:14:17.182141+0000","last_clean":"2024-09-21T02:14:17.182141+0000","last_became_active":"2024-09-21T02:14:14.184685+0000","last_became_peered":"2024-09-21T02:14:14.184685+0000","last_unstale":"2024-09-21T02:14:17.182141+0000","last_undegraded":"2024-09-21T02:14:17.182141+0000","last_fullsized":"2024-09-21T02:14:17.182141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:08.234402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181931+0000","last_change":"2024-09-21T02:14:15.184350+0000","last_active":"2024-09-21T02:14:17.181931+0000","last_peered":"2024-09-21T02:14:17.181931+0000","last_clean":"2024-09-21T02:14:17.181931+0000","last_became_active":"2024-09-21T02:14:15.183887+0000","last_became_peered":"2024-09-21T02:14:15.183887+0000","last_unstale":"2024-09-21T02:14:17.181931+0000","last_undegraded":"2024-09-21T02:14:17.181931+0000","last_fullsized":"2024-09-21T02:14:17.181931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:12.109910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182606+0000","last_change":"2024-09-21T02:14:14.169524+0000","last_active":"2024-09-21T02:14:17.182606+0000","last_peered":"2024-09-21T02:14:17.182606+0000","last_clean":"2024-09-21T02:14:17.182606+0000","last_became_active":"2024-09-21T02:14:14.169203+0000","last_became_peered":"2024-09-21T02:14:14.169203+0000","last_unstale":"2024-09-21T02:14:17.182606+0000","last_undegraded":"2024-09-21T02:14:17.182606+0000","last_fullsized":"2024-09-21T02:14:17.182606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:59.463695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179448+0000","last_change":"2024-09-21T02:14:15.184514+0000","last_active":"2024-09-21T02:14:17.179448+0000","last_peered":"2024-09-21T02:14:17.179448+0000","last_clean":"2024-09-21T02:14:17.179448+0000","last_became_active":"2024-09-21T02:14:15.183053+0000","last_became_peered":"2024-09-21T02:14:15.183053+0000","last_unstale":"2024-09-21T02:14:17.179448+0000","last_undegraded":"2024-09-21T02:14:17.179448+0000","last_fullsized":"2024-09-21T02:14:17.179448+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:56.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168926+0000","last_change":"2024-09-21T02:14:14.195177+0000","last_active":"2024-09-21T02:14:16.168926+0000","last_peered":"2024-09-21T02:14:16.168926+0000","last_clean":"2024-09-21T02:14:16.168926+0000","last_became_active":"2024-09-21T02:14:14.194325+0000","last_became_peered":"2024-09-21T02:14:14.194325+0000","last_unstale":"2024-09-21T02:14:16.168926+0000","last_undegraded":"2024-09-21T02:14:16.168926+0000","last_fullsized":"2024-09-21T02:14:16.168926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:47.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179304+0000","last_change":"2024-09-21T02:14:15.188344+0000","last_active":"2024-09-21T02:14:17.179304+0000","last_peered":"2024-09-21T02:14:17.179304+0000","last_clean":"2024-09-21T02:14:17.179304+0000","last_became_active":"2024-09-21T02:14:15.187467+0000","last_became_peered":"2024-09-21T02:14:15.187467+0000","last_unstale":"2024-09-21T02:14:17.179304+0000","last_undegraded":"2024-09-21T02:14:17.179304+0000","last_fullsized":"2024-09-21T02:14:17.179304+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:22.010859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182170+0000","last_change":"2024-09-21T02:14:14.188869+0000","last_active":"2024-09-21T02:14:17.182170+0000","last_peered":"2024-09-21T02:14:17.182170+0000","last_clean":"2024-09-21T02:14:17.182170+0000","last_became_active":"2024-09-21T02:14:14.188597+0000","last_became_peered":"2024-09-21T02:14:14.188597+0000","last_unstale":"2024-09-21T02:14:17.182170+0000","last_undegraded":"2024-09-21T02:14:17.182170+0000","last_fullsized":"2024-09-21T02:14:17.182170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:39.147803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162228+0000","last_change":"2024-09-21T02:14:15.201068+0000","last_active":"2024-09-21T02:14:16.162228+0000","last_peered":"2024-09-21T02:14:16.162228+0000","last_clean":"2024-09-21T02:14:16.162228+0000","last_became_active":"2024-09-21T02:14:15.200639+0000","last_became_peered":"2024-09-21T02:14:15.200639+0000","last_unstale":"2024-09-21T02:14:16.162228+0000","last_undegraded":"2024-09-21T02:14:16.162228+0000","last_fullsized":"2024-09-21T02:14:16.162228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:56.023114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167676+0000","last_change":"2024-09-21T02:14:14.172476+0000","last_active":"2024-09-21T02:14:16.167676+0000","last_peered":"2024-09-21T02:14:16.167676+0000","last_clean":"2024-09-21T02:14:16.167676+0000","last_became_active":"2024-09-21T02:14:14.171464+0000","last_became_peered":"2024-09-21T02:14:14.171464+0000","last_unstale":"2024-09-21T02:14:16.167676+0000","last_undegraded":"2024-09-21T02:14:16.167676+0000","last_fullsized":"2024-09-21T02:14:16.167676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:36.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181442+0000","last_change":"2024-09-21T02:14:15.186681+0000","last_active":"2024-09-21T02:14:17.181442+0000","last_peered":"2024-09-21T02:14:17.181442+0000","last_clean":"2024-09-21T02:14:17.181442+0000","last_became_active":"2024-09-21T02:14:15.185804+0000","last_became_peered":"2024-09-21T02:14:15.185804+0000","last_unstale":"2024-09-21T02:14:17.181442+0000","last_undegraded":"2024-09-21T02:14:17.181442+0000","last_fullsized":"2024-09-21T02:14:17.181442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:12.770185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167766+0000","last_change":"2024-09-21T02:14:14.195269+0000","last_active":"2024-09-21T02:14:16.167766+0000","last_peered":"2024-09-21T02:14:16.167766+0000","last_clean":"2024-09-21T02:14:16.167766+0000","last_became_active":"2024-09-21T02:14:14.194411+0000","last_became_peered":"2024-09-21T02:14:14.194411+0000","last_unstale":"2024-09-21T02:14:16.167766+0000","last_undegraded":"2024-09-21T02:14:16.167766+0000","last_fullsized":"2024-09-21T02:14:16.167766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:57.402889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180186+0000","last_change":"2024-09-21T02:14:15.222984+0000","last_active":"2024-09-21T02:14:17.180186+0000","last_peered":"2024-09-21T02:14:17.180186+0000","last_clean":"2024-09-21T02:14:17.180186+0000","last_became_active":"2024-09-21T02:14:15.220840+0000","last_became_peered":"2024-09-21T02:14:15.220840+0000","last_unstale":"2024-09-21T02:14:17.180186+0000","last_undegraded":"2024-09-21T02:14:17.180186+0000","last_fullsized":"2024-09-21T02:14:17.180186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.417152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162015+0000","last_change":"2024-09-21T02:14:14.175499+0000","last_active":"2024-09-21T02:14:16.162015+0000","last_peered":"2024-09-21T02:14:16.162015+0000","last_clean":"2024-09-21T02:14:16.162015+0000","last_became_active":"2024-09-21T02:14:14.175116+0000","last_became_peered":"2024-09-21T02:14:14.175116+0000","last_unstale":"2024-09-21T02:14:16.162015+0000","last_undegraded":"2024-09-21T02:14:16.162015+0000","last_fullsized":"2024-09-21T02:14:16.162015+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:02.892607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162005+0000","last_change":"2024-09-21T02:14:15.214472+0000","last_active":"2024-09-21T02:14:16.162005+0000","last_peered":"2024-09-21T02:14:16.162005+0000","last_clean":"2024-09-21T02:14:16.162005+0000","last_became_active":"2024-09-21T02:14:15.214029+0000","last_became_peered":"2024-09-21T02:14:15.214029+0000","last_unstale":"2024-09-21T02:14:16.162005+0000","last_undegraded":"2024-09-21T02:14:16.162005+0000","last_fullsized":"2024-09-21T02:14:16.162005+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:36.089332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181328+0000","last_change":"2024-09-21T02:14:14.189722+0000","last_active":"2024-09-21T02:14:17.181328+0000","last_peered":"2024-09-21T02:14:17.181328+0000","last_clean":"2024-09-21T02:14:17.181328+0000","last_became_active":"2024-09-21T02:14:14.187906+0000","last_became_peered":"2024-09-21T02:14:14.187906+0000","last_unstale":"2024-09-21T02:14:17.181328+0000","last_undegraded":"2024-09-21T02:14:17.181328+0000","last_fullsized":"2024-09-21T02:14:17.181328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:55.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178572+0000","last_change":"2024-09-21T02:14:15.193675+0000","last_active":"2024-09-21T02:14:17.178572+0000","last_peered":"2024-09-21T02:14:17.178572+0000","last_clean":"2024-09-21T02:14:17.178572+0000","last_became_active":"2024-09-21T02:14:15.191752+0000","last_became_peered":"2024-09-21T02:14:15.191752+0000","last_unstale":"2024-09-21T02:14:17.178572+0000","last_undegraded":"2024-09-21T02:14:17.178572+0000","last_fullsized":"2024-09-21T02:14:17.178572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:53.689471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182917+0000","last_change":"2024-09-21T02:14:14.172530+0000","last_active":"2024-09-21T02:14:17.182917+0000","last_peered":"2024-09-21T02:14:17.182917+0000","last_clean":"2024-09-21T02:14:17.182917+0000","last_became_active":"2024-09-21T02:14:14.172101+0000","last_became_peered":"2024-09-21T02:14:14.172101+0000","last_unstale":"2024-09-21T02:14:17.182917+0000","last_undegraded":"2024-09-21T02:14:17.182917+0000","last_fullsized":"2024-09-21T02:14:17.182917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:20.517402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180319+0000","last_change":"2024-09-21T02:14:15.206105+0000","last_active":"2024-09-21T02:14:17.180319+0000","last_peered":"2024-09-21T02:14:17.180319+0000","last_clean":"2024-09-21T02:14:17.180319+0000","last_became_active":"2024-09-21T02:14:15.205311+0000","last_became_peered":"2024-09-21T02:14:15.205311+0000","last_unstale":"2024-09-21T02:14:17.180319+0000","last_undegraded":"2024-09-21T02:14:17.180319+0000","last_fullsized":"2024-09-21T02:14:17.180319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:57.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179485+0000","last_change":"2024-09-21T02:14:14.180926+0000","last_active":"2024-09-21T02:14:17.179485+0000","last_peered":"2024-09-21T02:14:17.179485+0000","last_clean":"2024-09-21T02:14:17.179485+0000","last_became_active":"2024-09-21T02:14:14.179759+0000","last_became_peered":"2024-09-21T02:14:14.179759+0000","last_unstale":"2024-09-21T02:14:17.179485+0000","last_undegraded":"2024-09-21T02:14:17.179485+0000","last_fullsized":"2024-09-21T02:14:17.179485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:34.042299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.184450+0000","last_change":"2024-09-21T02:14:15.193038+0000","last_active":"2024-09-21T02:14:17.184450+0000","last_peered":"2024-09-21T02:14:17.184450+0000","last_clean":"2024-09-21T02:14:17.184450+0000","last_became_active":"2024-09-21T02:14:15.192236+0000","last_became_peered":"2024-09-21T02:14:15.192236+0000","last_unstale":"2024-09-21T02:14:17.184450+0000","last_undegraded":"2024-09-21T02:14:17.184450+0000","last_fullsized":"2024-09-21T02:14:17.184450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:00.074804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179525+0000","last_change":"2024-09-21T02:14:14.182658+0000","last_active":"2024-09-21T02:14:17.179525+0000","last_peered":"2024-09-21T02:14:17.179525+0000","last_clean":"2024-09-21T02:14:17.179525+0000","last_became_active":"2024-09-21T02:14:14.181767+0000","last_became_peered":"2024-09-21T02:14:14.181767+0000","last_unstale":"2024-09-21T02:14:17.179525+0000","last_undegraded":"2024-09-21T02:14:17.179525+0000","last_fullsized":"2024-09-21T02:14:17.179525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:32.022100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161899+0000","last_change":"2024-09-21T02:14:15.195531+0000","last_active":"2024-09-21T02:14:16.161899+0000","last_peered":"2024-09-21T02:14:16.161899+0000","last_clean":"2024-09-21T02:14:16.161899+0000","last_became_active":"2024-09-21T02:14:15.195215+0000","last_became_peered":"2024-09-21T02:14:15.195215+0000","last_unstale":"2024-09-21T02:14:16.161899+0000","last_undegraded":"2024-09-21T02:14:16.161899+0000","last_fullsized":"2024-09-21T02:14:16.161899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:27.908301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.166865+0000","last_change":"2024-09-21T02:14:14.192522+0000","last_active":"2024-09-21T02:14:16.166865+0000","last_peered":"2024-09-21T02:14:16.166865+0000","last_clean":"2024-09-21T02:14:16.166865+0000","last_became_active":"2024-09-21T02:14:14.191538+0000","last_became_peered":"2024-09-21T02:14:14.191538+0000","last_unstale":"2024-09-21T02:14:16.166865+0000","last_undegraded":"2024-09-21T02:14:16.166865+0000","last_fullsized":"2024-09-21T02:14:16.166865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:06.591082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162384+0000","last_change":"2024-09-21T02:14:15.213713+0000","last_active":"2024-09-21T02:14:16.162384+0000","last_peered":"2024-09-21T02:14:16.162384+0000","last_clean":"2024-09-21T02:14:16.162384+0000","last_became_active":"2024-09-21T02:14:15.213314+0000","last_became_peered":"2024-09-21T02:14:15.213314+0000","last_unstale":"2024-09-21T02:14:16.162384+0000","last_undegraded":"2024-09-21T02:14:16.162384+0000","last_fullsized":"2024-09-21T02:14:16.162384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:56.976743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.169408+0000","last_change":"2024-09-21T02:14:14.190309+0000","last_active":"2024-09-21T02:14:16.169408+0000","last_peered":"2024-09-21T02:14:16.169408+0000","last_clean":"2024-09-21T02:14:16.169408+0000","last_became_active":"2024-09-21T02:14:14.189460+0000","last_became_peered":"2024-09-21T02:14:14.189460+0000","last_unstale":"2024-09-21T02:14:16.169408+0000","last_undegraded":"2024-09-21T02:14:16.169408+0000","last_fullsized":"2024-09-21T02:14:16.169408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:56.982713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181931+0000","last_change":"2024-09-21T02:14:15.170769+0000","last_active":"2024-09-21T02:14:17.181931+0000","last_peered":"2024-09-21T02:14:17.181931+0000","last_clean":"2024-09-21T02:14:17.181931+0000","last_became_active":"2024-09-21T02:14:15.169962+0000","last_became_peered":"2024-09-21T02:14:15.169962+0000","last_unstale":"2024-09-21T02:14:17.181931+0000","last_undegraded":"2024-09-21T02:14:17.181931+0000","last_fullsized":"2024-09-21T02:14:17.181931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:22.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181341+0000","last_change":"2024-09-21T02:14:14.175339+0000","last_active":"2024-09-21T02:14:17.181341+0000","last_peered":"2024-09-21T02:14:17.181341+0000","last_clean":"2024-09-21T02:14:17.181341+0000","last_became_active":"2024-09-21T02:14:14.175001+0000","last_became_peered":"2024-09-21T02:14:14.175001+0000","last_unstale":"2024-09-21T02:14:17.181341+0000","last_undegraded":"2024-09-21T02:14:17.181341+0000","last_fullsized":"2024-09-21T02:14:17.181341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:07.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167494+0000","last_change":"2024-09-21T02:14:15.175776+0000","last_active":"2024-09-21T02:14:16.167494+0000","last_peered":"2024-09-21T02:14:16.167494+0000","last_clean":"2024-09-21T02:14:16.167494+0000","last_became_active":"2024-09-21T02:14:15.173231+0000","last_became_peered":"2024-09-21T02:14:15.173231+0000","last_unstale":"2024-09-21T02:14:16.167494+0000","last_undegraded":"2024-09-21T02:14:16.167494+0000","last_fullsized":"2024-09-21T02:14:16.167494+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:00.074804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162150+0000","last_change":"2024-09-21T02:14:14.170919+0000","last_active":"2024-09-21T02:14:16.162150+0000","last_peered":"2024-09-21T02:14:16.162150+0000","last_clean":"2024-09-21T02:14:16.162150+0000","last_became_active":"2024-09-21T02:14:14.170489+0000","last_became_peered":"2024-09-21T02:14:14.170489+0000","last_unstale":"2024-09-21T02:14:16.162150+0000","last_undegraded":"2024-09-21T02:14:16.162150+0000","last_fullsized":"2024-09-21T02:14:16.162150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:59.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.170135+0000","last_change":"2024-09-21T02:14:15.213284+0000","last_active":"2024-09-21T02:14:16.170135+0000","last_peered":"2024-09-21T02:14:16.170135+0000","last_clean":"2024-09-21T02:14:16.170135+0000","last_became_active":"2024-09-21T02:14:15.210729+0000","last_became_peered":"2024-09-21T02:14:15.210729+0000","last_unstale":"2024-09-21T02:14:16.170135+0000","last_undegraded":"2024-09-21T02:14:16.170135+0000","last_fullsized":"2024-09-21T02:14:16.170135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:20.809947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180154+0000","last_change":"2024-09-21T02:14:14.189463+0000","last_active":"2024-09-21T02:14:17.180154+0000","last_peered":"2024-09-21T02:14:17.180154+0000","last_clean":"2024-09-21T02:14:17.180154+0000","last_became_active":"2024-09-21T02:14:14.188788+0000","last_became_peered":"2024-09-21T02:14:14.188788+0000","last_unstale":"2024-09-21T02:14:17.180154+0000","last_undegraded":"2024-09-21T02:14:17.180154+0000","last_fullsized":"2024-09-21T02:14:17.180154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:11.699831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178370+0000","last_change":"2024-09-21T02:14:15.194757+0000","last_active":"2024-09-21T02:14:17.178370+0000","last_peered":"2024-09-21T02:14:17.178370+0000","last_clean":"2024-09-21T02:14:17.178370+0000","last_became_active":"2024-09-21T02:14:15.192774+0000","last_became_peered":"2024-09-21T02:14:15.192774+0000","last_unstale":"2024-09-21T02:14:17.178370+0000","last_undegraded":"2024-09-21T02:14:17.178370+0000","last_fullsized":"2024-09-21T02:14:17.178370+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:24.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167616+0000","last_change":"2024-09-21T02:14:14.190856+0000","last_active":"2024-09-21T02:14:16.167616+0000","last_peered":"2024-09-21T02:14:16.167616+0000","last_clean":"2024-09-21T02:14:16.167616+0000","last_became_active":"2024-09-21T02:14:14.189986+0000","last_became_peered":"2024-09-21T02:14:14.189986+0000","last_unstale":"2024-09-21T02:14:16.167616+0000","last_undegraded":"2024-09-21T02:14:16.167616+0000","last_fullsized":"2024-09-21T02:14:16.167616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:37.724200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167824+0000","last_change":"2024-09-21T02:14:15.177019+0000","last_active":"2024-09-21T02:14:16.167824+0000","last_peered":"2024-09-21T02:14:16.167824+0000","last_clean":"2024-09-21T02:14:16.167824+0000","last_became_active":"2024-09-21T02:14:15.176179+0000","last_became_peered":"2024-09-21T02:14:15.176179+0000","last_unstale":"2024-09-21T02:14:16.167824+0000","last_undegraded":"2024-09-21T02:14:16.167824+0000","last_fullsized":"2024-09-21T02:14:16.167824+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:12.601935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182991+0000","last_change":"2024-09-21T02:14:14.173484+0000","last_active":"2024-09-21T02:14:17.182991+0000","last_peered":"2024-09-21T02:14:17.182991+0000","last_clean":"2024-09-21T02:14:17.182991+0000","last_became_active":"2024-09-21T02:14:14.173112+0000","last_became_peered":"2024-09-21T02:14:14.173112+0000","last_unstale":"2024-09-21T02:14:17.182991+0000","last_undegraded":"2024-09-21T02:14:17.182991+0000","last_fullsized":"2024-09-21T02:14:17.182991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:28.093076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178311+0000","last_change":"2024-09-21T02:14:15.224490+0000","last_active":"2024-09-21T02:14:17.178311+0000","last_peered":"2024-09-21T02:14:17.178311+0000","last_clean":"2024-09-21T02:14:17.178311+0000","last_became_active":"2024-09-21T02:14:15.219388+0000","last_became_peered":"2024-09-21T02:14:15.219388+0000","last_unstale":"2024-09-21T02:14:17.178311+0000","last_undegraded":"2024-09-21T02:14:17.178311+0000","last_fullsized":"2024-09-21T02:14:17.178311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:28.963430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180056+0000","last_change":"2024-09-21T02:14:14.194963+0000","last_active":"2024-09-21T02:14:17.180056+0000","last_peered":"2024-09-21T02:14:17.180056+0000","last_clean":"2024-09-21T02:14:17.180056+0000","last_became_active":"2024-09-21T02:14:14.194477+0000","last_became_peered":"2024-09-21T02:14:14.194477+0000","last_unstale":"2024-09-21T02:14:17.180056+0000","last_undegraded":"2024-09-21T02:14:17.180056+0000","last_fullsized":"2024-09-21T02:14:17.180056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:04.278971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180436+0000","last_change":"2024-09-21T02:14:15.185697+0000","last_active":"2024-09-21T02:14:17.180436+0000","last_peered":"2024-09-21T02:14:17.180436+0000","last_clean":"2024-09-21T02:14:17.180436+0000","last_became_active":"2024-09-21T02:14:15.183668+0000","last_became_peered":"2024-09-21T02:14:15.183668+0000","last_unstale":"2024-09-21T02:14:17.180436+0000","last_undegraded":"2024-09-21T02:14:17.180436+0000","last_fullsized":"2024-09-21T02:14:17.180436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:24.838687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161640+0000","last_change":"2024-09-21T02:14:14.174185+0000","last_active":"2024-09-21T02:14:16.161640+0000","last_peered":"2024-09-21T02:14:16.161640+0000","last_clean":"2024-09-21T02:14:16.161640+0000","last_became_active":"2024-09-21T02:14:14.173632+0000","last_became_peered":"2024-09-21T02:14:14.173632+0000","last_unstale":"2024-09-21T02:14:16.161640+0000","last_undegraded":"2024-09-21T02:14:16.161640+0000","last_fullsized":"2024-09-21T02:14:16.161640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.661562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168792+0000","last_change":"2024-09-21T02:14:15.212782+0000","last_active":"2024-09-21T02:14:16.168792+0000","last_peered":"2024-09-21T02:14:16.168792+0000","last_clean":"2024-09-21T02:14:16.168792+0000","last_became_active":"2024-09-21T02:14:15.211937+0000","last_became_peered":"2024-09-21T02:14:15.211937+0000","last_unstale":"2024-09-21T02:14:16.168792+0000","last_undegraded":"2024-09-21T02:14:16.168792+0000","last_fullsized":"2024-09-21T02:14:16.168792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:19.944730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181308+0000","last_change":"2024-09-21T02:14:14.175791+0000","last_active":"2024-09-21T02:14:17.181308+0000","last_peered":"2024-09-21T02:14:17.181308+0000","last_clean":"2024-09-21T02:14:17.181308+0000","last_became_active":"2024-09-21T02:14:14.174962+0000","last_became_peered":"2024-09-21T02:14:14.174962+0000","last_unstale":"2024-09-21T02:14:17.181308+0000","last_undegraded":"2024-09-21T02:14:17.181308+0000","last_fullsized":"2024-09-21T02:14:17.181308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:37.666797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162331+0000","last_change":"2024-09-21T02:14:15.185845+0000","last_active":"2024-09-21T02:14:16.162331+0000","last_peered":"2024-09-21T02:14:16.162331+0000","last_clean":"2024-09-21T02:14:16.162331+0000","last_became_active":"2024-09-21T02:14:15.185563+0000","last_became_peered":"2024-09-21T02:14:15.185563+0000","last_unstale":"2024-09-21T02:14:16.162331+0000","last_undegraded":"2024-09-21T02:14:16.162331+0000","last_fullsized":"2024-09-21T02:14:16.162331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:57.860005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168015+0000","last_change":"2024-09-21T02:14:14.194882+0000","last_active":"2024-09-21T02:14:16.168015+0000","last_peered":"2024-09-21T02:14:16.168015+0000","last_clean":"2024-09-21T02:14:16.168015+0000","last_became_active":"2024-09-21T02:14:14.194002+0000","last_became_peered":"2024-09-21T02:14:14.194002+0000","last_unstale":"2024-09-21T02:14:16.168015+0000","last_undegraded":"2024-09-21T02:14:16.168015+0000","last_fullsized":"2024-09-21T02:14:16.168015+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.730357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.170750+0000","last_change":"2024-09-21T02:14:15.220848+0000","last_active":"2024-09-21T02:14:16.170750+0000","last_peered":"2024-09-21T02:14:16.170750+0000","last_clean":"2024-09-21T02:14:16.170750+0000","last_became_active":"2024-09-21T02:14:15.219476+0000","last_became_peered":"2024-09-21T02:14:15.219476+0000","last_unstale":"2024-09-21T02:14:16.170750+0000","last_undegraded":"2024-09-21T02:14:16.170750+0000","last_fullsized":"2024-09-21T02:14:16.170750+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181594+0000","last_change":"2024-09-21T02:14:14.186532+0000","last_active":"2024-09-21T02:14:17.181594+0000","last_peered":"2024-09-21T02:14:17.181594+0000","last_clean":"2024-09-21T02:14:17.181594+0000","last_became_active":"2024-09-21T02:14:14.185831+0000","last_became_peered":"2024-09-21T02:14:14.185831+0000","last_unstale":"2024-09-21T02:14:17.181594+0000","last_undegraded":"2024-09-21T02:14:17.181594+0000","last_fullsized":"2024-09-21T02:14:17.181594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:22.319742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179804+0000","last_change":"2024-09-21T02:14:15.176750+0000","last_active":"2024-09-21T02:14:17.179804+0000","last_peered":"2024-09-21T02:14:17.179804+0000","last_clean":"2024-09-21T02:14:17.179804+0000","last_became_active":"2024-09-21T02:14:15.176379+0000","last_became_peered":"2024-09-21T02:14:15.176379+0000","last_unstale":"2024-09-21T02:14:17.179804+0000","last_undegraded":"2024-09-21T02:14:17.179804+0000","last_fullsized":"2024-09-21T02:14:17.179804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:11.129900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181234+0000","last_change":"2024-09-21T02:14:14.175812+0000","last_active":"2024-09-21T02:14:17.181234+0000","last_peered":"2024-09-21T02:14:17.181234+0000","last_clean":"2024-09-21T02:14:17.181234+0000","last_became_active":"2024-09-21T02:14:14.175065+0000","last_became_peered":"2024-09-21T02:14:14.175065+0000","last_unstale":"2024-09-21T02:14:17.181234+0000","last_undegraded":"2024-09-21T02:14:17.181234+0000","last_fullsized":"2024-09-21T02:14:17.181234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:23.827792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180922+0000","last_change":"2024-09-21T02:14:15.199669+0000","last_active":"2024-09-21T02:14:17.180922+0000","last_peered":"2024-09-21T02:14:17.180922+0000","last_clean":"2024-09-21T02:14:17.180922+0000","last_became_active":"2024-09-21T02:14:15.198854+0000","last_became_peered":"2024-09-21T02:14:15.198854+0000","last_unstale":"2024-09-21T02:14:17.180922+0000","last_undegraded":"2024-09-21T02:14:17.180922+0000","last_fullsized":"2024-09-21T02:14:17.180922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:19.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178770+0000","last_change":"2024-09-21T02:14:14.177436+0000","last_active":"2024-09-21T02:14:17.178770+0000","last_peered":"2024-09-21T02:14:17.178770+0000","last_clean":"2024-09-21T02:14:17.178770+0000","last_became_active":"2024-09-21T02:14:14.176075+0000","last_became_peered":"2024-09-21T02:14:14.176075+0000","last_unstale":"2024-09-21T02:14:17.178770+0000","last_undegraded":"2024-09-21T02:14:17.178770+0000","last_fullsized":"2024-09-21T02:14:17.178770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.058048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179768+0000","last_change":"2024-09-21T02:14:15.200391+0000","last_active":"2024-09-21T02:14:17.179768+0000","last_peered":"2024-09-21T02:14:17.179768+0000","last_clean":"2024-09-21T02:14:17.179768+0000","last_became_active":"2024-09-21T02:14:15.196024+0000","last_became_peered":"2024-09-21T02:14:15.196024+0000","last_unstale":"2024-09-21T02:14:17.179768+0000","last_undegraded":"2024-09-21T02:14:17.179768+0000","last_fullsized":"2024-09-21T02:14:17.179768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:16.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182955+0000","last_change":"2024-09-21T02:14:14.172390+0000","last_active":"2024-09-21T02:14:17.182955+0000","last_peered":"2024-09-21T02:14:17.182955+0000","last_clean":"2024-09-21T02:14:17.182955+0000","last_became_active":"2024-09-21T02:14:14.172030+0000","last_became_peered":"2024-09-21T02:14:14.172030+0000","last_unstale":"2024-09-21T02:14:17.182955+0000","last_undegraded":"2024-09-21T02:14:17.182955+0000","last_fullsized":"2024-09-21T02:14:17.182955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.950716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161928+0000","last_change":"2024-09-21T02:14:15.185342+0000","last_active":"2024-09-21T02:14:16.161928+0000","last_peered":"2024-09-21T02:14:16.161928+0000","last_clean":"2024-09-21T02:14:16.161928+0000","last_became_active":"2024-09-21T02:14:15.184980+0000","last_became_peered":"2024-09-21T02:14:15.184980+0000","last_unstale":"2024-09-21T02:14:16.161928+0000","last_undegraded":"2024-09-21T02:14:16.161928+0000","last_fullsized":"2024-09-21T02:14:16.161928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:44.328587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179042+0000","last_change":"2024-09-21T02:14:14.180582+0000","last_active":"2024-09-21T02:14:17.179042+0000","last_peered":"2024-09-21T02:14:17.179042+0000","last_clean":"2024-09-21T02:14:17.179042+0000","last_became_active":"2024-09-21T02:14:14.179876+0000","last_became_peered":"2024-09-21T02:14:14.179876+0000","last_unstale":"2024-09-21T02:14:17.179042+0000","last_undegraded":"2024-09-21T02:14:17.179042+0000","last_fullsized":"2024-09-21T02:14:17.179042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:29.774228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179980+0000","last_change":"2024-09-21T02:14:15.201426+0000","last_active":"2024-09-21T02:14:17.179980+0000","last_peered":"2024-09-21T02:14:17.179980+0000","last_clean":"2024-09-21T02:14:17.179980+0000","last_became_active":"2024-09-21T02:14:15.201036+0000","last_became_peered":"2024-09-21T02:14:15.201036+0000","last_unstale":"2024-09-21T02:14:17.179980+0000","last_undegraded":"2024-09-21T02:14:17.179980+0000","last_fullsized":"2024-09-21T02:14:17.179980+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:22.182160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162236+0000","last_change":"2024-09-21T02:14:14.170824+0000","last_active":"2024-09-21T02:14:16.162236+0000","last_peered":"2024-09-21T02:14:16.162236+0000","last_clean":"2024-09-21T02:14:16.162236+0000","last_became_active":"2024-09-21T02:14:14.170474+0000","last_became_peered":"2024-09-21T02:14:14.170474+0000","last_unstale":"2024-09-21T02:14:16.162236+0000","last_undegraded":"2024-09-21T02:14:16.162236+0000","last_fullsized":"2024-09-21T02:14:16.162236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:10.119005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":132,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182486+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:17.182486+0000","last_peered":"2024-09-21T02:14:17.182486+0000","last_clean":"2024-09-21T02:14:17.182486+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:17.182486+0000","last_undegraded":"2024-09-21T02:14:17.182486+0000","last_fullsized":"2024-09-21T02:14:17.182486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":987136,"data_stored":741953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":405504,"data_stored":151779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":151037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":987136,"data_stored":741441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":151093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":146337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:23.649 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:14:23.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:14:23.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.144+0000 7f565eb69640 1 -- 172.21.15.170:0/3313116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 msgr2=0x7f56580b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.144+0000 7f565eb69640 1 --2- 172.21.15.170:0/3313116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 0x7f56580b3350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56580b3a10 tx=0x7f564c01c820 comp rx=0 tx=0).stop 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- 172.21.15.170:0/3313116751 shutdown_connections 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 --2- 172.21.15.170:0/3313116751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 0x7f56580b3350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 --2- 172.21.15.170:0/3313116751 >> v2:172.21.15.191:3300/0 conn(0x7f56580b5410 0x7f565806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- 172.21.15.170:0/3313116751 >> 172.21.15.170:0/3313116751 conn(0x7f56580b1c40 msgr2=0x7f56580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- 172.21.15.170:0/3313116751 shutdown_connections 2024-09-21T02:14:24.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- 172.21.15.170:0/3313116751 wait complete. 2024-09-21T02:14:24.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 Processor -- start 2024-09-21T02:14:24.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- start start 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 0x7f56581f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56580b5410 0x7f5658155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 0x7f565815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5658169fd0 con 0x7f56580ccb00 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5658169e50 con 0x7f56580b5410 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565eb69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f565816a150 con 0x7f56580b2f40 2024-09-21T02:14:24.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565d106640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 0x7f565815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565d106640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 0x7f565815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48010/0 (socket says 172.21.15.170:48010) 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565d106640 1 -- 172.21.15.170:0/186176932 learned_addr learned my addr 172.21.15.170:0/186176932 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f5657fff640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56580b5410 0x7f5658155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.148+0000 7f565c905640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 0x7f56581f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565d106640 1 -- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 msgr2=0x7f56581f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565d106640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 0x7f56581f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565d106640 1 -- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56580b5410 msgr2=0x7f5658155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565d106640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56580b5410 0x7f5658155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565d106640 1 -- 172.21.15.170:0/186176932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5658156f40 con 0x7f56580ccb00 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f5657fff640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56580b5410 0x7f5658155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565d106640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 0x7f565815bd40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f565000cb10 tx=0x7f565000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650016020 con 0x7f56580ccb00 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5658155ea0 con 0x7f56580ccb00 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f565000ce50 con 0x7f56580ccb00 2024-09-21T02:14:24.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565000ade0 con 0x7f56580ccb00 2024-09-21T02:14:24.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5658156460 con 0x7f56580ccb00 2024-09-21T02:14:24.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56580b3350 con 0x7f56580ccb00 2024-09-21T02:14:24.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5650019020 con 0x7f56580ccb00 2024-09-21T02:14:24.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f56557fa640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5644073230 0x7f5644075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f56500945b0 con 0x7f56580ccb00 2024-09-21T02:14:24.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.152+0000 7f565c905640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5644073230 0x7f5644075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.156+0000 7f565c905640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5644073230 0x7f5644075740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5648009ad0 tx=0x7f56480091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:24.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.156+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5650020030 con 0x7f56580ccb00 2024-09-21T02:14:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.328+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56580c3a10 con 0x7f5644073230 2024-09-21T02:14:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.340+0000 7f56557fa640 1 -- 172.21.15.170:0/186176932 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251956 (secure 0 0 0) 0x7f56580c3a10 con 0x7f5644073230 2024-09-21T02:14:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:24.346 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5644073230 msgr2=0x7f5644075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5644073230 0x7f5644075740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5648009ad0 tx=0x7f56480091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 msgr2=0x7f565815bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 0x7f565815bd40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f565000cb10 tx=0x7f565000cc00 comp rx=0 tx=0).stop 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 shutdown_connections 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5644073230 0x7f5644075740 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56580ccb00 0x7f565815bd40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56580b5410 0x7f5658155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 --2- 172.21.15.170:0/186176932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56580b2f40 0x7f56581f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 >> 172.21.15.170:0/186176932 conn(0x7f56580b1c40 msgr2=0x7f56580c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 shutdown_connections 2024-09-21T02:14:24.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.344+0000 7f565eb69640 1 -- 172.21.15.170:0/186176932 wait complete. 2024-09-21T02:14:24.367 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-21T02:14:22.885052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245452,"kb_used_data":4252,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754729268,"statfs":{"total":773094113280,"available":772842770432,"internally_reserved":0,"allocated":4354048,"data_stored":2387280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003147"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182931+0000","last_change":"2024-09-21T02:14:15.183292+0000","last_active":"2024-09-21T02:14:17.182931+0000","last_peered":"2024-09-21T02:14:17.182931+0000","last_clean":"2024-09-21T02:14:17.182931+0000","last_became_active":"2024-09-21T02:14:15.182677+0000","last_became_peered":"2024-09-21T02:14:15.182677+0000","last_unstale":"2024-09-21T02:14:17.182931+0000","last_undegraded":"2024-09-21T02:14:17.182931+0000","last_fullsized":"2024-09-21T02:14:17.182931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:01.672457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167597+0000","last_change":"2024-09-21T02:14:15.214151+0000","last_active":"2024-09-21T02:14:16.167597+0000","last_peered":"2024-09-21T02:14:16.167597+0000","last_clean":"2024-09-21T02:14:16.167597+0000","last_became_active":"2024-09-21T02:14:15.213317+0000","last_became_peered":"2024-09-21T02:14:15.213317+0000","last_unstale":"2024-09-21T02:14:16.167597+0000","last_undegraded":"2024-09-21T02:14:16.167597+0000","last_fullsized":"2024-09-21T02:14:16.167597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:48.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180231+0000","last_change":"2024-09-21T02:14:15.183780+0000","last_active":"2024-09-21T02:14:17.180231+0000","last_peered":"2024-09-21T02:14:17.180231+0000","last_clean":"2024-09-21T02:14:17.180231+0000","last_became_active":"2024-09-21T02:14:15.183335+0000","last_became_peered":"2024-09-21T02:14:15.183335+0000","last_unstale":"2024-09-21T02:14:17.180231+0000","last_undegraded":"2024-09-21T02:14:17.180231+0000","last_fullsized":"2024-09-21T02:14:17.180231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:46.068943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180103+0000","last_change":"2024-09-21T02:14:15.221703+0000","last_active":"2024-09-21T02:14:17.180103+0000","last_peered":"2024-09-21T02:14:17.180103+0000","last_clean":"2024-09-21T02:14:17.180103+0000","last_became_active":"2024-09-21T02:14:15.219448+0000","last_became_peered":"2024-09-21T02:14:15.219448+0000","last_unstale":"2024-09-21T02:14:17.180103+0000","last_undegraded":"2024-09-21T02:14:17.180103+0000","last_fullsized":"2024-09-21T02:14:17.180103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:27.282489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167414+0000","last_change":"2024-09-21T02:14:15.204321+0000","last_active":"2024-09-21T02:14:16.167414+0000","last_peered":"2024-09-21T02:14:16.167414+0000","last_clean":"2024-09-21T02:14:16.167414+0000","last_became_active":"2024-09-21T02:14:15.203516+0000","last_became_peered":"2024-09-21T02:14:15.203516+0000","last_unstale":"2024-09-21T02:14:16.167414+0000","last_undegraded":"2024-09-21T02:14:16.167414+0000","last_fullsized":"2024-09-21T02:14:16.167414+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:03.903502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168814+0000","last_change":"2024-09-21T02:14:15.210136+0000","last_active":"2024-09-21T02:14:16.168814+0000","last_peered":"2024-09-21T02:14:16.168814+0000","last_clean":"2024-09-21T02:14:16.168814+0000","last_became_active":"2024-09-21T02:14:15.209438+0000","last_became_peered":"2024-09-21T02:14:15.209438+0000","last_unstale":"2024-09-21T02:14:16.168814+0000","last_undegraded":"2024-09-21T02:14:16.168814+0000","last_fullsized":"2024-09-21T02:14:16.168814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:00.474590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181828+0000","last_change":"2024-09-21T02:14:15.194359+0000","last_active":"2024-09-21T02:14:17.181828+0000","last_peered":"2024-09-21T02:14:17.181828+0000","last_clean":"2024-09-21T02:14:17.181828+0000","last_became_active":"2024-09-21T02:14:15.193926+0000","last_became_peered":"2024-09-21T02:14:15.193926+0000","last_unstale":"2024-09-21T02:14:17.181828+0000","last_undegraded":"2024-09-21T02:14:17.181828+0000","last_fullsized":"2024-09-21T02:14:17.181828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:26.736916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181780+0000","last_change":"2024-09-21T02:14:15.198361+0000","last_active":"2024-09-21T02:14:17.181780+0000","last_peered":"2024-09-21T02:14:17.181780+0000","last_clean":"2024-09-21T02:14:17.181780+0000","last_became_active":"2024-09-21T02:14:15.198032+0000","last_became_peered":"2024-09-21T02:14:15.198032+0000","last_unstale":"2024-09-21T02:14:17.181780+0000","last_undegraded":"2024-09-21T02:14:17.181780+0000","last_fullsized":"2024-09-21T02:14:17.181780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:25.771631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181300+0000","last_change":"2024-09-21T02:14:15.191098+0000","last_active":"2024-09-21T02:14:17.181300+0000","last_peered":"2024-09-21T02:14:17.181300+0000","last_clean":"2024-09-21T02:14:17.181300+0000","last_became_active":"2024-09-21T02:14:15.190688+0000","last_became_peered":"2024-09-21T02:14:15.190688+0000","last_unstale":"2024-09-21T02:14:17.181300+0000","last_undegraded":"2024-09-21T02:14:17.181300+0000","last_fullsized":"2024-09-21T02:14:17.181300+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:03:12.511846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182169+0000","last_change":"2024-09-21T02:14:15.218508+0000","last_active":"2024-09-21T02:14:17.182169+0000","last_peered":"2024-09-21T02:14:17.182169+0000","last_clean":"2024-09-21T02:14:17.182169+0000","last_became_active":"2024-09-21T02:14:15.217625+0000","last_became_peered":"2024-09-21T02:14:15.217625+0000","last_unstale":"2024-09-21T02:14:17.182169+0000","last_undegraded":"2024-09-21T02:14:17.182169+0000","last_fullsized":"2024-09-21T02:14:17.182169+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.766624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162156+0000","last_change":"2024-09-21T02:14:15.220694+0000","last_active":"2024-09-21T02:14:16.162156+0000","last_peered":"2024-09-21T02:14:16.162156+0000","last_clean":"2024-09-21T02:14:16.162156+0000","last_became_active":"2024-09-21T02:14:15.220312+0000","last_became_peered":"2024-09-21T02:14:15.220312+0000","last_unstale":"2024-09-21T02:14:16.162156+0000","last_undegraded":"2024-09-21T02:14:16.162156+0000","last_fullsized":"2024-09-21T02:14:16.162156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:42.304704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181094+0000","last_change":"2024-09-21T02:14:15.221471+0000","last_active":"2024-09-21T02:14:17.181094+0000","last_peered":"2024-09-21T02:14:17.181094+0000","last_clean":"2024-09-21T02:14:17.181094+0000","last_became_active":"2024-09-21T02:14:15.221130+0000","last_became_peered":"2024-09-21T02:14:15.221130+0000","last_unstale":"2024-09-21T02:14:17.181094+0000","last_undegraded":"2024-09-21T02:14:17.181094+0000","last_fullsized":"2024-09-21T02:14:17.181094+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:38.704450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180801+0000","last_change":"2024-09-21T02:14:15.218389+0000","last_active":"2024-09-21T02:14:17.180801+0000","last_peered":"2024-09-21T02:14:17.180801+0000","last_clean":"2024-09-21T02:14:17.180801+0000","last_became_active":"2024-09-21T02:14:15.217790+0000","last_became_peered":"2024-09-21T02:14:15.217790+0000","last_unstale":"2024-09-21T02:14:17.180801+0000","last_undegraded":"2024-09-21T02:14:17.180801+0000","last_fullsized":"2024-09-21T02:14:17.180801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:13.765471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179430+0000","last_change":"2024-09-21T02:14:15.203118+0000","last_active":"2024-09-21T02:14:17.179430+0000","last_peered":"2024-09-21T02:14:17.179430+0000","last_clean":"2024-09-21T02:14:17.179430+0000","last_became_active":"2024-09-21T02:14:15.202380+0000","last_became_peered":"2024-09-21T02:14:15.202380+0000","last_unstale":"2024-09-21T02:14:17.179430+0000","last_undegraded":"2024-09-21T02:14:17.179430+0000","last_fullsized":"2024-09-21T02:14:17.179430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:30.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178955+0000","last_change":"2024-09-21T02:14:15.183321+0000","last_active":"2024-09-21T02:14:17.178955+0000","last_peered":"2024-09-21T02:14:17.178955+0000","last_clean":"2024-09-21T02:14:17.178955+0000","last_became_active":"2024-09-21T02:14:15.182939+0000","last_became_peered":"2024-09-21T02:14:15.182939+0000","last_unstale":"2024-09-21T02:14:17.178955+0000","last_undegraded":"2024-09-21T02:14:17.178955+0000","last_fullsized":"2024-09-21T02:14:17.178955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:01.672457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178558+0000","last_change":"2024-09-21T02:14:15.196638+0000","last_active":"2024-09-21T02:14:17.178558+0000","last_peered":"2024-09-21T02:14:17.178558+0000","last_clean":"2024-09-21T02:14:17.178558+0000","last_became_active":"2024-09-21T02:14:15.195665+0000","last_became_peered":"2024-09-21T02:14:15.195665+0000","last_unstale":"2024-09-21T02:14:17.178558+0000","last_undegraded":"2024-09-21T02:14:17.178558+0000","last_fullsized":"2024-09-21T02:14:17.178558+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:03.893645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162078+0000","last_change":"2024-09-21T02:14:15.190824+0000","last_active":"2024-09-21T02:14:16.162078+0000","last_peered":"2024-09-21T02:14:16.162078+0000","last_clean":"2024-09-21T02:14:16.162078+0000","last_became_active":"2024-09-21T02:14:15.190212+0000","last_became_peered":"2024-09-21T02:14:15.190212+0000","last_unstale":"2024-09-21T02:14:16.162078+0000","last_undegraded":"2024-09-21T02:14:16.162078+0000","last_fullsized":"2024-09-21T02:14:16.162078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:17.761345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168548+0000","last_change":"2024-09-21T02:14:15.222180+0000","last_active":"2024-09-21T02:14:16.168548+0000","last_peered":"2024-09-21T02:14:16.168548+0000","last_clean":"2024-09-21T02:14:16.168548+0000","last_became_active":"2024-09-21T02:14:15.221412+0000","last_became_peered":"2024-09-21T02:14:15.221412+0000","last_unstale":"2024-09-21T02:14:16.168548+0000","last_undegraded":"2024-09-21T02:14:16.168548+0000","last_fullsized":"2024-09-21T02:14:16.168548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:29.103971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.169541+0000","last_change":"2024-09-21T02:14:15.211355+0000","last_active":"2024-09-21T02:14:16.169541+0000","last_peered":"2024-09-21T02:14:16.169541+0000","last_clean":"2024-09-21T02:14:16.169541+0000","last_became_active":"2024-09-21T02:14:15.210408+0000","last_became_peered":"2024-09-21T02:14:15.210408+0000","last_unstale":"2024-09-21T02:14:16.169541+0000","last_undegraded":"2024-09-21T02:14:16.169541+0000","last_fullsized":"2024-09-21T02:14:16.169541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:11.338884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181159+0000","last_change":"2024-09-21T02:14:15.226510+0000","last_active":"2024-09-21T02:14:17.181159+0000","last_peered":"2024-09-21T02:14:17.181159+0000","last_clean":"2024-09-21T02:14:17.181159+0000","last_became_active":"2024-09-21T02:14:15.225677+0000","last_became_peered":"2024-09-21T02:14:15.225677+0000","last_unstale":"2024-09-21T02:14:17.181159+0000","last_undegraded":"2024-09-21T02:14:17.181159+0000","last_fullsized":"2024-09-21T02:14:17.181159+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:49.578216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179469+0000","last_change":"2024-09-21T02:14:15.198346+0000","last_active":"2024-09-21T02:14:17.179469+0000","last_peered":"2024-09-21T02:14:17.179469+0000","last_clean":"2024-09-21T02:14:17.179469+0000","last_became_active":"2024-09-21T02:14:15.197977+0000","last_became_peered":"2024-09-21T02:14:15.197977+0000","last_unstale":"2024-09-21T02:14:17.179469+0000","last_undegraded":"2024-09-21T02:14:17.179469+0000","last_fullsized":"2024-09-21T02:14:17.179469+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:00.074804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.183680+0000","last_change":"2024-09-21T02:14:15.213671+0000","last_active":"2024-09-21T02:14:17.183680+0000","last_peered":"2024-09-21T02:14:17.183680+0000","last_clean":"2024-09-21T02:14:17.183680+0000","last_became_active":"2024-09-21T02:14:15.213347+0000","last_became_peered":"2024-09-21T02:14:15.213347+0000","last_unstale":"2024-09-21T02:14:17.183680+0000","last_undegraded":"2024-09-21T02:14:17.183680+0000","last_fullsized":"2024-09-21T02:14:17.183680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:03.903502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178417+0000","last_change":"2024-09-21T02:14:15.194252+0000","last_active":"2024-09-21T02:14:17.178417+0000","last_peered":"2024-09-21T02:14:17.178417+0000","last_clean":"2024-09-21T02:14:17.178417+0000","last_became_active":"2024-09-21T02:14:15.191411+0000","last_became_peered":"2024-09-21T02:14:15.191411+0000","last_unstale":"2024-09-21T02:14:17.178417+0000","last_undegraded":"2024-09-21T02:14:17.178417+0000","last_fullsized":"2024-09-21T02:14:17.178417+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:55.467722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181995+0000","last_change":"2024-09-21T02:14:15.219221+0000","last_active":"2024-09-21T02:14:17.181995+0000","last_peered":"2024-09-21T02:14:17.181995+0000","last_clean":"2024-09-21T02:14:17.181995+0000","last_became_active":"2024-09-21T02:14:15.218863+0000","last_became_peered":"2024-09-21T02:14:15.218863+0000","last_unstale":"2024-09-21T02:14:17.181995+0000","last_undegraded":"2024-09-21T02:14:17.181995+0000","last_fullsized":"2024-09-21T02:14:17.181995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:11:46.551654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180846+0000","last_change":"2024-09-21T02:14:15.192842+0000","last_active":"2024-09-21T02:14:17.180846+0000","last_peered":"2024-09-21T02:14:17.180846+0000","last_clean":"2024-09-21T02:14:17.180846+0000","last_became_active":"2024-09-21T02:14:15.190820+0000","last_became_peered":"2024-09-21T02:14:15.190820+0000","last_unstale":"2024-09-21T02:14:17.180846+0000","last_undegraded":"2024-09-21T02:14:17.180846+0000","last_fullsized":"2024-09-21T02:14:17.180846+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:07.544915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182194+0000","last_change":"2024-09-21T02:14:15.212128+0000","last_active":"2024-09-21T02:14:17.182194+0000","last_peered":"2024-09-21T02:14:17.182194+0000","last_clean":"2024-09-21T02:14:17.182194+0000","last_became_active":"2024-09-21T02:14:15.211547+0000","last_became_peered":"2024-09-21T02:14:15.211547+0000","last_unstale":"2024-09-21T02:14:17.182194+0000","last_undegraded":"2024-09-21T02:14:17.182194+0000","last_fullsized":"2024-09-21T02:14:17.182194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:20.509138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167938+0000","last_change":"2024-09-21T02:14:15.172726+0000","last_active":"2024-09-21T02:14:16.167938+0000","last_peered":"2024-09-21T02:14:16.167938+0000","last_clean":"2024-09-21T02:14:16.167938+0000","last_became_active":"2024-09-21T02:14:15.171830+0000","last_became_peered":"2024-09-21T02:14:15.171830+0000","last_unstale":"2024-09-21T02:14:16.167938+0000","last_undegraded":"2024-09-21T02:14:16.167938+0000","last_fullsized":"2024-09-21T02:14:16.167938+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:11.129900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.185631+0000","last_change":"2024-09-21T02:14:15.227725+0000","last_active":"2024-09-21T02:14:17.185631+0000","last_peered":"2024-09-21T02:14:17.185631+0000","last_clean":"2024-09-21T02:14:17.185631+0000","last_became_active":"2024-09-21T02:14:15.227445+0000","last_became_peered":"2024-09-21T02:14:15.227445+0000","last_unstale":"2024-09-21T02:14:17.185631+0000","last_undegraded":"2024-09-21T02:14:17.185631+0000","last_fullsized":"2024-09-21T02:14:17.185631+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:11.129900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179092+0000","last_change":"2024-09-21T02:14:15.189074+0000","last_active":"2024-09-21T02:14:17.179092+0000","last_peered":"2024-09-21T02:14:17.179092+0000","last_clean":"2024-09-21T02:14:17.179092+0000","last_became_active":"2024-09-21T02:14:15.188662+0000","last_became_peered":"2024-09-21T02:14:15.188662+0000","last_unstale":"2024-09-21T02:14:17.179092+0000","last_undegraded":"2024-09-21T02:14:17.179092+0000","last_fullsized":"2024-09-21T02:14:17.179092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:32.044091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179152+0000","last_change":"2024-09-21T02:14:15.171330+0000","last_active":"2024-09-21T02:14:17.179152+0000","last_peered":"2024-09-21T02:14:17.179152+0000","last_clean":"2024-09-21T02:14:17.179152+0000","last_became_active":"2024-09-21T02:14:15.170440+0000","last_became_peered":"2024-09-21T02:14:15.170440+0000","last_unstale":"2024-09-21T02:14:17.179152+0000","last_undegraded":"2024-09-21T02:14:17.179152+0000","last_fullsized":"2024-09-21T02:14:17.179152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:03.903502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.183445+0000","last_change":"2024-09-21T02:14:15.210382+0000","last_active":"2024-09-21T02:14:17.183445+0000","last_peered":"2024-09-21T02:14:17.183445+0000","last_clean":"2024-09-21T02:14:17.183445+0000","last_became_active":"2024-09-21T02:14:15.210082+0000","last_became_peered":"2024-09-21T02:14:15.210082+0000","last_unstale":"2024-09-21T02:14:17.183445+0000","last_undegraded":"2024-09-21T02:14:17.183445+0000","last_fullsized":"2024-09-21T02:14:17.183445+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:03.893645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161967+0000","last_change":"2024-09-21T02:14:15.185248+0000","last_active":"2024-09-21T02:14:16.161967+0000","last_peered":"2024-09-21T02:14:16.161967+0000","last_clean":"2024-09-21T02:14:16.161967+0000","last_became_active":"2024-09-21T02:14:15.184881+0000","last_became_peered":"2024-09-21T02:14:15.184881+0000","last_unstale":"2024-09-21T02:14:16.161967+0000","last_undegraded":"2024-09-21T02:14:16.161967+0000","last_fullsized":"2024-09-21T02:14:16.161967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:25.642713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181083+0000","last_change":"2024-09-21T02:14:15.197089+0000","last_active":"2024-09-21T02:14:17.181083+0000","last_peered":"2024-09-21T02:14:17.181083+0000","last_clean":"2024-09-21T02:14:17.181083+0000","last_became_active":"2024-09-21T02:14:15.196068+0000","last_became_peered":"2024-09-21T02:14:15.196068+0000","last_unstale":"2024-09-21T02:14:17.181083+0000","last_undegraded":"2024-09-21T02:14:17.181083+0000","last_fullsized":"2024-09-21T02:14:17.181083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:46.961964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162311+0000","last_change":"2024-09-21T02:14:14.162293+0000","last_active":"2024-09-21T02:14:16.162311+0000","last_peered":"2024-09-21T02:14:16.162311+0000","last_clean":"2024-09-21T02:14:16.162311+0000","last_became_active":"2024-09-21T02:14:14.161294+0000","last_became_peered":"2024-09-21T02:14:14.161294+0000","last_unstale":"2024-09-21T02:14:16.162311+0000","last_undegraded":"2024-09-21T02:14:16.162311+0000","last_fullsized":"2024-09-21T02:14:16.162311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:31.033196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.183081+0000","last_change":"2024-09-21T02:14:15.199491+0000","last_active":"2024-09-21T02:14:17.183081+0000","last_peered":"2024-09-21T02:14:17.183081+0000","last_clean":"2024-09-21T02:14:17.183081+0000","last_became_active":"2024-09-21T02:14:15.199000+0000","last_became_peered":"2024-09-21T02:14:15.199000+0000","last_unstale":"2024-09-21T02:14:17.183081+0000","last_undegraded":"2024-09-21T02:14:17.183081+0000","last_fullsized":"2024-09-21T02:14:17.183081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:32.044091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182364+0000","last_change":"2024-09-21T02:14:14.180675+0000","last_active":"2024-09-21T02:14:17.182364+0000","last_peered":"2024-09-21T02:14:17.182364+0000","last_clean":"2024-09-21T02:14:17.182364+0000","last_became_active":"2024-09-21T02:14:14.179791+0000","last_became_peered":"2024-09-21T02:14:14.179791+0000","last_unstale":"2024-09-21T02:14:17.182364+0000","last_undegraded":"2024-09-21T02:14:17.182364+0000","last_fullsized":"2024-09-21T02:14:17.182364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:47.635791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182358+0000","last_change":"2024-09-21T02:14:15.222639+0000","last_active":"2024-09-21T02:14:17.182358+0000","last_peered":"2024-09-21T02:14:17.182358+0000","last_clean":"2024-09-21T02:14:17.182358+0000","last_became_active":"2024-09-21T02:14:15.222297+0000","last_became_peered":"2024-09-21T02:14:15.222297+0000","last_unstale":"2024-09-21T02:14:17.182358+0000","last_undegraded":"2024-09-21T02:14:17.182358+0000","last_fullsized":"2024-09-21T02:14:17.182358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:25.225188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.170956+0000","last_change":"2024-09-21T02:14:14.195324+0000","last_active":"2024-09-21T02:14:16.170956+0000","last_peered":"2024-09-21T02:14:16.170956+0000","last_clean":"2024-09-21T02:14:16.170956+0000","last_became_active":"2024-09-21T02:14:14.194573+0000","last_became_peered":"2024-09-21T02:14:14.194573+0000","last_unstale":"2024-09-21T02:14:16.170956+0000","last_undegraded":"2024-09-21T02:14:16.170956+0000","last_fullsized":"2024-09-21T02:14:16.170956+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:00.785055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181257+0000","last_change":"2024-09-21T02:14:15.201013+0000","last_active":"2024-09-21T02:14:17.181257+0000","last_peered":"2024-09-21T02:14:17.181257+0000","last_clean":"2024-09-21T02:14:17.181257+0000","last_became_active":"2024-09-21T02:14:15.199797+0000","last_became_peered":"2024-09-21T02:14:15.199797+0000","last_unstale":"2024-09-21T02:14:17.181257+0000","last_undegraded":"2024-09-21T02:14:17.181257+0000","last_fullsized":"2024-09-21T02:14:17.181257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:33.202747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180405+0000","last_change":"2024-09-21T02:14:14.195211+0000","last_active":"2024-09-21T02:14:17.180405+0000","last_peered":"2024-09-21T02:14:17.180405+0000","last_clean":"2024-09-21T02:14:17.180405+0000","last_became_active":"2024-09-21T02:14:14.194442+0000","last_became_peered":"2024-09-21T02:14:14.194442+0000","last_unstale":"2024-09-21T02:14:17.180405+0000","last_undegraded":"2024-09-21T02:14:17.180405+0000","last_fullsized":"2024-09-21T02:14:17.180405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:22.063757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179287+0000","last_change":"2024-09-21T02:14:15.175120+0000","last_active":"2024-09-21T02:14:17.179287+0000","last_peered":"2024-09-21T02:14:17.179287+0000","last_clean":"2024-09-21T02:14:17.179287+0000","last_became_active":"2024-09-21T02:14:15.174114+0000","last_became_peered":"2024-09-21T02:14:15.174114+0000","last_unstale":"2024-09-21T02:14:17.179287+0000","last_undegraded":"2024-09-21T02:14:17.179287+0000","last_fullsized":"2024-09-21T02:14:17.179287+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:57.464186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182141+0000","last_change":"2024-09-21T02:14:14.185340+0000","last_active":"2024-09-21T02:14:17.182141+0000","last_peered":"2024-09-21T02:14:17.182141+0000","last_clean":"2024-09-21T02:14:17.182141+0000","last_became_active":"2024-09-21T02:14:14.184685+0000","last_became_peered":"2024-09-21T02:14:14.184685+0000","last_unstale":"2024-09-21T02:14:17.182141+0000","last_undegraded":"2024-09-21T02:14:17.182141+0000","last_fullsized":"2024-09-21T02:14:17.182141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:08.234402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181931+0000","last_change":"2024-09-21T02:14:15.184350+0000","last_active":"2024-09-21T02:14:17.181931+0000","last_peered":"2024-09-21T02:14:17.181931+0000","last_clean":"2024-09-21T02:14:17.181931+0000","last_became_active":"2024-09-21T02:14:15.183887+0000","last_became_peered":"2024-09-21T02:14:15.183887+0000","last_unstale":"2024-09-21T02:14:17.181931+0000","last_undegraded":"2024-09-21T02:14:17.181931+0000","last_fullsized":"2024-09-21T02:14:17.181931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:12.109910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182606+0000","last_change":"2024-09-21T02:14:14.169524+0000","last_active":"2024-09-21T02:14:17.182606+0000","last_peered":"2024-09-21T02:14:17.182606+0000","last_clean":"2024-09-21T02:14:17.182606+0000","last_became_active":"2024-09-21T02:14:14.169203+0000","last_became_peered":"2024-09-21T02:14:14.169203+0000","last_unstale":"2024-09-21T02:14:17.182606+0000","last_undegraded":"2024-09-21T02:14:17.182606+0000","last_fullsized":"2024-09-21T02:14:17.182606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:59.463695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179448+0000","last_change":"2024-09-21T02:14:15.184514+0000","last_active":"2024-09-21T02:14:17.179448+0000","last_peered":"2024-09-21T02:14:17.179448+0000","last_clean":"2024-09-21T02:14:17.179448+0000","last_became_active":"2024-09-21T02:14:15.183053+0000","last_became_peered":"2024-09-21T02:14:15.183053+0000","last_unstale":"2024-09-21T02:14:17.179448+0000","last_undegraded":"2024-09-21T02:14:17.179448+0000","last_fullsized":"2024-09-21T02:14:17.179448+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:56.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168926+0000","last_change":"2024-09-21T02:14:14.195177+0000","last_active":"2024-09-21T02:14:16.168926+0000","last_peered":"2024-09-21T02:14:16.168926+0000","last_clean":"2024-09-21T02:14:16.168926+0000","last_became_active":"2024-09-21T02:14:14.194325+0000","last_became_peered":"2024-09-21T02:14:14.194325+0000","last_unstale":"2024-09-21T02:14:16.168926+0000","last_undegraded":"2024-09-21T02:14:16.168926+0000","last_fullsized":"2024-09-21T02:14:16.168926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:47.035638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179304+0000","last_change":"2024-09-21T02:14:15.188344+0000","last_active":"2024-09-21T02:14:17.179304+0000","last_peered":"2024-09-21T02:14:17.179304+0000","last_clean":"2024-09-21T02:14:17.179304+0000","last_became_active":"2024-09-21T02:14:15.187467+0000","last_became_peered":"2024-09-21T02:14:15.187467+0000","last_unstale":"2024-09-21T02:14:17.179304+0000","last_undegraded":"2024-09-21T02:14:17.179304+0000","last_fullsized":"2024-09-21T02:14:17.179304+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:22.010859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182170+0000","last_change":"2024-09-21T02:14:14.188869+0000","last_active":"2024-09-21T02:14:17.182170+0000","last_peered":"2024-09-21T02:14:17.182170+0000","last_clean":"2024-09-21T02:14:17.182170+0000","last_became_active":"2024-09-21T02:14:14.188597+0000","last_became_peered":"2024-09-21T02:14:14.188597+0000","last_unstale":"2024-09-21T02:14:17.182170+0000","last_undegraded":"2024-09-21T02:14:17.182170+0000","last_fullsized":"2024-09-21T02:14:17.182170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:39.147803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162228+0000","last_change":"2024-09-21T02:14:15.201068+0000","last_active":"2024-09-21T02:14:16.162228+0000","last_peered":"2024-09-21T02:14:16.162228+0000","last_clean":"2024-09-21T02:14:16.162228+0000","last_became_active":"2024-09-21T02:14:15.200639+0000","last_became_peered":"2024-09-21T02:14:15.200639+0000","last_unstale":"2024-09-21T02:14:16.162228+0000","last_undegraded":"2024-09-21T02:14:16.162228+0000","last_fullsized":"2024-09-21T02:14:16.162228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:56.023114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167676+0000","last_change":"2024-09-21T02:14:14.172476+0000","last_active":"2024-09-21T02:14:16.167676+0000","last_peered":"2024-09-21T02:14:16.167676+0000","last_clean":"2024-09-21T02:14:16.167676+0000","last_became_active":"2024-09-21T02:14:14.171464+0000","last_became_peered":"2024-09-21T02:14:14.171464+0000","last_unstale":"2024-09-21T02:14:16.167676+0000","last_undegraded":"2024-09-21T02:14:16.167676+0000","last_fullsized":"2024-09-21T02:14:16.167676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:36.340848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181442+0000","last_change":"2024-09-21T02:14:15.186681+0000","last_active":"2024-09-21T02:14:17.181442+0000","last_peered":"2024-09-21T02:14:17.181442+0000","last_clean":"2024-09-21T02:14:17.181442+0000","last_became_active":"2024-09-21T02:14:15.185804+0000","last_became_peered":"2024-09-21T02:14:15.185804+0000","last_unstale":"2024-09-21T02:14:17.181442+0000","last_undegraded":"2024-09-21T02:14:17.181442+0000","last_fullsized":"2024-09-21T02:14:17.181442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:12.770185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167766+0000","last_change":"2024-09-21T02:14:14.195269+0000","last_active":"2024-09-21T02:14:16.167766+0000","last_peered":"2024-09-21T02:14:16.167766+0000","last_clean":"2024-09-21T02:14:16.167766+0000","last_became_active":"2024-09-21T02:14:14.194411+0000","last_became_peered":"2024-09-21T02:14:14.194411+0000","last_unstale":"2024-09-21T02:14:16.167766+0000","last_undegraded":"2024-09-21T02:14:16.167766+0000","last_fullsized":"2024-09-21T02:14:16.167766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:57.402889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180186+0000","last_change":"2024-09-21T02:14:15.222984+0000","last_active":"2024-09-21T02:14:17.180186+0000","last_peered":"2024-09-21T02:14:17.180186+0000","last_clean":"2024-09-21T02:14:17.180186+0000","last_became_active":"2024-09-21T02:14:15.220840+0000","last_became_peered":"2024-09-21T02:14:15.220840+0000","last_unstale":"2024-09-21T02:14:17.180186+0000","last_undegraded":"2024-09-21T02:14:17.180186+0000","last_fullsized":"2024-09-21T02:14:17.180186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.417152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162015+0000","last_change":"2024-09-21T02:14:14.175499+0000","last_active":"2024-09-21T02:14:16.162015+0000","last_peered":"2024-09-21T02:14:16.162015+0000","last_clean":"2024-09-21T02:14:16.162015+0000","last_became_active":"2024-09-21T02:14:14.175116+0000","last_became_peered":"2024-09-21T02:14:14.175116+0000","last_unstale":"2024-09-21T02:14:16.162015+0000","last_undegraded":"2024-09-21T02:14:16.162015+0000","last_fullsized":"2024-09-21T02:14:16.162015+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:02.892607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162005+0000","last_change":"2024-09-21T02:14:15.214472+0000","last_active":"2024-09-21T02:14:16.162005+0000","last_peered":"2024-09-21T02:14:16.162005+0000","last_clean":"2024-09-21T02:14:16.162005+0000","last_became_active":"2024-09-21T02:14:15.214029+0000","last_became_peered":"2024-09-21T02:14:15.214029+0000","last_unstale":"2024-09-21T02:14:16.162005+0000","last_undegraded":"2024-09-21T02:14:16.162005+0000","last_fullsized":"2024-09-21T02:14:16.162005+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:36.089332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181328+0000","last_change":"2024-09-21T02:14:14.189722+0000","last_active":"2024-09-21T02:14:17.181328+0000","last_peered":"2024-09-21T02:14:17.181328+0000","last_clean":"2024-09-21T02:14:17.181328+0000","last_became_active":"2024-09-21T02:14:14.187906+0000","last_became_peered":"2024-09-21T02:14:14.187906+0000","last_unstale":"2024-09-21T02:14:17.181328+0000","last_undegraded":"2024-09-21T02:14:17.181328+0000","last_fullsized":"2024-09-21T02:14:17.181328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:55.597023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178572+0000","last_change":"2024-09-21T02:14:15.193675+0000","last_active":"2024-09-21T02:14:17.178572+0000","last_peered":"2024-09-21T02:14:17.178572+0000","last_clean":"2024-09-21T02:14:17.178572+0000","last_became_active":"2024-09-21T02:14:15.191752+0000","last_became_peered":"2024-09-21T02:14:15.191752+0000","last_unstale":"2024-09-21T02:14:17.178572+0000","last_undegraded":"2024-09-21T02:14:17.178572+0000","last_fullsized":"2024-09-21T02:14:17.178572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:53.689471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182917+0000","last_change":"2024-09-21T02:14:14.172530+0000","last_active":"2024-09-21T02:14:17.182917+0000","last_peered":"2024-09-21T02:14:17.182917+0000","last_clean":"2024-09-21T02:14:17.182917+0000","last_became_active":"2024-09-21T02:14:14.172101+0000","last_became_peered":"2024-09-21T02:14:14.172101+0000","last_unstale":"2024-09-21T02:14:17.182917+0000","last_undegraded":"2024-09-21T02:14:17.182917+0000","last_fullsized":"2024-09-21T02:14:17.182917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:20.517402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180319+0000","last_change":"2024-09-21T02:14:15.206105+0000","last_active":"2024-09-21T02:14:17.180319+0000","last_peered":"2024-09-21T02:14:17.180319+0000","last_clean":"2024-09-21T02:14:17.180319+0000","last_became_active":"2024-09-21T02:14:15.205311+0000","last_became_peered":"2024-09-21T02:14:15.205311+0000","last_unstale":"2024-09-21T02:14:17.180319+0000","last_undegraded":"2024-09-21T02:14:17.180319+0000","last_fullsized":"2024-09-21T02:14:17.180319+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:57.419153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179485+0000","last_change":"2024-09-21T02:14:14.180926+0000","last_active":"2024-09-21T02:14:17.179485+0000","last_peered":"2024-09-21T02:14:17.179485+0000","last_clean":"2024-09-21T02:14:17.179485+0000","last_became_active":"2024-09-21T02:14:14.179759+0000","last_became_peered":"2024-09-21T02:14:14.179759+0000","last_unstale":"2024-09-21T02:14:17.179485+0000","last_undegraded":"2024-09-21T02:14:17.179485+0000","last_fullsized":"2024-09-21T02:14:17.179485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:34.042299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.184450+0000","last_change":"2024-09-21T02:14:15.193038+0000","last_active":"2024-09-21T02:14:17.184450+0000","last_peered":"2024-09-21T02:14:17.184450+0000","last_clean":"2024-09-21T02:14:17.184450+0000","last_became_active":"2024-09-21T02:14:15.192236+0000","last_became_peered":"2024-09-21T02:14:15.192236+0000","last_unstale":"2024-09-21T02:14:17.184450+0000","last_undegraded":"2024-09-21T02:14:17.184450+0000","last_fullsized":"2024-09-21T02:14:17.184450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:00.074804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179525+0000","last_change":"2024-09-21T02:14:14.182658+0000","last_active":"2024-09-21T02:14:17.179525+0000","last_peered":"2024-09-21T02:14:17.179525+0000","last_clean":"2024-09-21T02:14:17.179525+0000","last_became_active":"2024-09-21T02:14:14.181767+0000","last_became_peered":"2024-09-21T02:14:14.181767+0000","last_unstale":"2024-09-21T02:14:17.179525+0000","last_undegraded":"2024-09-21T02:14:17.179525+0000","last_fullsized":"2024-09-21T02:14:17.179525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:46:32.022100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161899+0000","last_change":"2024-09-21T02:14:15.195531+0000","last_active":"2024-09-21T02:14:16.161899+0000","last_peered":"2024-09-21T02:14:16.161899+0000","last_clean":"2024-09-21T02:14:16.161899+0000","last_became_active":"2024-09-21T02:14:15.195215+0000","last_became_peered":"2024-09-21T02:14:15.195215+0000","last_unstale":"2024-09-21T02:14:16.161899+0000","last_undegraded":"2024-09-21T02:14:16.161899+0000","last_fullsized":"2024-09-21T02:14:16.161899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:27.908301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.166865+0000","last_change":"2024-09-21T02:14:14.192522+0000","last_active":"2024-09-21T02:14:16.166865+0000","last_peered":"2024-09-21T02:14:16.166865+0000","last_clean":"2024-09-21T02:14:16.166865+0000","last_became_active":"2024-09-21T02:14:14.191538+0000","last_became_peered":"2024-09-21T02:14:14.191538+0000","last_unstale":"2024-09-21T02:14:16.166865+0000","last_undegraded":"2024-09-21T02:14:16.166865+0000","last_fullsized":"2024-09-21T02:14:16.166865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:06.591082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162384+0000","last_change":"2024-09-21T02:14:15.213713+0000","last_active":"2024-09-21T02:14:16.162384+0000","last_peered":"2024-09-21T02:14:16.162384+0000","last_clean":"2024-09-21T02:14:16.162384+0000","last_became_active":"2024-09-21T02:14:15.213314+0000","last_became_peered":"2024-09-21T02:14:15.213314+0000","last_unstale":"2024-09-21T02:14:16.162384+0000","last_undegraded":"2024-09-21T02:14:16.162384+0000","last_fullsized":"2024-09-21T02:14:16.162384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:56.976743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.169408+0000","last_change":"2024-09-21T02:14:14.190309+0000","last_active":"2024-09-21T02:14:16.169408+0000","last_peered":"2024-09-21T02:14:16.169408+0000","last_clean":"2024-09-21T02:14:16.169408+0000","last_became_active":"2024-09-21T02:14:14.189460+0000","last_became_peered":"2024-09-21T02:14:14.189460+0000","last_unstale":"2024-09-21T02:14:16.169408+0000","last_undegraded":"2024-09-21T02:14:16.169408+0000","last_fullsized":"2024-09-21T02:14:16.169408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:56.982713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181931+0000","last_change":"2024-09-21T02:14:15.170769+0000","last_active":"2024-09-21T02:14:17.181931+0000","last_peered":"2024-09-21T02:14:17.181931+0000","last_clean":"2024-09-21T02:14:17.181931+0000","last_became_active":"2024-09-21T02:14:15.169962+0000","last_became_peered":"2024-09-21T02:14:15.169962+0000","last_unstale":"2024-09-21T02:14:17.181931+0000","last_undegraded":"2024-09-21T02:14:17.181931+0000","last_fullsized":"2024-09-21T02:14:17.181931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:22.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181341+0000","last_change":"2024-09-21T02:14:14.175339+0000","last_active":"2024-09-21T02:14:17.181341+0000","last_peered":"2024-09-21T02:14:17.181341+0000","last_clean":"2024-09-21T02:14:17.181341+0000","last_became_active":"2024-09-21T02:14:14.175001+0000","last_became_peered":"2024-09-21T02:14:14.175001+0000","last_unstale":"2024-09-21T02:14:17.181341+0000","last_undegraded":"2024-09-21T02:14:17.181341+0000","last_fullsized":"2024-09-21T02:14:17.181341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:07.175995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167494+0000","last_change":"2024-09-21T02:14:15.175776+0000","last_active":"2024-09-21T02:14:16.167494+0000","last_peered":"2024-09-21T02:14:16.167494+0000","last_clean":"2024-09-21T02:14:16.167494+0000","last_became_active":"2024-09-21T02:14:15.173231+0000","last_became_peered":"2024-09-21T02:14:15.173231+0000","last_unstale":"2024-09-21T02:14:16.167494+0000","last_undegraded":"2024-09-21T02:14:16.167494+0000","last_fullsized":"2024-09-21T02:14:16.167494+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:00.074804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162150+0000","last_change":"2024-09-21T02:14:14.170919+0000","last_active":"2024-09-21T02:14:16.162150+0000","last_peered":"2024-09-21T02:14:16.162150+0000","last_clean":"2024-09-21T02:14:16.162150+0000","last_became_active":"2024-09-21T02:14:14.170489+0000","last_became_peered":"2024-09-21T02:14:14.170489+0000","last_unstale":"2024-09-21T02:14:16.162150+0000","last_undegraded":"2024-09-21T02:14:16.162150+0000","last_fullsized":"2024-09-21T02:14:16.162150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:59.063909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.170135+0000","last_change":"2024-09-21T02:14:15.213284+0000","last_active":"2024-09-21T02:14:16.170135+0000","last_peered":"2024-09-21T02:14:16.170135+0000","last_clean":"2024-09-21T02:14:16.170135+0000","last_became_active":"2024-09-21T02:14:15.210729+0000","last_became_peered":"2024-09-21T02:14:15.210729+0000","last_unstale":"2024-09-21T02:14:16.170135+0000","last_undegraded":"2024-09-21T02:14:16.170135+0000","last_fullsized":"2024-09-21T02:14:16.170135+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:20.809947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180154+0000","last_change":"2024-09-21T02:14:14.189463+0000","last_active":"2024-09-21T02:14:17.180154+0000","last_peered":"2024-09-21T02:14:17.180154+0000","last_clean":"2024-09-21T02:14:17.180154+0000","last_became_active":"2024-09-21T02:14:14.188788+0000","last_became_peered":"2024-09-21T02:14:14.188788+0000","last_unstale":"2024-09-21T02:14:17.180154+0000","last_undegraded":"2024-09-21T02:14:17.180154+0000","last_fullsized":"2024-09-21T02:14:17.180154+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:59:11.699831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178370+0000","last_change":"2024-09-21T02:14:15.194757+0000","last_active":"2024-09-21T02:14:17.178370+0000","last_peered":"2024-09-21T02:14:17.178370+0000","last_clean":"2024-09-21T02:14:17.178370+0000","last_became_active":"2024-09-21T02:14:15.192774+0000","last_became_peered":"2024-09-21T02:14:15.192774+0000","last_unstale":"2024-09-21T02:14:17.178370+0000","last_undegraded":"2024-09-21T02:14:17.178370+0000","last_fullsized":"2024-09-21T02:14:17.178370+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:24.486565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167616+0000","last_change":"2024-09-21T02:14:14.190856+0000","last_active":"2024-09-21T02:14:16.167616+0000","last_peered":"2024-09-21T02:14:16.167616+0000","last_clean":"2024-09-21T02:14:16.167616+0000","last_became_active":"2024-09-21T02:14:14.189986+0000","last_became_peered":"2024-09-21T02:14:14.189986+0000","last_unstale":"2024-09-21T02:14:16.167616+0000","last_undegraded":"2024-09-21T02:14:16.167616+0000","last_fullsized":"2024-09-21T02:14:16.167616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:37.724200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.167824+0000","last_change":"2024-09-21T02:14:15.177019+0000","last_active":"2024-09-21T02:14:16.167824+0000","last_peered":"2024-09-21T02:14:16.167824+0000","last_clean":"2024-09-21T02:14:16.167824+0000","last_became_active":"2024-09-21T02:14:15.176179+0000","last_became_peered":"2024-09-21T02:14:15.176179+0000","last_unstale":"2024-09-21T02:14:16.167824+0000","last_undegraded":"2024-09-21T02:14:16.167824+0000","last_fullsized":"2024-09-21T02:14:16.167824+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:12.601935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182991+0000","last_change":"2024-09-21T02:14:14.173484+0000","last_active":"2024-09-21T02:14:17.182991+0000","last_peered":"2024-09-21T02:14:17.182991+0000","last_clean":"2024-09-21T02:14:17.182991+0000","last_became_active":"2024-09-21T02:14:14.173112+0000","last_became_peered":"2024-09-21T02:14:14.173112+0000","last_unstale":"2024-09-21T02:14:17.182991+0000","last_undegraded":"2024-09-21T02:14:17.182991+0000","last_fullsized":"2024-09-21T02:14:17.182991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:28.093076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178311+0000","last_change":"2024-09-21T02:14:15.224490+0000","last_active":"2024-09-21T02:14:17.178311+0000","last_peered":"2024-09-21T02:14:17.178311+0000","last_clean":"2024-09-21T02:14:17.178311+0000","last_became_active":"2024-09-21T02:14:15.219388+0000","last_became_peered":"2024-09-21T02:14:15.219388+0000","last_unstale":"2024-09-21T02:14:17.178311+0000","last_undegraded":"2024-09-21T02:14:17.178311+0000","last_fullsized":"2024-09-21T02:14:17.178311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:28.963430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180056+0000","last_change":"2024-09-21T02:14:14.194963+0000","last_active":"2024-09-21T02:14:17.180056+0000","last_peered":"2024-09-21T02:14:17.180056+0000","last_clean":"2024-09-21T02:14:17.180056+0000","last_became_active":"2024-09-21T02:14:14.194477+0000","last_became_peered":"2024-09-21T02:14:14.194477+0000","last_unstale":"2024-09-21T02:14:17.180056+0000","last_undegraded":"2024-09-21T02:14:17.180056+0000","last_fullsized":"2024-09-21T02:14:17.180056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:29:04.278971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180436+0000","last_change":"2024-09-21T02:14:15.185697+0000","last_active":"2024-09-21T02:14:17.180436+0000","last_peered":"2024-09-21T02:14:17.180436+0000","last_clean":"2024-09-21T02:14:17.180436+0000","last_became_active":"2024-09-21T02:14:15.183668+0000","last_became_peered":"2024-09-21T02:14:15.183668+0000","last_unstale":"2024-09-21T02:14:17.180436+0000","last_undegraded":"2024-09-21T02:14:17.180436+0000","last_fullsized":"2024-09-21T02:14:17.180436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:24.838687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161640+0000","last_change":"2024-09-21T02:14:14.174185+0000","last_active":"2024-09-21T02:14:16.161640+0000","last_peered":"2024-09-21T02:14:16.161640+0000","last_clean":"2024-09-21T02:14:16.161640+0000","last_became_active":"2024-09-21T02:14:14.173632+0000","last_became_peered":"2024-09-21T02:14:14.173632+0000","last_unstale":"2024-09-21T02:14:16.161640+0000","last_undegraded":"2024-09-21T02:14:16.161640+0000","last_fullsized":"2024-09-21T02:14:16.161640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:00.661562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168792+0000","last_change":"2024-09-21T02:14:15.212782+0000","last_active":"2024-09-21T02:14:16.168792+0000","last_peered":"2024-09-21T02:14:16.168792+0000","last_clean":"2024-09-21T02:14:16.168792+0000","last_became_active":"2024-09-21T02:14:15.211937+0000","last_became_peered":"2024-09-21T02:14:15.211937+0000","last_unstale":"2024-09-21T02:14:16.168792+0000","last_undegraded":"2024-09-21T02:14:16.168792+0000","last_fullsized":"2024-09-21T02:14:16.168792+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:19.944730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181308+0000","last_change":"2024-09-21T02:14:14.175791+0000","last_active":"2024-09-21T02:14:17.181308+0000","last_peered":"2024-09-21T02:14:17.181308+0000","last_clean":"2024-09-21T02:14:17.181308+0000","last_became_active":"2024-09-21T02:14:14.174962+0000","last_became_peered":"2024-09-21T02:14:14.174962+0000","last_unstale":"2024-09-21T02:14:17.181308+0000","last_undegraded":"2024-09-21T02:14:17.181308+0000","last_fullsized":"2024-09-21T02:14:17.181308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:37.666797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162331+0000","last_change":"2024-09-21T02:14:15.185845+0000","last_active":"2024-09-21T02:14:16.162331+0000","last_peered":"2024-09-21T02:14:16.162331+0000","last_clean":"2024-09-21T02:14:16.162331+0000","last_became_active":"2024-09-21T02:14:15.185563+0000","last_became_peered":"2024-09-21T02:14:15.185563+0000","last_unstale":"2024-09-21T02:14:16.162331+0000","last_undegraded":"2024-09-21T02:14:16.162331+0000","last_fullsized":"2024-09-21T02:14:16.162331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:57.860005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.168015+0000","last_change":"2024-09-21T02:14:14.194882+0000","last_active":"2024-09-21T02:14:16.168015+0000","last_peered":"2024-09-21T02:14:16.168015+0000","last_clean":"2024-09-21T02:14:16.168015+0000","last_became_active":"2024-09-21T02:14:14.194002+0000","last_became_peered":"2024-09-21T02:14:14.194002+0000","last_unstale":"2024-09-21T02:14:16.168015+0000","last_undegraded":"2024-09-21T02:14:16.168015+0000","last_fullsized":"2024-09-21T02:14:16.168015+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.730357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.170750+0000","last_change":"2024-09-21T02:14:15.220848+0000","last_active":"2024-09-21T02:14:16.170750+0000","last_peered":"2024-09-21T02:14:16.170750+0000","last_clean":"2024-09-21T02:14:16.170750+0000","last_became_active":"2024-09-21T02:14:15.219476+0000","last_became_peered":"2024-09-21T02:14:15.219476+0000","last_unstale":"2024-09-21T02:14:16.170750+0000","last_undegraded":"2024-09-21T02:14:16.170750+0000","last_fullsized":"2024-09-21T02:14:16.170750+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:57.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181594+0000","last_change":"2024-09-21T02:14:14.186532+0000","last_active":"2024-09-21T02:14:17.181594+0000","last_peered":"2024-09-21T02:14:17.181594+0000","last_clean":"2024-09-21T02:14:17.181594+0000","last_became_active":"2024-09-21T02:14:14.185831+0000","last_became_peered":"2024-09-21T02:14:14.185831+0000","last_unstale":"2024-09-21T02:14:17.181594+0000","last_undegraded":"2024-09-21T02:14:17.181594+0000","last_fullsized":"2024-09-21T02:14:17.181594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:22.319742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179804+0000","last_change":"2024-09-21T02:14:15.176750+0000","last_active":"2024-09-21T02:14:17.179804+0000","last_peered":"2024-09-21T02:14:17.179804+0000","last_clean":"2024-09-21T02:14:17.179804+0000","last_became_active":"2024-09-21T02:14:15.176379+0000","last_became_peered":"2024-09-21T02:14:15.176379+0000","last_unstale":"2024-09-21T02:14:17.179804+0000","last_undegraded":"2024-09-21T02:14:17.179804+0000","last_fullsized":"2024-09-21T02:14:17.179804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:11.129900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.181234+0000","last_change":"2024-09-21T02:14:14.175812+0000","last_active":"2024-09-21T02:14:17.181234+0000","last_peered":"2024-09-21T02:14:17.181234+0000","last_clean":"2024-09-21T02:14:17.181234+0000","last_became_active":"2024-09-21T02:14:14.175065+0000","last_became_peered":"2024-09-21T02:14:14.175065+0000","last_unstale":"2024-09-21T02:14:17.181234+0000","last_undegraded":"2024-09-21T02:14:17.181234+0000","last_fullsized":"2024-09-21T02:14:17.181234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:23.827792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.180922+0000","last_change":"2024-09-21T02:14:15.199669+0000","last_active":"2024-09-21T02:14:17.180922+0000","last_peered":"2024-09-21T02:14:17.180922+0000","last_clean":"2024-09-21T02:14:17.180922+0000","last_became_active":"2024-09-21T02:14:15.198854+0000","last_became_peered":"2024-09-21T02:14:15.198854+0000","last_unstale":"2024-09-21T02:14:17.180922+0000","last_undegraded":"2024-09-21T02:14:17.180922+0000","last_fullsized":"2024-09-21T02:14:17.180922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:19.959216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.178770+0000","last_change":"2024-09-21T02:14:14.177436+0000","last_active":"2024-09-21T02:14:17.178770+0000","last_peered":"2024-09-21T02:14:17.178770+0000","last_clean":"2024-09-21T02:14:17.178770+0000","last_became_active":"2024-09-21T02:14:14.176075+0000","last_became_peered":"2024-09-21T02:14:14.176075+0000","last_unstale":"2024-09-21T02:14:17.178770+0000","last_undegraded":"2024-09-21T02:14:17.178770+0000","last_fullsized":"2024-09-21T02:14:17.178770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:45.058048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179768+0000","last_change":"2024-09-21T02:14:15.200391+0000","last_active":"2024-09-21T02:14:17.179768+0000","last_peered":"2024-09-21T02:14:17.179768+0000","last_clean":"2024-09-21T02:14:17.179768+0000","last_became_active":"2024-09-21T02:14:15.196024+0000","last_became_peered":"2024-09-21T02:14:15.196024+0000","last_unstale":"2024-09-21T02:14:17.179768+0000","last_undegraded":"2024-09-21T02:14:17.179768+0000","last_fullsized":"2024-09-21T02:14:17.179768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:16.889915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182955+0000","last_change":"2024-09-21T02:14:14.172390+0000","last_active":"2024-09-21T02:14:17.182955+0000","last_peered":"2024-09-21T02:14:17.182955+0000","last_clean":"2024-09-21T02:14:17.182955+0000","last_became_active":"2024-09-21T02:14:14.172030+0000","last_became_peered":"2024-09-21T02:14:14.172030+0000","last_unstale":"2024-09-21T02:14:17.182955+0000","last_undegraded":"2024-09-21T02:14:17.182955+0000","last_fullsized":"2024-09-21T02:14:17.182955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.950716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.161928+0000","last_change":"2024-09-21T02:14:15.185342+0000","last_active":"2024-09-21T02:14:16.161928+0000","last_peered":"2024-09-21T02:14:16.161928+0000","last_clean":"2024-09-21T02:14:16.161928+0000","last_became_active":"2024-09-21T02:14:15.184980+0000","last_became_peered":"2024-09-21T02:14:15.184980+0000","last_unstale":"2024-09-21T02:14:16.161928+0000","last_undegraded":"2024-09-21T02:14:16.161928+0000","last_fullsized":"2024-09-21T02:14:16.161928+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:44.328587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179042+0000","last_change":"2024-09-21T02:14:14.180582+0000","last_active":"2024-09-21T02:14:17.179042+0000","last_peered":"2024-09-21T02:14:17.179042+0000","last_clean":"2024-09-21T02:14:17.179042+0000","last_became_active":"2024-09-21T02:14:14.179876+0000","last_became_peered":"2024-09-21T02:14:14.179876+0000","last_unstale":"2024-09-21T02:14:17.179042+0000","last_undegraded":"2024-09-21T02:14:17.179042+0000","last_fullsized":"2024-09-21T02:14:17.179042+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:29.774228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.179980+0000","last_change":"2024-09-21T02:14:15.201426+0000","last_active":"2024-09-21T02:14:17.179980+0000","last_peered":"2024-09-21T02:14:17.179980+0000","last_clean":"2024-09-21T02:14:17.179980+0000","last_became_active":"2024-09-21T02:14:15.201036+0000","last_became_peered":"2024-09-21T02:14:15.201036+0000","last_unstale":"2024-09-21T02:14:17.179980+0000","last_undegraded":"2024-09-21T02:14:17.179980+0000","last_fullsized":"2024-09-21T02:14:17.179980+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:14.134329+0000","last_clean_scrub_stamp":"2024-09-21T02:14:14.134329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:22.182160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:16.162236+0000","last_change":"2024-09-21T02:14:14.170824+0000","last_active":"2024-09-21T02:14:16.162236+0000","last_peered":"2024-09-21T02:14:16.162236+0000","last_clean":"2024-09-21T02:14:16.162236+0000","last_became_active":"2024-09-21T02:14:14.170474+0000","last_became_peered":"2024-09-21T02:14:14.170474+0000","last_unstale":"2024-09-21T02:14:16.162236+0000","last_undegraded":"2024-09-21T02:14:16.162236+0000","last_fullsized":"2024-09-21T02:14:16.162236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:13.123434+0000","last_clean_scrub_stamp":"2024-09-21T02:14:13.123434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:10.119005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":132,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T02:14:17.182486+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:17.182486+0000","last_peered":"2024-09-21T02:14:17.182486+0000","last_clean":"2024-09-21T02:14:17.182486+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:17.182486+0000","last_undegraded":"2024-09-21T02:14:17.182486+0000","last_fullsized":"2024-09-21T02:14:17.182486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":987136,"data_stored":741953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":405504,"data_stored":151779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":151037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":987136,"data_stored":741441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":393216,"data_stored":151093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":146337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:24.371 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:14:24.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:14:24.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 -- 172.21.15.170:0/3437476874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 msgr2=0x7f280c0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 --2- 172.21.15.170:0/3437476874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0ce540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f280c0720f0 tx=0x7f280801c8a0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 -- 172.21.15.170:0/3437476874 shutdown_connections 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 --2- 172.21.15.170:0/3437476874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280c06ef40 0x7f280c06f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 --2- 172.21.15.170:0/3437476874 >> v2:172.21.15.191:3300/0 conn(0x7f280c0cea80 0x7f280c0cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 --2- 172.21.15.170:0/3437476874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0ce540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 -- 172.21.15.170:0/3437476874 >> 172.21.15.170:0/3437476874 conn(0x7f280c0d0170 msgr2=0x7f280c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 -- 172.21.15.170:0/3437476874 shutdown_connections 2024-09-21T02:14:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 -- 172.21.15.170:0/3437476874 wait complete. 2024-09-21T02:14:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 Processor -- start 2024-09-21T02:14:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.580+0000 7f2814188640 1 -- start start 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280c06ef40 0x7f280c1fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f280c0cea80 0x7f280c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f280c070220 con 0x7f280c0b2fc0 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0700a0 con 0x7f280c0cea80 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0703a0 con 0x7f280c06ef40 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48028/0 (socket says 172.21.15.170:48028) 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 -- 172.21.15.170:0/3543197000 learned_addr learned my addr 172.21.15.170:0/3543197000 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2812725640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f280c0cea80 0x7f280c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811f24640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280c06ef40 0x7f280c1fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 -- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280c06ef40 msgr2=0x7f280c1fd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280c06ef40 0x7f280c1fd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 -- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f280c0cea80 msgr2=0x7f280c1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f280c0cea80 0x7f280c1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 -- 172.21.15.170:0/3543197000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c1f9260 con 0x7f280c0b2fc0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2812725640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f280c0cea80 0x7f280c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2811723640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0b24d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e7d0 tx=0x7f27fc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00ccc0 con 0x7f280c0b2fc0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27fc010430 con 0x7f280c0b2fc0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c1f9540 con 0x7f280c0b2fc0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc010860 con 0x7f280c0b2fc0 2024-09-21T02:14:24.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1f9b00 con 0x7f280c0b2fc0 2024-09-21T02:14:24.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c0ad1e0 con 0x7f280c0b2fc0 2024-09-21T02:14:24.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27fc010a00 con 0x7f280c0b2fc0 2024-09-21T02:14:24.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.584+0000 7f28027fc640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f27f0073200 0x7f27f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:24.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.588+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f27fc0953d0 con 0x7f280c0b2fc0 2024-09-21T02:14:24.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.588+0000 7f2811f24640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f27f0073200 0x7f27f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:24.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.588+0000 7f2811f24640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f27f0073200 0x7f27f0075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28080af000 tx=0x7f280801b0e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:24.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.588+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f27fc0604d0 con 0x7f280c0b2fc0 2024-09-21T02:14:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.828+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f280c15cbf0 con 0x7f280c0b2fc0 2024-09-21T02:14:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.832+0000 7f28027fc640 1 -- 172.21.15.170:0/3543197000 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f27fc05fdd0 con 0x7f280c0b2fc0 2024-09-21T02:14:24.838 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:14:24.838 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:14:24.838 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:14:24.838 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:14:24.838 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:14:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:14:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f27f0073200 msgr2=0x7f27f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f27f0073200 0x7f27f0075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28080af000 tx=0x7f280801b0e0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 msgr2=0x7f280c0b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0b24d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e7d0 tx=0x7f27fc00ec90 comp rx=0 tx=0).stop 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 shutdown_connections 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f27f0073200 0x7f27f0075710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f280c0cea80 0x7f280c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f280c0b2fc0 0x7f280c0b24d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 --2- 172.21.15.170:0/3543197000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280c06ef40 0x7f280c1fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 >> 172.21.15.170:0/3543197000 conn(0x7f280c0d0170 msgr2=0x7f280c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 shutdown_connections 2024-09-21T02:14:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:24.836+0000 7f2814188640 1 -- 172.21.15.170:0/3543197000 wait complete. 2024-09-21T02:14:24.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:25.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 -- 172.21.15.170:0/678293867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c40cadf0 msgr2=0x7fd1c40cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 --2- 172.21.15.170:0/678293867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c40cadf0 0x7fd1c40cb200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40b2d00 tx=0x7fd1bc01c6b0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 -- 172.21.15.170:0/678293867 shutdown_connections 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 --2- 172.21.15.170:0/678293867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c40cadf0 0x7fd1c40cb200 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 --2- 172.21.15.170:0/678293867 >> v2:172.21.15.191:3300/0 conn(0x7fd1c415f990 0x7fd1c4070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 -- 172.21.15.170:0/678293867 >> 172.21.15.170:0/678293867 conn(0x7fd1c40b1c40 msgr2=0x7fd1c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 -- 172.21.15.170:0/678293867 shutdown_connections 2024-09-21T02:14:25.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 -- 172.21.15.170:0/678293867 wait complete. 2024-09-21T02:14:25.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 Processor -- start 2024-09-21T02:14:25.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.012+0000 7fd1cad69640 1 -- start start 2024-09-21T02:14:25.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1c4070570 0x7fd1c415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1c40cadf0 0x7fd1c4156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 0x7fd1c40adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4165fe0 con 0x7fd1c415f990 2024-09-21T02:14:25.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c40cede0 con 0x7fd1c40cadf0 2024-09-21T02:14:25.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c4166160 con 0x7fd1c4070570 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 0x7fd1c40adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 0x7fd1c40adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48042/0 (socket says 172.21.15.170:48042) 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 -- 172.21.15.170:0/824243070 learned_addr learned my addr 172.21.15.170:0/824243070 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c8b05640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1c4070570 0x7fd1c415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c3fff640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1c40cadf0 0x7fd1c4156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 -- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1c4070570 msgr2=0x7fd1c415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1c4070570 0x7fd1c415ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 -- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1c40cadf0 msgr2=0x7fd1c4156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1c40cadf0 0x7fd1c4156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 -- 172.21.15.170:0/824243070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c415f5d0 con 0x7fd1c415f990 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c9306640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 0x7fd1c40adf30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009850 tx=0x7fd1bc0a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc0041c0 con 0x7fd1c415f990 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1bc004450 con 0x7fd1c415f990 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c409aaf0 con 0x7fd1c415f990 2024-09-21T02:14:25.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc005290 con 0x7fd1c415f990 2024-09-21T02:14:25.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c409b0b0 con 0x7fd1c415f990 2024-09-21T02:14:25.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.016+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c40cb200 con 0x7fd1c415f990 2024-09-21T02:14:25.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.020+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1bc0be020 con 0x7fd1c415f990 2024-09-21T02:14:25.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.020+0000 7fd1c17fa640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd1a4073200 0x7fd1a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.020+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd1bc12f2e0 con 0x7fd1c415f990 2024-09-21T02:14:25.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.020+0000 7fd1c8b05640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd1a4073200 0x7fd1a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.020+0000 7fd1c8b05640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd1a4073200 0x7fd1a4075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005fd0 tx=0x7fd1b400a780 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.024+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd1bc0c5020 con 0x7fd1c415f990 2024-09-21T02:14:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.228+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1c415f360 con 0x7fd1c415f990 2024-09-21T02:14:25.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1c17fa640 1 -- 172.21.15.170:0/824243070 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7fd1c40cb740 con 0x7fd1c415f990 2024-09-21T02:14:25.235 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:25.235 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"btime":"2024-09-21T02:14:18:191863+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:14.576721+0000","modified":"2024-09-21T02:14:17.607956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4914":{"gid":4914,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2504310121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2504310121},{"type":"v1","addr":"172.21.15.170:6835","nonce":2504310121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4914,"qdb_cluster":[4914]},"id":2}]} 2024-09-21T02:14:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 17 2024-09-21T02:14:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd1a4073200 msgr2=0x7fd1a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd1a4073200 0x7fd1a4075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005fd0 tx=0x7fd1b400a780 comp rx=0 tx=0).stop 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 msgr2=0x7fd1c40adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 0x7fd1c40adf30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009850 tx=0x7fd1bc0a6f60 comp rx=0 tx=0).stop 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 shutdown_connections 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd1a4073200 0x7fd1a4075710 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1c415f990 0x7fd1c40adf30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1c40cadf0 0x7fd1c4156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 --2- 172.21.15.170:0/824243070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1c4070570 0x7fd1c415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 >> 172.21.15.170:0/824243070 conn(0x7fd1c40b1c40 msgr2=0x7fd1c40cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 shutdown_connections 2024-09-21T02:14:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.232+0000 7fd1cad69640 1 -- 172.21.15.170:0/824243070 wait complete. 2024-09-21T02:14:25.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:14:14.576721+0000', 'modified': '2024-09-21T02:14:17.607956+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4914}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4914': {'gid': 4914, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/2504310121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2504310121}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2504310121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4914, 'qdb_cluster': [4914]} 2024-09-21T02:14:25.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:14:25.659 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:14:25.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 -- 172.21.15.170:0/3830368352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0ccb00 msgr2=0x7fb0dc0ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/3830368352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0ccb00 0x7fb0dc0ccf10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc070e90 tx=0x7fb0d401cd80 comp rx=0 tx=0).stop 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 -- 172.21.15.170:0/3830368352 shutdown_connections 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/3830368352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0ccb00 0x7fb0dc0ccf10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/3830368352 >> v2:172.21.15.191:3300/0 conn(0x7fb0dc0b2f40 0x7fb0dc0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 -- 172.21.15.170:0/3830368352 >> 172.21.15.170:0/3830368352 conn(0x7fb0dc0b1c40 msgr2=0x7fb0dc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 -- 172.21.15.170:0/3830368352 shutdown_connections 2024-09-21T02:14:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.820+0000 7fb0e2e62640 1 -- 172.21.15.170:0/3830368352 wait complete. 2024-09-21T02:14:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 Processor -- start 2024-09-21T02:14:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 -- start start 2024-09-21T02:14:25.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0b2f40 0x7fb0dc156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 0x7fb0dc14ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0ccb00 0x7fb0dc14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc169e50 con 0x7fb0dc0b5410 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc0cf970 con 0x7fb0dc0b2f40 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc169fd0 con 0x7fb0dc0ccb00 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 0x7fb0dc14ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 0x7fb0dc14ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56668/0 (socket says 172.21.15.170:56668) 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 -- 172.21.15.170:0/916889410 learned_addr learned my addr 172.21.15.170:0/916889410 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0b2f40 0x7fb0dc156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0ccb00 0x7fb0dc14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 -- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0ccb00 msgr2=0x7fb0dc14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0ccb00 0x7fb0dc14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 -- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0b2f40 msgr2=0x7fb0dc156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0b2f40 0x7fb0dc156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 -- 172.21.15.170:0/916889410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc14d4f0 con 0x7fb0dc0b5410 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 0x7fb0dc14ce40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80058d0 tx=0x7fb0c800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0d97fa640 1 -- 172.21.15.170:0/916889410 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8017070 con 0x7fb0dc0b5410 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0d97fa640 1 -- 172.21.15.170:0/916889410 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0c8009d20 con 0x7fb0dc0b5410 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc14d7d0 con 0x7fb0dc0b5410 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0d97fa640 1 -- 172.21.15.170:0/916889410 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80106c0 con 0x7fb0dc0b5410 2024-09-21T02:14:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.824+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc14dd90 con 0x7fb0dc0b5410 2024-09-21T02:14:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0d97fa640 1 -- 172.21.15.170:0/916889410 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0c801a070 con 0x7fb0dc0b5410 2024-09-21T02:14:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0d97fa640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb0c0073200 0x7fb0c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0d97fa640 1 -- 172.21.15.170:0/916889410 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c80943c0 con 0x7fb0dc0b5410 2024-09-21T02:14:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb0c0073200 0x7fb0c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0ea12d000 1 Processor -- start 2024-09-21T02:14:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0ea12d000 1 -- start start 2024-09-21T02:14:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.828+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb0c0073200 0x7fb0c0075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0000f70 tx=0x7fb0d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef2956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef144950 0x5612ef295be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef296120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612ef16a410 con 0x5612ef12dca0 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5612ef0a7b40 con 0x5612ef144950 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5612ef096f70 con 0x5612eed54460 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef296120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef296120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56670/0 (socket says 172.21.15.170:56670) 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 -- 172.21.15.170:0/1087701452 learned_addr learned my addr 172.21.15.170:0/1087701452 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef2956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef144950 0x5612ef295be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 -- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 msgr2=0x5612ef2956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef2956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 -- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef144950 msgr2=0x5612ef295be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef144950 0x5612ef295be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 -- 172.21.15.170:0/1087701452 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ef05e5b0 con 0x5612ef12dca0 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef144950 0x5612ef295be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef296120 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4019990 tx=0x7fb0d4019b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41922559bd5ce23a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0c67fc640 1 -- 172.21.15.170:0/1087701452 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40bd070 con 0x5612ef12dca0 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0c67fc640 1 -- 172.21.15.170:0/1087701452 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0d4004a50 con 0x5612ef12dca0 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0c67fc640 1 -- 172.21.15.170:0/1087701452 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40b2040 con 0x5612ef12dca0 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1087701452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 msgr2=0x5612ef296120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef296120 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4019990 tx=0x7fb0d4019b20 comp rx=0 tx=0).stop 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1087701452 shutdown_connections 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef296120 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef144950 0x5612ef295be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1087701452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef2956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1087701452 >> 172.21.15.170:0/1087701452 conn(0x5612ef21ad90 msgr2=0x5612ef24a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1087701452 shutdown_connections 2024-09-21T02:14:25.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1087701452 wait complete. 2024-09-21T02:14:25.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 -- start start 2024-09-21T02:14:25.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.832+0000 7fb0ea12d000 1 Processor -- start 2024-09-21T02:14:25.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612eed54460 0x5612ef29c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef29ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef144950 0x5612ef29d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612ef0315f0 con 0x5612ef12dca0 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5612ef09ad80 con 0x5612eed54460 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5612ef03d520 con 0x5612ef144950 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 10 client.0 _ll_register_callbacks cb 0x5612ef041830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef29ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef29ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56676/0 (socket says 172.21.15.170:56676) 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 -- 172.21.15.170:0/1195790920 learned_addr learned my addr 172.21.15.170:0/1195790920 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef144950 0x5612ef29d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612eed54460 0x5612ef29c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 -- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef144950 msgr2=0x5612ef29d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef144950 0x5612ef29d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 -- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612eed54460 msgr2=0x5612ef29c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612eed54460 0x5612ef29c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 -- 172.21.15.170:0/1195790920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ef035190 con 0x5612ef12dca0 2024-09-21T02:14:25.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef29ce60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8092910 tx=0x7fb0c805ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01972ff4d3772b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:14:25.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1195790920 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8021040 con 0x5612ef12dca0 2024-09-21T02:14:25.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1195790920 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c8049da0 con 0x5612ef12dca0 2024-09-21T02:14:25.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612ef02f110 con 0x5612ef12dca0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1195790920 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb0c805f3e0 con 0x5612ef12dca0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 10 client.5124 fetch_fsmap learned FSMap version 17 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5612ef11bf10 con 0x5612ef12dca0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1195790920 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5612ef02f110 con 0x5612ef12dca0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1195790920 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fb0c804c110 con 0x5612ef12dca0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 10 client.5124 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 10 client.5124 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 20 client.5124 populate_metadata read hostname 'smithi170' 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed397d0 0x5612ef2a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 4 client.5124 mds_command: new command op to 4914 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 --> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612ef0a9a90 con 0x5612eed397d0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.836+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed397d0 0x5612ef2a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0bb7fe640 10 client.5124 ms_handle_connect on v2:172.21.15.170:6834/2504310121 2024-09-21T02:14:25.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed397d0 0x5612ef2a6d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f773bb5ec059966c server_cookie=7e9d46eb94c30e57 in_seq=0 out_seq=0 2024-09-21T02:14:25.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1195790920 <== mds.0 v2:172.21.15.170:6834/2504310121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612ef0a9a90 con 0x5612eed397d0 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0bb7fe640 10 client.5124 handle_command_reply: tid=0 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 client.5124 shutdown 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 msgr2=0x5612ef29ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef29ce60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8092910 tx=0x7fb0c805ebe0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 shutdown_connections 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed397d0 0x5612ef2a6d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef144950 0x5612ef29d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef29ce60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0bb7fe640 0 client.5124 ms_handle_reset on v2:172.21.15.170:6834/2504310121 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1195790920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612eed54460 0x5612ef29c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 >> 172.21.15.170:0/1195790920 conn(0x5612ef21ad90 msgr2=0x5612ef251440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 shutdown_connections 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1195790920 wait complete. 2024-09-21T02:14:25.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.840+0000 7fb0ea12d000 20 client.5124 trim_cache size 0 max 16384 2024-09-21T02:14:25.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 Processor -- start 2024-09-21T02:14:25.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- start start 2024-09-21T02:14:25.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef307620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef307b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef3080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc203160 con 0x5612ef12dca0 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc2021c0 con 0x5612ef21ad90 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc157ca0 con 0x5612eed54460 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef3080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef3080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56682/0 (socket says 172.21.15.170:56682) 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 -- 172.21.15.170:0/3401715689 learned_addr learned my addr 172.21.15.170:0/3401715689 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef307620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef307b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 -- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 msgr2=0x5612ef307b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef307b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 -- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 msgr2=0x5612ef307620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef307620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 -- 172.21.15.170:0/3401715689 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612ef0f2db0 con 0x5612ef12dca0 2024-09-21T02:14:25.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef3080a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb0d401c7f0 tx=0x7fb0d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7d959013d89b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0c5ffb640 1 -- 172.21.15.170:0/3401715689 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40bd020 con 0x5612ef12dca0 2024-09-21T02:14:25.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0c5ffb640 1 -- 172.21.15.170:0/3401715689 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0d4004d40 con 0x5612ef12dca0 2024-09-21T02:14:25.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0c5ffb640 1 -- 172.21.15.170:0/3401715689 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40b2040 con 0x5612ef12dca0 2024-09-21T02:14:25.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- 172.21.15.170:0/3401715689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 msgr2=0x5612ef3080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef3080a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb0d401c7f0 tx=0x7fb0d40095c0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- 172.21.15.170:0/3401715689 shutdown_connections 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612eed54460 0x5612ef307b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612ef12dca0 0x5612ef3080a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/3401715689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef307620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- 172.21.15.170:0/3401715689 >> 172.21.15.170:0/3401715689 conn(0x5612ef251740 msgr2=0x5612ef293620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- 172.21.15.170:0/3401715689 shutdown_connections 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.860+0000 7fb0ea12d000 1 -- 172.21.15.170:0/3401715689 wait complete. 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- start start 2024-09-21T02:14:25.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 Processor -- start 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 0x5612ef30b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef12dca0 0x5612ef30bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef30c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612ef041760 con 0x5612eed54460 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5612ef0a6910 con 0x5612ef21ad90 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5612ef062ee0 con 0x5612ef12dca0 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 10 client.0 _ll_register_callbacks cb 0x7fb0c8009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 0x5612ef30b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 0x5612ef30b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56690/0 (socket says 172.21.15.170:56690) 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 -- 172.21.15.170:0/1913592413 learned_addr learned my addr 172.21.15.170:0/1913592413 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef12dca0 0x5612ef30bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef30c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 -- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef12dca0 msgr2=0x5612ef30bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef12dca0 0x5612ef30bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 -- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 msgr2=0x5612ef30c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef30c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 -- 172.21.15.170:0/1913592413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612ef05dab0 con 0x5612eed54460 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e13ff640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef30c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0e0bfe640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 0x5612ef30b7b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5612ef119f60 tx=0x7fb0d0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78e9fba0af9233d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1913592413 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00267d0 con 0x5612eed54460 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1913592413 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb0d0010040 con 0x5612eed54460 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d4004d40 con 0x5612eed54460 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1913592413 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb0d0037050 con 0x5612eed54460 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 10 client.5130 fetch_fsmap learned FSMap version 17 2024-09-21T02:14:25.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5612ef03c070 con 0x5612eed54460 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1913592413 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0d4004d40 con 0x5612eed54460 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1913592413 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fb0d003a020 con 0x5612eed54460 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 10 client.5130 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 10 client.5130 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 20 client.5130 populate_metadata read hostname 'smithi170' 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed52e80 0x5612ef2c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 4 client.5130 mds_command: new command op to 4914 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 --> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] -- command(tid 0: {"prefix": "status"}) -- 0x5612ef0a5540 con 0x5612eed52e80 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed52e80 0x5612ef2c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0bb7fe640 10 client.5130 ms_handle_connect on v2:172.21.15.170:6834/2504310121 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.864+0000 7fb0dbfff640 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed52e80 0x5612ef2c7950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2065ae14e57973f4 server_cookie=43228331111d4aa0 in_seq=0 out_seq=0 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0bb7fe640 1 -- 172.21.15.170:0/1913592413 <== mds.0 v2:172.21.15.170:6834/2504310121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5612ef0a5540 con 0x5612eed52e80 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0bb7fe640 10 client.5130 handle_command_reply: tid=0 2024-09-21T02:14:25.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 client.5130 shutdown 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 msgr2=0x5612ef30b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 0x5612ef30b7b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5612ef119f60 tx=0x7fb0d0000ee0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 shutdown_connections 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:6834/2504310121,v1:172.21.15.170:6835/2504310121] conn(0x5612eed52e80 0x5612ef2c7950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5612ef12dca0 0x5612ef30bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5612ef21ad90 0x5612ef30c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 --2- 172.21.15.170:0/1913592413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612eed54460 0x5612ef30b7b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0bb7fe640 0 client.5130 ms_handle_reset on v2:172.21.15.170:6834/2504310121 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 >> 172.21.15.170:0/1913592413 conn(0x5612ef251740 msgr2=0x5612ef2a66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 shutdown_connections 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 1 -- 172.21.15.170:0/1913592413 wait complete. 2024-09-21T02:14:25.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0ea12d000 20 client.5130 trim_cache size 0 max 16384 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4914, 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.696609279, 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 17, 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 35, 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 35, 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 17.581484482 2024-09-21T02:14:25.875 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:14:25.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb0c0073200 msgr2=0x7fb0c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb0c0073200 0x7fb0c0075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0000f70 tx=0x7fb0d0009040 comp rx=0 tx=0).stop 2024-09-21T02:14:25.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 msgr2=0x7fb0dc14ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:25.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 0x7fb0dc14ce40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80058d0 tx=0x7fb0c800dde0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 shutdown_connections 2024-09-21T02:14:25.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb0c0073200 0x7fb0c0075710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0ccb00 0x7fb0dc14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0dc0b5410 0x7fb0dc14ce40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 --2- 172.21.15.170:0/916889410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0b2f40 0x7fb0dc156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:25.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 >> 172.21.15.170:0/916889410 conn(0x7fb0dc0b1c40 msgr2=0x7fb0dc15c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:25.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 shutdown_connections 2024-09-21T02:14:25.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:25.868+0000 7fb0e2e62640 1 -- 172.21.15.170:0/916889410 wait complete. 2024-09-21T02:14:25.890 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/3679627166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7e0004d50 msgr2=0x7fd7e0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/3679627166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7e0004d50 0x7fd7e0005160 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c770 tx=0x7fd7ec01faa0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7f9e58640 1 -- 172.21.15.170:0/3679627166 reap_dead start 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/3679627166 shutdown_connections 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/3679627166 >> 172.21.15.170:0/3679627166 conn(0x7fd7f408bf50 msgr2=0x7fd7f409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/3679627166 shutdown_connections 2024-09-21T02:14:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/3679627166 wait complete. 2024-09-21T02:14:26.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 Processor -- start 2024-09-21T02:14:26.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.040+0000 7fd7fb8bb640 1 -- start start 2024-09-21T02:14:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f41f9830 0x7fd7f40cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 0x7fd7f41f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f41f9160 0x7fd7f4151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40d30e0 con 0x7fd7f414f440 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40d2f60 con 0x7fd7f41f9830 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f40d3260 con 0x7fd7f41f9160 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 0x7fd7f41f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 0x7fd7f41f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56708/0 (socket says 172.21.15.170:56708) 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 -- 172.21.15.170:0/1743362391 learned_addr learned my addr 172.21.15.170:0/1743362391 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f9657640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f41f9830 0x7fd7f40cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f9e58640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f41f9160 0x7fd7f4151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 -- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f41f9160 msgr2=0x7fd7f4151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f41f9160 0x7fd7f4151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 -- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f41f9830 msgr2=0x7fd7f40cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f41f9830 0x7fd7f40cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 -- 172.21.15.170:0/1743362391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f41f9570 con 0x7fd7f414f440 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f9e58640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f41f9160 0x7fd7f4151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7f8e56640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 0x7fd7f41f8c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4011a80 tx=0x7fd7e4011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400fe10 con 0x7fd7f414f440 2024-09-21T02:14:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7e4016070 con 0x7fd7f414f440 2024-09-21T02:14:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f40cdf10 con 0x7fd7f414f440 2024-09-21T02:14:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4012780 con 0x7fd7f414f440 2024-09-21T02:14:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f40cd150 con 0x7fd7f414f440 2024-09-21T02:14:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f40cf970 con 0x7fd7f414f440 2024-09-21T02:14:26.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd7e4018030 con 0x7fd7f414f440 2024-09-21T02:14:26.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.044+0000 7fd7e9ffb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7d8073230 0x7fd7d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:26.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.048+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd7e4094a20 con 0x7fd7f414f440 2024-09-21T02:14:26.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.048+0000 7fd7f9657640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7d8073230 0x7fd7d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:26.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.048+0000 7fd7f9657640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7d8073230 0x7fd7d8075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40a08e0 tx=0x7fd7f00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:26.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.052+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd7f41f9570 con 0x7fd7f414f440 2024-09-21T02:14:26.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.244+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd7f406fa30 con 0x7fd7f414f440 2024-09-21T02:14:26.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.244+0000 7fd7e9ffb640 1 -- 172.21.15.170:0/1743362391 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fd7e405fad0 con 0x7fd7f414f440 2024-09-21T02:14:26.249 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:14:26.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7d8073230 msgr2=0x7fd7d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:26.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7d8073230 0x7fd7d8075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40a08e0 tx=0x7fd7f00091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 msgr2=0x7fd7f41f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 0x7fd7f41f8c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4011a80 tx=0x7fd7e4011f40 comp rx=0 tx=0).stop 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 shutdown_connections 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7d8073230 0x7fd7d8075740 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7f41f9160 0x7fd7f4151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7f414f440 0x7fd7f41f8c20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 --2- 172.21.15.170:0/1743362391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7f41f9830 0x7fd7f40cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 >> 172.21.15.170:0/1743362391 conn(0x7fd7f408bf50 msgr2=0x7fd7f40b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 shutdown_connections 2024-09-21T02:14:26.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26.248+0000 7fd7fb8bb640 1 -- 172.21.15.170:0/1743362391 wait complete. 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:14:26.269 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:14:26.270 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:14:26.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.270 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:14:26.276 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:14:26.277 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:14:26.278 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:26.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.279 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:14:26.330 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:14:26.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.331 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:14:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:14:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:14:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:14:26.379 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:14:26.380 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:14:26.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:14:26.380 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:14:26.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:14:26.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:14:26.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:14:26.380 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:14:26.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:14:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:14:26.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.497 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:14:26.502 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:14:26.502 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:14:26.502 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:14:26.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:14:26.556 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:14:26.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:14:26.556 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:26.606 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:14:26.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:14:26.606 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:26.654 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:14:26.711 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:14:26.759 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:14:26.760 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:14:26.761 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:26.762 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:26.762 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:26.762 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:26.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.763 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:14:26.810 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:14:26.810 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:14:26.855 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:14:26.870 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:14:26.922 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:14:26.923 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:26.924 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:26.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:26.925 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:14:26.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:14:26.960+0000 7f911e286580 -1 init, newargv = 0x7f9108001260 newargc=15 2024-09-21T02:14:26.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[36947]: starting ceph client 2024-09-21T02:14:26.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[36947]: starting fuse 2024-09-21T02:14:28.070 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:14:28.087 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:14:28.137 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:14:28.137 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:14:28.137 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:14:28.137 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:14:28.138 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:28.139 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:14:28.140 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:28.140 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:14:28.140 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:28.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:28.141 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:14:28.187 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:14:28.187 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:14:28.187 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:14:28.187 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:14:28.187 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:14:28.187 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:14:28.187 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:14:28.187 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:14:28.188 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:14:28.189 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:14:28.189 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:14:28.189 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:14:28.189 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:14:28.189 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:14:28.311 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.36947.asok 2024-09-21T02:14:28.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:14:28.323 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36947.asok 2024-09-21T02:14:28.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:28.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36947.asok status 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "36947", 2024-09-21T02:14:28.503 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 5145, 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 5145 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 1627125716 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:28.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 1627125716 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.5145 192.168.0.1:0/1627125716", 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/1627125716", 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 17, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 35, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:14:28.505 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:14:28.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:28.520 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:14:28.520 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:28.526 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:14:28.527 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:28.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:28.527 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:28.586 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:14:28.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 -- 172.21.15.170:0/1672653500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8004810 msgr2=0x7f1ca8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:28.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/1672653500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8004810 0x7f1ca8004c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0016f0 tx=0x7f1cac01c770 comp rx=0 tx=0).stop 2024-09-21T02:14:28.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 -- 172.21.15.170:0/1672653500 shutdown_connections 2024-09-21T02:14:28.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/1672653500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8004810 0x7f1ca8004c20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:28.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 -- 172.21.15.170:0/1672653500 >> 172.21.15.170:0/1672653500 conn(0x7f1cbc09d7d0 msgr2=0x7f1cbc0d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:28.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 -- 172.21.15.170:0/1672653500 shutdown_connections 2024-09-21T02:14:28.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.784+0000 7f1cc32ed640 1 -- 172.21.15.170:0/1672653500 wait complete. 2024-09-21T02:14:28.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 Processor -- start 2024-09-21T02:14:28.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 -- start start 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ca8004810 0x7f1cbc0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cbc0ca470 0x7f1cbc0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 0x7f1cbc1f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc06df00 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc06dd80 con 0x7f1cbc0ca470 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc06e080 con 0x7f1ca8004810 2024-09-21T02:14:28.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 0x7f1cbc1f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 0x7f1cbc1f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56726/0 (socket says 172.21.15.170:56726) 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 -- 172.21.15.170:0/4045976167 learned_addr learned my addr 172.21.15.170:0/4045976167 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc0888640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cbc0ca470 0x7f1cbc0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 -- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ca8004810 msgr2=0x7f1cbc0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ca8004810 0x7f1cbc0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 -- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cbc0ca470 msgr2=0x7f1cbc0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cbc0ca470 0x7f1cbc0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 -- 172.21.15.170:0/4045976167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc0cda50 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc0888640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cbc0ca470 0x7f1cbc0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc188a640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 0x7f1cbc1f5170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0b6dd0 tx=0x7f1cb400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4003db0 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb4018070 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc0cdcf0 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400cc10 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.788+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc0ce2b0 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.792+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc0b5fd0 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.792+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1cb4004030 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.792+0000 7f1cb1ffb640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ca0073230 0x7f1ca0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:28.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.792+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1cb4095270 con 0x7f1cbc0cadc0 2024-09-21T02:14:28.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.792+0000 7f1cc1089640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ca0073230 0x7f1ca0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:28.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.792+0000 7f1cc1089640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ca0073230 0x7f1ca0075740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8001250 tx=0x7f1cb8001120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:28.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:28.796+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1cb4060290 con 0x7f1cbc0cadc0 2024-09-21T02:14:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.012+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1cbc0d4be0 con 0x7f1cbc0cadc0 2024-09-21T02:14:29.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.012+0000 7f1cb1ffb640 1 -- 172.21.15.170:0/4045976167 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f1cb405fb90 con 0x7f1cbc0cadc0 2024-09-21T02:14:29.017 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:14:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ca0073230 msgr2=0x7f1ca0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ca0073230 0x7f1ca0075740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8001250 tx=0x7f1cb8001120 comp rx=0 tx=0).stop 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 msgr2=0x7f1cbc1f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 0x7f1cbc1f5170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0b6dd0 tx=0x7f1cb400ee10 comp rx=0 tx=0).stop 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 shutdown_connections 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1ca0073230 0x7f1ca0075740 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cbc0cadc0 0x7f1cbc1f5170 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cbc0ca470 0x7f1cbc0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 --2- 172.21.15.170:0/4045976167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ca8004810 0x7f1cbc0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 >> 172.21.15.170:0/4045976167 conn(0x7f1cbc09d7d0 msgr2=0x7f1cbc0b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 shutdown_connections 2024-09-21T02:14:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:29.016+0000 7f1cc32ed640 1 -- 172.21.15.170:0/4045976167 wait complete. 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:14:29.039 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:14:29.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.039 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:14:29.047 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42440sec preferred_lft 42440sec 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.048 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:14:29.049 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:14:29.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.050 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:14:29.103 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:14:29.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.105 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:14:29.157 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:14:29.158 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:14:29.159 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:14:29.159 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:14:29.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:14:29.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:14:29.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:14:29.159 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:14:29.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:14:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:14:29.272 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:14:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:14:29.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.280 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:14:29.286 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:14:29.286 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:14:29.286 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:14:29.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.286 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:14:29.339 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:14:29.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:14:29.339 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:29.392 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:14:29.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:14:29.392 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:29.439 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:14:29.497 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:14:29.544 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:14:29.545 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:14:29.546 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:29.547 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:29.547 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:29.547 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:29.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.548 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:14:29.596 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:14:29.596 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:14:29.641 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:14:29.655 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:14:29.703 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:14:29.704 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:14:29.705 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:14:29.706 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:29.706 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:29.706 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:29.706 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:29.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:29.707 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:14:29.749 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:14:29.745+0000 7f7adf1cd580 -1 init, newargv = 0x7f7acc004bb0 newargc=15 2024-09-21T02:14:29.749 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34110]: starting ceph client 2024-09-21T02:14:29.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34110]: starting fuse 2024-09-21T02:14:30.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:14:30.869 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:14:30.917 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:14:30.918 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:14:30.919 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:30.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:30.920 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:14:30.968 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:14:30.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:14:30.968 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:14:30.969 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:14:30.970 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:14:31.094 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.34110.asok 2024-09-21T02:14:31.096 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:14:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:14:31.104 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34110.asok 2024-09-21T02:14:31.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:31.105 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34110.asok status 2024-09-21T02:14:31.288 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:14:31.288 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:14:31.288 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "34110", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 4721, 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 4721 2024-09-21T02:14:31.289 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 193300708 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 193300708 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.4721 192.168.0.1:0/193300708", 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/193300708", 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:14:31.290 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 17, 2024-09-21T02:14:31.291 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 35, 2024-09-21T02:14:31.291 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:14:31.291 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:14:31.291 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:14:31.291 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:14:31.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:31.304 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:14:31.304 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:31.309 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:14:31.310 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:31.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:31.310 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:31.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 -- 172.21.15.170:0/4052613440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 msgr2=0x7fad400cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 --2- 172.21.15.170:0/4052613440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 0x7fad400cab30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad3400aa10 tx=0x7fad3401e690 comp rx=0 tx=0).stop 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 -- 172.21.15.170:0/4052613440 shutdown_connections 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 --2- 172.21.15.170:0/4052613440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 0x7fad400cab30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 --2- 172.21.15.170:0/4052613440 >> v2:172.21.15.191:3300/0 conn(0x7fad4009dc00 0x7fad4015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 -- 172.21.15.170:0/4052613440 >> 172.21.15.170:0/4052613440 conn(0x7fad400b1c40 msgr2=0x7fad400b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 -- 172.21.15.170:0/4052613440 shutdown_connections 2024-09-21T02:14:31.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 -- 172.21.15.170:0/4052613440 wait complete. 2024-09-21T02:14:31.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 Processor -- start 2024-09-21T02:14:31.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.531+0000 7fad44d69640 1 -- start start 2024-09-21T02:14:31.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad4009dc00 0x7fad400741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:31.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 0x7fad401643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:31.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 0x7fad4009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:31.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fad40168440 con 0x7fad4015e9f0 2024-09-21T02:14:31.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad401682c0 con 0x7fad4009dc00 2024-09-21T02:14:31.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fad401685c0 con 0x7fad400ca720 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 0x7fad4009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 0x7fad4009a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56732/0 (socket says 172.21.15.170:56732) 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 -- 172.21.15.170:0/3857537745 learned_addr learned my addr 172.21.15.170:0/3857537745 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3e59c640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad4009dc00 0x7fad400741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3dd9b640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 0x7fad401643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 -- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 msgr2=0x7fad401643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 0x7fad401643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 -- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad4009dc00 msgr2=0x7fad400741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad4009dc00 0x7fad400741d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 -- 172.21.15.170:0/3857537745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4015b690 con 0x7fad4015e9f0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3e59c640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad4009dc00 0x7fad400741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3ed9d640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 0x7fad4009a9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c6e0 tx=0x7fad3800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38003b80 con 0x7fad4015e9f0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad38003d20 con 0x7fad4015e9f0 2024-09-21T02:14:31.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad400b4ac0 con 0x7fad4015e9f0 2024-09-21T02:14:31.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38005330 con 0x7fad4015e9f0 2024-09-21T02:14:31.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad400b4d60 con 0x7fad4015e9f0 2024-09-21T02:14:31.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad40151e30 con 0x7fad4015e9f0 2024-09-21T02:14:31.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad380054d0 con 0x7fad4015e9f0 2024-09-21T02:14:31.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad2effd640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fad20073200 0x7fad20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fad38093fb0 con 0x7fad4015e9f0 2024-09-21T02:14:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.535+0000 7fad3e59c640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fad20073200 0x7fad20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.539+0000 7fad3e59c640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fad20073200 0x7fad20075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad30005fd0 tx=0x7fad30005f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:31.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.543+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fad400cb070 con 0x7fad4015e9f0 2024-09-21T02:14:31.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.751+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fad4015ad30 con 0x7fad4015e9f0 2024-09-21T02:14:31.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.959+0000 7fad2effd640 1 -- 172.21.15.170:0/3857537745 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7fad3805f060 con 0x7fad4015e9f0 2024-09-21T02:14:31.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fad20073200 msgr2=0x7fad20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:31.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fad20073200 0x7fad20075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad30005fd0 tx=0x7fad30005f40 comp rx=0 tx=0).stop 2024-09-21T02:14:31.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 msgr2=0x7fad4009a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:31.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 0x7fad4009a9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c6e0 tx=0x7fad3800cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 shutdown_connections 2024-09-21T02:14:31.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fad20073200 0x7fad20075710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad400ca720 0x7fad401643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4015e9f0 0x7fad4009a9c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 --2- 172.21.15.170:0/3857537745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad4009dc00 0x7fad400741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:31.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 >> 172.21.15.170:0/3857537745 conn(0x7fad400b1c40 msgr2=0x7fad4015a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:31.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 shutdown_connections 2024-09-21T02:14:31.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:31.963+0000 7fad44d69640 1 -- 172.21.15.170:0/3857537745 wait complete. 2024-09-21T02:14:31.993 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 -- 172.21.15.170:0/2875299835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6400b25f0 msgr2=0x7fc6400cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 --2- 172.21.15.170:0/2875299835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6400b25f0 0x7fc6400cf950 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc638009c70 tx=0x7fc63801cd20 comp rx=0 tx=0).stop 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 -- 172.21.15.170:0/2875299835 shutdown_connections 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 --2- 172.21.15.170:0/2875299835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc640160610 0x7fc6400b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 --2- 172.21.15.170:0/2875299835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6400b25f0 0x7fc6400cf950 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 -- 172.21.15.170:0/2875299835 >> 172.21.15.170:0/2875299835 conn(0x7fc64009a9c0 msgr2=0x7fc6400d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 -- 172.21.15.170:0/2875299835 shutdown_connections 2024-09-21T02:14:32.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.139+0000 7fc647742640 1 -- 172.21.15.170:0/2875299835 wait complete. 2024-09-21T02:14:32.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 Processor -- start 2024-09-21T02:14:32.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 -- start start 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6400b25f0 0x7fc6400cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64015faf0 0x7fc640154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 0x7fc640154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc640168f40 con 0x7fc640160610 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc640168dc0 con 0x7fc6400b25f0 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc6401690c0 con 0x7fc64015faf0 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 0x7fc640154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 0x7fc640154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56748/0 (socket says 172.21.15.170:56748) 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 -- 172.21.15.170:0/451142492 learned_addr learned my addr 172.21.15.170:0/451142492 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc6454de640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6400b25f0 0x7fc6400cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc644cdd640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64015faf0 0x7fc640154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 -- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64015faf0 msgr2=0x7fc640154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64015faf0 0x7fc640154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 -- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6400b25f0 msgr2=0x7fc6400cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6400b25f0 0x7fc6400cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 -- 172.21.15.170:0/451142492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc640150960 con 0x7fc640160610 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc6454de640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6400b25f0 0x7fc6400cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc645cdf640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 0x7fc640154ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00b440 tx=0x7fc63c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c015020 con 0x7fc640160610 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc63c011940 con 0x7fc640160610 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc640151460 con 0x7fc640160610 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c011d90 con 0x7fc640160610 2024-09-21T02:14:32.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.143+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc640151700 con 0x7fc640160610 2024-09-21T02:14:32.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.147+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6400cf950 con 0x7fc640160610 2024-09-21T02:14:32.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.147+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc63c018020 con 0x7fc640160610 2024-09-21T02:14:32.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.147+0000 7fc631ffb640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc624073200 0x7fc624075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.147+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc63c0944c0 con 0x7fc640160610 2024-09-21T02:14:32.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.147+0000 7fc6454de640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc624073200 0x7fc624075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.147+0000 7fc6454de640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc624073200 0x7fc624075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6380af000 tx=0x7fc63801b560 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:32.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.151+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc63c01f020 con 0x7fc640160610 2024-09-21T02:14:32.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.343+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc640150bd0 con 0x7fc624073200 2024-09-21T02:14:32.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.347+0000 7fc631ffb640 1 -- 172.21.15.170:0/451142492 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc640150bd0 con 0x7fc624073200 2024-09-21T02:14:32.352 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:14:32.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:32.352 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:14:32.352 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:32.352 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:14:32.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc624073200 msgr2=0x7fc624075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc624073200 0x7fc624075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6380af000 tx=0x7fc63801b560 comp rx=0 tx=0).stop 2024-09-21T02:14:32.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 msgr2=0x7fc640154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 0x7fc640154ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00b440 tx=0x7fc63c00b900 comp rx=0 tx=0).stop 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 shutdown_connections 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc624073200 0x7fc624075710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64015faf0 0x7fc640154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640160610 0x7fc640154ab0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 --2- 172.21.15.170:0/451142492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6400b25f0 0x7fc6400cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 >> 172.21.15.170:0/451142492 conn(0x7fc64009a9c0 msgr2=0x7fc64006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 shutdown_connections 2024-09-21T02:14:32.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.351+0000 7fc647742640 1 -- 172.21.15.170:0/451142492 wait complete. 2024-09-21T02:14:32.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 -- 172.21.15.170:0/1738189521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80cf7d0 msgr2=0x7f10e8073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/1738189521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80cf7d0 0x7f10e8073b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f10e80b61b0 tx=0x7f10d801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 -- 172.21.15.170:0/1738189521 shutdown_connections 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/1738189521 >> v2:172.21.15.191:3300/0 conn(0x7f10e80744b0 0x7f10e80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/1738189521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80cf7d0 0x7f10e8073b60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 -- 172.21.15.170:0/1738189521 >> 172.21.15.170:0/1738189521 conn(0x7f10e80d0170 msgr2=0x7f10e80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:32.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 -- 172.21.15.170:0/1738189521 shutdown_connections 2024-09-21T02:14:32.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 -- 172.21.15.170:0/1738189521 wait complete. 2024-09-21T02:14:32.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 Processor -- start 2024-09-21T02:14:32.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.511+0000 7f10ecdf2640 1 -- start start 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 0x7f10e8165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10e80744b0 0x7f10e806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e80cf7d0 0x7f10e81637f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80cee40 con 0x7f10e80740a0 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10e80cecc0 con 0x7f10e80cf7d0 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f10e8168cb0 con 0x7f10e80744b0 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 0x7f10e8165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 0x7f10e8165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56776/0 (socket says 172.21.15.170:56776) 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 -- 172.21.15.170:0/3687563665 learned_addr learned my addr 172.21.15.170:0/3687563665 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e5d9b640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10e80744b0 0x7f10e806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 -- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10e80744b0 msgr2=0x7f10e806fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10e80744b0 0x7f10e806fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 -- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e80cf7d0 msgr2=0x7f10e81637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e6d9d640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e80cf7d0 0x7f10e81637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e80cf7d0 0x7f10e81637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 -- 172.21.15.170:0/3687563665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81fc330 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10e659c640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 0x7f10e8165b60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10d80b0040 tx=0x7f10d80a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80b2040 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d80042b0 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81fc610 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8003600 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81fcbd0 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e8073b60 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.515+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f10d80bf020 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.519+0000 7f10d6ffd640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f10c8073230 0x7f10c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.519+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f10d812e5e0 con 0x7f10e80740a0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.519+0000 7f10e5d9b640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f10c8073230 0x7f10c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:32.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.519+0000 7f10e5d9b640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f10c8073230 0x7f10c8075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f10e8165470 tx=0x7f10dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:32.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.523+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f10d80c6020 con 0x7f10e80740a0 2024-09-21T02:14:32.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.687+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f10e80b3ae0 con 0x7f10e80740a0 2024-09-21T02:14:32.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.687+0000 7f10d6ffd640 1 -- 172.21.15.170:0/3687563665 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f10d80164b0 con 0x7f10e80740a0 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f10c8073230 msgr2=0x7f10c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f10c8073230 0x7f10c8075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f10e8165470 tx=0x7f10dc008040 comp rx=0 tx=0).stop 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 msgr2=0x7f10e8165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 0x7f10e8165b60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10d80b0040 tx=0x7f10d80a5790 comp rx=0 tx=0).stop 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 shutdown_connections 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f10c8073230 0x7f10c8075740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f10e80744b0 0x7f10e806fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e80cf7d0 0x7f10e81637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 --2- 172.21.15.170:0/3687563665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10e80740a0 0x7f10e8165b60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 >> 172.21.15.170:0/3687563665 conn(0x7f10e80d0170 msgr2=0x7f10e80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 shutdown_connections 2024-09-21T02:14:32.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:32.691+0000 7f10ecdf2640 1 -- 172.21.15.170:0/3687563665 wait complete. 2024-09-21T02:14:32.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:32.713 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp72220/subvol78810) 2024-09-21T02:14:32.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:32.737 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:14:32.792 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:14:32.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:32.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp72220/subvol78810) 2024-09-21T02:14:32.854 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:32.854 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp72220/subvol78810/.meta 2024-09-21T02:14:32.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol78810 subvol_grp72220 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 -- 172.21.15.170:0/3109679093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569006f2d0 msgr2=0x7f569006f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 --2- 172.21.15.170:0/3109679093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569006f2d0 0x7f569006f6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5688009c80 tx=0x7f568801cda0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 -- 172.21.15.170:0/3109679093 shutdown_connections 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 --2- 172.21.15.170:0/3109679093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569006f2d0 0x7f569006f6e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 --2- 172.21.15.170:0/3109679093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56900ca720 0x7f56900cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 -- 172.21.15.170:0/3109679093 >> 172.21.15.170:0/3109679093 conn(0x7f56900b1c40 msgr2=0x7f56900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 -- 172.21.15.170:0/3109679093 shutdown_connections 2024-09-21T02:14:33.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.119+0000 7f5695907640 1 -- 172.21.15.170:0/3109679093 wait complete. 2024-09-21T02:14:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 Processor -- start 2024-09-21T02:14:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 -- start start 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569006f2d0 0x7f569015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 0x7f56900b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56900ca720 0x7f56900b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5690168850 con 0x7f569009dc00 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f56901686d0 con 0x7f569006f2d0 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5695907640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f56901689d0 con 0x7f56900ca720 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 0x7f56900b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 0x7f56900b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56804/0 (socket says 172.21.15.170:56804) 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 -- 172.21.15.170:0/4224105512 learned_addr learned my addr 172.21.15.170:0/4224105512 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568effd640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569006f2d0 0x7f569015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568f7fe640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56900ca720 0x7f56900b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 -- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56900ca720 msgr2=0x7f56900b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56900ca720 0x7f56900b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 -- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569006f2d0 msgr2=0x7f569015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569006f2d0 0x7f569015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 -- 172.21.15.170:0/4224105512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569015de80 con 0x7f569009dc00 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568effd640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569006f2d0 0x7f569015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f568e7fc640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 0x7f56900b48d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f568400c970 tx=0x7f568400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.123+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684004220 con 0x7f569009dc00 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56840079b0 con 0x7f569009dc00 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569015e160 con 0x7f569009dc00 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840044b0 con 0x7f569009dc00 2024-09-21T02:14:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5690159da0 con 0x7f569009dc00 2024-09-21T02:14:33.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569015e680 con 0x7f569009dc00 2024-09-21T02:14:33.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5684003600 con 0x7f569009dc00 2024-09-21T02:14:33.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5677fff640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f567c073200 0x7f567c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5684094340 con 0x7f569009dc00 2024-09-21T02:14:33.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f568effd640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f567c073200 0x7f567c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.127+0000 7f568effd640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f567c073200 0x7f567c075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5680001700 tx=0x7f5680009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.131+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5684014030 con 0x7f569009dc00 2024-09-21T02:14:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.331+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol78810", "group_name": "subvol_grp72220", "target": ["mon-mgr", ""]}) -- 0x7f569015dc10 con 0x7f567c073200 2024-09-21T02:14:33.346 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:33.341+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:33.346 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:33.341+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:33.346 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:33.341+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:33.346 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:33.341+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:33.346 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:33.341+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:33.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.487+0000 7f5677fff640 1 -- 172.21.15.170:0/4224105512 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f569015dc10 con 0x7f567c073200 2024-09-21T02:14:33.492 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp72220/subvol78810 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f567c073200 msgr2=0x7f567c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f567c073200 0x7f567c075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5680001700 tx=0x7f5680009040 comp rx=0 tx=0).stop 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 msgr2=0x7f56900b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 0x7f56900b48d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f568400c970 tx=0x7f568400ce30 comp rx=0 tx=0).stop 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 shutdown_connections 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f567c073200 0x7f567c075710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56900ca720 0x7f56900b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f569009dc00 0x7f56900b48d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 --2- 172.21.15.170:0/4224105512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569006f2d0 0x7f569015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 >> 172.21.15.170:0/4224105512 conn(0x7f56900b1c40 msgr2=0x7f56901698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 shutdown_connections 2024-09-21T02:14:33.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.491+0000 7f5695907640 1 -- 172.21.15.170:0/4224105512 wait complete. 2024-09-21T02:14:33.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol78810 subvol_grp72220 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 -- 172.21.15.170:0/4197663291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a00ca720 msgr2=0x7fa5a00cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/4197663291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a00ca720 0x7fa5a00cab30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa588009900 tx=0x7fa58801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 -- 172.21.15.170:0/4197663291 shutdown_connections 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/4197663291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a006f2d0 0x7fa5a006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/4197663291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a00ca720 0x7fa5a00cab30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 -- 172.21.15.170:0/4197663291 >> 172.21.15.170:0/4197663291 conn(0x7fa5a00b1c40 msgr2=0x7fa5a00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 -- 172.21.15.170:0/4197663291 shutdown_connections 2024-09-21T02:14:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 -- 172.21.15.170:0/4197663291 wait complete. 2024-09-21T02:14:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 Processor -- start 2024-09-21T02:14:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.675+0000 7fa5a5e30640 1 -- start start 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a006f2d0 0x7fa5a015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 0x7fa5a015eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a00ca720 0x7fa5a009ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0165ee0 con 0x7fa5a009dc00 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0165d60 con 0x7fa5a006f2d0 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a0166060 con 0x7fa5a00ca720 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 0x7fa5a015eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 0x7fa5a015eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56818/0 (socket says 172.21.15.170:56818) 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 -- 172.21.15.170:0/1994339270 learned_addr learned my addr 172.21.15.170:0/1994339270 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59ffff640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a00ca720 0x7fa5a009ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59f7fe640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a006f2d0 0x7fa5a015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 -- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a00ca720 msgr2=0x7fa5a009ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a00ca720 0x7fa5a009ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 -- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a006f2d0 msgr2=0x7fa5a015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a006f2d0 0x7fa5a015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 -- 172.21.15.170:0/1994339270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a0159b40 con 0x7fa5a009dc00 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa59effd640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 0x7fa5a015eec0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5880b0040 tx=0x7fa588005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5880b2040 con 0x7fa5a009dc00 2024-09-21T02:14:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa588004330 con 0x7fa5a009dc00 2024-09-21T02:14:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a015b6d0 con 0x7fa5a009dc00 2024-09-21T02:14:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588005280 con 0x7fa5a009dc00 2024-09-21T02:14:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a015bc90 con 0x7fa5a009dc00 2024-09-21T02:14:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.679+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a006f6e0 con 0x7fa5a009dc00 2024-09-21T02:14:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.683+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa5880be060 con 0x7fa5a009dc00 2024-09-21T02:14:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.683+0000 7fa597fff640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa580073230 0x7fa580075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.683+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa58812d920 con 0x7fa5a009dc00 2024-09-21T02:14:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.683+0000 7fa59f7fe640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa580073230 0x7fa580075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.683+0000 7fa59f7fe640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa580073230 0x7fa580075740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa598001e80 tx=0x7fa598009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:33.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.687+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa5a0159b40 con 0x7fa5a009dc00 2024-09-21T02:14:33.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.891+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol78810", "group_name": "subvol_grp72220", "target": ["mon-mgr", ""]}) -- 0x7fa5a00cb3c0 con 0x7fa580073230 2024-09-21T02:14:33.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.903+0000 7fa597fff640 1 -- 172.21.15.170:0/1994339270 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa5a00cb3c0 con 0x7fa580073230 2024-09-21T02:14:33.907 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp72220/subvol78810 2024-09-21T02:14:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.903+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa580073230 msgr2=0x7fa580075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.903+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa580073230 0x7fa580075740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa598001e80 tx=0x7fa598009040 comp rx=0 tx=0).stop 2024-09-21T02:14:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.903+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 msgr2=0x7fa5a015eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.903+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 0x7fa5a015eec0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5880b0040 tx=0x7fa588005e70 comp rx=0 tx=0).stop 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 shutdown_connections 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa580073230 0x7fa580075740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a00ca720 0x7fa5a009ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a009dc00 0x7fa5a015eec0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 --2- 172.21.15.170:0/1994339270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a006f2d0 0x7fa5a015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 >> 172.21.15.170:0/1994339270 conn(0x7fa5a00b1c40 msgr2=0x7fa5a00ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 shutdown_connections 2024-09-21T02:14:33.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:33.907+0000 7fa5a5e30640 1 -- 172.21.15.170:0/1994339270 wait complete. 2024-09-21T02:14:33.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:33.929 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/b837dc2299c527aa5472b590b3280378.meta) 2024-09-21T02:14:38.305 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:14:38.305 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:14:38.306 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:14:38.306 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/subvol_grp72220/subvol78810 2024-09-21T02:14:38.306 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:14:38.306 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:38.307 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol78810 subvol_grp72220 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- 172.21.15.170:0/2207056246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f4004d50 msgr2=0x7f36f4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 --2- 172.21.15.170:0/2207056246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f4004d50 0x7f36f4005200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37080c8480 tx=0x7f370001d5b0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f3707fff640 1 -- 172.21.15.170:0/2207056246 reap_dead start 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- 172.21.15.170:0/2207056246 shutdown_connections 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- 172.21.15.170:0/2207056246 >> 172.21.15.170:0/2207056246 conn(0x7f3708074320 msgr2=0x7f37080d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- 172.21.15.170:0/2207056246 shutdown_connections 2024-09-21T02:14:38.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- 172.21.15.170:0/2207056246 wait complete. 2024-09-21T02:14:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 Processor -- start 2024-09-21T02:14:38.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- start start 2024-09-21T02:14:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f4004d50 0x7f370806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 0x7f37081f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37081f75a0 0x7f37080cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37080ae960 con 0x7f37081f3f70 2024-09-21T02:14:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37080ae7e0 con 0x7f36f4004d50 2024-09-21T02:14:38.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.475+0000 7f370e235640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f37080aeae0 con 0x7f37081f75a0 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 0x7f37081f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 0x7f37081f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53596/0 (socket says 172.21.15.170:53596) 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 -- 172.21.15.170:0/2998727368 learned_addr learned my addr 172.21.15.170:0/2998727368 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f37077fe640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f4004d50 0x7f370806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3707fff640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37081f75a0 0x7f37080cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 -- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37081f75a0 msgr2=0x7f37080cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37081f75a0 0x7f37080cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 -- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f4004d50 msgr2=0x7f370806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f4004d50 0x7f370806d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 -- 172.21.15.170:0/2998727368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37081f34d0 con 0x7f37081f3f70 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3707fff640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37081f75a0 0x7f37080cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f3706ffd640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 0x7f37081f7060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36f0000fe0 tx=0x7f36f000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00118a0 con 0x7f37081f3f70 2024-09-21T02:14:38.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f0011b30 con 0x7f37081f3f70 2024-09-21T02:14:38.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370809e060 con 0x7f37081f3f70 2024-09-21T02:14:38.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0012470 con 0x7f37081f3f70 2024-09-21T02:14:38.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081f3a50 con 0x7f37081f3f70 2024-09-21T02:14:38.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370809ddc0 con 0x7f37081f3f70 2024-09-21T02:14:38.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 97285+0+0 (secure 0 0 0) 0x7f36f0012610 con 0x7f37081f3f70 2024-09-21T02:14:38.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f36fffff640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f36e80732c0 0x7f36e80757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f36f00979b0 con 0x7f37081f3f70 2024-09-21T02:14:38.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.479+0000 7f37077fe640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f36e80732c0 0x7f36e80757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.483+0000 7f37077fe640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f36e80732c0 0x7f36e80757d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000c10 tx=0x7f36f8006f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:38.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.483+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f370809ddc0 con 0x7f37081f3f70 2024-09-21T02:14:38.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.675+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol78810", "group_name": "subvol_grp72220", "target": ["mon-mgr", ""]}) -- 0x7f37080b4f10 con 0x7f36e80732c0 2024-09-21T02:14:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.711+0000 7f36fffff640 1 -- 172.21.15.170:0/2998727368 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37080b4f10 con 0x7f36e80732c0 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f36e80732c0 msgr2=0x7f36e80757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f36e80732c0 0x7f36e80757d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000c10 tx=0x7f36f8006f00 comp rx=0 tx=0).stop 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 msgr2=0x7f37081f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 0x7f37081f7060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f36f0000fe0 tx=0x7f36f000fa30 comp rx=0 tx=0).stop 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 shutdown_connections 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f36e80732c0 0x7f36e80757d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37081f75a0 0x7f37080cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37081f3f70 0x7f37081f7060 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 --2- 172.21.15.170:0/2998727368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f4004d50 0x7f370806d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 >> 172.21.15.170:0/2998727368 conn(0x7f3708074320 msgr2=0x7f370815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 shutdown_connections 2024-09-21T02:14:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.715+0000 7f370e235640 1 -- 172.21.15.170:0/2998727368 wait complete. 2024-09-21T02:14:38.725 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.721+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.725 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.721+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.725 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.721+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.725 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.721+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.725 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.721+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.737 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:14:38.746 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.741+0000 7f1a44df9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.747 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.741+0000 7f1a44df9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.747 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.741+0000 7f1a44df9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.747 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.741+0000 7f1a44df9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.747 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:14:38.741+0000 7f1a44df9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:14:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:14:38.779 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:14:38.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:14:38.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp72220 2024-09-21T02:14:38.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.975+0000 7f0e30172640 1 -- 172.21.15.170:0/3023504798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 msgr2=0x7f0e2806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.975+0000 7f0e30172640 1 --2- 172.21.15.170:0/3023504798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2806ff50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e28072940 tx=0x7f0e1801c800 comp rx=0 tx=0).stop 2024-09-21T02:14:38.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.975+0000 7f0e30172640 1 -- 172.21.15.170:0/3023504798 shutdown_connections 2024-09-21T02:14:38.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.975+0000 7f0e30172640 1 --2- 172.21.15.170:0/3023504798 >> v2:172.21.15.191:3300/0 conn(0x7f0e280cf060 0x7f0e28070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.975+0000 7f0e30172640 1 --2- 172.21.15.170:0/3023504798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2806ff50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.975+0000 7f0e30172640 1 -- 172.21.15.170:0/3023504798 >> 172.21.15.170:0/3023504798 conn(0x7f0e280d0170 msgr2=0x7f0e280d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 -- 172.21.15.170:0/3023504798 shutdown_connections 2024-09-21T02:14:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 -- 172.21.15.170:0/3023504798 wait complete. 2024-09-21T02:14:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 Processor -- start 2024-09-21T02:14:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 -- start start 2024-09-21T02:14:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e280b2fc0 0x7f0e280afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e280cf060 0x7f0e2815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28169f50 con 0x7f0e280cec50 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28169dd0 con 0x7f0e280cf060 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e30172640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2816a0d0 con 0x7f0e280b2fc0 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53612/0 (socket says 172.21.15.170:53612) 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 -- 172.21.15.170:0/1014781253 learned_addr learned my addr 172.21.15.170:0/1014781253 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2e70f640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e280cf060 0x7f0e2815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 -- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e280b2fc0 msgr2=0x7f0e280afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e280b2fc0 0x7f0e280afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 -- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e280cf060 msgr2=0x7f0e2815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e280cf060 0x7f0e2815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2d70d640 1 -- 172.21.15.170:0/1014781253 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e281f7990 con 0x7f0e280cec50 2024-09-21T02:14:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.979+0000 7f0e2e70f640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e280cf060 0x7f0e2815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e2d70d640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2815c4c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e180982f0 tx=0x7f0e18005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e180b2040 con 0x7f0e280cec50 2024-09-21T02:14:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e180bb070 con 0x7f0e280cec50 2024-09-21T02:14:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18003600 con 0x7f0e280cec50 2024-09-21T02:14:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e280b0110 con 0x7f0e280cec50 2024-09-21T02:14:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281f6eb0 con 0x7f0e280cec50 2024-09-21T02:14:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e280b0390 con 0x7f0e280cec50 2024-09-21T02:14:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 97285+0+0 (secure 0 0 0) 0x7f0e180037a0 con 0x7f0e280cec50 2024-09-21T02:14:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e1e7fc640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0e0c073290 0x7f0e0c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0e1812e750 con 0x7f0e280cec50 2024-09-21T02:14:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e2df0e640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0e0c073290 0x7f0e0c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.983+0000 7f0e2df0e640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0e0c073290 0x7f0e0c0757a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e240046c0 tx=0x7f0e24007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:38.987+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0e180163f0 con 0x7f0e280cec50 2024-09-21T02:14:39.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.171+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp72220", "target": ["mon-mgr", ""]}) -- 0x7f0e280b3cf0 con 0x7f0e0c073290 2024-09-21T02:14:39.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.183+0000 7f0e1e7fc640 1 -- 172.21.15.170:0/1014781253 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e280b3cf0 con 0x7f0e0c073290 2024-09-21T02:14:39.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0e0c073290 msgr2=0x7f0e0c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0e0c073290 0x7f0e0c0757a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e240046c0 tx=0x7f0e24007910 comp rx=0 tx=0).stop 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 msgr2=0x7f0e2815c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2815c4c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e180982f0 tx=0x7f0e18005d20 comp rx=0 tx=0).stop 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 shutdown_connections 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0e0c073290 0x7f0e0c0757a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e280cf060 0x7f0e2815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e280cec50 0x7f0e2815c4c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 --2- 172.21.15.170:0/1014781253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e280b2fc0 0x7f0e280afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 >> 172.21.15.170:0/1014781253 conn(0x7f0e280d0170 msgr2=0x7f0e280c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 shutdown_connections 2024-09-21T02:14:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.187+0000 7f0e30172640 1 -- 172.21.15.170:0/1014781253 wait complete. 2024-09-21T02:14:39.211 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:14:39.211 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:14:39.283 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:14:39.353 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:14:39.353 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:14:39.353 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:39.353 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:39.359 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:39.359 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:39.407 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- 172.21.15.170:0/2631207862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0b3fa0 msgr2=0x7f35bc0cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 --2- 172.21.15.170:0/2631207862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0b3fa0 0x7f35bc0cf970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009840 tx=0x7f35ac01c790 comp rx=0 tx=0).stop 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- 172.21.15.170:0/2631207862 shutdown_connections 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 --2- 172.21.15.170:0/2631207862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0b3fa0 0x7f35bc0cf970 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- 172.21.15.170:0/2631207862 >> 172.21.15.170:0/2631207862 conn(0x7f35bc08bf50 msgr2=0x7f35bc09d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- 172.21.15.170:0/2631207862 shutdown_connections 2024-09-21T02:14:39.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- 172.21.15.170:0/2631207862 wait complete. 2024-09-21T02:14:39.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 Processor -- start 2024-09-21T02:14:39.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- start start 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35bc0b1230 0x7f35bc1f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35bc0b3fa0 0x7f35bc154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 0x7f35bc0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc16d020 con 0x7f35bc0d0e50 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc16cea0 con 0x7f35bc0b1230 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35c11c1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f35bc16d1a0 con 0x7f35bc0b3fa0 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35bb59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 0x7f35bc0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35bb59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 0x7f35bc0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53628/0 (socket says 172.21.15.170:53628) 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.611+0000 7f35bb59e640 1 -- 172.21.15.170:0/1610913428 learned_addr learned my addr 172.21.15.170:0/1610913428 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bad9d640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35bc0b1230 0x7f35bc1f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35ba59c640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35bc0b3fa0 0x7f35bc154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bb59e640 1 -- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35bc0b3fa0 msgr2=0x7f35bc154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bb59e640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35bc0b3fa0 0x7f35bc154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bb59e640 1 -- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35bc0b1230 msgr2=0x7f35bc1f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bb59e640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35bc0b1230 0x7f35bc1f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bb59e640 1 -- 172.21.15.170:0/1610913428 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc15acd0 con 0x7f35bc0d0e50 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bb59e640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 0x7f35bc0cb5d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f35b400caa0 tx=0x7f35b400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4016020 con 0x7f35bc0d0e50 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35b400a8d0 con 0x7f35bc0d0e50 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b400ad20 con 0x7f35bc0d0e50 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc15afb0 con 0x7f35bc0d0e50 2024-09-21T02:14:39.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc1f90b0 con 0x7f35bc0d0e50 2024-09-21T02:14:39.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35bc0cf970 con 0x7f35bc0d0e50 2024-09-21T02:14:39.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 97285+0+0 (secure 0 0 0) 0x7f35b4019020 con 0x7f35bc0d0e50 2024-09-21T02:14:39.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35a37fe640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f35a8073290 0x7f35a80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:39.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f35b40941b0 con 0x7f35bc0d0e50 2024-09-21T02:14:39.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.615+0000 7f35bad9d640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f35a8073290 0x7f35a80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:39.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.619+0000 7f35bad9d640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f35a8073290 0x7f35a80757a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0af000 tx=0x7f35ac01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:39.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.623+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f35b4020070 con 0x7f35bc0d0e50 2024-09-21T02:14:39.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.807+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35bc15b180 con 0x7f35bc0d0e50 2024-09-21T02:14:39.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.807+0000 7f35a37fe640 1 -- 172.21.15.170:0/1610913428 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f35bc0cfeb0 con 0x7f35bc0d0e50 2024-09-21T02:14:39.812 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:39.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f35a8073290 msgr2=0x7f35a80757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f35a8073290 0x7f35a80757a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0af000 tx=0x7f35ac01afd0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 msgr2=0x7f35bc0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 0x7f35bc0cb5d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f35b400caa0 tx=0x7f35b400cb90 comp rx=0 tx=0).stop 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 shutdown_connections 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f35a8073290 0x7f35a80757a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35bc0b3fa0 0x7f35bc154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0d0e50 0x7f35bc0cb5d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 --2- 172.21.15.170:0/1610913428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35bc0b1230 0x7f35bc1f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 >> 172.21.15.170:0/1610913428 conn(0x7f35bc08bf50 msgr2=0x7f35bc0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 shutdown_connections 2024-09-21T02:14:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:39.811+0000 7f35c11c1640 1 -- 172.21.15.170:0/1610913428 wait complete. 2024-09-21T02:14:39.833 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:14:39.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:39.833 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:14:39.842 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[36947]: fuse finished with error 0 and tester_r 0 2024-09-21T02:14:39.843 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:14:39.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:39.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:14:39.937 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:14:39.937 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:39.937 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:39.943 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:39.944 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:39.996 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:14:40.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2882349290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cb460 msgr2=0x7fab280cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2882349290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cb460 0x7fab280cb870 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009840 tx=0x7fab1c01c820 comp rx=0 tx=0).stop 2024-09-21T02:14:40.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2882349290 shutdown_connections 2024-09-21T02:14:40.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2882349290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab280ce1c0 0x7fab280ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2882349290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cb460 0x7fab280cb870 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2882349290 >> 172.21.15.170:0/2882349290 conn(0x7fab280b1c40 msgr2=0x7fab280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:40.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2882349290 shutdown_connections 2024-09-21T02:14:40.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2882349290 wait complete. 2024-09-21T02:14:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.143+0000 7fab2d0cc640 1 Processor -- start 2024-09-21T02:14:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- start start 2024-09-21T02:14:40.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab280cb460 0x7fab281f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab280ce1c0 0x7fab28148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 0x7fab28149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab281690d0 con 0x7fab280cf7e0 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fab28168f50 con 0x7fab280ce1c0 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fab28169250 con 0x7fab280cb460 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 0x7fab28149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 0x7fab28149390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53646/0 (socket says 172.21.15.170:53646) 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 -- 172.21.15.170:0/2355833684 learned_addr learned my addr 172.21.15.170:0/2355833684 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab26d9d640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab280cb460 0x7fab281f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2659c640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab280ce1c0 0x7fab28148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 -- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab280cb460 msgr2=0x7fab281f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab280cb460 0x7fab281f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 -- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab280ce1c0 msgr2=0x7fab28148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab280ce1c0 0x7fab28148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 -- 172.21.15.170:0/2355833684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2806a8a0 con 0x7fab280cf7e0 2024-09-21T02:14:40.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2759e640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 0x7fab28149390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fab2000ca80 tx=0x7fab2000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:40.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20007d90 con 0x7fab280cf7e0 2024-09-21T02:14:40.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab20003a90 con 0x7fab280cf7e0 2024-09-21T02:14:40.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab280cd4c0 con 0x7fab280cf7e0 2024-09-21T02:14:40.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20004020 con 0x7fab280cf7e0 2024-09-21T02:14:40.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab280c71c0 con 0x7fab280cf7e0 2024-09-21T02:14:40.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.147+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab280c64d0 con 0x7fab280cf7e0 2024-09-21T02:14:40.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.151+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fab200041c0 con 0x7fab280cf7e0 2024-09-21T02:14:40.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.151+0000 7fab0f7fe640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab140733e0 0x7fab140758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.151+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fab20095130 con 0x7fab280cf7e0 2024-09-21T02:14:40.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.151+0000 7fab26d9d640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab140733e0 0x7fab140758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.151+0000 7fab26d9d640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab140733e0 0x7fab140758f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab180015f0 tx=0x7fab18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:40.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.155+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fab20014070 con 0x7fab280cf7e0 2024-09-21T02:14:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.323+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fab280b2210 con 0x7fab280cf7e0 2024-09-21T02:14:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab0f7fe640 1 -- 172.21.15.170:0/2355833684 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7fab2005fdc0 con 0x7fab280cf7e0 2024-09-21T02:14:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:14:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab140733e0 msgr2=0x7fab140758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab140733e0 0x7fab140758f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab180015f0 tx=0x7fab18009040 comp rx=0 tx=0).stop 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 msgr2=0x7fab28149390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 0x7fab28149390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fab2000ca80 tx=0x7fab2000cf40 comp rx=0 tx=0).stop 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 shutdown_connections 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab140733e0 0x7fab140758f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab280cf7e0 0x7fab28149390 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab280ce1c0 0x7fab28148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 --2- 172.21.15.170:0/2355833684 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab280cb460 0x7fab281f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 >> 172.21.15.170:0/2355833684 conn(0x7fab280b1c40 msgr2=0x7fab280b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 shutdown_connections 2024-09-21T02:14:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.327+0000 7fab2d0cc640 1 -- 172.21.15.170:0/2355833684 wait complete. 2024-09-21T02:14:40.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:14:40.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:40.351 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:14:40.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34110]: fuse finished with error 0 and tester_r 0 2024-09-21T02:14:40.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:14:40.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:14:40.365 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:14:40.525 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:14:40.525 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:40.525 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:40.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:14:40.531 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:14:40.531 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:40.531 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:40.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:14:40.573 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f4ba31640 1 -- 172.21.15.170:0/2379570269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f30003a60 msgr2=0x7f1f30003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/2379570269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f30003a60 0x7f1f30003ed0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f40006f90 tx=0x7f1f4001cf20 comp rx=0 tx=0).stop 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f49fce640 1 -- 172.21.15.170:0/2379570269 reap_dead start 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f4ba31640 1 -- 172.21.15.170:0/2379570269 shutdown_connections 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f4ba31640 1 -- 172.21.15.170:0/2379570269 >> 172.21.15.170:0/2379570269 conn(0x7f1f440b1c40 msgr2=0x7f1f440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f4ba31640 1 -- 172.21.15.170:0/2379570269 shutdown_connections 2024-09-21T02:14:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.723+0000 7f1f4ba31640 1 -- 172.21.15.170:0/2379570269 wait complete. 2024-09-21T02:14:40.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 Processor -- start 2024-09-21T02:14:40.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 -- start start 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f30003a60 0x7f1f4415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 0x7f1f4415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4409ba10 0x7f1f4415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44166f40 con 0x7f1f4415ec50 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f440cf160 con 0x7f1f4409ba10 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1f441670c0 con 0x7f1f30003a60 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 0x7f1f4415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 0x7f1f4415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53658/0 (socket says 172.21.15.170:53658) 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 -- 172.21.15.170:0/606140670 learned_addr learned my addr 172.21.15.170:0/606140670 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f49fce640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4409ba10 0x7f1f4415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 -- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f30003a60 msgr2=0x7f1f4415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f497cd640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f30003a60 0x7f1f4415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f30003a60 0x7f1f4415e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 -- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4409ba10 msgr2=0x7f1f4415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4409ba10 0x7f1f4415b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 -- 172.21.15.170:0/606140670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4415c860 con 0x7f1f4415ec50 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f49fce640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4409ba10 0x7f1f4415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f48fcc640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 0x7f1f4415b130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9a0 tx=0x7f1f3400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400ccc0 con 0x7f1f4415ec50 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f3400ce60 con 0x7f1f4415ec50 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34005260 con 0x7f1f4415ec50 2024-09-21T02:14:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f44161ee0 con 0x7f1f4415ec50 2024-09-21T02:14:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.727+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f440c9620 con 0x7f1f4415ec50 2024-09-21T02:14:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.731+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f440ce5d0 con 0x7f1f4415ec50 2024-09-21T02:14:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.731+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f1f34005400 con 0x7f1f4415ec50 2024-09-21T02:14:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.731+0000 7f1f39ffb640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f280733e0 0x7f1f280758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.731+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1f34094f50 con 0x7f1f4415ec50 2024-09-21T02:14:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.731+0000 7f1f497cd640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f280733e0 0x7f1f280758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.731+0000 7f1f497cd640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f280733e0 0x7f1f280758f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f400ab000 tx=0x7f1f400a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:40.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.735+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1f3405fbf0 con 0x7f1f4415ec50 2024-09-21T02:14:40.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.927+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f4415caf0 con 0x7f1f4415ec50 2024-09-21T02:14:40.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f39ffb640 1 -- 172.21.15.170:0/606140670 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f1f3405f4f0 con 0x7f1f4415ec50 2024-09-21T02:14:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:40.935 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":18,"btime":"2024-09-21T02:14:31:962739+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:14.576721+0000","modified":"2024-09-21T02:14:17.607956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4914":{"gid":4914,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2504310121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2504310121},{"type":"v1","addr":"172.21.15.170:6835","nonce":2504310121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4914,"qdb_cluster":[4914]},"id":2}]} 2024-09-21T02:14:40.935 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 18 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f280733e0 msgr2=0x7f1f280758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f280733e0 0x7f1f280758f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f400ab000 tx=0x7f1f400a7040 comp rx=0 tx=0).stop 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 msgr2=0x7f1f4415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 0x7f1f4415b130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9a0 tx=0x7f1f3400ee60 comp rx=0 tx=0).stop 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 shutdown_connections 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f280733e0 0x7f1f280758f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4409ba10 0x7f1f4415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4415ec50 0x7f1f4415b130 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 --2- 172.21.15.170:0/606140670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f30003a60 0x7f1f4415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:40.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 >> 172.21.15.170:0/606140670 conn(0x7f1f440b1c40 msgr2=0x7f1f440b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:40.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 shutdown_connections 2024-09-21T02:14:40.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:40.931+0000 7f1f4ba31640 1 -- 172.21.15.170:0/606140670 wait complete. 2024-09-21T02:14:40.956 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:41.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 -- 172.21.15.170:0/2311268741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540cadf0 msgr2=0x7fbe540cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/2311268741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540cadf0 0x7fbe540cb200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009c80 tx=0x7fbe4c01cbc0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 -- 172.21.15.170:0/2311268741 shutdown_connections 2024-09-21T02:14:41.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/2311268741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540cadf0 0x7fbe540cb200 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/2311268741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe540ce4c0 0x7fbe540ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 -- 172.21.15.170:0/2311268741 >> 172.21.15.170:0/2311268741 conn(0x7fbe540b1c40 msgr2=0x7fbe540b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:41.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 -- 172.21.15.170:0/2311268741 shutdown_connections 2024-09-21T02:14:41.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.099+0000 7fbe5b55b640 1 -- 172.21.15.170:0/2311268741 wait complete. 2024-09-21T02:14:41.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 Processor -- start 2024-09-21T02:14:41.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- start start 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540cadf0 0x7fbe541f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 0x7fbe5415af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe5415f990 0x7fbe5409cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54169280 con 0x7fbe540ce4c0 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54169100 con 0x7fbe540cadf0 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbe54169400 con 0x7fbe5415f990 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 0x7fbe5415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 0x7fbe5415af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53686/0 (socket says 172.21.15.170:53686) 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 -- 172.21.15.170:0/978818407 learned_addr learned my addr 172.21.15.170:0/978818407 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe59af8640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe5415f990 0x7fbe5409cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe592f7640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540cadf0 0x7fbe541f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 -- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe5415f990 msgr2=0x7fbe5409cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe5415f990 0x7fbe5409cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 -- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540cadf0 msgr2=0x7fbe541f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540cadf0 0x7fbe541f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 -- 172.21.15.170:0/978818407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5415b930 con 0x7fbe540ce4c0 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe59af8640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe5415f990 0x7fbe5409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe592f7640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540cadf0 0x7fbe541f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe58af6640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 0x7fbe5415af50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c970 tx=0x7fbe4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44007bb0 con 0x7fbe540ce4c0 2024-09-21T02:14:41.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe44007e40 con 0x7fbe540ce4c0 2024-09-21T02:14:41.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5409ac10 con 0x7fbe540ce4c0 2024-09-21T02:14:41.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400f6b0 con 0x7fbe540ce4c0 2024-09-21T02:14:41.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe541571c0 con 0x7fbe540ce4c0 2024-09-21T02:14:41.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.103+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe540cb200 con 0x7fbe540ce4c0 2024-09-21T02:14:41.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.107+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fbe44018400 con 0x7fbe540ce4c0 2024-09-21T02:14:41.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.107+0000 7fbe49ffb640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbe380733b0 0x7fbe380758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.107+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fbe44095410 con 0x7fbe540ce4c0 2024-09-21T02:14:41.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.107+0000 7fbe592f7640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbe380733b0 0x7fbe380758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.107+0000 7fbe592f7640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbe380733b0 0x7fbe380758c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000a850 tx=0x7fbe50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:41.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.111+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbe44026020 con 0x7fbe540ce4c0 2024-09-21T02:14:41.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.311+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe540ce2a0 con 0x7fbe540ce4c0 2024-09-21T02:14:41.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.311+0000 7fbe49ffb640 1 -- 172.21.15.170:0/978818407 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fbe44014030 con 0x7fbe540ce4c0 2024-09-21T02:14:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:41.318 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":18,"btime":"2024-09-21T02:14:31:962739+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:14.576721+0000","modified":"2024-09-21T02:14:17.607956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4914":{"gid":4914,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2504310121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2504310121},{"type":"v1","addr":"172.21.15.170:6835","nonce":2504310121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4914,"qdb_cluster":[4914]},"id":2}]} 2024-09-21T02:14:41.318 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 18 2024-09-21T02:14:41.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbe380733b0 msgr2=0x7fbe380758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbe380733b0 0x7fbe380758c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000a850 tx=0x7fbe50008040 comp rx=0 tx=0).stop 2024-09-21T02:14:41.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 msgr2=0x7fbe5415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 0x7fbe5415af50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c970 tx=0x7fbe4400ce30 comp rx=0 tx=0).stop 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 shutdown_connections 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbe380733b0 0x7fbe380758c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe5415f990 0x7fbe5409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540ce4c0 0x7fbe5415af50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 --2- 172.21.15.170:0/978818407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540cadf0 0x7fbe541f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 >> 172.21.15.170:0/978818407 conn(0x7fbe540b1c40 msgr2=0x7fbe540b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 shutdown_connections 2024-09-21T02:14:41.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.315+0000 7fbe5b55b640 1 -- 172.21.15.170:0/978818407 wait complete. 2024-09-21T02:14:41.338 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:41.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 -- 172.21.15.170:0/4155194380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0cd210 msgr2=0x7f7b8c0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 --2- 172.21.15.170:0/4155194380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0cd210 0x7f7b8c0b3350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0b3a10 tx=0x7f7b8401cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 -- 172.21.15.170:0/4155194380 shutdown_connections 2024-09-21T02:14:41.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 --2- 172.21.15.170:0/4155194380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0cd210 0x7f7b8c0b3350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 --2- 172.21.15.170:0/4155194380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8c0b5410 0x7f7b8c06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 -- 172.21.15.170:0/4155194380 >> 172.21.15.170:0/4155194380 conn(0x7f7b8c0b1c40 msgr2=0x7f7b8c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:41.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 -- 172.21.15.170:0/4155194380 shutdown_connections 2024-09-21T02:14:41.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.483+0000 7f7b91149640 1 -- 172.21.15.170:0/4155194380 wait complete. 2024-09-21T02:14:41.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 Processor -- start 2024-09-21T02:14:41.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 -- start start 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 0x7f7b8c0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8c0b5410 0x7f7b8c154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c154990 0x7f7b8c154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c16a060 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c070160 con 0x7f7b8c154990 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b91149640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8c16a1e0 con 0x7f7b8c0b5410 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 0x7f7b8c0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 0x7f7b8c0c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53706/0 (socket says 172.21.15.170:53706) 2024-09-21T02:14:41.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 -- 172.21.15.170:0/3180494989 learned_addr learned my addr 172.21.15.170:0/3180494989 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8b59e640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c154990 0x7f7b8c154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8a59c640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8c0b5410 0x7f7b8c154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 -- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8c0b5410 msgr2=0x7f7b8c154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8c0b5410 0x7f7b8c154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 -- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c154990 msgr2=0x7f7b8c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c154990 0x7f7b8c154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 -- 172.21.15.170:0/3180494989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c0c98e0 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8b59e640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c154990 0x7f7b8c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b8ad9d640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 0x7f7b8c0c9160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b800058d0 tx=0x7f7b8000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017070 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b80009d20 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.487+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800106c0 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8c155790 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8c0c9e30 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8c06fa30 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f7b8001a070 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b737fe640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7b780733e0 0x7f7b780758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7b800947c0 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b8a59c640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7b780733e0 0x7f7b780758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.491+0000 7f7b8a59c640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7b780733e0 0x7f7b780758f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0b19b0 tx=0x7f7b7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:41.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.495+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7b8c155790 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.671+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b8c0b49a0 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.675+0000 7f7b737fe640 1 -- 172.21.15.170:0/3180494989 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f7b8005f4e0 con 0x7f7b8c0b2f40 2024-09-21T02:14:41.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:41.681 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":35,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:16.149601+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:12.737491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:13.529023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:41.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7b780733e0 msgr2=0x7f7b780758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7b780733e0 0x7f7b780758f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0b19b0 tx=0x7f7b7c009040 comp rx=0 tx=0).stop 2024-09-21T02:14:41.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 msgr2=0x7f7b8c0c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 0x7f7b8c0c9160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b800058d0 tx=0x7f7b8000dde0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 shutdown_connections 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7b780733e0 0x7f7b780758f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8c0b5410 0x7f7b8c154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b8c154990 0x7f7b8c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 --2- 172.21.15.170:0/3180494989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b8c0b2f40 0x7f7b8c0c9160 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 >> 172.21.15.170:0/3180494989 conn(0x7f7b8c0b1c40 msgr2=0x7f7b8c155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 shutdown_connections 2024-09-21T02:14:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.679+0000 7f7b91149640 1 -- 172.21.15.170:0/3180494989 wait complete. 2024-09-21T02:14:41.703 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:14:41.703 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:41.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f789a67f640 1 -- 172.21.15.170:0/2872283268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787c003a60 msgr2=0x7f787c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f789a67f640 1 --2- 172.21.15.170:0/2872283268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787c003a60 0x7f787c003ef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f788400c310 tx=0x7f788401cf90 comp rx=0 tx=0).stop 2024-09-21T02:14:41.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f7898c1c640 1 -- 172.21.15.170:0/2872283268 reap_dead start 2024-09-21T02:14:41.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f789a67f640 1 -- 172.21.15.170:0/2872283268 shutdown_connections 2024-09-21T02:14:41.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f789a67f640 1 -- 172.21.15.170:0/2872283268 >> 172.21.15.170:0/2872283268 conn(0x7f78940cd3c0 msgr2=0x7f78940cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:41.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f789a67f640 1 -- 172.21.15.170:0/2872283268 shutdown_connections 2024-09-21T02:14:41.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.847+0000 7f789a67f640 1 -- 172.21.15.170:0/2872283268 wait complete. 2024-09-21T02:14:41.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 Processor -- start 2024-09-21T02:14:41.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 -- start start 2024-09-21T02:14:41.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78941f7680 0x7f78941f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 0x7f78941f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78941f6310 0x7f78941f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7894166450 con 0x7f78941f9780 2024-09-21T02:14:41.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7894071df0 con 0x7f78941f6310 2024-09-21T02:14:41.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f789a67f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f78941665d0 con 0x7f78941f7680 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 0x7f78941f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 0x7f78941f9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53710/0 (socket says 172.21.15.170:53710) 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 -- 172.21.15.170:0/3428435643 learned_addr learned my addr 172.21.15.170:0/3428435643 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 -- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78941f7680 msgr2=0x7f78941f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78941f7680 0x7f78941f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 -- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78941f6310 msgr2=0x7f78941f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f7898c1c640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78941f6310 0x7f78941f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78941f6310 0x7f78941f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 -- 172.21.15.170:0/3428435643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78941f73b0 con 0x7f78941f9780 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f78937fe640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 0x7f78941f9b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7888011a50 tx=0x7f7888011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.851+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788801f070 con 0x7f78941f9780 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f788800fe80 con 0x7f78941f9780 2024-09-21T02:14:41.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78941fa0d0 con 0x7f78941f9780 2024-09-21T02:14:41.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888012690 con 0x7f78941f9780 2024-09-21T02:14:41.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78940c4560 con 0x7f78941f9780 2024-09-21T02:14:41.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78940caf90 con 0x7f78941f9780 2024-09-21T02:14:41.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f7888022070 con 0x7f78941f9780 2024-09-21T02:14:41.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7890ff9640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78800733b0 0x7f78800758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:41.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f78880950b0 con 0x7f78941f9780 2024-09-21T02:14:41.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7893fff640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78800733b0 0x7f78800758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:41.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.855+0000 7f7893fff640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78800733b0 0x7f78800758c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78840ac000 tx=0x7f787c005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:41.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:41.859+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f78941fa0d0 con 0x7f78941f9780 2024-09-21T02:14:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.063+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78940aee20 con 0x7f78941f9780 2024-09-21T02:14:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.063+0000 7f7890ff9640 1 -- 172.21.15.170:0/3428435643 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f788805f780 con 0x7f78941f9780 2024-09-21T02:14:42.068 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:42.069 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":18,"btime":"2024-09-21T02:14:31:962739+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:14.576721+0000","modified":"2024-09-21T02:14:17.607956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4914":{"gid":4914,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2504310121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2504310121},{"type":"v1","addr":"172.21.15.170:6835","nonce":2504310121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4914,"qdb_cluster":[4914]},"id":2}]} 2024-09-21T02:14:42.069 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 18 2024-09-21T02:14:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78800733b0 msgr2=0x7f78800758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78800733b0 0x7f78800758c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78840ac000 tx=0x7f787c005840 comp rx=0 tx=0).stop 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 msgr2=0x7f78941f9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 0x7f78941f9b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7888011a50 tx=0x7f7888011f10 comp rx=0 tx=0).stop 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 shutdown_connections 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78800733b0 0x7f78800758c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78941f6310 0x7f78941f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78941f9780 0x7f78941f9b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 --2- 172.21.15.170:0/3428435643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78941f7680 0x7f78941f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 >> 172.21.15.170:0/3428435643 conn(0x7f78940cd3c0 msgr2=0x7f78940b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 shutdown_connections 2024-09-21T02:14:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.067+0000 7f789a67f640 1 -- 172.21.15.170:0/3428435643 wait complete. 2024-09-21T02:14:42.089 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 -- 172.21.15.170:0/2930945092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 msgr2=0x7fc5880d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 --2- 172.21.15.170:0/2930945092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880d4e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5880b39a0 tx=0x7fc57801c7d0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 -- 172.21.15.170:0/2930945092 shutdown_connections 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 --2- 172.21.15.170:0/2930945092 >> v2:172.21.15.191:3300/0 conn(0x7fc58806dd80 0x7fc5880b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 --2- 172.21.15.170:0/2930945092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880d4e00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 -- 172.21.15.170:0/2930945092 >> 172.21.15.170:0/2930945092 conn(0x7fc58809d7d0 msgr2=0x7fc5880d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 -- 172.21.15.170:0/2930945092 shutdown_connections 2024-09-21T02:14:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.223+0000 7fc590718640 1 -- 172.21.15.170:0/2930945092 wait complete. 2024-09-21T02:14:42.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 Processor -- start 2024-09-21T02:14:42.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- start start 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc58806dd80 0x7fc5880ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc58806e190 0x7fc5881fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc58816c730 con 0x7fc5880b9ba0 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc58816c5b0 con 0x7fc58806dd80 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc58816c8b0 con 0x7fc58806e190 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53722/0 (socket says 172.21.15.170:53722) 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 -- 172.21.15.170:0/1740950543 learned_addr learned my addr 172.21.15.170:0/1740950543 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58e4b4640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc58806dd80 0x7fc5880ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 -- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc58806e190 msgr2=0x7fc5881fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc58806e190 0x7fc5881fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 -- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc58806dd80 msgr2=0x7fc5880ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc58806dd80 0x7fc5880ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 -- 172.21.15.170:0/1740950543 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58815bde0 con 0x7fc5880b9ba0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc58ecb5640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880b3e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc58000cae0 tx=0x7fc58000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580016020 con 0x7fc5880b9ba0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc58000ab40 con 0x7fc5880b9ba0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58815c310 con 0x7fc5880b9ba0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58000ce20 con 0x7fc5880b9ba0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58815c8d0 con 0x7fc5880b9ba0 2024-09-21T02:14:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.227+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58815c080 con 0x7fc5880b9ba0 2024-09-21T02:14:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.231+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fc580019020 con 0x7fc5880b9ba0 2024-09-21T02:14:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.231+0000 7fc57effd640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc56c0733b0 0x7fc56c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.231+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc580094e10 con 0x7fc5880b9ba0 2024-09-21T02:14:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.231+0000 7fc58e4b4640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc56c0733b0 0x7fc56c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.231+0000 7fc58e4b4640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc56c0733b0 0x7fc56c0758c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5780af000 tx=0x7fc57801b010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:42.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.235+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc580020030 con 0x7fc5880b9ba0 2024-09-21T02:14:42.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.427+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5880d4be0 con 0x7fc5880b9ba0 2024-09-21T02:14:42.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.427+0000 7fc57effd640 1 -- 172.21.15.170:0/1740950543 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fc58005fa80 con 0x7fc5880b9ba0 2024-09-21T02:14:42.432 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:42.432 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":18,"btime":"2024-09-21T02:14:31:962739+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:14.576721+0000","modified":"2024-09-21T02:14:17.607956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4914":{"gid":4914,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2504310121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2504310121},{"type":"v1","addr":"172.21.15.170:6835","nonce":2504310121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4914,"qdb_cluster":[4914]},"id":2}]} 2024-09-21T02:14:42.433 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 18 2024-09-21T02:14:42.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc56c0733b0 msgr2=0x7fc56c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc56c0733b0 0x7fc56c0758c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5780af000 tx=0x7fc57801b010 comp rx=0 tx=0).stop 2024-09-21T02:14:42.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 msgr2=0x7fc5880b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880b3e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc58000cae0 tx=0x7fc58000cbd0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 shutdown_connections 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc56c0733b0 0x7fc56c0758c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc58806e190 0x7fc5881fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5880b9ba0 0x7fc5880b3e60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 --2- 172.21.15.170:0/1740950543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc58806dd80 0x7fc5880ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 >> 172.21.15.170:0/1740950543 conn(0x7fc58809d7d0 msgr2=0x7fc5880b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 shutdown_connections 2024-09-21T02:14:42.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.431+0000 7fc590718640 1 -- 172.21.15.170:0/1740950543 wait complete. 2024-09-21T02:14:42.452 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5c057640 1 -- 172.21.15.170:0/2326754328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b40003a10 msgr2=0x7f6b40003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5c057640 1 --2- 172.21.15.170:0/2326754328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b40003a10 0x7f6b40003e80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009b60 tx=0x7f6b5001d280 comp rx=0 tx=0).stop 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5a5f4640 1 -- 172.21.15.170:0/2326754328 reap_dead start 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5c057640 1 -- 172.21.15.170:0/2326754328 shutdown_connections 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5c057640 1 -- 172.21.15.170:0/2326754328 >> 172.21.15.170:0/2326754328 conn(0x7f6b540cee30 msgr2=0x7f6b540cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5c057640 1 -- 172.21.15.170:0/2326754328 shutdown_connections 2024-09-21T02:14:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.599+0000 7f6b5c057640 1 -- 172.21.15.170:0/2326754328 wait complete. 2024-09-21T02:14:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 Processor -- start 2024-09-21T02:14:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 -- start start 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b40003a10 0x7f6b541fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b540cabc0 0x7f6b5414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 0x7f6b5406f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54168030 con 0x7f6b5406f090 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54167eb0 con 0x7f6b40003a10 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6b541681b0 con 0x7f6b540cabc0 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 0x7f6b5406f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 0x7f6b5406f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53742/0 (socket says 172.21.15.170:53742) 2024-09-21T02:14:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 -- 172.21.15.170:0/1017605956 learned_addr learned my addr 172.21.15.170:0/1017605956 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b595f2640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b540cabc0 0x7f6b5414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b59df3640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b40003a10 0x7f6b541fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 -- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b540cabc0 msgr2=0x7f6b5414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b540cabc0 0x7f6b5414b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 -- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b40003a10 msgr2=0x7f6b541fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b40003a10 0x7f6b541fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 -- 172.21.15.170:0/1017605956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b540cd320 con 0x7f6b5406f090 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5a5f4640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 0x7f6b5406f4a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b54074ef0 tx=0x7f6b4c00dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c003d90 con 0x7f6b5406f090 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b4c00b8f0 con 0x7f6b5406f090 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5406f9e0 con 0x7f6b5406f090 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.603+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00bd20 con 0x7f6b5406f090 2024-09-21T02:14:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b540cb620 con 0x7f6b5406f090 2024-09-21T02:14:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5415c850 con 0x7f6b5406f090 2024-09-21T02:14:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f6b4c006070 con 0x7f6b5406f090 2024-09-21T02:14:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b4a7fc640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b380733b0 0x7f6b380758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6b4c095890 con 0x7f6b5406f090 2024-09-21T02:14:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b59df3640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b380733b0 0x7f6b380758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.607+0000 7f6b59df3640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b380733b0 0x7f6b380758c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ae000 tx=0x7f6b5001a020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:42.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.611+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6b5406f9e0 con 0x7f6b5406f090 2024-09-21T02:14:42.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.787+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b540d0e20 con 0x7f6b5406f090 2024-09-21T02:14:42.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.787+0000 7f6b4a7fc640 1 -- 172.21.15.170:0/1017605956 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16717 (secure 0 0 0) 0x7f6b540b4780 con 0x7f6b5406f090 2024-09-21T02:14:42.793 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:42.793 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":35,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:16.149601+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:12.737491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:13.529023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b380733b0 msgr2=0x7f6b380758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b380733b0 0x7f6b380758c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ae000 tx=0x7f6b5001a020 comp rx=0 tx=0).stop 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 msgr2=0x7f6b5406f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 0x7f6b5406f4a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b54074ef0 tx=0x7f6b4c00dd80 comp rx=0 tx=0).stop 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 shutdown_connections 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b380733b0 0x7f6b380758c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b540cabc0 0x7f6b5414b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5406f090 0x7f6b5406f4a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 --2- 172.21.15.170:0/1017605956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b40003a10 0x7f6b541fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 >> 172.21.15.170:0/1017605956 conn(0x7f6b540cee30 msgr2=0x7f6b5415b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 shutdown_connections 2024-09-21T02:14:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.791+0000 7f6b5c057640 1 -- 172.21.15.170:0/1017605956 wait complete. 2024-09-21T02:14:42.817 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 -- 172.21.15.170:0/676184209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680ce1c0 msgr2=0x7efc680ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 --2- 172.21.15.170:0/676184209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680ce1c0 0x7efc680ce5d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc68070cc0 tx=0x7efc6001cb80 comp rx=0 tx=0).stop 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 -- 172.21.15.170:0/676184209 shutdown_connections 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 --2- 172.21.15.170:0/676184209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680ce1c0 0x7efc680ce5d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 --2- 172.21.15.170:0/676184209 >> v2:172.21.15.191:3300/0 conn(0x7efc680cb460 0x7efc680cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 -- 172.21.15.170:0/676184209 >> 172.21.15.170:0/676184209 conn(0x7efc680b1c40 msgr2=0x7efc680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 -- 172.21.15.170:0/676184209 shutdown_connections 2024-09-21T02:14:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 -- 172.21.15.170:0/676184209 wait complete. 2024-09-21T02:14:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 Processor -- start 2024-09-21T02:14:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.971+0000 7efc6d684640 1 -- start start 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc680cb460 0x7efc680c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc680ce1c0 0x7efc680c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 0x7efc680b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efc68168f50 con 0x7efc680cf7e0 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efc680cf140 con 0x7efc680cb460 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efc681690d0 con 0x7efc680ce1c0 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 0x7efc680b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 0x7efc680b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53752/0 (socket says 172.21.15.170:53752) 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 -- 172.21.15.170:0/749543456 learned_addr learned my addr 172.21.15.170:0/749543456 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc66ffd640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc680cb460 0x7efc680c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc667fc640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc680ce1c0 0x7efc680c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 -- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc680ce1c0 msgr2=0x7efc680c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc680ce1c0 0x7efc680c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 -- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc680cb460 msgr2=0x7efc680c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc680cb460 0x7efc680c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 -- 172.21.15.170:0/749543456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc680ae3f0 con 0x7efc680cf7e0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc66ffd640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc680cb460 0x7efc680c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc677fe640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 0x7efc680b0d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc6001c5f0 tx=0x7efc60005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600b2040 con 0x7efc680cf7e0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc600bb070 con 0x7efc680cf7e0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681f9f30 con 0x7efc680cf7e0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600045f0 con 0x7efc680cf7e0 2024-09-21T02:14:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc680b0500 con 0x7efc680cf7e0 2024-09-21T02:14:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc680ce5d0 con 0x7efc680cf7e0 2024-09-21T02:14:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7efc600c0460 con 0x7efc680cf7e0 2024-09-21T02:14:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc5f7fe640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc480733e0 0x7efc480758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.975+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7efc6012f050 con 0x7efc680cf7e0 2024-09-21T02:14:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.979+0000 7efc66ffd640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc480733e0 0x7efc480758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:42.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.979+0000 7efc66ffd640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc480733e0 0x7efc480758f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc50000f70 tx=0x7efc50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:42.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:42.983+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efc60016770 con 0x7efc680cf7e0 2024-09-21T02:14:43.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.195+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efc680aed30 con 0x7efc680cf7e0 2024-09-21T02:14:43.200 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:14:43.195+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:14:43.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.199+0000 7efc5f7fe640 1 -- 172.21.15.170:0/749543456 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7efc600f9df0 con 0x7efc680cf7e0 2024-09-21T02:14:43.203 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:14:43.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.199+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc480733e0 msgr2=0x7efc480758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:43.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.199+0000 7efc6d684640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc480733e0 0x7efc480758f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc50000f70 tx=0x7efc50009040 comp rx=0 tx=0).stop 2024-09-21T02:14:43.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.199+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 msgr2=0x7efc680b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:43.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.199+0000 7efc6d684640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 0x7efc680b0d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc6001c5f0 tx=0x7efc60005ae0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 shutdown_connections 2024-09-21T02:14:43.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efc480733e0 0x7efc480758f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc680ce1c0 0x7efc680c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc680cf7e0 0x7efc680b0d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 --2- 172.21.15.170:0/749543456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc680cb460 0x7efc680c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 >> 172.21.15.170:0/749543456 conn(0x7efc680b1c40 msgr2=0x7efc680ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:43.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 shutdown_connections 2024-09-21T02:14:43.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.203+0000 7efc6d684640 1 -- 172.21.15.170:0/749543456 wait complete. 2024-09-21T02:14:43.229 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:14:43.332 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:14:43.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- 172.21.15.170:0/3695019455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 msgr2=0x7f1da4003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:43.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/3695019455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1da4003f50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1db400c770 tx=0x7f1db401d320 comp rx=0 tx=0).stop 2024-09-21T02:14:43.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbe69b640 1 -- 172.21.15.170:0/3695019455 reap_dead start 2024-09-21T02:14:43.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- 172.21.15.170:0/3695019455 shutdown_connections 2024-09-21T02:14:43.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- 172.21.15.170:0/3695019455 >> 172.21.15.170:0/3695019455 conn(0x7f1db808bf50 msgr2=0x7f1db809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:43.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- 172.21.15.170:0/3695019455 shutdown_connections 2024-09-21T02:14:43.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- 172.21.15.170:0/3695019455 wait complete. 2024-09-21T02:14:43.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 Processor -- start 2024-09-21T02:14:43.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- start start 2024-09-21T02:14:43.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1db80b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1db80b6710 0x7f1db80c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1db80cae70 0x7f1db80ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81697d0 con 0x7f1da4003ae0 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8169650 con 0x7f1db80cae70 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dc00fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1db8169950 con 0x7f1db80b6710 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1db80b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1db80b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53780/0 (socket says 172.21.15.170:53780) 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 -- 172.21.15.170:0/646350562 learned_addr learned my addr 172.21.15.170:0/646350562 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbe69b640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1db80cae70 0x7f1db80ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbd699640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1db80b6710 0x7f1db80c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 -- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1db80b6710 msgr2=0x7f1db80c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1db80b6710 0x7f1db80c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 -- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1db80cae70 msgr2=0x7f1db80ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:43.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1db80cae70 0x7f1db80ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbde9a640 1 -- 172.21.15.170:0/646350562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db80c9510 con 0x7f1da4003ae0 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.371+0000 7f1dbe69b640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1db80cae70 0x7f1db80ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dbde9a640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1db80b61d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1db401bbf0 tx=0x7f1db4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4003b80 con 0x7f1da4003ae0 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1db4003e10 con 0x7f1da4003ae0 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4006330 con 0x7f1da4003ae0 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db81f89b0 con 0x7f1da4003ae0 2024-09-21T02:14:43.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81f8f70 con 0x7f1da4003ae0 2024-09-21T02:14:43.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db80c96e0 con 0x7f1da4003ae0 2024-09-21T02:14:43.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f1db40064d0 con 0x7f1da4003ae0 2024-09-21T02:14:43.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dae7fc640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1d9c0733b0 0x7f1d9c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:43.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f1db412f330 con 0x7f1da4003ae0 2024-09-21T02:14:43.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dbd699640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1d9c0733b0 0x7f1d9c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:43.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.375+0000 7f1dbd699640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1d9c0733b0 0x7f1d9c0758c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005ec0 tx=0x7f1da80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:43.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.379+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1db40bf060 con 0x7f1da4003ae0 2024-09-21T02:14:43.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:43.575+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1db80cff20 con 0x7f1da4003ae0 2024-09-21T02:14:44.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dae7fc640 1 -- 172.21.15.170:0/646350562 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f1db80d4e60 con 0x7f1da4003ae0 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1d9c0733b0 msgr2=0x7f1d9c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1d9c0733b0 0x7f1d9c0758c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005ec0 tx=0x7f1da80091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 msgr2=0x7f1db80b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1db80b61d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1db401bbf0 tx=0x7f1db4004e60 comp rx=0 tx=0).stop 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 shutdown_connections 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1d9c0733b0 0x7f1d9c0758c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1db80b6710 0x7f1db80c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1db80cae70 0x7f1db80ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 --2- 172.21.15.170:0/646350562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1da4003ae0 0x7f1db80b61d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 >> 172.21.15.170:0/646350562 conn(0x7f1db808bf50 msgr2=0x7f1db81610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:44.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 shutdown_connections 2024-09-21T02:14:44.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.203+0000 7f1dc00fe640 1 -- 172.21.15.170:0/646350562 wait complete. 2024-09-21T02:14:44.232 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/291860920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b4460 msgr2=0x7fa7f00b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/291860920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b4460 0x7fa7f00b5120 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00d0ad0 tx=0x7fa7e801cb80 comp rx=0 tx=0).stop 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/291860920 shutdown_connections 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/291860920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b4460 0x7fa7f00b5120 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/291860920 >> 172.21.15.170:0/291860920 conn(0x7fa7f00ced50 msgr2=0x7fa7f00cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/291860920 shutdown_connections 2024-09-21T02:14:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.391+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/291860920 wait complete. 2024-09-21T02:14:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 Processor -- start 2024-09-21T02:14:44.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- start start 2024-09-21T02:14:44.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 0x7fa7f0158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:44.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7f015c770 0x7fa7f014a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:44.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7f0156130 0x7fa7f0156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:44.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f016b130 con 0x7fa7f00b7f20 2024-09-21T02:14:44.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f016afb0 con 0x7fa7f015c770 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f016b2b0 con 0x7fa7f0156130 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 0x7fa7f0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 0x7fa7f0158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53808/0 (socket says 172.21.15.170:53808) 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 -- 172.21.15.170:0/1963367649 learned_addr learned my addr 172.21.15.170:0/1963367649 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7edd9b640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7f015c770 0x7fa7f014a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7eed9d640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7f0156130 0x7fa7f0156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 -- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7f0156130 msgr2=0x7fa7f0156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7f0156130 0x7fa7f0156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 -- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7f015c770 msgr2=0x7fa7f014a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7f015c770 0x7fa7f014a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:44.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 -- 172.21.15.170:0/1963367649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0156a80 con 0x7fa7f00b7f20 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7edd9b640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7f015c770 0x7fa7f014a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7ee59c640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 0x7fa7f0158600 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b410 tx=0x7fa7e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0015030 con 0x7fa7f00b7f20 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7e0004020 con 0x7fa7f00b7f20 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f015a0f0 con 0x7fa7f00b7f20 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0004470 con 0x7fa7f00b7f20 2024-09-21T02:14:44.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f0159050 con 0x7fa7f00b7f20 2024-09-21T02:14:44.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.395+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f00b5120 con 0x7fa7f00b7f20 2024-09-21T02:14:44.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.399+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fa7e0018070 con 0x7fa7f00b7f20 2024-09-21T02:14:44.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.399+0000 7fa7deffd640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7d00733b0 0x7fa7d00758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:44.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.399+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fa7e0094670 con 0x7fa7f00b7f20 2024-09-21T02:14:44.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.399+0000 7fa7edd9b640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7d00733b0 0x7fa7d00758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:44.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.399+0000 7fa7edd9b640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7d00733b0 0x7fa7d00758c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7f014adb0 tx=0x7fa7e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.403+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa7f0156a80 con 0x7fa7f00b7f20 2024-09-21T02:14:44.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:44.579+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa7f0157d00 con 0x7fa7f00b7f20 2024-09-21T02:14:45.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.211+0000 7fa7deffd640 1 -- 172.21.15.170:0/1963367649 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7fa7e005f380 con 0x7fa7f00b7f20 2024-09-21T02:14:45.215 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:14:45.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7d00733b0 msgr2=0x7fa7d00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:45.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7d00733b0 0x7fa7d00758c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa7f014adb0 tx=0x7fa7e4009040 comp rx=0 tx=0).stop 2024-09-21T02:14:45.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 msgr2=0x7fa7f0158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:45.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 0x7fa7f0158600 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b410 tx=0x7fa7e000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 shutdown_connections 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7d00733b0 0x7fa7d00758c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7f0156130 0x7fa7f0156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7f015c770 0x7fa7f014a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 --2- 172.21.15.170:0/1963367649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7f00b7f20 0x7fa7f0158600 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 >> 172.21.15.170:0/1963367649 conn(0x7fa7f00ced50 msgr2=0x7fa7f00b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 shutdown_connections 2024-09-21T02:14:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.215+0000 7fa7f4ec3640 1 -- 172.21.15.170:0/1963367649 wait complete. 2024-09-21T02:14:45.248 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d1852f640 1 -- 172.21.15.170:0/4228998768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc003a10 msgr2=0x7f4cfc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d1852f640 1 --2- 172.21.15.170:0/4228998768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc003a10 0x7f4cfc003e80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00baf0 tx=0x7f4d0c01cda0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d16acc640 1 -- 172.21.15.170:0/4228998768 reap_dead start 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d1852f640 1 -- 172.21.15.170:0/4228998768 shutdown_connections 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d1852f640 1 -- 172.21.15.170:0/4228998768 >> 172.21.15.170:0/4228998768 conn(0x7f4d100b1c40 msgr2=0x7f4d100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d1852f640 1 -- 172.21.15.170:0/4228998768 shutdown_connections 2024-09-21T02:14:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.411+0000 7f4d1852f640 1 -- 172.21.15.170:0/4228998768 wait complete. 2024-09-21T02:14:45.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 Processor -- start 2024-09-21T02:14:45.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 -- start start 2024-09-21T02:14:45.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc003a10 0x7f4d10156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:45.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d101fc910 0x7f4d100cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:45.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 0x7f4d10151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1006fc70 con 0x7f4d10151530 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1006faf0 con 0x7f4d101fc910 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d1852f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4d10167610 con 0x7f4cfc003a10 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 0x7f4d10151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 0x7f4d10151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57922/0 (socket says 172.21.15.170:57922) 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 -- 172.21.15.170:0/1218382190 learned_addr learned my addr 172.21.15.170:0/1218382190 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d162cb640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc003a10 0x7f4d10156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d15aca640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d101fc910 0x7f4d100cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 -- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc003a10 msgr2=0x7f4d10156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc003a10 0x7f4d10156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 -- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d101fc910 msgr2=0x7f4d100cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d101fc910 0x7f4d100cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:45.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 -- 172.21.15.170:0/1218382190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d101f8990 con 0x7f4d10151530 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d162cb640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc003a10 0x7f4d10156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d16acc640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 0x7f4d10151940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4d10074170 tx=0x7f4d0800fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.415+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08004d30 con 0x7f4d10151530 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d0800eab0 con 0x7f4d10151530 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08005020 con 0x7f4d10151530 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d101fcd20 con 0x7f4d10151530 2024-09-21T02:14:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d101f8c00 con 0x7f4d10151530 2024-09-21T02:14:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d100af440 con 0x7f4d10151530 2024-09-21T02:14:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f4d08019030 con 0x7f4d10151530 2024-09-21T02:14:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d06ffd640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4cf40733b0 0x7f4cf40758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5357+0+0 (secure 0 0 0) 0x7f4d08094cc0 con 0x7f4d10151530 2024-09-21T02:14:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d162cb640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4cf40733b0 0x7f4cf40758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.419+0000 7f4d162cb640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4cf40733b0 0x7f4cf40758c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0b0000 tx=0x7f4d0c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:45.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.423+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4d0805f4f0 con 0x7f4d10151530 2024-09-21T02:14:45.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:45.595+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d10157470 con 0x7f4d10151530 2024-09-21T02:14:46.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.211+0000 7f4d06ffd640 1 -- 172.21.15.170:0/1218382190 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7f4d08063f20 con 0x7f4d10151530 2024-09-21T02:14:46.217 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:14:46.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4cf40733b0 msgr2=0x7f4cf40758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4cf40733b0 0x7f4cf40758c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0b0000 tx=0x7f4d0c0a7040 comp rx=0 tx=0).stop 2024-09-21T02:14:46.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 msgr2=0x7f4d10151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 0x7f4d10151940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4d10074170 tx=0x7f4d0800fe10 comp rx=0 tx=0).stop 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 shutdown_connections 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4cf40733b0 0x7f4cf40758c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d10151530 0x7f4d10151940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d101fc910 0x7f4d100cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 --2- 172.21.15.170:0/1218382190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc003a10 0x7f4d10156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 >> 172.21.15.170:0/1218382190 conn(0x7f4d100b1c40 msgr2=0x7f4d1015c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 shutdown_connections 2024-09-21T02:14:46.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.219+0000 7f4d1852f640 1 -- 172.21.15.170:0/1218382190 wait complete. 2024-09-21T02:14:46.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:14:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- 172.21.15.170:0/1414198533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 msgr2=0x7f3a680cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 --2- 172.21.15.170:0/1414198533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a680cd0f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009900 tx=0x7f3a5c01c7f0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- 172.21.15.170:0/1414198533 shutdown_connections 2024-09-21T02:14:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 --2- 172.21.15.170:0/1414198533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a680cd0f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 --2- 172.21.15.170:0/1414198533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b21a0 0x7f3a680b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- 172.21.15.170:0/1414198533 >> 172.21.15.170:0/1414198533 conn(0x7f3a6809a9c0 msgr2=0x7f3a680d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- 172.21.15.170:0/1414198533 shutdown_connections 2024-09-21T02:14:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- 172.21.15.170:0/1414198533 wait complete. 2024-09-21T02:14:46.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 Processor -- start 2024-09-21T02:14:46.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- start start 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a680b21a0 0x7f3a681567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b60e0 0x7f3a68166570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a68166ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a680b1750 con 0x7f3a680ccce0 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3a680b15d0 con 0x7f3a680b21a0 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a70328640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3a680b18d0 con 0x7f3a680b60e0 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a6e8c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a68166ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a6e8c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a68166ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57948/0 (socket says 172.21.15.170:57948) 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a6e8c5640 1 -- 172.21.15.170:0/3428959780 learned_addr learned my addr 172.21.15.170:0/3428959780 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a6d8c3640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b60e0 0x7f3a68166570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.407+0000 7f3a6e0c4640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a680b21a0 0x7f3a681567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e8c5640 1 -- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b60e0 msgr2=0x7f3a68166570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e8c5640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b60e0 0x7f3a68166570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e8c5640 1 -- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a680b21a0 msgr2=0x7f3a681567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e8c5640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a680b21a0 0x7f3a681567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e8c5640 1 -- 172.21.15.170:0/3428959780 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a680b00e0 con 0x7f3a680ccce0 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e0c4640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a680b21a0 0x7f3a681567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6d8c3640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b60e0 0x7f3a68166570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e8c5640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a68166ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c3f0 tx=0x7f3a6000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60017070 con 0x7f3a680ccce0 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a600053e0 con 0x7f3a680ccce0 2024-09-21T02:14:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a68156d00 con 0x7f3a680ccce0 2024-09-21T02:14:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60003860 con 0x7f3a680ccce0 2024-09-21T02:14:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a68166330 con 0x7f3a680ccce0 2024-09-21T02:14:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680b25b0 con 0x7f3a680ccce0 2024-09-21T02:14:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f3a6001a070 con 0x7f3a680ccce0 2024-09-21T02:14:46.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a66ffd640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a540733b0 0x7f3a540758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f3a60093670 con 0x7f3a680ccce0 2024-09-21T02:14:46.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.411+0000 7f3a6e0c4640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a540733b0 0x7f3a540758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.415+0000 7f3a6e0c4640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a540733b0 0x7f3a540758c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a68074d80 tx=0x7f3a580094b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.415+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3a680b00e0 con 0x7f3a680ccce0 2024-09-21T02:14:46.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.603+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f3a6806d340 con 0x7f3a680ccce0 2024-09-21T02:14:46.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a66ffd640 1 -- 172.21.15.170:0/3428959780 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f3a680d1cb0 con 0x7f3a680ccce0 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a540733b0 msgr2=0x7f3a540758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a540733b0 0x7f3a540758c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a68074d80 tx=0x7f3a580094b0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 msgr2=0x7f3a68166ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a68166ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c3f0 tx=0x7f3a6000c7f0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 shutdown_connections 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3a540733b0 0x7f3a540758c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a680b60e0 0x7f3a68166570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a680ccce0 0x7f3a68166ab0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 --2- 172.21.15.170:0/3428959780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a680b21a0 0x7f3a681567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 >> 172.21.15.170:0/3428959780 conn(0x7f3a6809a9c0 msgr2=0x7f3a680b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 shutdown_connections 2024-09-21T02:14:46.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.607+0000 7f3a70328640 1 -- 172.21.15.170:0/3428959780 wait complete. 2024-09-21T02:14:46.634 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 -- 172.21.15.170:0/3227563443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 msgr2=0x7f14800b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 --2- 172.21.15.170:0/3227563443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f14800b3350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1480071d80 tx=0x7f147401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 -- 172.21.15.170:0/3227563443 shutdown_connections 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 --2- 172.21.15.170:0/3227563443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f14800b3350 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 --2- 172.21.15.170:0/3227563443 >> v2:172.21.15.191:3300/0 conn(0x7f14800b5410 0x7f148006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 -- 172.21.15.170:0/3227563443 >> 172.21.15.170:0/3227563443 conn(0x7f14800b1c40 msgr2=0x7f14800b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 -- 172.21.15.170:0/3227563443 shutdown_connections 2024-09-21T02:14:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.783+0000 7f14866f5640 1 -- 172.21.15.170:0/3227563443 wait complete. 2024-09-21T02:14:46.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 Processor -- start 2024-09-21T02:14:46.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 -- start start 2024-09-21T02:14:46.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f1480147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14800b5410 0x7f1480148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14800ccb00 0x7f148009ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1480169fd0 con 0x7f14800b2f40 2024-09-21T02:14:46.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1480169e50 con 0x7f14800b5410 2024-09-21T02:14:46.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f14866f5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f148016a150 con 0x7f14800ccb00 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f147ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f1480147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f147ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f1480147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57968/0 (socket says 172.21.15.170:57968) 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.787+0000 7f147ffff640 1 -- 172.21.15.170:0/862541263 learned_addr learned my addr 172.21.15.170:0/862541263 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147f7fe640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14800b5410 0x7f1480148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f1484c92640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14800ccb00 0x7f148009ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147ffff640 1 -- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14800ccb00 msgr2=0x7f148009ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147ffff640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14800ccb00 0x7f148009ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147ffff640 1 -- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14800b5410 msgr2=0x7f1480148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147ffff640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14800b5410 0x7f1480148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147ffff640 1 -- 172.21.15.170:0/862541263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1480145fd0 con 0x7f14800b2f40 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f1484c92640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14800ccb00 0x7f148009ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147ffff640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f1480147fc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f147000c970 tx=0x7f147000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470004220 con 0x7f14800b2f40 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14700079b0 con 0x7f14800b2f40 2024-09-21T02:14:46.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1480146240 con 0x7f14800b2f40 2024-09-21T02:14:46.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700044b0 con 0x7f14800b2f40 2024-09-21T02:14:46.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1480146800 con 0x7f14800b2f40 2024-09-21T02:14:46.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14800b3350 con 0x7f14800b2f40 2024-09-21T02:14:46.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f1470005020 con 0x7f14800b2f40 2024-09-21T02:14:46.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147cff9640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f146c0733e0 0x7f146c0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:46.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f14700947d0 con 0x7f14800b2f40 2024-09-21T02:14:46.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.791+0000 7f147f7fe640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f146c0733e0 0x7f146c0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:46.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.795+0000 7f147f7fe640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f146c0733e0 0x7f146c0758f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f14740af000 tx=0x7f147401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:46.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:46.795+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1470014030 con 0x7f14800b2f40 2024-09-21T02:14:47.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.035+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f14800b4d60 con 0x7f14800b2f40 2024-09-21T02:14:47.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.231+0000 7f147cff9640 1 -- 172.21.15.170:0/862541263 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v95) ==== 158+0+0 (secure 0 0 0) 0x7f147005f7f0 con 0x7f14800b2f40 2024-09-21T02:14:47.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f146c0733e0 msgr2=0x7f146c0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f146c0733e0 0x7f146c0758f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f14740af000 tx=0x7f147401b100 comp rx=0 tx=0).stop 2024-09-21T02:14:47.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 msgr2=0x7f1480147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f1480147fc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f147000c970 tx=0x7f147000ce30 comp rx=0 tx=0).stop 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 shutdown_connections 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f146c0733e0 0x7f146c0758f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14800ccb00 0x7f148009ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14800b5410 0x7f1480148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 --2- 172.21.15.170:0/862541263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14800b2f40 0x7f1480147fc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 >> 172.21.15.170:0/862541263 conn(0x7f14800b1c40 msgr2=0x7f14800b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 shutdown_connections 2024-09-21T02:14:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.235+0000 7f14866f5640 1 -- 172.21.15.170:0/862541263 wait complete. 2024-09-21T02:14:47.264 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-21T02:14:47.264 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:14:47.265 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 -- 172.21.15.170:0/3950998096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 msgr2=0x7fc0000caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 --2- 172.21.15.170:0/3950998096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc0000caf90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009840 tx=0x7fbff401f010 comp rx=0 tx=0).stop 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 -- 172.21.15.170:0/3950998096 shutdown_connections 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 --2- 172.21.15.170:0/3950998096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc0000caf90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 -- 172.21.15.170:0/3950998096 >> 172.21.15.170:0/3950998096 conn(0x7fc0000cd3c0 msgr2=0x7fc0000cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 -- 172.21.15.170:0/3950998096 shutdown_connections 2024-09-21T02:14:47.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.411+0000 7fc0080ac640 1 -- 172.21.15.170:0/3950998096 wait complete. 2024-09-21T02:14:47.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 Processor -- start 2024-09-21T02:14:47.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- start start 2024-09-21T02:14:47.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc000070330 0x7fc0000ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc000151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0000cb4d0 0x7fc000151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0000cc830 con 0x7fc0000b5520 2024-09-21T02:14:47.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0000cc6b0 con 0x7fc000070330 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc0000cc9b0 con 0x7fc0000cb4d0 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc000151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc000151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57972/0 (socket says 172.21.15.170:57972) 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 -- 172.21.15.170:0/639137383 learned_addr learned my addr 172.21.15.170:0/639137383 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005e48640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc000070330 0x7fc0000ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc006649640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0000cb4d0 0x7fc000151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 -- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0000cb4d0 msgr2=0x7fc000151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0000cb4d0 0x7fc000151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 -- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc000070330 msgr2=0x7fc0000ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc000070330 0x7fc0000ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 -- 172.21.15.170:0/639137383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0000cef10 con 0x7fc0000b5520 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc005647640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc000151850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009790 tx=0x7fbff40a8d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff40b5040 con 0x7fc0000b5520 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbff4005030 con 0x7fc0000b5520 2024-09-21T02:14:47.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00015a800 con 0x7fc0000b5520 2024-09-21T02:14:47.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4005460 con 0x7fc0000b5520 2024-09-21T02:14:47.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0000cf490 con 0x7fc0000b5520 2024-09-21T02:14:47.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.415+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0000caf90 con 0x7fc0000b5520 2024-09-21T02:14:47.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.419+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fbff40c0020 con 0x7fc0000b5520 2024-09-21T02:14:47.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.419+0000 7fbffe7fc640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbfec0733b0 0x7fbfec0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.419+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fbff41316f0 con 0x7fc0000b5520 2024-09-21T02:14:47.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.419+0000 7fc005e48640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbfec0733b0 0x7fbfec0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.419+0000 7fc005e48640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbfec0733b0 0x7fbfec0758c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc0000b3960 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.423+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbff40c7020 con 0x7fc0000b5520 2024-09-21T02:14:47.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.587+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc00015f4d0 con 0x7fc0000b5520 2024-09-21T02:14:47.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.587+0000 7fbffe7fc640 1 -- 172.21.15.170:0/639137383 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13082 (secure 0 0 0) 0x7fbff4018b70 con 0x7fc0000b5520 2024-09-21T02:14:47.593 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:47.593 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:46.214486+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6835/2504310121":"2024-09-22T02:14:43.200827+0000","172.21.15.170:6834/2504310121":"2024-09-22T02:14:43.200827+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbfec0733b0 msgr2=0x7fbfec0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbfec0733b0 0x7fbfec0758c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc0000b3960 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 msgr2=0x7fc000151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc000151850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009790 tx=0x7fbff40a8d30 comp rx=0 tx=0).stop 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 shutdown_connections 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbfec0733b0 0x7fbfec0758c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0000cb4d0 0x7fc000151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0000b5520 0x7fc000151850 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 --2- 172.21.15.170:0/639137383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc000070330 0x7fc0000ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 >> 172.21.15.170:0/639137383 conn(0x7fc0000cd3c0 msgr2=0x7fc0000cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 shutdown_connections 2024-09-21T02:14:47.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.591+0000 7fc0080ac640 1 -- 172.21.15.170:0/639137383 wait complete. 2024-09-21T02:14:47.614 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:14:47.615 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- 172.21.15.170:0/1998498582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 msgr2=0x7fafd8073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 --2- 172.21.15.170:0/1998498582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd8073b60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009840 tx=0x7fafc801c820 comp rx=0 tx=0).stop 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- 172.21.15.170:0/1998498582 shutdown_connections 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 --2- 172.21.15.170:0/1998498582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd8073b60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- 172.21.15.170:0/1998498582 >> 172.21.15.170:0/1998498582 conn(0x7fafd80d0170 msgr2=0x7fafd80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- 172.21.15.170:0/1998498582 shutdown_connections 2024-09-21T02:14:47.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- 172.21.15.170:0/1998498582 wait complete. 2024-09-21T02:14:47.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 Processor -- start 2024-09-21T02:14:47.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- start start 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80740a0 0x7fafd809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd80744b0 0x7fafd8166db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd80b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8169ed0 con 0x7fafd80cf7d0 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80b5c90 con 0x7fafd80740a0 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafe02c1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fafd816a050 con 0x7fafd80744b0 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafde85e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd80b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafde85e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd80b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58000/0 (socket says 172.21.15.170:58000) 2024-09-21T02:14:47.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.771+0000 7fafde85e640 1 -- 172.21.15.170:0/2963425476 learned_addr learned my addr 172.21.15.170:0/2963425476 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde05d640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80740a0 0x7fafd809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafdd85c640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd80744b0 0x7fafd8166db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde85e640 1 -- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd80744b0 msgr2=0x7fafd8166db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde85e640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd80744b0 0x7fafd8166db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde85e640 1 -- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80740a0 msgr2=0x7fafd809a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde85e640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80740a0 0x7fafd809a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde85e640 1 -- 172.21.15.170:0/2963425476 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd815ee20 con 0x7fafd80cf7d0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde85e640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd80b1d70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fafd000cae0 tx=0x7fafd000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0016020 con 0x7fafd80cf7d0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafd000aae0 con 0x7fafd80cf7d0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd80b0fa0 con 0x7fafd80cf7d0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000ce20 con 0x7fafd80cf7d0 2024-09-21T02:14:47.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd809af90 con 0x7fafd80cf7d0 2024-09-21T02:14:47.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd80b11e0 con 0x7fafd80cf7d0 2024-09-21T02:14:47.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fafd0019020 con 0x7fafd80cf7d0 2024-09-21T02:14:47.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafceffd640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fafbc0733e0 0x7fafbc0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:47.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fafd0094110 con 0x7fafd80cf7d0 2024-09-21T02:14:47.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.775+0000 7fafde05d640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fafbc0733e0 0x7fafbc0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:47.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.779+0000 7fafde05d640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fafbc0733e0 0x7fafbc0758f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fafc80af000 tx=0x7fafc801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:47.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.779+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fafd80b11e0 con 0x7fafd80cf7d0 2024-09-21T02:14:47.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.951+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafd8073b60 con 0x7fafd80cf7d0 2024-09-21T02:14:47.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.951+0000 7fafceffd640 1 -- 172.21.15.170:0/2963425476 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7fafd0020030 con 0x7fafd80cf7d0 2024-09-21T02:14:47.958 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fafbc0733e0 msgr2=0x7fafbc0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fafbc0733e0 0x7fafbc0758f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fafc80af000 tx=0x7fafc801b060 comp rx=0 tx=0).stop 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 msgr2=0x7fafd80b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd80b1d70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fafd000cae0 tx=0x7fafd000cbd0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 shutdown_connections 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fafbc0733e0 0x7fafbc0758f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd80744b0 0x7fafd8166db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80cf7d0 0x7fafd80b1d70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 --2- 172.21.15.170:0/2963425476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80740a0 0x7fafd809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 >> 172.21.15.170:0/2963425476 conn(0x7fafd80d0170 msgr2=0x7fafd80b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 shutdown_connections 2024-09-21T02:14:47.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:47.955+0000 7fafe02c1640 1 -- 172.21.15.170:0/2963425476 wait complete. 2024-09-21T02:14:47.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/684653555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9a8004000 msgr2=0x7ff9a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/684653555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9a8004000 0x7ff9a8004810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009b60 tx=0x7ff9b001d290 comp rx=0 tx=0).stop 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c206f640 1 -- 172.21.15.170:0/684653555 reap_dead start 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/684653555 shutdown_connections 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/684653555 >> 172.21.15.170:0/684653555 conn(0x7ff9bc0b1c40 msgr2=0x7ff9bc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/684653555 shutdown_connections 2024-09-21T02:14:48.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.123+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/684653555 wait complete. 2024-09-21T02:14:48.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 Processor -- start 2024-09-21T02:14:48.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 -- start start 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a8004000 0x7ff9bc09d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 0x7ff9bc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9bc14a770 0x7ff9bc15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0cb9f0 con 0x7ff9bc14a360 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0cb870 con 0x7ff9bc14a770 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc0cbb70 con 0x7ff9a8004000 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 0x7ff9bc15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 0x7ff9bc15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58018/0 (socket says 172.21.15.170:58018) 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 -- 172.21.15.170:0/897507454 learned_addr learned my addr 172.21.15.170:0/897507454 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:48.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c186e640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a8004000 0x7ff9bc09d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c206f640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9bc14a770 0x7ff9bc15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 -- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a8004000 msgr2=0x7ff9bc09d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a8004000 0x7ff9bc09d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 -- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9bc14a770 msgr2=0x7ff9bc15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9bc14a770 0x7ff9bc15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 -- 172.21.15.170:0/897507454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc15d280 con 0x7ff9bc14a360 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c106d640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 0x7ff9bc15e040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff9b001d0c0 tx=0x7ff9b0005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00b1040 con 0x7ff9bc14a360 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b00ac7b0 con 0x7ff9bc14a360 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc15d560 con 0x7ff9bc14a360 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00acbe0 con 0x7ff9bc14a360 2024-09-21T02:14:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.127+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc09a9c0 con 0x7ff9bc14a360 2024-09-21T02:14:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.131+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9bc0ce5d0 con 0x7ff9bc14a360 2024-09-21T02:14:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.131+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7ff9b00be020 con 0x7ff9bc14a360 2024-09-21T02:14:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.131+0000 7ff9ae7fc640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff9a00733b0 0x7ff9a00758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.131+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff9b012e9c0 con 0x7ff9bc14a360 2024-09-21T02:14:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.131+0000 7ff9c186e640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff9a00733b0 0x7ff9a00758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:48.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.131+0000 7ff9c186e640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff9a00733b0 0x7ff9a00758c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000f70 tx=0x7ff9b8001550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:48.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.135+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff9b0130e20 con 0x7ff9bc14a360 2024-09-21T02:14:48.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:48.371+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7ff9bc15ad40 con 0x7ff9bc14a360 2024-09-21T02:14:49.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.243+0000 7ff9ae7fc640 1 -- 172.21.15.170:0/897507454 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v97) ==== 143+0+0 (secure 0 0 0) 0x7ff9b00c5020 con 0x7ff9bc14a360 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff9a00733b0 msgr2=0x7ff9a00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff9a00733b0 0x7ff9a00758c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8000f70 tx=0x7ff9b8001550 comp rx=0 tx=0).stop 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 msgr2=0x7ff9bc15e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 0x7ff9bc15e040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff9b001d0c0 tx=0x7ff9b0005640 comp rx=0 tx=0).stop 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 shutdown_connections 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff9a00733b0 0x7ff9a00758c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9bc14a770 0x7ff9bc15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc14a360 0x7ff9bc15e040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 --2- 172.21.15.170:0/897507454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9a8004000 0x7ff9bc09d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 >> 172.21.15.170:0/897507454 conn(0x7ff9bc0b1c40 msgr2=0x7ff9bc0cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 shutdown_connections 2024-09-21T02:14:49.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.247+0000 7ff9c3ad2640 1 -- 172.21.15.170:0/897507454 wait complete. 2024-09-21T02:14:49.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:14:49.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 -- 172.21.15.170:0/838144702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 msgr2=0x7fa0680cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 --2- 172.21.15.170:0/838144702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0680cb870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa0680d1160 tx=0x7fa05001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 -- 172.21.15.170:0/838144702 shutdown_connections 2024-09-21T02:14:49.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 --2- 172.21.15.170:0/838144702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cc8c0 0x7fa0680cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 --2- 172.21.15.170:0/838144702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0680cb870 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 --2- 172.21.15.170:0/838144702 >> v2:172.21.15.191:3300/0 conn(0x7fa0680cf7e0 0x7fa068070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 -- 172.21.15.170:0/838144702 >> 172.21.15.170:0/838144702 conn(0x7fa0680b1c40 msgr2=0x7fa0680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 -- 172.21.15.170:0/838144702 shutdown_connections 2024-09-21T02:14:49.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 -- 172.21.15.170:0/838144702 wait complete. 2024-09-21T02:14:49.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.423+0000 7fa06db26640 1 Processor -- start 2024-09-21T02:14:49.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- start start 2024-09-21T02:14:49.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0681f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0680cc8c0 0x7fa06815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cf7e0 0x7fa0680b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0680ced60 con 0x7fa0680cb460 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0680cebe0 con 0x7fa0680cc8c0 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa0680ceee0 con 0x7fa0680cf7e0 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0681f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0681f9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58038/0 (socket says 172.21.15.170:58038) 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 -- 172.21.15.170:0/1814977575 learned_addr learned my addr 172.21.15.170:0/1814977575 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa05ffff640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0680cc8c0 0x7fa06815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa067fff640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cf7e0 0x7fa0680b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 -- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cf7e0 msgr2=0x7fa0680b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cf7e0 0x7fa0680b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 -- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0680cc8c0 msgr2=0x7fa06815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0680cc8c0 0x7fa06815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 -- 172.21.15.170:0/1814977575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0681f6ef0 con 0x7fa0680cb460 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa05ffff640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0680cc8c0 0x7fa06815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa067fff640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cf7e0 0x7fa0680b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa0677fe640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0681f9b00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa05800c9f0 tx=0x7fa05800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058007bb0 con 0x7fa0680cb460 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa058007e40 con 0x7fa0680cb460 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05800f690 con 0x7fa0680cb460 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0681f71d0 con 0x7fa0680cb460 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0681f7790 con 0x7fa0680cb460 2024-09-21T02:14:49.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.427+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0680cccd0 con 0x7fa0680cb460 2024-09-21T02:14:49.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.431+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fa05800f830 con 0x7fa0680cb460 2024-09-21T02:14:49.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.431+0000 7fa064ff9640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa0480733b0 0x7fa0480758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.431+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fa058094b10 con 0x7fa0680cb460 2024-09-21T02:14:49.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.431+0000 7fa05ffff640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa0480733b0 0x7fa0480758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.431+0000 7fa05ffff640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa0480733b0 0x7fa0480758c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0500af000 tx=0x7fa05001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:49.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.435+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa058014030 con 0x7fa0680cb460 2024-09-21T02:14:49.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.595+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa0680b0680 con 0x7fa0680cb460 2024-09-21T02:14:49.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.595+0000 7fa064ff9640 1 -- 172.21.15.170:0/1814977575 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7fa05805fa90 con 0x7fa0680cb460 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:14:49.603 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:14:49.604 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:14:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:14:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:14:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:14:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:14:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:14:49.605 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa0480733b0 msgr2=0x7fa0480758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa0480733b0 0x7fa0480758c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0500af000 tx=0x7fa05001b100 comp rx=0 tx=0).stop 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 msgr2=0x7fa0681f9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0681f9b00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa05800c9f0 tx=0x7fa05800ceb0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 shutdown_connections 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa0480733b0 0x7fa0480758c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0680cf7e0 0x7fa0680b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0680cc8c0 0x7fa06815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 --2- 172.21.15.170:0/1814977575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680cb460 0x7fa0681f9b00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 >> 172.21.15.170:0/1814977575 conn(0x7fa0680b1c40 msgr2=0x7fa0680afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 shutdown_connections 2024-09-21T02:14:49.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.599+0000 7fa06db26640 1 -- 172.21.15.170:0/1814977575 wait complete. 2024-09-21T02:14:49.624 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:14:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 -- 172.21.15.170:0/2641615677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80d0c40 msgr2=0x7fe6d80b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 --2- 172.21.15.170:0/2641615677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80d0c40 0x7fe6d80b22b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80740d0 tx=0x7fe6d001ccf0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 -- 172.21.15.170:0/2641615677 shutdown_connections 2024-09-21T02:14:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 --2- 172.21.15.170:0/2641615677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80d0c40 0x7fe6d80b22b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 --2- 172.21.15.170:0/2641615677 >> v2:172.21.15.191:3300/0 conn(0x7fe6d80cce90 0x7fe6d80b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 -- 172.21.15.170:0/2641615677 >> 172.21.15.170:0/2641615677 conn(0x7fe6d80cee00 msgr2=0x7fe6d80cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 -- 172.21.15.170:0/2641615677 shutdown_connections 2024-09-21T02:14:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 -- 172.21.15.170:0/2641615677 wait complete. 2024-09-21T02:14:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 Processor -- start 2024-09-21T02:14:49.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.763+0000 7fe6dde14640 1 -- start start 2024-09-21T02:14:49.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 0x7fe6d806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6d80d0830 0x7fe6d8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d80d0c40 0x7fe6d80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d80b17d0 con 0x7fe6d80cce90 2024-09-21T02:14:49.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d80b1650 con 0x7fe6d80d0c40 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d80b27f0 con 0x7fe6d80d0830 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 0x7fe6d806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 0x7fe6d806f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58050/0 (socket says 172.21.15.170:58050) 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4169694722 learned_addr learned my addr 172.21.15.170:0/4169694722 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d7fff640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d80d0c40 0x7fe6d80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6d80d0830 msgr2=0x7fe6d8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6d80d0830 0x7fe6d8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d80d0c40 msgr2=0x7fe6d80b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d80d0c40 0x7fe6d80b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 -- 172.21.15.170:0/4169694722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d8164da0 con 0x7fe6d80cce90 2024-09-21T02:14:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6d77fe640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 0x7fe6d806f8c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800c970 tx=0x7fe6c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8004220 con 0x7fe6d80cce90 2024-09-21T02:14:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c80044b0 con 0x7fe6d80cce90 2024-09-21T02:14:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81fc620 con 0x7fe6d80cce90 2024-09-21T02:14:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8007bb0 con 0x7fe6d80cce90 2024-09-21T02:14:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81fcbe0 con 0x7fe6d80cce90 2024-09-21T02:14:49.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d80b7e40 con 0x7fe6d80cce90 2024-09-21T02:14:49.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.767+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fe6c8005030 con 0x7fe6d80cce90 2024-09-21T02:14:49.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.771+0000 7fe6bffff640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6c40733e0 0x7fe6c40758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:49.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.771+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fe6c8094ab0 con 0x7fe6d80cce90 2024-09-21T02:14:49.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.771+0000 7fe6d6ffd640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6c40733e0 0x7fe6c40758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:49.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.771+0000 7fe6d6ffd640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6c40733e0 0x7fe6c40758f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80b5ef0 tx=0x7fe6cc006bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:49.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.775+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe6c8014070 con 0x7fe6d80cce90 2024-09-21T02:14:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.931+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe6d80b77e0 con 0x7fe6d80cce90 2024-09-21T02:14:49.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6bffff640 1 -- 172.21.15.170:0/4169694722 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fe6c805fbb0 con 0x7fe6d80cce90 2024-09-21T02:14:49.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6c40733e0 msgr2=0x7fe6c40758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6c40733e0 0x7fe6c40758f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80b5ef0 tx=0x7fe6cc006bb0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 msgr2=0x7fe6d806f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 0x7fe6d806f8c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800c970 tx=0x7fe6c800ce30 comp rx=0 tx=0).stop 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 shutdown_connections 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6c40733e0 0x7fe6c40758f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6d80d0830 0x7fe6d8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d80d0c40 0x7fe6d80b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 --2- 172.21.15.170:0/4169694722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6d80cce90 0x7fe6d806f8c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 >> 172.21.15.170:0/4169694722 conn(0x7fe6d80cee00 msgr2=0x7fe6d80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 shutdown_connections 2024-09-21T02:14:49.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:49.939+0000 7fe6dde14640 1 -- 172.21.15.170:0/4169694722 wait complete. 2024-09-21T02:14:49.964 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:14:49.965 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:14:50.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:14:50.117 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:14:50.117 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:50.117 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:50.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:14:50.123 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:50.123 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:50.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:14:50.129 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:14:50.129 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:50.171 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:14:50.171 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:14:50.171 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:14:50.177 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:14:50.177 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:14:50.178 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 -- 172.21.15.170:0/3944094238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b5410 msgr2=0x7f060006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 --2- 172.21.15.170:0/3944094238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b5410 0x7f060006fa30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0600071d80 tx=0x7f05f401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 -- 172.21.15.170:0/3944094238 shutdown_connections 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 --2- 172.21.15.170:0/3944094238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06000cd210 0x7f06000b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 --2- 172.21.15.170:0/3944094238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b5410 0x7f060006fa30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 -- 172.21.15.170:0/3944094238 >> 172.21.15.170:0/3944094238 conn(0x7f06000b1c40 msgr2=0x7f06000b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 -- 172.21.15.170:0/3944094238 shutdown_connections 2024-09-21T02:14:50.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 -- 172.21.15.170:0/3944094238 wait complete. 2024-09-21T02:14:50.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 Processor -- start 2024-09-21T02:14:50.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.367+0000 7f0607197640 1 -- start start 2024-09-21T02:14:50.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 0x7f0600147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06000b5410 0x7f06001fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0600154670 0x7f06001fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f060016a060 con 0x7f06000b2f40 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f06000701b0 con 0x7f06000b5410 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f060016a1e0 con 0x7f0600154670 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 0x7f0600147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 0x7f0600147330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58070/0 (socket says 172.21.15.170:58070) 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 -- 172.21.15.170:0/3063861066 learned_addr learned my addr 172.21.15.170:0/3063861066 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 -- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0600154670 msgr2=0x7f06001fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0600154670 0x7f06001fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 -- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06000b5410 msgr2=0x7f06001fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fffff640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06000b5410 0x7f06001fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06000b5410 0x7f06001fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 -- 172.21.15.170:0/3063861066 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060015c300 con 0x7f06000b2f40 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0604f33640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 0x7f0600147330 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009850 tx=0x7f05f40a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:50.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40bc070 con 0x7f06000b2f40 2024-09-21T02:14:50.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05f40a6ca0 con 0x7f06000b2f40 2024-09-21T02:14:50.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0600155550 con 0x7f06000b2f40 2024-09-21T02:14:50.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f40b2040 con 0x7f06000b2f40 2024-09-21T02:14:50.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060015a320 con 0x7f06000b2f40 2024-09-21T02:14:50.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060006fa30 con 0x7f06000b2f40 2024-09-21T02:14:50.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f05f40bf070 con 0x7f06000b2f40 2024-09-21T02:14:50.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fd7fa640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05ec0733b0 0x7f05ec0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.371+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f05f412dda0 con 0x7f06000b2f40 2024-09-21T02:14:50.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.375+0000 7f05fffff640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05ec0733b0 0x7f05ec0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.375+0000 7f05fffff640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05ec0733b0 0x7f05ec0758c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0600154ce0 tx=0x7f05f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:50.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.379+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f05f40164d0 con 0x7f06000b2f40 2024-09-21T02:14:50.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.579+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06000b3350 con 0x7f06000b2f40 2024-09-21T02:14:50.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.579+0000 7f05fd7fa640 1 -- 172.21.15.170:0/3063861066 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3920 (secure 0 0 0) 0x7f05f40f8e00 con 0x7f06000b2f40 2024-09-21T02:14:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:50.586 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":20,"btime":"2024-09-21T02:14:44:203516+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2990869342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2990869342},{"type":"v1","addr":"172.21.15.191:6835","nonce":2990869342}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4729,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1814828339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1814828339},{"type":"v1","addr":"172.21.15.191:6837","nonce":1814828339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/301016117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":301016117},{"type":"v1","addr":"172.21.15.170:6833","nonce":301016117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1916016529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1916016529},{"type":"v1","addr":"172.21.15.170:6835","nonce":1916016529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-21T02:14:50.586 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 20 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05ec0733b0 msgr2=0x7f05ec0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05ec0733b0 0x7f05ec0758c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0600154ce0 tx=0x7f05f0008040 comp rx=0 tx=0).stop 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 msgr2=0x7f0600147330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 0x7f0600147330 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009850 tx=0x7f05f40a56b0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 shutdown_connections 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f05ec0733b0 0x7f05ec0758c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0600154670 0x7f06001fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06000b5410 0x7f06001fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 --2- 172.21.15.170:0/3063861066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06000b2f40 0x7f0600147330 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 >> 172.21.15.170:0/3063861066 conn(0x7f06000b1c40 msgr2=0x7f06000c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:50.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 shutdown_connections 2024-09-21T02:14:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.583+0000 7f0607197640 1 -- 172.21.15.170:0/3063861066 wait complete. 2024-09-21T02:14:50.606 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:14:50.606 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:14:50.607 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:14:50.607 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:50.607 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:14:50.607 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:14:50.607 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:14:50.607 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:50.607 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:14:50.608 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:14:50.608 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:14:50.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:50.608 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:14:50.608 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:14:50.608 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:14:50.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:14:50.620 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:14:50.620 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:14:50.624 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:14:50.624 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:14:50.626 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:14:50.626 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:14:50.626 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:14:50.626 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:14:50.669 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:14:50.669 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:50.670 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:14:50.670 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:50.671 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:14:50.671 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:50.672 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:14:50.672 DEBUG:teuthology.parallel:result is None 2024-09-21T02:14:50.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:14:50.757 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:14:50.757 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:14:50.764 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:14:50.765 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 -- 172.21.15.170:0/4000703118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78740b25f0 msgr2=0x7f78740cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 --2- 172.21.15.170:0/4000703118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78740b25f0 0x7f78740cf950 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7870009c70 tx=0x7f787001cd20 comp rx=0 tx=0).stop 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 -- 172.21.15.170:0/4000703118 shutdown_connections 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 --2- 172.21.15.170:0/4000703118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7874160610 0x7f78740b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 --2- 172.21.15.170:0/4000703118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78740b25f0 0x7f78740cf950 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 -- 172.21.15.170:0/4000703118 >> 172.21.15.170:0/4000703118 conn(0x7f787409a9c0 msgr2=0x7f78740d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 -- 172.21.15.170:0/4000703118 shutdown_connections 2024-09-21T02:14:50.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 -- 172.21.15.170:0/4000703118 wait complete. 2024-09-21T02:14:50.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 Processor -- start 2024-09-21T02:14:50.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.823+0000 7f787b332640 1 -- start start 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78740b25f0 0x7f78741503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f787415faf0 0x7f7874155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 0x7f787406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7874168f40 con 0x7f7874160610 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7874168dc0 con 0x7f78740b25f0 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f78741690c0 con 0x7f787415faf0 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 0x7f787406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 0x7f787406d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58134/0 (socket says 172.21.15.170:58134) 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 -- 172.21.15.170:0/989222772 learned_addr learned my addr 172.21.15.170:0/989222772 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78790ce640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78740b25f0 0x7f78741503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78788cd640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f787415faf0 0x7f7874155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 -- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f787415faf0 msgr2=0x7f7874155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f787415faf0 0x7f7874155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 -- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78740b25f0 msgr2=0x7f78741503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78740b25f0 0x7f78741503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 -- 172.21.15.170:0/989222772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7874151140 con 0x7f7874160610 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78798cf640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 0x7f787406d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f786c00d630 tx=0x7f786c00daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c017070 con 0x7f7874160610 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f786c005310 con 0x7f7874160610 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7874155840 con 0x7f7874160610 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0047b0 con 0x7f7874160610 2024-09-21T02:14:50.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78741f7220 con 0x7f7874160610 2024-09-21T02:14:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7874156700 con 0x7f7874160610 2024-09-21T02:14:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f786c004950 con 0x7f7874160610 2024-09-21T02:14:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f7869ffb640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78580733b0 0x7f78580758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f786c094750 con 0x7f7874160610 2024-09-21T02:14:50.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.827+0000 7f78790ce640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78580733b0 0x7f78580758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:50.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.831+0000 7f78790ce640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78580733b0 0x7f78580758c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78700af000 tx=0x7f787001b560 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:50.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:50.831+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f786c05f170 con 0x7f7874160610 2024-09-21T02:14:51.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.003+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f78740ce9d0 con 0x7f7874160610 2024-09-21T02:14:51.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.263+0000 7f7869ffb640 1 -- 172.21.15.170:0/989222772 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f786c063ba0 con 0x7f7874160610 2024-09-21T02:14:51.269 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:14:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78580733b0 msgr2=0x7f78580758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78580733b0 0x7f78580758c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f78700af000 tx=0x7f787001b560 comp rx=0 tx=0).stop 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 msgr2=0x7f787406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 0x7f787406d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f786c00d630 tx=0x7f786c00daf0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 shutdown_connections 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f78580733b0 0x7f78580758c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f787415faf0 0x7f7874155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7874160610 0x7f787406d300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 --2- 172.21.15.170:0/989222772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78740b25f0 0x7f78741503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 >> 172.21.15.170:0/989222772 conn(0x7f787409a9c0 msgr2=0x7f7874071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 shutdown_connections 2024-09-21T02:14:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.267+0000 7f787b332640 1 -- 172.21.15.170:0/989222772 wait complete. 2024-09-21T02:14:51.299 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:14:51.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.447+0000 7fca8ad9c640 1 -- 172.21.15.170:0/444023607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 msgr2=0x7fca8406ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.447+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/444023607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca8406ce00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009bc0 tx=0x7fca6c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- 172.21.15.170:0/444023607 shutdown_connections 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/444023607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca8406ce00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/444023607 >> v2:172.21.15.191:3300/0 conn(0x7fca840b22b0 0x7fca840b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- 172.21.15.170:0/444023607 >> 172.21.15.170:0/444023607 conn(0x7fca8409a9c0 msgr2=0x7fca840d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- 172.21.15.170:0/444023607 shutdown_connections 2024-09-21T02:14:51.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- 172.21.15.170:0/444023607 wait complete. 2024-09-21T02:14:51.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 Processor -- start 2024-09-21T02:14:51.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- start start 2024-09-21T02:14:51.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 0x7fca840ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca840b22b0 0x7fca84159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca841f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fca8415c4d0 con 0x7fca8406d340 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca8415c350 con 0x7fca840b22b0 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca8ad9c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fca8415c650 con 0x7fca840b6930 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 0x7fca840ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 0x7fca840ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58146/0 (socket says 172.21.15.170:58146) 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 -- 172.21.15.170:0/2305071571 learned_addr learned my addr 172.21.15.170:0/2305071571 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca89339640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca841f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca83fff640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca840b22b0 0x7fca84159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 -- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 msgr2=0x7fca841f9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca841f9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 -- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca840b22b0 msgr2=0x7fca84159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca840b22b0 0x7fca84159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 -- 172.21.15.170:0/2305071571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca840c4f40 con 0x7fca8406d340 2024-09-21T02:14:51.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca83fff640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca840b22b0 0x7fca84159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca89339640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca841f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.451+0000 7fca88b38640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 0x7fca840ca030 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00ea40 tx=0x7fca7c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00ce40 con 0x7fca8406d340 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca7c004020 con 0x7fca8406d340 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca840c5220 con 0x7fca8406d340 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004470 con 0x7fca8406d340 2024-09-21T02:14:51.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca840c57e0 con 0x7fca8406d340 2024-09-21T02:14:51.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8415cb00 con 0x7fca8406d340 2024-09-21T02:14:51.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fca7c005030 con 0x7fca8406d340 2024-09-21T02:14:51.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca817fa640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca64073410 0x7fca64075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fca7c094ce0 con 0x7fca8406d340 2024-09-21T02:14:51.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca83fff640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca64073410 0x7fca64075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.455+0000 7fca83fff640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca64073410 0x7fca64075920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009bc0 tx=0x7fca6c01b3e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.459+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fca7c05fd50 con 0x7fca8406d340 2024-09-21T02:14:51.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.695+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fca8415bc90 con 0x7fca8406d340 2024-09-21T02:14:51.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.695+0000 7fca817fa640 1 -- 172.21.15.170:0/2305071571 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fca7c05f650 con 0x7fca8406d340 2024-09-21T02:14:51.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:51.700 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:14:51.700 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:14:51.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.702 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.703 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:14:51.706 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:14:51.708 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:14:51.709 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:51.710 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:14:51.711 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:14:51.713 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:14:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca64073410 msgr2=0x7fca64075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca64073410 0x7fca64075920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009bc0 tx=0x7fca6c01b3e0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 msgr2=0x7fca840ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 0x7fca840ca030 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00ea40 tx=0x7fca7c00ef00 comp rx=0 tx=0).stop 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 shutdown_connections 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fca64073410 0x7fca64075920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca840b6930 0x7fca841f9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca840b22b0 0x7fca84159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 --2- 172.21.15.170:0/2305071571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8406d340 0x7fca840ca030 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 >> 172.21.15.170:0/2305071571 conn(0x7fca8409a9c0 msgr2=0x7fca840b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 shutdown_connections 2024-09-21T02:14:51.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.699+0000 7fca8ad9c640 1 -- 172.21.15.170:0/2305071571 wait complete. 2024-09-21T02:14:51.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa86182a640 1 -- 172.21.15.170:0/334748420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 msgr2=0x7fa83c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa86182a640 1 --2- 172.21.15.170:0/334748420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa83c005160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c610 tx=0x7fa85401d300 comp rx=0 tx=0).stop 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa85b7fe640 1 -- 172.21.15.170:0/334748420 reap_dead start 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa86182a640 1 -- 172.21.15.170:0/334748420 shutdown_connections 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa86182a640 1 -- 172.21.15.170:0/334748420 >> 172.21.15.170:0/334748420 conn(0x7fa85c0b1c40 msgr2=0x7fa85c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa86182a640 1 -- 172.21.15.170:0/334748420 shutdown_connections 2024-09-21T02:14:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.855+0000 7fa86182a640 1 -- 172.21.15.170:0/334748420 wait complete. 2024-09-21T02:14:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 Processor -- start 2024-09-21T02:14:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 -- start start 2024-09-21T02:14:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa85c074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c15c620 0x7fa85c1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa85c15d0f0 0x7fa85c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c0c39f0 con 0x7fa83c004d50 2024-09-21T02:14:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c0cf970 con 0x7fa85c15c620 2024-09-21T02:14:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c06fa30 con 0x7fa85c15d0f0 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa85c074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa85c074700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58170/0 (socket says 172.21.15.170:58170) 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 -- 172.21.15.170:0/3768587089 learned_addr learned my addr 172.21.15.170:0/3768587089 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa853fff640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c15c620 0x7fa85c1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85b7fe640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa85c15d0f0 0x7fa85c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 -- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa85c15d0f0 msgr2=0x7fa85c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa85c15d0f0 0x7fa85c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 -- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c15c620 msgr2=0x7fa85c1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c15c620 0x7fa85c1f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 -- 172.21.15.170:0/3768587089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c073c60 con 0x7fa83c004d50 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa853fff640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c15c620 0x7fa85c1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa85affd640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa85c074700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011a80 tx=0x7fa84c011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00fe10 con 0x7fa83c004d50 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c016070 con 0x7fa83c004d50 2024-09-21T02:14:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1f6160 con 0x7fa83c004d50 2024-09-21T02:14:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c012780 con 0x7fa83c004d50 2024-09-21T02:14:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.859+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c073f00 con 0x7fa83c004d50 2024-09-21T02:14:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.863+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c06fee0 con 0x7fa83c004d50 2024-09-21T02:14:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.863+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fa84c018030 con 0x7fa83c004d50 2024-09-21T02:14:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.863+0000 7fa858ff9640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa8400733b0 0x7fa8400758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.863+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fa84c094c00 con 0x7fa83c004d50 2024-09-21T02:14:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.863+0000 7fa853fff640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa8400733b0 0x7fa8400758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.863+0000 7fa853fff640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa8400733b0 0x7fa8400758c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa85c073bf0 tx=0x7fa8440091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:51.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:51.867+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa84c09b080 con 0x7fa83c004d50 2024-09-21T02:14:52.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.039+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa85c0b4e70 con 0x7fa83c004d50 2024-09-21T02:14:52.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.039+0000 7fa858ff9640 1 -- 172.21.15.170:0/3768587089 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7fa84c05fc70 con 0x7fa83c004d50 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:14:52.046 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:14:52.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa8400733b0 msgr2=0x7fa8400758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa8400733b0 0x7fa8400758c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa85c073bf0 tx=0x7fa8440091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 msgr2=0x7fa85c074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa85c074700 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011a80 tx=0x7fa84c011f40 comp rx=0 tx=0).stop 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 shutdown_connections 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa8400733b0 0x7fa8400758c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa85c15d0f0 0x7fa85c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c15c620 0x7fa85c1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 --2- 172.21.15.170:0/3768587089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83c004d50 0x7fa85c074700 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 >> 172.21.15.170:0/3768587089 conn(0x7fa85c0b1c40 msgr2=0x7fa85c0cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 shutdown_connections 2024-09-21T02:14:52.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.043+0000 7fa86182a640 1 -- 172.21.15.170:0/3768587089 wait complete. 2024-09-21T02:14:52.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:14:52.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:14:52.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 -- 172.21.15.170:0/1219810186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015c8a0 msgr2=0x7f157015ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 --2- 172.21.15.170:0/1219810186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015c8a0 0x7f157015ccb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1558009bc0 tx=0x7f155801f3b0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 -- 172.21.15.170:0/1219810186 shutdown_connections 2024-09-21T02:14:52.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 --2- 172.21.15.170:0/1219810186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015c8a0 0x7f157015ccb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 -- 172.21.15.170:0/1219810186 >> 172.21.15.170:0/1219810186 conn(0x7f15700ced80 msgr2=0x7f15700cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:52.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 -- 172.21.15.170:0/1219810186 shutdown_connections 2024-09-21T02:14:52.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 -- 172.21.15.170:0/1219810186 wait complete. 2024-09-21T02:14:52.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 Processor -- start 2024-09-21T02:14:52.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.207+0000 7f1576a0a640 1 -- start start 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1570071150 0x7f15700b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f157015c8a0 0x7f15701f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 0x7f1570158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15700d0aa0 con 0x7f157015d1f0 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15700d0920 con 0x7f1570071150 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1570168440 con 0x7f157015c8a0 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 0x7f1570158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 0x7f1570158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58182/0 (socket says 172.21.15.170:58182) 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 -- 172.21.15.170:0/2358157736 learned_addr learned my addr 172.21.15.170:0/2358157736 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156f7fe640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f157015c8a0 0x7f15701f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156ffff640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1570071150 0x7f15700b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 -- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f157015c8a0 msgr2=0x7f15701f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f157015c8a0 0x7f15701f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 -- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1570071150 msgr2=0x7f15700b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1570071150 0x7f15700b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 -- 172.21.15.170:0/2358157736 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15700b4b60 con 0x7f157015d1f0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156ffff640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1570071150 0x7f15700b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1574fa7640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 0x7f1570158270 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f156000b4c0 tx=0x7f156000b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560015020 con 0x7f157015d1f0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1560011920 con 0x7f157015d1f0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560011d70 con 0x7f157015d1f0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15700b1cf0 con 0x7f157015d1f0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15700b22b0 con 0x7f157015d1f0 2024-09-21T02:14:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157015ccb0 con 0x7f157015d1f0 2024-09-21T02:14:52.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.211+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f1560018020 con 0x7f157015d1f0 2024-09-21T02:14:52.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.215+0000 7f156cff9640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f15500733b0 0x7f15500758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:52.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.215+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f1560093df0 con 0x7f157015d1f0 2024-09-21T02:14:52.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.215+0000 7f156ffff640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f15500733b0 0x7f15500758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:52.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.215+0000 7f156ffff640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f15500733b0 0x7f15500758c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1568001630 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:52.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.219+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f156001f030 con 0x7f157015d1f0 2024-09-21T02:14:52.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:52.391+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f15700cc490 con 0x7f157015d1f0 2024-09-21T02:14:53.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.275+0000 7f156cff9640 1 -- 172.21.15.170:0/2358157736 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f156005ee60 con 0x7f157015d1f0 2024-09-21T02:14:53.282 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:14:53.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f15500733b0 msgr2=0x7f15500758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:53.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f15500733b0 0x7f15500758c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1568001630 tx=0x7f1568009040 comp rx=0 tx=0).stop 2024-09-21T02:14:53.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 msgr2=0x7f1570158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:53.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 0x7f1570158270 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f156000b4c0 tx=0x7f156000b980 comp rx=0 tx=0).stop 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 shutdown_connections 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f15500733b0 0x7f15500758c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f157015c8a0 0x7f15701f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f157015d1f0 0x7f1570158270 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 --2- 172.21.15.170:0/2358157736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1570071150 0x7f15700b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 >> 172.21.15.170:0/2358157736 conn(0x7f15700ced80 msgr2=0x7f15700b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 shutdown_connections 2024-09-21T02:14:53.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.279+0000 7f1576a0a640 1 -- 172.21.15.170:0/2358157736 wait complete. 2024-09-21T02:14:53.316 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:14:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 -- 172.21.15.170:0/3730949573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f553c003a60 msgr2=0x7f553c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 --2- 172.21.15.170:0/3730949573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f553c003a60 0x7f553c003ed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5544009b60 tx=0x7f554401d270 comp rx=0 tx=0).stop 2024-09-21T02:14:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 -- 172.21.15.170:0/3730949573 shutdown_connections 2024-09-21T02:14:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 --2- 172.21.15.170:0/3730949573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f553c003a60 0x7f553c003ed0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 -- 172.21.15.170:0/3730949573 >> 172.21.15.170:0/3730949573 conn(0x7f55540d0170 msgr2=0x7f55540d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 -- 172.21.15.170:0/3730949573 shutdown_connections 2024-09-21T02:14:53.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 -- 172.21.15.170:0/3730949573 wait complete. 2024-09-21T02:14:53.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 Processor -- start 2024-09-21T02:14:53.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.475+0000 7f5559aa6640 1 -- start start 2024-09-21T02:14:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f553c003a60 0x7f5554154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55541fad60 0x7f55541fb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 0x7f55541595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55540afa10 con 0x7f55541fb6b0 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f55540b35e0 con 0x7f553c003a60 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f555406f750 con 0x7f55541fad60 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 0x7f55541595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 0x7f55541595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58200/0 (socket says 172.21.15.170:58200) 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 -- 172.21.15.170:0/237085401 learned_addr learned my addr 172.21.15.170:0/237085401 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f55537fe640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f553c003a60 0x7f5554154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5552ffd640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55541fad60 0x7f55541fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 -- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55541fad60 msgr2=0x7f55541fb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55541fad60 0x7f55541fb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 -- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f553c003a60 msgr2=0x7f5554154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f553c003a60 0x7f5554154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 -- 172.21.15.170:0/237085401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55540b2780 con 0x7f55541fb6b0 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f55537fe640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f553c003a60 0x7f5554154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5553fff640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 0x7f55541595e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f555409e220 tx=0x7f554c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:53.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c011870 con 0x7f55541fb6b0 2024-09-21T02:14:53.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f554c017070 con 0x7f55541fb6b0 2024-09-21T02:14:53.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55540b2a60 con 0x7f55541fb6b0 2024-09-21T02:14:53.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c011da0 con 0x7f55541fb6b0 2024-09-21T02:14:53.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5554155220 con 0x7f55541fb6b0 2024-09-21T02:14:53.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.479+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55540c9fd0 con 0x7f55541fb6b0 2024-09-21T02:14:53.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.483+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f554c026430 con 0x7f55541fb6b0 2024-09-21T02:14:53.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.483+0000 7f553bfff640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55400733b0 0x7f55400758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:53.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.483+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7f554c095740 con 0x7f55541fb6b0 2024-09-21T02:14:53.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.483+0000 7f55537fe640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55400733b0 0x7f55400758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:53.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.483+0000 7f55537fe640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55400733b0 0x7f55400758c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55440ae000 tx=0x7f554401a010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:53.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.487+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f554c060690 con 0x7f55541fb6b0 2024-09-21T02:14:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:53.659+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f555406fc50 con 0x7f55541fb6b0 2024-09-21T02:14:53.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:14:53.702 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:14:54.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.287+0000 7f553bfff640 1 -- 172.21.15.170:0/237085401 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7f554c05ff90 con 0x7f55541fb6b0 2024-09-21T02:14:54.294 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55400733b0 msgr2=0x7f55400758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55400733b0 0x7f55400758c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55440ae000 tx=0x7f554401a010 comp rx=0 tx=0).stop 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 msgr2=0x7f55541595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 0x7f55541595e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f555409e220 tx=0x7f554c0019c0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 shutdown_connections 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f55400733b0 0x7f55400758c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55541fad60 0x7f55541fb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55541fb6b0 0x7f55541595e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 --2- 172.21.15.170:0/237085401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f553c003a60 0x7f5554154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 >> 172.21.15.170:0/237085401 conn(0x7f55540d0170 msgr2=0x7f555415c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 shutdown_connections 2024-09-21T02:14:54.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.291+0000 7f5559aa6640 1 -- 172.21.15.170:0/237085401 wait complete. 2024-09-21T02:14:54.338 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:14:54.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 -- 172.21.15.170:0/4004861018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 msgr2=0x7f041806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 --2- 172.21.15.170:0/4004861018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f041806e7c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0408009bc0 tx=0x7f040801f3b0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 -- 172.21.15.170:0/4004861018 shutdown_connections 2024-09-21T02:14:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 --2- 172.21.15.170:0/4004861018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f041806e7c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 -- 172.21.15.170:0/4004861018 >> 172.21.15.170:0/4004861018 conn(0x7f04180d2a30 msgr2=0x7f04180cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 -- 172.21.15.170:0/4004861018 shutdown_connections 2024-09-21T02:14:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.479+0000 7f041dddd640 1 -- 172.21.15.170:0/4004861018 wait complete. 2024-09-21T02:14:54.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 Processor -- start 2024-09-21T02:14:54.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 -- start start 2024-09-21T02:14:54.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f04181f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041815fa80 0x7f04181f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0418162530 0x7f04181f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f041806f090 con 0x7f041815f670 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f041806ef10 con 0x7f041815fa80 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f041816a8d0 con 0x7f0418162530 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f04181f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f04181f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58216/0 (socket says 172.21.15.170:58216) 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 -- 172.21.15.170:0/1733827173 learned_addr learned my addr 172.21.15.170:0/1733827173 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f0417fff640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0418162530 0x7f04181f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f0416ffd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041815fa80 0x7f04181f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 -- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0418162530 msgr2=0x7f04181f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0418162530 0x7f04181f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 -- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041815fa80 msgr2=0x7f04181f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041815fa80 0x7f04181f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 -- 172.21.15.170:0/1733827173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418159c70 con 0x7f041815f670 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f04177fe640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f04181f6ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0410001ec0 tx=0x7f041000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410017070 con 0x7f041815f670 2024-09-21T02:14:54.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0410003600 con 0x7f041815f670 2024-09-21T02:14:54.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0418158230 con 0x7f041815f670 2024-09-21T02:14:54.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410003a50 con 0x7f041815f670 2024-09-21T02:14:54.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.483+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181587f0 con 0x7f041815f670 2024-09-21T02:14:54.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.487+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418158050 con 0x7f041815f670 2024-09-21T02:14:54.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.487+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f041001a070 con 0x7f041815f670 2024-09-21T02:14:54.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.487+0000 7f040ffff640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f03f80733b0 0x7f03f80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:54.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.487+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7f0410093df0 con 0x7f041815f670 2024-09-21T02:14:54.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.487+0000 7f0416ffd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f03f80733b0 0x7f03f80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:54.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.487+0000 7f0416ffd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f03f80733b0 0x7f03f80758c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04080b1000 tx=0x7f040801dbf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:54.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.491+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0418158050 con 0x7f041815f670 2024-09-21T02:14:54.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:54.687+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f04180b8350 con 0x7f041815f670 2024-09-21T02:14:55.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.291+0000 7f040ffff640 1 -- 172.21.15.170:0/1733827173 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f0410021020 con 0x7f041815f670 2024-09-21T02:14:55.297 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 6 and data pool 7 2024-09-21T02:14:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f03f80733b0 msgr2=0x7f03f80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f03f80733b0 0x7f03f80758c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04080b1000 tx=0x7f040801dbf0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 msgr2=0x7f04181f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f04181f6ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0410001ec0 tx=0x7f041000c890 comp rx=0 tx=0).stop 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 shutdown_connections 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f03f80733b0 0x7f03f80758c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0418162530 0x7f04181f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041815fa80 0x7f04181f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 --2- 172.21.15.170:0/1733827173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f041815f670 0x7f04181f6ff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 >> 172.21.15.170:0/1733827173 conn(0x7f04180d2a30 msgr2=0x7f041815e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 shutdown_connections 2024-09-21T02:14:55.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.295+0000 7f041dddd640 1 -- 172.21.15.170:0/1733827173 wait complete. 2024-09-21T02:14:55.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4000956586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 msgr2=0x7ffbc80ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4000956586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc80ce5d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80d1160 tx=0x7ffbc001cb80 comp rx=0 tx=0).stop 2024-09-21T02:14:55.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4000956586 shutdown_connections 2024-09-21T02:14:55.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4000956586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc80ce5d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4000956586 >> v2:172.21.15.191:3300/0 conn(0x7ffbc80cf7e0 0x7ffbc8070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4000956586 >> 172.21.15.170:0/4000956586 conn(0x7ffbc80b1c40 msgr2=0x7ffbc80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4000956586 shutdown_connections 2024-09-21T02:14:55.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.491+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4000956586 wait complete. 2024-09-21T02:14:55.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 Processor -- start 2024-09-21T02:14:55.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- start start 2024-09-21T02:14:55.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc80cb460 0x7ffbc80c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc80cf7e0 0x7ffbc8152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81690d0 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8168f50 con 0x7ffbc80cb460 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc8169250 con 0x7ffbc80cf7e0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc8151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59720/0 (socket says 172.21.15.170:59720) 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 -- 172.21.15.170:0/4157608214 learned_addr learned my addr 172.21.15.170:0/4157608214 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc77fe640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc80cb460 0x7ffbc80c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc7fff640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc80cf7e0 0x7ffbc8152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 -- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc80cf7e0 msgr2=0x7ffbc8152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc80cf7e0 0x7ffbc8152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 -- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc80cb460 msgr2=0x7ffbc80c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc80cb460 0x7ffbc80c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 -- 172.21.15.170:0/4157608214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc815bc90 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbc6ffd640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc8151b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800c3f0 tx=0x7ffbb800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8017070 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbb8003600 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc8156810 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003a50 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc815eb00 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.495+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc8152830 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.499+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7ffbb801a070 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.499+0000 7ffbaffff640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffbb40733e0 0x7ffbb40758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.499+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb8093830 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.499+0000 7ffbc77fe640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffbb40733e0 0x7ffbb40758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.499+0000 7ffbc77fe640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffbb40733e0 0x7ffbb40758f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0058d0 tx=0x7ffbbc005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.503+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffbb805e5e0 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.679+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbc80ce5d0 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.679+0000 7ffbaffff640 1 -- 172.21.15.170:0/4157608214 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16730 (secure 0 0 0) 0x7ffbb805dee0 con 0x7ffbc80ce1c0 2024-09-21T02:14:55.686 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:55.686 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":45,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:54.692564+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:52.409252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:53.674497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:55.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffbb40733e0 msgr2=0x7ffbb40758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffbb40733e0 0x7ffbb40758f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0058d0 tx=0x7ffbbc005860 comp rx=0 tx=0).stop 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 msgr2=0x7ffbc8151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc8151b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800c3f0 tx=0x7ffbb800c8b0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 shutdown_connections 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffbb40733e0 0x7ffbb40758f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbc80cf7e0 0x7ffbc8152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbc80ce1c0 0x7ffbc8151b00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 --2- 172.21.15.170:0/4157608214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbc80cb460 0x7ffbc80c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 >> 172.21.15.170:0/4157608214 conn(0x7ffbc80b1c40 msgr2=0x7ffbc80cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 shutdown_connections 2024-09-21T02:14:55.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.683+0000 7ffbcdabb640 1 -- 172.21.15.170:0/4157608214 wait complete. 2024-09-21T02:14:55.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 -- 172.21.15.170:0/1658263785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b2f40 msgr2=0x7fce5c0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 --2- 172.21.15.170:0/1658263785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b2f40 0x7fce5c0b3350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0c7150 tx=0x7fce4c01c800 comp rx=0 tx=0).stop 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 -- 172.21.15.170:0/1658263785 shutdown_connections 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 --2- 172.21.15.170:0/1658263785 >> v2:172.21.15.191:3300/0 conn(0x7fce5c0ccb00 0x7fce5c0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 --2- 172.21.15.170:0/1658263785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b2f40 0x7fce5c0b3350 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 -- 172.21.15.170:0/1658263785 >> 172.21.15.170:0/1658263785 conn(0x7fce5c0b1c40 msgr2=0x7fce5c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 -- 172.21.15.170:0/1658263785 shutdown_connections 2024-09-21T02:14:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.863+0000 7fce64057640 1 -- 172.21.15.170:0/1658263785 wait complete. 2024-09-21T02:14:55.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 Processor -- start 2024-09-21T02:14:55.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- start start 2024-09-21T02:14:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 0x7fce5c14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5c0ccb00 0x7fce5c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5c09dd70 0x7fce5c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c169e50 con 0x7fce5c0b5410 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0cf970 con 0x7fce5c0ccb00 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c169fd0 con 0x7fce5c09dd70 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 0x7fce5c14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 0x7fce5c14b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59736/0 (socket says 172.21.15.170:59736) 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 -- 172.21.15.170:0/660190081 learned_addr learned my addr 172.21.15.170:0/660190081 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 -- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5c09dd70 msgr2=0x7fce5c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5c09dd70 0x7fce5c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 -- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5c0ccb00 msgr2=0x7fce5c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce615f2640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5c0ccb00 0x7fce5c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5c0ccb00 0x7fce5c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 -- 172.21.15.170:0/660190081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c1fa120 con 0x7fce5c0b5410 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce61df3640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 0x7fce5c14b260 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c730 tx=0x7fce5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58016020 con 0x7fce5c0b5410 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce5800a950 con 0x7fce5c0b5410 2024-09-21T02:14:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5800ada0 con 0x7fce5c0b5410 2024-09-21T02:14:55.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1fa400 con 0x7fce5c0b5410 2024-09-21T02:14:55.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1fa9c0 con 0x7fce5c0b5410 2024-09-21T02:14:55.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.867+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c0cd230 con 0x7fce5c0b5410 2024-09-21T02:14:55.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.871+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fce58019020 con 0x7fce5c0b5410 2024-09-21T02:14:55.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.871+0000 7fce527fc640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce400733b0 0x7fce400758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:55.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.871+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fce580949d0 con 0x7fce5c0b5410 2024-09-21T02:14:55.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.871+0000 7fce615f2640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce400733b0 0x7fce400758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:55.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.871+0000 7fce615f2640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce400733b0 0x7fce400758c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0af000 tx=0x7fce4c01b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:55.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:55.875+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fce58020020 con 0x7fce5c0b5410 2024-09-21T02:14:56.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.047+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce5c0b44c0 con 0x7fce5c0b5410 2024-09-21T02:14:56.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce527fc640 1 -- 172.21.15.170:0/660190081 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16730 (secure 0 0 0) 0x7fce5c0cd450 con 0x7fce5c0b5410 2024-09-21T02:14:56.055 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:56.055 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":45,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:54.692564+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:52.409252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:53.674497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:56.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce400733b0 msgr2=0x7fce400758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:56.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce400733b0 0x7fce400758c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0af000 tx=0x7fce4c01b040 comp rx=0 tx=0).stop 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 msgr2=0x7fce5c14b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 0x7fce5c14b260 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c730 tx=0x7fce5800cbf0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 shutdown_connections 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce400733b0 0x7fce400758c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce5c09dd70 0x7fce5c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce5c0ccb00 0x7fce5c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 --2- 172.21.15.170:0/660190081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce5c0b5410 0x7fce5c14b260 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 >> 172.21.15.170:0/660190081 conn(0x7fce5c0b1c40 msgr2=0x7fce5c0b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 shutdown_connections 2024-09-21T02:14:56.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.051+0000 7fce64057640 1 -- 172.21.15.170:0/660190081 wait complete. 2024-09-21T02:14:56.077 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:14:56.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 -- 172.21.15.170:0/981550954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 msgr2=0x7f1f800ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 --2- 172.21.15.170:0/981550954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800ce480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1f800726b0 tx=0x7f1f7801cd80 comp rx=0 tx=0).stop 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 -- 172.21.15.170:0/981550954 shutdown_connections 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 --2- 172.21.15.170:0/981550954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800ce480 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 --2- 172.21.15.170:0/981550954 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800740a0 0x7f1f800744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 --2- 172.21.15.170:0/981550954 >> v2:172.21.15.191:3300/0 conn(0x7f1f800cf7d0 0x7f1f80073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 -- 172.21.15.170:0/981550954 >> 172.21.15.170:0/981550954 conn(0x7f1f800d0170 msgr2=0x7f1f800d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 -- 172.21.15.170:0/981550954 shutdown_connections 2024-09-21T02:14:56.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.223+0000 7f1f859bd640 1 -- 172.21.15.170:0/981550954 wait complete. 2024-09-21T02:14:56.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 Processor -- start 2024-09-21T02:14:56.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- start start 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800740a0 0x7f1f8009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf7d0 0x7f1f800c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800cbe20 con 0x7f1f800ce070 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800cbca0 con 0x7f1f800740a0 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1f800cbfa0 con 0x7f1f800cf7d0 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59764/0 (socket says 172.21.15.170:59764) 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/2249627296 learned_addr learned my addr 172.21.15.170:0/2249627296 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf7d0 msgr2=0x7f1f800c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7f7fe640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf7d0 0x7f1f800c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7effd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800740a0 0x7f1f8009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf7d0 0x7f1f800c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800740a0 msgr2=0x7f1f8009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800740a0 0x7f1f8009a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/2249627296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8009b330 con 0x7f1f800ce070 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f7e7fc640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800c5d50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1f70005950 tx=0x7f1f7000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70007bb0 con 0x7f1f800ce070 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f70007e40 con 0x7f1f800ce070 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8009b610 con 0x7f1f800ce070 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000f690 con 0x7f1f800ce070 2024-09-21T02:14:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f800aff50 con 0x7f1f800ce070 2024-09-21T02:14:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8009b080 con 0x7f1f800ce070 2024-09-21T02:14:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.227+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f1f7000f830 con 0x7f1f800ce070 2024-09-21T02:14:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.231+0000 7f1f67fff640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f6c0733b0 0x7f1f6c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.231+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1f70094dd0 con 0x7f1f800ce070 2024-09-21T02:14:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.231+0000 7f1f7effd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f6c0733b0 0x7f1f6c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.231+0000 7f1f7effd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f6c0733b0 0x7f1f6c0758c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b9a0 tx=0x7f1f74005c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.235+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1f70014030 con 0x7f1f800ce070 2024-09-21T02:14:56.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:56.439+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1f800ce480 con 0x7f1f800ce070 2024-09-21T02:14:57.285 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-09-21T02:14:57.281+0000 7f2420f37640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:14:57.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.326+0000 7f1f67fff640 1 -- 172.21.15.170:0/2249627296 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f1f7005faf0 con 0x7f1f800ce070 2024-09-21T02:14:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.326+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f6c0733b0 msgr2=0x7f1f6c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f6c0733b0 0x7f1f6c0758c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b9a0 tx=0x7f1f74005c30 comp rx=0 tx=0).stop 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 msgr2=0x7f1f800c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800c5d50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1f70005950 tx=0x7f1f7000cdd0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 shutdown_connections 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1f6c0733b0 0x7f1f6c0758c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f800cf7d0 0x7f1f800c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f800ce070 0x7f1f800c5d50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 --2- 172.21.15.170:0/2249627296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f800740a0 0x7f1f8009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 >> 172.21.15.170:0/2249627296 conn(0x7f1f800d0170 msgr2=0x7f1f800b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 shutdown_connections 2024-09-21T02:14:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.330+0000 7f1f859bd640 1 -- 172.21.15.170:0/2249627296 wait complete. 2024-09-21T02:14:57.360 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:14:57.360 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 -- 172.21.15.170:0/888111464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 msgr2=0x7f30680cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 --2- 172.21.15.170:0/888111464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680cf950 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f306808d520 tx=0x7f306001cba0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 -- 172.21.15.170:0/888111464 shutdown_connections 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 --2- 172.21.15.170:0/888111464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680cf950 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 --2- 172.21.15.170:0/888111464 >> v2:172.21.15.191:3300/0 conn(0x7f30680d4ed0 0x7f306806fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 -- 172.21.15.170:0/888111464 >> 172.21.15.170:0/888111464 conn(0x7f306808bf50 msgr2=0x7f30680d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:57.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 -- 172.21.15.170:0/888111464 shutdown_connections 2024-09-21T02:14:57.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.514+0000 7f306cfcc640 1 -- 172.21.15.170:0/888111464 wait complete. 2024-09-21T02:14:57.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 Processor -- start 2024-09-21T02:14:57.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 -- start start 2024-09-21T02:14:57.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30680d0df0 0x7f306815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30680d4ed0 0x7f306815af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f306816c4a0 con 0x7f30680cf540 2024-09-21T02:14:57.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f306816c320 con 0x7f30680d0df0 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f306816c620 con 0x7f30680d4ed0 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59778/0 (socket says 172.21.15.170:59778) 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 -- 172.21.15.170:0/2839721014 learned_addr learned my addr 172.21.15.170:0/2839721014 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f3066d9d640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30680d4ed0 0x7f306815af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f3065d9b640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30680d0df0 0x7f306815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 -- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30680d4ed0 msgr2=0x7f306815af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30680d4ed0 0x7f306815af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 -- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30680d0df0 msgr2=0x7f306815a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30680d0df0 0x7f306815a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 -- 172.21.15.170:0/2839721014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30680c8210 con 0x7f30680cf540 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f3065d9b640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30680d0df0 0x7f306815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306659c640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680b57d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f305800d950 tx=0x7f305800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058017070 con 0x7f30680cf540 2024-09-21T02:14:57.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3058009d20 con 0x7f30680cf540 2024-09-21T02:14:57.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30680c84f0 con 0x7f30680cf540 2024-09-21T02:14:57.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580106c0 con 0x7f30680cf540 2024-09-21T02:14:57.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.518+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30680c8ab0 con 0x7f30680cf540 2024-09-21T02:14:57.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.522+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30680cf950 con 0x7f30680cf540 2024-09-21T02:14:57.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.522+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f305801a070 con 0x7f30680cf540 2024-09-21T02:14:57.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.522+0000 7f3056ffd640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30480733e0 0x7f30480758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:57.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.522+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f3058094b80 con 0x7f30680cf540 2024-09-21T02:14:57.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.522+0000 7f3065d9b640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30480733e0 0x7f30480758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:57.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.522+0000 7f3065d9b640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30480733e0 0x7f30480758f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f30680c9330 tx=0x7f305c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:57.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.526+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f305805f910 con 0x7f30680cf540 2024-09-21T02:14:57.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:57.734+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f30680d1ee0 con 0x7f30680cf540 2024-09-21T02:14:58.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.326+0000 7f3056ffd640 1 -- 172.21.15.170:0/2839721014 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f305805f210 con 0x7f30680cf540 2024-09-21T02:14:58.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30480733e0 msgr2=0x7f30480758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30480733e0 0x7f30480758f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f30680c9330 tx=0x7f305c008040 comp rx=0 tx=0).stop 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 msgr2=0x7f30680b57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680b57d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f305800d950 tx=0x7f305800de10 comp rx=0 tx=0).stop 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 shutdown_connections 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f30480733e0 0x7f30480758f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30680d4ed0 0x7f306815af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30680d0df0 0x7f306815a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 --2- 172.21.15.170:0/2839721014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30680cf540 0x7f30680b57d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 >> 172.21.15.170:0/2839721014 conn(0x7f306808bf50 msgr2=0x7f30680d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 shutdown_connections 2024-09-21T02:14:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.330+0000 7f306cfcc640 1 -- 172.21.15.170:0/2839721014 wait complete. 2024-09-21T02:14:58.363 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:14:58.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:14:58.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 -- 172.21.15.170:0/2473248973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 msgr2=0x7f631c0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 --2- 172.21.15.170:0/2473248973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0cccd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f631c0d1160 tx=0x7f630c01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 -- 172.21.15.170:0/2473248973 shutdown_connections 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 --2- 172.21.15.170:0/2473248973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0cccd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 --2- 172.21.15.170:0/2473248973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f631c0cb460 0x7f631c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 --2- 172.21.15.170:0/2473248973 >> v2:172.21.15.191:3300/0 conn(0x7f631c0cf7e0 0x7f631c070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 -- 172.21.15.170:0/2473248973 >> 172.21.15.170:0/2473248973 conn(0x7f631c0b1c40 msgr2=0x7f631c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.510+0000 7f6322975640 1 -- 172.21.15.170:0/2473248973 shutdown_connections 2024-09-21T02:14:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.514+0000 7f6322975640 1 -- 172.21.15.170:0/2473248973 wait complete. 2024-09-21T02:14:58.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.514+0000 7f6322975640 1 Processor -- start 2024-09-21T02:14:58.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.514+0000 7f6322975640 1 -- start start 2024-09-21T02:14:58.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f631c0cb460 0x7f631c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f631c0cf7e0 0x7f631c0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0ceb50 con 0x7f631c0cc8c0 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0ce9d0 con 0x7f631c0cf7e0 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f631c0cecd0 con 0x7f631c0cb460 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59804/0 (socket says 172.21.15.170:59804) 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 -- 172.21.15.170:0/2195833998 learned_addr learned my addr 172.21.15.170:0/2195833998 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631bfff640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f631c0cb460 0x7f631c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6320f12640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f631c0cf7e0 0x7f631c0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 -- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f631c0cb460 msgr2=0x7f631c14a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f631c0cb460 0x7f631c14a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 -- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f631c0cf7e0 msgr2=0x7f631c0c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f631c0cf7e0 0x7f631c0c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 -- 172.21.15.170:0/2195833998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c0c9500 con 0x7f631c0cc8c0 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f631b7fe640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0c8fc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f631000ca50 tx=0x7f631000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310007b90 con 0x7f631c0cc8c0 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6310007e20 con 0x7f631c0cc8c0 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000f690 con 0x7f631c0cc8c0 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c0c9980 con 0x7f631c0cc8c0 2024-09-21T02:14:58.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c0c9f40 con 0x7f631c0cc8c0 2024-09-21T02:14:58.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c0c96d0 con 0x7f631c0cc8c0 2024-09-21T02:14:58.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.518+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f6310018400 con 0x7f631c0cc8c0 2024-09-21T02:14:58.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.522+0000 7f6318ff9640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63080733e0 0x7f63080758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.522+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6310095260 con 0x7f631c0cc8c0 2024-09-21T02:14:58.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.522+0000 7f631bfff640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63080733e0 0x7f63080758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.522+0000 7f631bfff640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63080733e0 0x7f63080758f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6314005d10 tx=0x7f6314005c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:58.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.522+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6310026020 con 0x7f631c0cc8c0 2024-09-21T02:14:58.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.726+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f631c0cccd0 con 0x7f631c0cc8c0 2024-09-21T02:14:58.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.726+0000 7f6318ff9640 1 -- 172.21.15.170:0/2195833998 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5235 (secure 0 0 0) 0x7f6310014030 con 0x7f631c0cc8c0 2024-09-21T02:14:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:58.734 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":27,"btime":"2024-09-21T02:14:58:331015+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:14:58.734 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 27 2024-09-21T02:14:58.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63080733e0 msgr2=0x7f63080758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63080733e0 0x7f63080758f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6314005d10 tx=0x7f6314005c80 comp rx=0 tx=0).stop 2024-09-21T02:14:58.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 msgr2=0x7f631c0c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0c8fc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f631000ca50 tx=0x7f631000cf10 comp rx=0 tx=0).stop 2024-09-21T02:14:58.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 shutdown_connections 2024-09-21T02:14:58.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63080733e0 0x7f63080758f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f631c0cf7e0 0x7f631c0c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f631c0cc8c0 0x7f631c0c8fc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 --2- 172.21.15.170:0/2195833998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f631c0cb460 0x7f631c14a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 >> 172.21.15.170:0/2195833998 conn(0x7f631c0b1c40 msgr2=0x7f631c15b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:58.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 shutdown_connections 2024-09-21T02:14:58.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.730+0000 7f6322975640 1 -- 172.21.15.170:0/2195833998 wait complete. 2024-09-21T02:14:58.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:14:58.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.898+0000 7fe0d4b82640 1 -- 172.21.15.170:0/119452190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d00b3e30 msgr2=0x7fe0d00b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.898+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/119452190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d00b3e30 0x7fe0d00b4240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009900 tx=0x7fe0b801c8b0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.898+0000 7fe0d4b82640 1 -- 172.21.15.170:0/119452190 shutdown_connections 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.898+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/119452190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d00b3e30 0x7fe0d00b4240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.898+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/119452190 >> v2:172.21.15.191:3300/0 conn(0x7fe0d00b7ab0 0x7fe0d015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.898+0000 7fe0d4b82640 1 -- 172.21.15.170:0/119452190 >> 172.21.15.170:0/119452190 conn(0x7fe0d00cee30 msgr2=0x7fe0d00cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.902+0000 7fe0d4b82640 1 -- 172.21.15.170:0/119452190 shutdown_connections 2024-09-21T02:14:58.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.902+0000 7fe0d4b82640 1 -- 172.21.15.170:0/119452190 wait complete. 2024-09-21T02:14:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.902+0000 7fe0d4b82640 1 Processor -- start 2024-09-21T02:14:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.902+0000 7fe0d4b82640 1 -- start start 2024-09-21T02:14:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d0072900 0x7fe0d00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0d00b3e30 0x7fe0d00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 0x7fe0d015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d016a510 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d016a390 con 0x7fe0d00b3e30 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d016a690 con 0x7fe0d0072900 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 0x7fe0d015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 0x7fe0d015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59812/0 (socket says 172.21.15.170:59812) 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 -- 172.21.15.170:0/2052562953 learned_addr learned my addr 172.21.15.170:0/2052562953 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ce59c640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d0072900 0x7fe0d00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c7fff640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0d00b3e30 0x7fe0d00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 -- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d0072900 msgr2=0x7fe0d00caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d0072900 0x7fe0d00caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 -- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0d00b3e30 msgr2=0x7fe0d00cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0d00b3e30 0x7fe0d00cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 -- 172.21.15.170:0/2052562953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d00ca4e0 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ced9d640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 0x7fe0d015e6e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c6b0 tx=0x7fe0c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8003b80 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0c8003d20 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d0143110 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8005330 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d00caa60 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d00b4240 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fe0c80054d0 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c6ffd640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe0b40733b0 0x7fe0b40758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe0c80944b0 con 0x7fe0d00b7ab0 2024-09-21T02:14:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.906+0000 7fe0ce59c640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe0b40733b0 0x7fe0b40758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:58.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.910+0000 7fe0ce59c640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe0b40733b0 0x7fe0b40758c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009ad0 tx=0x7fe0c00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:58.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:58.914+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe0d00b4780 con 0x7fe0d00b7ab0 2024-09-21T02:14:59.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.090+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0d0154910 con 0x7fe0d00b7ab0 2024-09-21T02:14:59.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.090+0000 7fe0c6ffd640 1 -- 172.21.15.170:0/2052562953 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16717 (secure 0 0 0) 0x7fe0d0154910 con 0x7fe0d00b7ab0 2024-09-21T02:14:59.097 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:59.097 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":46,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:14:56.311925+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:52.409252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:53.674497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe0b40733b0 msgr2=0x7fe0b40758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe0b40733b0 0x7fe0b40758c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009ad0 tx=0x7fe0c00091e0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 msgr2=0x7fe0d015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 0x7fe0d015e6e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c6b0 tx=0x7fe0c800cb70 comp rx=0 tx=0).stop 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 shutdown_connections 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe0b40733b0 0x7fe0b40758c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0d00b7ab0 0x7fe0d015e6e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0d00b3e30 0x7fe0d00cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 --2- 172.21.15.170:0/2052562953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0d0072900 0x7fe0d00caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 >> 172.21.15.170:0/2052562953 conn(0x7fe0d00cee30 msgr2=0x7fe0d00b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 shutdown_connections 2024-09-21T02:14:59.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.094+0000 7fe0d4b82640 1 -- 172.21.15.170:0/2052562953 wait complete. 2024-09-21T02:14:59.119 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:14:59.119 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:59.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 -- 172.21.15.170:0/2931663795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b4540 msgr2=0x7f337415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 --2- 172.21.15.170:0/2931663795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b4540 0x7f337415c850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3374075080 tx=0x7f336801c800 comp rx=0 tx=0).stop 2024-09-21T02:14:59.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 -- 172.21.15.170:0/2931663795 shutdown_connections 2024-09-21T02:14:59.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 --2- 172.21.15.170:0/2931663795 >> v2:172.21.15.191:3300/0 conn(0x7f33740d0aa0 0x7f33740d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 --2- 172.21.15.170:0/2931663795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b4540 0x7f337415c850 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 -- 172.21.15.170:0/2931663795 >> 172.21.15.170:0/2931663795 conn(0x7f33740cee30 msgr2=0x7f33740cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 -- 172.21.15.170:0/2931663795 shutdown_connections 2024-09-21T02:14:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.262+0000 7f3378959640 1 -- 172.21.15.170:0/2931663795 wait complete. 2024-09-21T02:14:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 Processor -- start 2024-09-21T02:14:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 -- start start 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 0x7f3374151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33740d0aa0 0x7f337406f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f337406faf0 0x7f3374165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f337416a470 con 0x7f33740b7ab0 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f337416a2f0 con 0x7f33740d0aa0 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f337416a5f0 con 0x7f337406faf0 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 0x7f3374151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 0x7f3374151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59824/0 (socket says 172.21.15.170:59824) 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 -- 172.21.15.170:0/2837309038 learned_addr learned my addr 172.21.15.170:0/2837309038 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3371d9b640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33740d0aa0 0x7f337406f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3372d9d640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f337406faf0 0x7f3374165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 -- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f337406faf0 msgr2=0x7f3374165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f337406faf0 0x7f3374165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 -- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33740d0aa0 msgr2=0x7f337406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33740d0aa0 0x7f337406f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 -- 172.21.15.170:0/2837309038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337406ff00 con 0x7f33740b7ab0 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3372d9d640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f337406faf0 0x7f3374165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3371d9b640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33740d0aa0 0x7f337406f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f337259c640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 0x7f3374151470 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f336400c3c0 tx=0x7f336400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640068a0 con 0x7f33740b7ab0 2024-09-21T02:14:59.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3364006b50 con 0x7f33740b7ab0 2024-09-21T02:14:59.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741fcc20 con 0x7f33740b7ab0 2024-09-21T02:14:59.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364007020 con 0x7f33740b7ab0 2024-09-21T02:14:59.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.266+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741fd1a0 con 0x7f33740b7ab0 2024-09-21T02:14:59.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.270+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337415c850 con 0x7f33740b7ab0 2024-09-21T02:14:59.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.270+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f3364019020 con 0x7f33740b7ab0 2024-09-21T02:14:59.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.270+0000 7f3362ffd640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33540733b0 0x7f33540758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.270+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f33640943d0 con 0x7f33740b7ab0 2024-09-21T02:14:59.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.270+0000 7f3371d9b640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33540733b0 0x7f33540758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.270+0000 7f3371d9b640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33540733b0 0x7f33540758c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33680af000 tx=0x7f336801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.274+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f336405f140 con 0x7f33740b7ab0 2024-09-21T02:14:59.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.446+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33740c8fd0 con 0x7f33540733b0 2024-09-21T02:14:59.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3362ffd640 1 -- 172.21.15.170:0/2837309038 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241609 (secure 0 0 0) 0x7f33680f7fe0 con 0x7f33540733b0 2024-09-21T02:14:59.468 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:59.469 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:59.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33540733b0 msgr2=0x7f33540758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33540733b0 0x7f33540758c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33680af000 tx=0x7f336801b040 comp rx=0 tx=0).stop 2024-09-21T02:14:59.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 msgr2=0x7f3374151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 0x7f3374151470 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f336400c3c0 tx=0x7f336400c880 comp rx=0 tx=0).stop 2024-09-21T02:14:59.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 shutdown_connections 2024-09-21T02:14:59.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33540733b0 0x7f33540758c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f337406faf0 0x7f3374165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33740d0aa0 0x7f337406f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 --2- 172.21.15.170:0/2837309038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33740b7ab0 0x7f3374151470 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 >> 172.21.15.170:0/2837309038 conn(0x7f33740cee30 msgr2=0x7f33740b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 shutdown_connections 2024-09-21T02:14:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.462+0000 7f3378959640 1 -- 172.21.15.170:0/2837309038 wait complete. 2024-09-21T02:14:59.488 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-21T02:14:58.898665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246320,"kb_used_data":5076,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728400,"statfs":{"total":773094113280,"available":772841881600,"internally_reserved":0,"allocated":5197824,"data_stored":2754797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.614834"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329777+0000","last_change":"2024-09-21T02:14:55.329046+0000","last_active":"2024-09-21T02:14:56.329777+0000","last_peered":"2024-09-21T02:14:56.329777+0000","last_clean":"2024-09-21T02:14:56.329777+0000","last_became_active":"2024-09-21T02:14:55.328689+0000","last_became_peered":"2024-09-21T02:14:55.328689+0000","last_unstale":"2024-09-21T02:14:56.329777+0000","last_undegraded":"2024-09-21T02:14:56.329777+0000","last_fullsized":"2024-09-21T02:14:56.329777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:50.567278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329725+0000","last_change":"2024-09-21T02:14:55.333330+0000","last_active":"2024-09-21T02:14:56.329725+0000","last_peered":"2024-09-21T02:14:56.329725+0000","last_clean":"2024-09-21T02:14:56.329725+0000","last_became_active":"2024-09-21T02:14:55.332877+0000","last_became_peered":"2024-09-21T02:14:55.332877+0000","last_unstale":"2024-09-21T02:14:56.329725+0000","last_undegraded":"2024-09-21T02:14:56.329725+0000","last_fullsized":"2024-09-21T02:14:56.329725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:06.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.298186+0000","last_change":"2024-09-21T02:14:53.298186+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.298186+0000","last_undegraded":"2024-09-21T02:14:53.298186+0000","last_fullsized":"2024-09-21T02:14:53.298186+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290793+0000","last_change":"2024-09-21T02:14:53.290793+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290793+0000","last_undegraded":"2024-09-21T02:14:53.290793+0000","last_fullsized":"2024-09-21T02:14:53.290793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.295546+0000","last_change":"2024-09-21T02:14:53.295546+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.295546+0000","last_undegraded":"2024-09-21T02:14:53.295546+0000","last_fullsized":"2024-09-21T02:14:53.295546+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290515+0000","last_change":"2024-09-21T02:14:53.290515+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290515+0000","last_undegraded":"2024-09-21T02:14:53.290515+0000","last_fullsized":"2024-09-21T02:14:53.290515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292284+0000","last_change":"2024-09-21T02:14:53.292284+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292284+0000","last_undegraded":"2024-09-21T02:14:53.292284+0000","last_fullsized":"2024-09-21T02:14:53.292284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329912+0000","last_change":"2024-09-21T02:14:55.339871+0000","last_active":"2024-09-21T02:14:56.329912+0000","last_peered":"2024-09-21T02:14:56.329912+0000","last_clean":"2024-09-21T02:14:56.329912+0000","last_became_active":"2024-09-21T02:14:55.339582+0000","last_became_peered":"2024-09-21T02:14:55.339582+0000","last_unstale":"2024-09-21T02:14:56.329912+0000","last_undegraded":"2024-09-21T02:14:56.329912+0000","last_fullsized":"2024-09-21T02:14:56.329912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:12.830203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292570+0000","last_change":"2024-09-21T02:14:53.292570+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292570+0000","last_undegraded":"2024-09-21T02:14:53.292570+0000","last_fullsized":"2024-09-21T02:14:53.292570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328781+0000","last_change":"2024-09-21T02:14:55.336714+0000","last_active":"2024-09-21T02:14:56.328781+0000","last_peered":"2024-09-21T02:14:56.328781+0000","last_clean":"2024-09-21T02:14:56.328781+0000","last_became_active":"2024-09-21T02:14:55.336144+0000","last_became_peered":"2024-09-21T02:14:55.336144+0000","last_unstale":"2024-09-21T02:14:56.328781+0000","last_undegraded":"2024-09-21T02:14:56.328781+0000","last_fullsized":"2024-09-21T02:14:56.328781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:34.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293126+0000","last_change":"2024-09-21T02:14:53.293126+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293126+0000","last_undegraded":"2024-09-21T02:14:53.293126+0000","last_fullsized":"2024-09-21T02:14:53.293126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329124+0000","last_change":"2024-09-21T02:14:54.329228+0000","last_active":"2024-09-21T02:14:56.329124+0000","last_peered":"2024-09-21T02:14:56.329124+0000","last_clean":"2024-09-21T02:14:56.329124+0000","last_became_active":"2024-09-21T02:14:54.328430+0000","last_became_peered":"2024-09-21T02:14:54.328430+0000","last_unstale":"2024-09-21T02:14:56.329124+0000","last_undegraded":"2024-09-21T02:14:56.329124+0000","last_fullsized":"2024-09-21T02:14:56.329124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:46.117802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286366+0000","last_change":"2024-09-21T02:14:53.286366+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286366+0000","last_undegraded":"2024-09-21T02:14:53.286366+0000","last_fullsized":"2024-09-21T02:14:53.286366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329961+0000","last_change":"2024-09-21T02:14:54.329034+0000","last_active":"2024-09-21T02:14:56.329961+0000","last_peered":"2024-09-21T02:14:56.329961+0000","last_clean":"2024-09-21T02:14:56.329961+0000","last_became_active":"2024-09-21T02:14:54.327414+0000","last_became_peered":"2024-09-21T02:14:54.327414+0000","last_unstale":"2024-09-21T02:14:56.329961+0000","last_undegraded":"2024-09-21T02:14:56.329961+0000","last_fullsized":"2024-09-21T02:14:56.329961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:29.369340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290240+0000","last_change":"2024-09-21T02:14:53.290240+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290240+0000","last_undegraded":"2024-09-21T02:14:53.290240+0000","last_fullsized":"2024-09-21T02:14:53.290240+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330095+0000","last_change":"2024-09-21T02:14:54.319009+0000","last_active":"2024-09-21T02:14:56.330095+0000","last_peered":"2024-09-21T02:14:56.330095+0000","last_clean":"2024-09-21T02:14:56.330095+0000","last_became_active":"2024-09-21T02:14:54.317796+0000","last_became_peered":"2024-09-21T02:14:54.317796+0000","last_unstale":"2024-09-21T02:14:56.330095+0000","last_undegraded":"2024-09-21T02:14:56.330095+0000","last_fullsized":"2024-09-21T02:14:56.330095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:16.231157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.291629+0000","last_change":"2024-09-21T02:14:53.291629+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.291629+0000","last_undegraded":"2024-09-21T02:14:53.291629+0000","last_fullsized":"2024-09-21T02:14:53.291629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.294638+0000","last_change":"2024-09-21T02:14:53.294638+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.294638+0000","last_undegraded":"2024-09-21T02:14:53.294638+0000","last_fullsized":"2024-09-21T02:14:53.294638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.297512+0000","last_change":"2024-09-21T02:14:53.297512+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.297512+0000","last_undegraded":"2024-09-21T02:14:53.297512+0000","last_fullsized":"2024-09-21T02:14:53.297512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329396+0000","last_change":"2024-09-21T02:14:54.319185+0000","last_active":"2024-09-21T02:14:56.329396+0000","last_peered":"2024-09-21T02:14:56.329396+0000","last_clean":"2024-09-21T02:14:56.329396+0000","last_became_active":"2024-09-21T02:14:54.318313+0000","last_became_peered":"2024-09-21T02:14:54.318313+0000","last_unstale":"2024-09-21T02:14:56.329396+0000","last_undegraded":"2024-09-21T02:14:56.329396+0000","last_fullsized":"2024-09-21T02:14:56.329396+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:08.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.287294+0000","last_change":"2024-09-21T02:14:53.287294+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.287294+0000","last_undegraded":"2024-09-21T02:14:53.287294+0000","last_fullsized":"2024-09-21T02:14:53.287294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286404+0000","last_change":"2024-09-21T02:14:53.286404+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286404+0000","last_undegraded":"2024-09-21T02:14:53.286404+0000","last_fullsized":"2024-09-21T02:14:53.286404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286381+0000","last_change":"2024-09-21T02:14:53.286381+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286381+0000","last_undegraded":"2024-09-21T02:14:53.286381+0000","last_fullsized":"2024-09-21T02:14:53.286381+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.296742+0000","last_change":"2024-09-21T02:14:53.296742+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.296742+0000","last_undegraded":"2024-09-21T02:14:53.296742+0000","last_fullsized":"2024-09-21T02:14:53.296742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328752+0000","last_change":"2024-09-21T02:14:55.328993+0000","last_active":"2024-09-21T02:14:56.328752+0000","last_peered":"2024-09-21T02:14:56.328752+0000","last_clean":"2024-09-21T02:14:56.328752+0000","last_became_active":"2024-09-21T02:14:55.328635+0000","last_became_peered":"2024-09-21T02:14:55.328635+0000","last_unstale":"2024-09-21T02:14:56.328752+0000","last_undegraded":"2024-09-21T02:14:56.328752+0000","last_fullsized":"2024-09-21T02:14:56.328752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:09.392936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328949+0000","last_change":"2024-09-21T02:14:54.323086+0000","last_active":"2024-09-21T02:14:56.328949+0000","last_peered":"2024-09-21T02:14:56.328949+0000","last_clean":"2024-09-21T02:14:56.328949+0000","last_became_active":"2024-09-21T02:14:54.321950+0000","last_became_peered":"2024-09-21T02:14:54.321950+0000","last_unstale":"2024-09-21T02:14:56.328949+0000","last_undegraded":"2024-09-21T02:14:56.328949+0000","last_fullsized":"2024-09-21T02:14:56.328949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.939679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.294058+0000","last_change":"2024-09-21T02:14:53.294058+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.294058+0000","last_undegraded":"2024-09-21T02:14:53.294058+0000","last_fullsized":"2024-09-21T02:14:53.294058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328456+0000","last_change":"2024-09-21T02:14:54.328647+0000","last_active":"2024-09-21T02:14:56.328456+0000","last_peered":"2024-09-21T02:14:56.328456+0000","last_clean":"2024-09-21T02:14:56.328456+0000","last_became_active":"2024-09-21T02:14:54.327622+0000","last_became_peered":"2024-09-21T02:14:54.327622+0000","last_unstale":"2024-09-21T02:14:56.328456+0000","last_undegraded":"2024-09-21T02:14:56.328456+0000","last_fullsized":"2024-09-21T02:14:56.328456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:34.074515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290333+0000","last_change":"2024-09-21T02:14:53.290333+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290333+0000","last_undegraded":"2024-09-21T02:14:53.290333+0000","last_fullsized":"2024-09-21T02:14:53.290333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.287290+0000","last_change":"2024-09-21T02:14:53.287290+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.287290+0000","last_undegraded":"2024-09-21T02:14:53.287290+0000","last_fullsized":"2024-09-21T02:14:53.287290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330281+0000","last_change":"2024-09-21T02:14:55.332453+0000","last_active":"2024-09-21T02:14:56.330281+0000","last_peered":"2024-09-21T02:14:56.330281+0000","last_clean":"2024-09-21T02:14:56.330281+0000","last_became_active":"2024-09-21T02:14:55.332018+0000","last_became_peered":"2024-09-21T02:14:55.332018+0000","last_unstale":"2024-09-21T02:14:56.330281+0000","last_undegraded":"2024-09-21T02:14:56.330281+0000","last_fullsized":"2024-09-21T02:14:56.330281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.636901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293553+0000","last_change":"2024-09-21T02:14:53.293553+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293553+0000","last_undegraded":"2024-09-21T02:14:53.293553+0000","last_fullsized":"2024-09-21T02:14:53.293553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292552+0000","last_change":"2024-09-21T02:14:53.292552+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292552+0000","last_undegraded":"2024-09-21T02:14:53.292552+0000","last_fullsized":"2024-09-21T02:14:53.292552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328821+0000","last_change":"2024-09-21T02:14:55.334564+0000","last_active":"2024-09-21T02:14:56.328821+0000","last_peered":"2024-09-21T02:14:56.328821+0000","last_clean":"2024-09-21T02:14:56.328821+0000","last_became_active":"2024-09-21T02:14:55.333693+0000","last_became_peered":"2024-09-21T02:14:55.333693+0000","last_unstale":"2024-09-21T02:14:56.328821+0000","last_undegraded":"2024-09-21T02:14:56.328821+0000","last_fullsized":"2024-09-21T02:14:56.328821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.259862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293139+0000","last_change":"2024-09-21T02:14:53.293139+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293139+0000","last_undegraded":"2024-09-21T02:14:53.293139+0000","last_fullsized":"2024-09-21T02:14:53.293139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290426+0000","last_change":"2024-09-21T02:14:53.290426+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290426+0000","last_undegraded":"2024-09-21T02:14:53.290426+0000","last_fullsized":"2024-09-21T02:14:53.290426+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292598+0000","last_change":"2024-09-21T02:14:53.292598+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292598+0000","last_undegraded":"2024-09-21T02:14:53.292598+0000","last_fullsized":"2024-09-21T02:14:53.292598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.291639+0000","last_change":"2024-09-21T02:14:53.291639+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.291639+0000","last_undegraded":"2024-09-21T02:14:53.291639+0000","last_fullsized":"2024-09-21T02:14:53.291639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290357+0000","last_change":"2024-09-21T02:14:53.290357+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290357+0000","last_undegraded":"2024-09-21T02:14:53.290357+0000","last_fullsized":"2024-09-21T02:14:53.290357+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T02:14:53.286936+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:53.286936+0000","last_peered":"2024-09-21T02:14:53.286936+0000","last_clean":"2024-09-21T02:14:53.286936+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:53.286936+0000","last_undegraded":"2024-09-21T02:14:53.286936+0000","last_fullsized":"2024-09-21T02:14:53.286936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1089536,"data_stored":785221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1077248,"data_stored":779404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":552960,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:59.492 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:14:59.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0deadd5640 1 -- 172.21.15.170:0/803517044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dcc003a10 msgr2=0x7f0dcc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0deadd5640 1 --2- 172.21.15.170:0/803517044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dcc003a10 0x7f0dcc003ea0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009b60 tx=0x7f0ddc01d290 comp rx=0 tx=0).stop 2024-09-21T02:14:59.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0de9372640 1 -- 172.21.15.170:0/803517044 reap_dead start 2024-09-21T02:14:59.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0deadd5640 1 -- 172.21.15.170:0/803517044 shutdown_connections 2024-09-21T02:14:59.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0deadd5640 1 -- 172.21.15.170:0/803517044 >> 172.21.15.170:0/803517044 conn(0x7f0de40b1c40 msgr2=0x7f0de40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0deadd5640 1 -- 172.21.15.170:0/803517044 shutdown_connections 2024-09-21T02:14:59.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.678+0000 7f0deadd5640 1 -- 172.21.15.170:0/803517044 wait complete. 2024-09-21T02:14:59.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 Processor -- start 2024-09-21T02:14:59.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 -- start start 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc003a10 0x7f0de4073860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0de4160010 0x7f0de415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 0x7f0de40c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4167ad0 con 0x7f0de40c74c0 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4167950 con 0x7f0dcc003a10 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0de4167c50 con 0x7f0de4160010 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 0x7f0de40c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 0x7f0de40c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59838/0 (socket says 172.21.15.170:59838) 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 -- 172.21.15.170:0/2405982261 learned_addr learned my addr 172.21.15.170:0/2405982261 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de3fff640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0de4160010 0x7f0de415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de8b71640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc003a10 0x7f0de4073860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 -- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0de4160010 msgr2=0x7f0de415a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0de4160010 0x7f0de415a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 -- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc003a10 msgr2=0x7f0de4073860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc003a10 0x7f0de4073860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 -- 172.21.15.170:0/2405982261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de40c7e10 con 0x7f0de40c74c0 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de8b71640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc003a10 0x7f0de4073860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de9372640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 0x7f0de40c78d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0de40c7150 tx=0x7f0dd400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400cba0 con 0x7f0de40c74c0 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dd4018070 con 0x7f0de40c74c0 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de40741d0 con 0x7f0de40c74c0 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40116c0 con 0x7f0de40c74c0 2024-09-21T02:14:59.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.682+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de4074790 con 0x7f0de40c74c0 2024-09-21T02:14:59.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.686+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de40b3350 con 0x7f0de40c74c0 2024-09-21T02:14:59.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.686+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f0dd4011860 con 0x7f0de40c74c0 2024-09-21T02:14:59.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.686+0000 7f0de17fa640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0dd00733e0 0x7f0dd00758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:14:59.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.686+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f0dd4095500 con 0x7f0de40c74c0 2024-09-21T02:14:59.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.686+0000 7f0de8b71640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0dd00733e0 0x7f0dd00758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:14:59.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.686+0000 7f0de8b71640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0dd00733e0 0x7f0dd00758f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0ae000 tx=0x7f0ddc01a030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:14:59.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.690+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0de40c7e10 con 0x7f0de40c74c0 2024-09-21T02:14:59.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.858+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0de4166890 con 0x7f0dd00733e0 2024-09-21T02:14:59.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.870+0000 7f0de17fa640 1 -- 172.21.15.170:0/2405982261 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241609 (secure 0 0 0) 0x7f0de4166890 con 0x7f0dd00733e0 2024-09-21T02:14:59.878 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:14:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0dd00733e0 msgr2=0x7f0dd00758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0dd00733e0 0x7f0dd00758f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0ae000 tx=0x7f0ddc01a030 comp rx=0 tx=0).stop 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 msgr2=0x7f0de40c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 0x7f0de40c78d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0de40c7150 tx=0x7f0dd400ed10 comp rx=0 tx=0).stop 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 shutdown_connections 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0dd00733e0 0x7f0dd00758f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0de4160010 0x7f0de415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0de40c74c0 0x7f0de40c78d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 --2- 172.21.15.170:0/2405982261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc003a10 0x7f0de4073860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:14:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 >> 172.21.15.170:0/2405982261 conn(0x7f0de40b1c40 msgr2=0x7f0de415bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:14:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 shutdown_connections 2024-09-21T02:14:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:14:59.874+0000 7f0deadd5640 1 -- 172.21.15.170:0/2405982261 wait complete. 2024-09-21T02:14:59.902 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-21T02:14:58.898665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246320,"kb_used_data":5076,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728400,"statfs":{"total":773094113280,"available":772841881600,"internally_reserved":0,"allocated":5197824,"data_stored":2754797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.614834"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329777+0000","last_change":"2024-09-21T02:14:55.329046+0000","last_active":"2024-09-21T02:14:56.329777+0000","last_peered":"2024-09-21T02:14:56.329777+0000","last_clean":"2024-09-21T02:14:56.329777+0000","last_became_active":"2024-09-21T02:14:55.328689+0000","last_became_peered":"2024-09-21T02:14:55.328689+0000","last_unstale":"2024-09-21T02:14:56.329777+0000","last_undegraded":"2024-09-21T02:14:56.329777+0000","last_fullsized":"2024-09-21T02:14:56.329777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:50.567278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329725+0000","last_change":"2024-09-21T02:14:55.333330+0000","last_active":"2024-09-21T02:14:56.329725+0000","last_peered":"2024-09-21T02:14:56.329725+0000","last_clean":"2024-09-21T02:14:56.329725+0000","last_became_active":"2024-09-21T02:14:55.332877+0000","last_became_peered":"2024-09-21T02:14:55.332877+0000","last_unstale":"2024-09-21T02:14:56.329725+0000","last_undegraded":"2024-09-21T02:14:56.329725+0000","last_fullsized":"2024-09-21T02:14:56.329725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:06.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.298186+0000","last_change":"2024-09-21T02:14:53.298186+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.298186+0000","last_undegraded":"2024-09-21T02:14:53.298186+0000","last_fullsized":"2024-09-21T02:14:53.298186+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290793+0000","last_change":"2024-09-21T02:14:53.290793+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290793+0000","last_undegraded":"2024-09-21T02:14:53.290793+0000","last_fullsized":"2024-09-21T02:14:53.290793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.295546+0000","last_change":"2024-09-21T02:14:53.295546+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.295546+0000","last_undegraded":"2024-09-21T02:14:53.295546+0000","last_fullsized":"2024-09-21T02:14:53.295546+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290515+0000","last_change":"2024-09-21T02:14:53.290515+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290515+0000","last_undegraded":"2024-09-21T02:14:53.290515+0000","last_fullsized":"2024-09-21T02:14:53.290515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292284+0000","last_change":"2024-09-21T02:14:53.292284+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292284+0000","last_undegraded":"2024-09-21T02:14:53.292284+0000","last_fullsized":"2024-09-21T02:14:53.292284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329912+0000","last_change":"2024-09-21T02:14:55.339871+0000","last_active":"2024-09-21T02:14:56.329912+0000","last_peered":"2024-09-21T02:14:56.329912+0000","last_clean":"2024-09-21T02:14:56.329912+0000","last_became_active":"2024-09-21T02:14:55.339582+0000","last_became_peered":"2024-09-21T02:14:55.339582+0000","last_unstale":"2024-09-21T02:14:56.329912+0000","last_undegraded":"2024-09-21T02:14:56.329912+0000","last_fullsized":"2024-09-21T02:14:56.329912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:12.830203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292570+0000","last_change":"2024-09-21T02:14:53.292570+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292570+0000","last_undegraded":"2024-09-21T02:14:53.292570+0000","last_fullsized":"2024-09-21T02:14:53.292570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328781+0000","last_change":"2024-09-21T02:14:55.336714+0000","last_active":"2024-09-21T02:14:56.328781+0000","last_peered":"2024-09-21T02:14:56.328781+0000","last_clean":"2024-09-21T02:14:56.328781+0000","last_became_active":"2024-09-21T02:14:55.336144+0000","last_became_peered":"2024-09-21T02:14:55.336144+0000","last_unstale":"2024-09-21T02:14:56.328781+0000","last_undegraded":"2024-09-21T02:14:56.328781+0000","last_fullsized":"2024-09-21T02:14:56.328781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:34.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293126+0000","last_change":"2024-09-21T02:14:53.293126+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293126+0000","last_undegraded":"2024-09-21T02:14:53.293126+0000","last_fullsized":"2024-09-21T02:14:53.293126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329124+0000","last_change":"2024-09-21T02:14:54.329228+0000","last_active":"2024-09-21T02:14:56.329124+0000","last_peered":"2024-09-21T02:14:56.329124+0000","last_clean":"2024-09-21T02:14:56.329124+0000","last_became_active":"2024-09-21T02:14:54.328430+0000","last_became_peered":"2024-09-21T02:14:54.328430+0000","last_unstale":"2024-09-21T02:14:56.329124+0000","last_undegraded":"2024-09-21T02:14:56.329124+0000","last_fullsized":"2024-09-21T02:14:56.329124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:46.117802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286366+0000","last_change":"2024-09-21T02:14:53.286366+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286366+0000","last_undegraded":"2024-09-21T02:14:53.286366+0000","last_fullsized":"2024-09-21T02:14:53.286366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329961+0000","last_change":"2024-09-21T02:14:54.329034+0000","last_active":"2024-09-21T02:14:56.329961+0000","last_peered":"2024-09-21T02:14:56.329961+0000","last_clean":"2024-09-21T02:14:56.329961+0000","last_became_active":"2024-09-21T02:14:54.327414+0000","last_became_peered":"2024-09-21T02:14:54.327414+0000","last_unstale":"2024-09-21T02:14:56.329961+0000","last_undegraded":"2024-09-21T02:14:56.329961+0000","last_fullsized":"2024-09-21T02:14:56.329961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:29.369340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290240+0000","last_change":"2024-09-21T02:14:53.290240+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290240+0000","last_undegraded":"2024-09-21T02:14:53.290240+0000","last_fullsized":"2024-09-21T02:14:53.290240+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330095+0000","last_change":"2024-09-21T02:14:54.319009+0000","last_active":"2024-09-21T02:14:56.330095+0000","last_peered":"2024-09-21T02:14:56.330095+0000","last_clean":"2024-09-21T02:14:56.330095+0000","last_became_active":"2024-09-21T02:14:54.317796+0000","last_became_peered":"2024-09-21T02:14:54.317796+0000","last_unstale":"2024-09-21T02:14:56.330095+0000","last_undegraded":"2024-09-21T02:14:56.330095+0000","last_fullsized":"2024-09-21T02:14:56.330095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:16.231157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.291629+0000","last_change":"2024-09-21T02:14:53.291629+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.291629+0000","last_undegraded":"2024-09-21T02:14:53.291629+0000","last_fullsized":"2024-09-21T02:14:53.291629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.294638+0000","last_change":"2024-09-21T02:14:53.294638+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.294638+0000","last_undegraded":"2024-09-21T02:14:53.294638+0000","last_fullsized":"2024-09-21T02:14:53.294638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.297512+0000","last_change":"2024-09-21T02:14:53.297512+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.297512+0000","last_undegraded":"2024-09-21T02:14:53.297512+0000","last_fullsized":"2024-09-21T02:14:53.297512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329396+0000","last_change":"2024-09-21T02:14:54.319185+0000","last_active":"2024-09-21T02:14:56.329396+0000","last_peered":"2024-09-21T02:14:56.329396+0000","last_clean":"2024-09-21T02:14:56.329396+0000","last_became_active":"2024-09-21T02:14:54.318313+0000","last_became_peered":"2024-09-21T02:14:54.318313+0000","last_unstale":"2024-09-21T02:14:56.329396+0000","last_undegraded":"2024-09-21T02:14:56.329396+0000","last_fullsized":"2024-09-21T02:14:56.329396+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:08.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.287294+0000","last_change":"2024-09-21T02:14:53.287294+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.287294+0000","last_undegraded":"2024-09-21T02:14:53.287294+0000","last_fullsized":"2024-09-21T02:14:53.287294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286404+0000","last_change":"2024-09-21T02:14:53.286404+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286404+0000","last_undegraded":"2024-09-21T02:14:53.286404+0000","last_fullsized":"2024-09-21T02:14:53.286404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286381+0000","last_change":"2024-09-21T02:14:53.286381+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286381+0000","last_undegraded":"2024-09-21T02:14:53.286381+0000","last_fullsized":"2024-09-21T02:14:53.286381+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.296742+0000","last_change":"2024-09-21T02:14:53.296742+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.296742+0000","last_undegraded":"2024-09-21T02:14:53.296742+0000","last_fullsized":"2024-09-21T02:14:53.296742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328752+0000","last_change":"2024-09-21T02:14:55.328993+0000","last_active":"2024-09-21T02:14:56.328752+0000","last_peered":"2024-09-21T02:14:56.328752+0000","last_clean":"2024-09-21T02:14:56.328752+0000","last_became_active":"2024-09-21T02:14:55.328635+0000","last_became_peered":"2024-09-21T02:14:55.328635+0000","last_unstale":"2024-09-21T02:14:56.328752+0000","last_undegraded":"2024-09-21T02:14:56.328752+0000","last_fullsized":"2024-09-21T02:14:56.328752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:09.392936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328949+0000","last_change":"2024-09-21T02:14:54.323086+0000","last_active":"2024-09-21T02:14:56.328949+0000","last_peered":"2024-09-21T02:14:56.328949+0000","last_clean":"2024-09-21T02:14:56.328949+0000","last_became_active":"2024-09-21T02:14:54.321950+0000","last_became_peered":"2024-09-21T02:14:54.321950+0000","last_unstale":"2024-09-21T02:14:56.328949+0000","last_undegraded":"2024-09-21T02:14:56.328949+0000","last_fullsized":"2024-09-21T02:14:56.328949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.939679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.294058+0000","last_change":"2024-09-21T02:14:53.294058+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.294058+0000","last_undegraded":"2024-09-21T02:14:53.294058+0000","last_fullsized":"2024-09-21T02:14:53.294058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328456+0000","last_change":"2024-09-21T02:14:54.328647+0000","last_active":"2024-09-21T02:14:56.328456+0000","last_peered":"2024-09-21T02:14:56.328456+0000","last_clean":"2024-09-21T02:14:56.328456+0000","last_became_active":"2024-09-21T02:14:54.327622+0000","last_became_peered":"2024-09-21T02:14:54.327622+0000","last_unstale":"2024-09-21T02:14:56.328456+0000","last_undegraded":"2024-09-21T02:14:56.328456+0000","last_fullsized":"2024-09-21T02:14:56.328456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:34.074515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290333+0000","last_change":"2024-09-21T02:14:53.290333+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290333+0000","last_undegraded":"2024-09-21T02:14:53.290333+0000","last_fullsized":"2024-09-21T02:14:53.290333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.287290+0000","last_change":"2024-09-21T02:14:53.287290+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.287290+0000","last_undegraded":"2024-09-21T02:14:53.287290+0000","last_fullsized":"2024-09-21T02:14:53.287290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330281+0000","last_change":"2024-09-21T02:14:55.332453+0000","last_active":"2024-09-21T02:14:56.330281+0000","last_peered":"2024-09-21T02:14:56.330281+0000","last_clean":"2024-09-21T02:14:56.330281+0000","last_became_active":"2024-09-21T02:14:55.332018+0000","last_became_peered":"2024-09-21T02:14:55.332018+0000","last_unstale":"2024-09-21T02:14:56.330281+0000","last_undegraded":"2024-09-21T02:14:56.330281+0000","last_fullsized":"2024-09-21T02:14:56.330281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.636901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293553+0000","last_change":"2024-09-21T02:14:53.293553+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293553+0000","last_undegraded":"2024-09-21T02:14:53.293553+0000","last_fullsized":"2024-09-21T02:14:53.293553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292552+0000","last_change":"2024-09-21T02:14:53.292552+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292552+0000","last_undegraded":"2024-09-21T02:14:53.292552+0000","last_fullsized":"2024-09-21T02:14:53.292552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328821+0000","last_change":"2024-09-21T02:14:55.334564+0000","last_active":"2024-09-21T02:14:56.328821+0000","last_peered":"2024-09-21T02:14:56.328821+0000","last_clean":"2024-09-21T02:14:56.328821+0000","last_became_active":"2024-09-21T02:14:55.333693+0000","last_became_peered":"2024-09-21T02:14:55.333693+0000","last_unstale":"2024-09-21T02:14:56.328821+0000","last_undegraded":"2024-09-21T02:14:56.328821+0000","last_fullsized":"2024-09-21T02:14:56.328821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.259862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293139+0000","last_change":"2024-09-21T02:14:53.293139+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293139+0000","last_undegraded":"2024-09-21T02:14:53.293139+0000","last_fullsized":"2024-09-21T02:14:53.293139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290426+0000","last_change":"2024-09-21T02:14:53.290426+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290426+0000","last_undegraded":"2024-09-21T02:14:53.290426+0000","last_fullsized":"2024-09-21T02:14:53.290426+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292598+0000","last_change":"2024-09-21T02:14:53.292598+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292598+0000","last_undegraded":"2024-09-21T02:14:53.292598+0000","last_fullsized":"2024-09-21T02:14:53.292598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.291639+0000","last_change":"2024-09-21T02:14:53.291639+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.291639+0000","last_undegraded":"2024-09-21T02:14:53.291639+0000","last_fullsized":"2024-09-21T02:14:53.291639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290357+0000","last_change":"2024-09-21T02:14:53.290357+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290357+0000","last_undegraded":"2024-09-21T02:14:53.290357+0000","last_fullsized":"2024-09-21T02:14:53.290357+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T02:14:53.286936+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:53.286936+0000","last_peered":"2024-09-21T02:14:53.286936+0000","last_clean":"2024-09-21T02:14:53.286936+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:53.286936+0000","last_undegraded":"2024-09-21T02:14:53.286936+0000","last_fullsized":"2024-09-21T02:14:53.286936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1089536,"data_stored":785221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1077248,"data_stored":779404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":552960,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:14:59.905 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 -- 172.21.15.170:0/301033040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d100cadf0 msgr2=0x7f9d100cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 --2- 172.21.15.170:0/301033040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d100cadf0 0x7f9d100cb200 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d100b2d00 tx=0x7f9d0801cb50 comp rx=0 tx=0).stop 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 -- 172.21.15.170:0/301033040 shutdown_connections 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 --2- 172.21.15.170:0/301033040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d100cadf0 0x7f9d100cb200 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 --2- 172.21.15.170:0/301033040 >> v2:172.21.15.191:3300/0 conn(0x7f9d1015f990 0x7f9d10070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 -- 172.21.15.170:0/301033040 >> 172.21.15.170:0/301033040 conn(0x7f9d100b1c40 msgr2=0x7f9d100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 -- 172.21.15.170:0/301033040 shutdown_connections 2024-09-21T02:15:00.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.098+0000 7f9d171ec640 1 -- 172.21.15.170:0/301033040 wait complete. 2024-09-21T02:15:00.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 Processor -- start 2024-09-21T02:15:00.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- start start 2024-09-21T02:15:00.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 0x7f9d1009ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d100cadf0 0x7f9d1015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d1015f990 0x7f9d1015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10165fe0 con 0x7f9d10070570 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100cede0 con 0x7f9d100cadf0 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9d10166160 con 0x7f9d1015f990 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 0x7f9d1009ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 0x7f9d1009ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59854/0 (socket says 172.21.15.170:59854) 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 -- 172.21.15.170:0/3584879674 learned_addr learned my addr 172.21.15.170:0/3584879674 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d0ffff640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d100cadf0 0x7f9d1015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d15789640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d1015f990 0x7f9d1015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 -- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d1015f990 msgr2=0x7f9d1015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d1015f990 0x7f9d1015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 -- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d100cadf0 msgr2=0x7f9d1015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d100cadf0 0x7f9d1015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 -- 172.21.15.170:0/3584879674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1015e9a0 con 0x7f9d10070570 2024-09-21T02:15:00.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d0ffff640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d100cadf0 0x7f9d1015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d15789640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d1015f990 0x7f9d1015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d14f88640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 0x7f9d1009ac50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000d750 tx=0x7f9d0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00017070 con 0x7f9d10070570 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d00009d20 con 0x7f9d10070570 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000de60 con 0x7f9d10070570 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1015ec80 con 0x7f9d10070570 2024-09-21T02:15:00.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1009b870 con 0x7f9d10070570 2024-09-21T02:15:00.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.102+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d100cb200 con 0x7f9d10070570 2024-09-21T02:15:00.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.106+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f9d00004280 con 0x7f9d10070570 2024-09-21T02:15:00.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.106+0000 7f9d0d7fa640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9cfc0733e0 0x7f9cfc0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.106+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f9d00094b40 con 0x7f9d10070570 2024-09-21T02:15:00.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.106+0000 7f9d0ffff640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9cfc0733e0 0x7f9cfc0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.106+0000 7f9d0ffff640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9cfc0733e0 0x7f9cfc0758f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d04001400 tx=0x7f9d04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.110+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9d00020020 con 0x7f9d10070570 2024-09-21T02:15:00.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.282+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d100ce2a0 con 0x7f9cfc0733e0 2024-09-21T02:15:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d0d7fa640 1 -- 172.21.15.170:0/3584879674 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241609 (secure 0 0 0) 0x7f9d100ce2a0 con 0x7f9cfc0733e0 2024-09-21T02:15:00.297 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:00.299 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:15:00.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9cfc0733e0 msgr2=0x7f9cfc0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9cfc0733e0 0x7f9cfc0758f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d04001400 tx=0x7f9d04009040 comp rx=0 tx=0).stop 2024-09-21T02:15:00.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 msgr2=0x7f9d1009ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 0x7f9d1009ac50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000d750 tx=0x7f9d0000dc10 comp rx=0 tx=0).stop 2024-09-21T02:15:00.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 shutdown_connections 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9cfc0733e0 0x7f9cfc0758f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d1015f990 0x7f9d1015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d100cadf0 0x7f9d1015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 --2- 172.21.15.170:0/3584879674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d10070570 0x7f9d1009ac50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 >> 172.21.15.170:0/3584879674 conn(0x7f9d100b1c40 msgr2=0x7f9d100c9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 shutdown_connections 2024-09-21T02:15:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.294+0000 7f9d171ec640 1 -- 172.21.15.170:0/3584879674 wait complete. 2024-09-21T02:15:00.318 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-21T02:14:58.898665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246320,"kb_used_data":5076,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728400,"statfs":{"total":773094113280,"available":772841881600,"internally_reserved":0,"allocated":5197824,"data_stored":2754797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.614834"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329777+0000","last_change":"2024-09-21T02:14:55.329046+0000","last_active":"2024-09-21T02:14:56.329777+0000","last_peered":"2024-09-21T02:14:56.329777+0000","last_clean":"2024-09-21T02:14:56.329777+0000","last_became_active":"2024-09-21T02:14:55.328689+0000","last_became_peered":"2024-09-21T02:14:55.328689+0000","last_unstale":"2024-09-21T02:14:56.329777+0000","last_undegraded":"2024-09-21T02:14:56.329777+0000","last_fullsized":"2024-09-21T02:14:56.329777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:50.567278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329725+0000","last_change":"2024-09-21T02:14:55.333330+0000","last_active":"2024-09-21T02:14:56.329725+0000","last_peered":"2024-09-21T02:14:56.329725+0000","last_clean":"2024-09-21T02:14:56.329725+0000","last_became_active":"2024-09-21T02:14:55.332877+0000","last_became_peered":"2024-09-21T02:14:55.332877+0000","last_unstale":"2024-09-21T02:14:56.329725+0000","last_undegraded":"2024-09-21T02:14:56.329725+0000","last_fullsized":"2024-09-21T02:14:56.329725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:06.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.298186+0000","last_change":"2024-09-21T02:14:53.298186+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.298186+0000","last_undegraded":"2024-09-21T02:14:53.298186+0000","last_fullsized":"2024-09-21T02:14:53.298186+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290793+0000","last_change":"2024-09-21T02:14:53.290793+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290793+0000","last_undegraded":"2024-09-21T02:14:53.290793+0000","last_fullsized":"2024-09-21T02:14:53.290793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.295546+0000","last_change":"2024-09-21T02:14:53.295546+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.295546+0000","last_undegraded":"2024-09-21T02:14:53.295546+0000","last_fullsized":"2024-09-21T02:14:53.295546+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290515+0000","last_change":"2024-09-21T02:14:53.290515+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290515+0000","last_undegraded":"2024-09-21T02:14:53.290515+0000","last_fullsized":"2024-09-21T02:14:53.290515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292284+0000","last_change":"2024-09-21T02:14:53.292284+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292284+0000","last_undegraded":"2024-09-21T02:14:53.292284+0000","last_fullsized":"2024-09-21T02:14:53.292284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329912+0000","last_change":"2024-09-21T02:14:55.339871+0000","last_active":"2024-09-21T02:14:56.329912+0000","last_peered":"2024-09-21T02:14:56.329912+0000","last_clean":"2024-09-21T02:14:56.329912+0000","last_became_active":"2024-09-21T02:14:55.339582+0000","last_became_peered":"2024-09-21T02:14:55.339582+0000","last_unstale":"2024-09-21T02:14:56.329912+0000","last_undegraded":"2024-09-21T02:14:56.329912+0000","last_fullsized":"2024-09-21T02:14:56.329912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:12.830203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292570+0000","last_change":"2024-09-21T02:14:53.292570+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292570+0000","last_undegraded":"2024-09-21T02:14:53.292570+0000","last_fullsized":"2024-09-21T02:14:53.292570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328781+0000","last_change":"2024-09-21T02:14:55.336714+0000","last_active":"2024-09-21T02:14:56.328781+0000","last_peered":"2024-09-21T02:14:56.328781+0000","last_clean":"2024-09-21T02:14:56.328781+0000","last_became_active":"2024-09-21T02:14:55.336144+0000","last_became_peered":"2024-09-21T02:14:55.336144+0000","last_unstale":"2024-09-21T02:14:56.328781+0000","last_undegraded":"2024-09-21T02:14:56.328781+0000","last_fullsized":"2024-09-21T02:14:56.328781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:34.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293126+0000","last_change":"2024-09-21T02:14:53.293126+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293126+0000","last_undegraded":"2024-09-21T02:14:53.293126+0000","last_fullsized":"2024-09-21T02:14:53.293126+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329124+0000","last_change":"2024-09-21T02:14:54.329228+0000","last_active":"2024-09-21T02:14:56.329124+0000","last_peered":"2024-09-21T02:14:56.329124+0000","last_clean":"2024-09-21T02:14:56.329124+0000","last_became_active":"2024-09-21T02:14:54.328430+0000","last_became_peered":"2024-09-21T02:14:54.328430+0000","last_unstale":"2024-09-21T02:14:56.329124+0000","last_undegraded":"2024-09-21T02:14:56.329124+0000","last_fullsized":"2024-09-21T02:14:56.329124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:46.117802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286366+0000","last_change":"2024-09-21T02:14:53.286366+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286366+0000","last_undegraded":"2024-09-21T02:14:53.286366+0000","last_fullsized":"2024-09-21T02:14:53.286366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329961+0000","last_change":"2024-09-21T02:14:54.329034+0000","last_active":"2024-09-21T02:14:56.329961+0000","last_peered":"2024-09-21T02:14:56.329961+0000","last_clean":"2024-09-21T02:14:56.329961+0000","last_became_active":"2024-09-21T02:14:54.327414+0000","last_became_peered":"2024-09-21T02:14:54.327414+0000","last_unstale":"2024-09-21T02:14:56.329961+0000","last_undegraded":"2024-09-21T02:14:56.329961+0000","last_fullsized":"2024-09-21T02:14:56.329961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:29.369340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290240+0000","last_change":"2024-09-21T02:14:53.290240+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290240+0000","last_undegraded":"2024-09-21T02:14:53.290240+0000","last_fullsized":"2024-09-21T02:14:53.290240+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330095+0000","last_change":"2024-09-21T02:14:54.319009+0000","last_active":"2024-09-21T02:14:56.330095+0000","last_peered":"2024-09-21T02:14:56.330095+0000","last_clean":"2024-09-21T02:14:56.330095+0000","last_became_active":"2024-09-21T02:14:54.317796+0000","last_became_peered":"2024-09-21T02:14:54.317796+0000","last_unstale":"2024-09-21T02:14:56.330095+0000","last_undegraded":"2024-09-21T02:14:56.330095+0000","last_fullsized":"2024-09-21T02:14:56.330095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:16.231157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.291629+0000","last_change":"2024-09-21T02:14:53.291629+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.291629+0000","last_undegraded":"2024-09-21T02:14:53.291629+0000","last_fullsized":"2024-09-21T02:14:53.291629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.294638+0000","last_change":"2024-09-21T02:14:53.294638+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.294638+0000","last_undegraded":"2024-09-21T02:14:53.294638+0000","last_fullsized":"2024-09-21T02:14:53.294638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.297512+0000","last_change":"2024-09-21T02:14:53.297512+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.297512+0000","last_undegraded":"2024-09-21T02:14:53.297512+0000","last_fullsized":"2024-09-21T02:14:53.297512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329396+0000","last_change":"2024-09-21T02:14:54.319185+0000","last_active":"2024-09-21T02:14:56.329396+0000","last_peered":"2024-09-21T02:14:56.329396+0000","last_clean":"2024-09-21T02:14:56.329396+0000","last_became_active":"2024-09-21T02:14:54.318313+0000","last_became_peered":"2024-09-21T02:14:54.318313+0000","last_unstale":"2024-09-21T02:14:56.329396+0000","last_undegraded":"2024-09-21T02:14:56.329396+0000","last_fullsized":"2024-09-21T02:14:56.329396+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:08.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.287294+0000","last_change":"2024-09-21T02:14:53.287294+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.287294+0000","last_undegraded":"2024-09-21T02:14:53.287294+0000","last_fullsized":"2024-09-21T02:14:53.287294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286404+0000","last_change":"2024-09-21T02:14:53.286404+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286404+0000","last_undegraded":"2024-09-21T02:14:53.286404+0000","last_fullsized":"2024-09-21T02:14:53.286404+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.286381+0000","last_change":"2024-09-21T02:14:53.286381+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.286381+0000","last_undegraded":"2024-09-21T02:14:53.286381+0000","last_fullsized":"2024-09-21T02:14:53.286381+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.296742+0000","last_change":"2024-09-21T02:14:53.296742+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.296742+0000","last_undegraded":"2024-09-21T02:14:53.296742+0000","last_fullsized":"2024-09-21T02:14:53.296742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328752+0000","last_change":"2024-09-21T02:14:55.328993+0000","last_active":"2024-09-21T02:14:56.328752+0000","last_peered":"2024-09-21T02:14:56.328752+0000","last_clean":"2024-09-21T02:14:56.328752+0000","last_became_active":"2024-09-21T02:14:55.328635+0000","last_became_peered":"2024-09-21T02:14:55.328635+0000","last_unstale":"2024-09-21T02:14:56.328752+0000","last_undegraded":"2024-09-21T02:14:56.328752+0000","last_fullsized":"2024-09-21T02:14:56.328752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:09.392936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328949+0000","last_change":"2024-09-21T02:14:54.323086+0000","last_active":"2024-09-21T02:14:56.328949+0000","last_peered":"2024-09-21T02:14:56.328949+0000","last_clean":"2024-09-21T02:14:56.328949+0000","last_became_active":"2024-09-21T02:14:54.321950+0000","last_became_peered":"2024-09-21T02:14:54.321950+0000","last_unstale":"2024-09-21T02:14:56.328949+0000","last_undegraded":"2024-09-21T02:14:56.328949+0000","last_fullsized":"2024-09-21T02:14:56.328949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.939679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.294058+0000","last_change":"2024-09-21T02:14:53.294058+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.294058+0000","last_undegraded":"2024-09-21T02:14:53.294058+0000","last_fullsized":"2024-09-21T02:14:53.294058+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328456+0000","last_change":"2024-09-21T02:14:54.328647+0000","last_active":"2024-09-21T02:14:56.328456+0000","last_peered":"2024-09-21T02:14:56.328456+0000","last_clean":"2024-09-21T02:14:56.328456+0000","last_became_active":"2024-09-21T02:14:54.327622+0000","last_became_peered":"2024-09-21T02:14:54.327622+0000","last_unstale":"2024-09-21T02:14:56.328456+0000","last_undegraded":"2024-09-21T02:14:56.328456+0000","last_fullsized":"2024-09-21T02:14:56.328456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:34.074515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290333+0000","last_change":"2024-09-21T02:14:53.290333+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290333+0000","last_undegraded":"2024-09-21T02:14:53.290333+0000","last_fullsized":"2024-09-21T02:14:53.290333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.287290+0000","last_change":"2024-09-21T02:14:53.287290+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.287290+0000","last_undegraded":"2024-09-21T02:14:53.287290+0000","last_fullsized":"2024-09-21T02:14:53.287290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330281+0000","last_change":"2024-09-21T02:14:55.332453+0000","last_active":"2024-09-21T02:14:56.330281+0000","last_peered":"2024-09-21T02:14:56.330281+0000","last_clean":"2024-09-21T02:14:56.330281+0000","last_became_active":"2024-09-21T02:14:55.332018+0000","last_became_peered":"2024-09-21T02:14:55.332018+0000","last_unstale":"2024-09-21T02:14:56.330281+0000","last_undegraded":"2024-09-21T02:14:56.330281+0000","last_fullsized":"2024-09-21T02:14:56.330281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.636901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293553+0000","last_change":"2024-09-21T02:14:53.293553+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293553+0000","last_undegraded":"2024-09-21T02:14:53.293553+0000","last_fullsized":"2024-09-21T02:14:53.293553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292552+0000","last_change":"2024-09-21T02:14:53.292552+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292552+0000","last_undegraded":"2024-09-21T02:14:53.292552+0000","last_fullsized":"2024-09-21T02:14:53.292552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328821+0000","last_change":"2024-09-21T02:14:55.334564+0000","last_active":"2024-09-21T02:14:56.328821+0000","last_peered":"2024-09-21T02:14:56.328821+0000","last_clean":"2024-09-21T02:14:56.328821+0000","last_became_active":"2024-09-21T02:14:55.333693+0000","last_became_peered":"2024-09-21T02:14:55.333693+0000","last_unstale":"2024-09-21T02:14:56.328821+0000","last_undegraded":"2024-09-21T02:14:56.328821+0000","last_fullsized":"2024-09-21T02:14:56.328821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.259862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.293139+0000","last_change":"2024-09-21T02:14:53.293139+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.293139+0000","last_undegraded":"2024-09-21T02:14:53.293139+0000","last_fullsized":"2024-09-21T02:14:53.293139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290426+0000","last_change":"2024-09-21T02:14:53.290426+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290426+0000","last_undegraded":"2024-09-21T02:14:53.290426+0000","last_fullsized":"2024-09-21T02:14:53.290426+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.292598+0000","last_change":"2024-09-21T02:14:53.292598+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.292598+0000","last_undegraded":"2024-09-21T02:14:53.292598+0000","last_fullsized":"2024-09-21T02:14:53.292598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.291639+0000","last_change":"2024-09-21T02:14:53.291639+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.291639+0000","last_undegraded":"2024-09-21T02:14:53.291639+0000","last_fullsized":"2024-09-21T02:14:53.291639+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:14:54.282109+0000","last_change":"2024-09-21T02:14:54.282109+0000","last_active":"2024-09-21T02:14:54.282109+0000","last_peered":"2024-09-21T02:14:54.282109+0000","last_clean":"2024-09-21T02:14:54.282109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:54.282109+0000","last_undegraded":"2024-09-21T02:14:54.282109+0000","last_fullsized":"2024-09-21T02:14:54.282109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-21T02:14:53.290357+0000","last_change":"2024-09-21T02:14:53.290357+0000","last_active":"2024-09-21T02:14:53.271564+0000","last_peered":"2024-09-21T02:14:53.271564+0000","last_clean":"2024-09-21T02:14:53.271564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:14:53.290357+0000","last_undegraded":"2024-09-21T02:14:53.290357+0000","last_fullsized":"2024-09-21T02:14:53.290357+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":146,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T02:14:53.286936+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:53.286936+0000","last_peered":"2024-09-21T02:14:53.286936+0000","last_clean":"2024-09-21T02:14:53.286936+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:53.286936+0000","last_undegraded":"2024-09-21T02:14:53.286936+0000","last_fullsized":"2024-09-21T02:14:53.286936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1089536,"data_stored":785221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1077248,"data_stored":779404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":495616,"data_stored":194853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":552960,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:15:00.323 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3f is not active+clean 2024-09-21T02:15:00.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.323 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3e is not active+clean 2024-09-21T02:15:00.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3d is not active+clean 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3c is not active+clean 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3b is not active+clean 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:PG 7.39 is not active+clean 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:PG 7.38 is not active+clean 2024-09-21T02:15:00.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:PG 7.37 is not active+clean 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:PG 7.35 is not active+clean 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:PG 7.34 is not active+clean 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:PG 7.32 is not active+clean 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:PG 7.31 is not active+clean 2024-09-21T02:15:00.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:PG 7.30 is not active+clean 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2f is not active+clean 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2e is not active+clean 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2d is not active+clean 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2b is not active+clean 2024-09-21T02:15:00.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2a is not active+clean 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:PG 7.29 is not active+clean 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:PG 7.28 is not active+clean 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:PG 7.27 is not active+clean 2024-09-21T02:15:00.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.328 INFO:tasks.ceph.ceph_manager.ceph:PG 7.26 is not active+clean 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:PG 7.25 is not active+clean 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:PG 7.24 is not active+clean 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:PG 7.23 is not active+clean 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:PG 7.22 is not active+clean 2024-09-21T02:15:00.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:PG 7.20 is not active+clean 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.298186+0000', 'last_change': '2024-09-21T02:14:53.298186+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.298186+0000', 'last_undegraded': '2024-09-21T02:14:53.298186+0000', 'last_fullsized': '2024-09-21T02:14:53.298186+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-21T02:15:00.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.290793+0000', 'last_change': '2024-09-21T02:14:53.290793+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.290793+0000', 'last_undegraded': '2024-09-21T02:14:53.290793+0000', 'last_fullsized': '2024-09-21T02:14:53.290793+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1c is not active+clean 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.295546+0000', 'last_change': '2024-09-21T02:14:53.295546+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.295546+0000', 'last_undegraded': '2024-09-21T02:14:53.295546+0000', 'last_fullsized': '2024-09-21T02:14:53.295546+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1d is not active+clean 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.290515+0000', 'last_change': '2024-09-21T02:14:53.290515+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.290515+0000', 'last_undegraded': '2024-09-21T02:14:53.290515+0000', 'last_fullsized': '2024-09-21T02:14:53.290515+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.292284+0000', 'last_change': '2024-09-21T02:14:53.292284+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.292284+0000', 'last_undegraded': '2024-09-21T02:14:53.292284+0000', 'last_fullsized': '2024-09-21T02:14:53.292284+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.292570+0000', 'last_change': '2024-09-21T02:14:53.292570+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.292570+0000', 'last_undegraded': '2024-09-21T02:14:53.292570+0000', 'last_fullsized': '2024-09-21T02:14:53.292570+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.293126+0000', 'last_change': '2024-09-21T02:14:53.293126+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.293126+0000', 'last_undegraded': '2024-09-21T02:14:53.293126+0000', 'last_fullsized': '2024-09-21T02:14:53.293126+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-21T02:15:00.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.286366+0000', 'last_change': '2024-09-21T02:14:53.286366+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.286366+0000', 'last_undegraded': '2024-09-21T02:14:53.286366+0000', 'last_fullsized': '2024-09-21T02:14:53.286366+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.290240+0000', 'last_change': '2024-09-21T02:14:53.290240+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.290240+0000', 'last_undegraded': '2024-09-21T02:14:53.290240+0000', 'last_fullsized': '2024-09-21T02:14:53.290240+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-21T02:15:00.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.291629+0000', 'last_change': '2024-09-21T02:14:53.291629+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.291629+0000', 'last_undegraded': '2024-09-21T02:14:53.291629+0000', 'last_fullsized': '2024-09-21T02:14:53.291629+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.294638+0000', 'last_change': '2024-09-21T02:14:53.294638+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.294638+0000', 'last_undegraded': '2024-09-21T02:14:53.294638+0000', 'last_fullsized': '2024-09-21T02:14:53.294638+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.297512+0000', 'last_change': '2024-09-21T02:14:53.297512+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.297512+0000', 'last_undegraded': '2024-09-21T02:14:53.297512+0000', 'last_fullsized': '2024-09-21T02:14:53.297512+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-09-21T02:15:00.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.287294+0000', 'last_change': '2024-09-21T02:14:53.287294+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.287294+0000', 'last_undegraded': '2024-09-21T02:14:53.287294+0000', 'last_fullsized': '2024-09-21T02:14:53.287294+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.286404+0000', 'last_change': '2024-09-21T02:14:53.286404+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.286404+0000', 'last_undegraded': '2024-09-21T02:14:53.286404+0000', 'last_fullsized': '2024-09-21T02:14:53.286404+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-21T02:15:00.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.286381+0000', 'last_change': '2024-09-21T02:14:53.286381+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.286381+0000', 'last_undegraded': '2024-09-21T02:14:53.286381+0000', 'last_fullsized': '2024-09-21T02:14:53.286381+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.296742+0000', 'last_change': '2024-09-21T02:14:53.296742+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.296742+0000', 'last_undegraded': '2024-09-21T02:14:53.296742+0000', 'last_fullsized': '2024-09-21T02:14:53.296742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.294058+0000', 'last_change': '2024-09-21T02:14:53.294058+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.294058+0000', 'last_undegraded': '2024-09-21T02:14:53.294058+0000', 'last_fullsized': '2024-09-21T02:14:53.294058+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.336 INFO:tasks.ceph.ceph_manager.ceph:PG 7.9 is not active+clean 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.290333+0000', 'last_change': '2024-09-21T02:14:53.290333+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.290333+0000', 'last_undegraded': '2024-09-21T02:14:53.290333+0000', 'last_fullsized': '2024-09-21T02:14:53.290333+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.287290+0000', 'last_change': '2024-09-21T02:14:53.287290+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.287290+0000', 'last_undegraded': '2024-09-21T02:14:53.287290+0000', 'last_fullsized': '2024-09-21T02:14:53.287290+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.293553+0000', 'last_change': '2024-09-21T02:14:53.293553+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.293553+0000', 'last_undegraded': '2024-09-21T02:14:53.293553+0000', 'last_fullsized': '2024-09-21T02:14:53.293553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.292552+0000', 'last_change': '2024-09-21T02:14:53.292552+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.292552+0000', 'last_undegraded': '2024-09-21T02:14:53.292552+0000', 'last_fullsized': '2024-09-21T02:14:53.292552+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.293139+0000', 'last_change': '2024-09-21T02:14:53.293139+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.293139+0000', 'last_undegraded': '2024-09-21T02:14:53.293139+0000', 'last_fullsized': '2024-09-21T02:14:53.293139+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.290426+0000', 'last_change': '2024-09-21T02:14:53.290426+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.290426+0000', 'last_undegraded': '2024-09-21T02:14:53.290426+0000', 'last_fullsized': '2024-09-21T02:14:53.290426+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:PG 7.0 is not active+clean 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-09-21T02:15:00.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.292598+0000', 'last_change': '2024-09-21T02:14:53.292598+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.292598+0000', 'last_undegraded': '2024-09-21T02:14:53.292598+0000', 'last_fullsized': '2024-09-21T02:14:53.292598+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.291639+0000', 'last_change': '2024-09-21T02:14:53.291639+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.291639+0000', 'last_undegraded': '2024-09-21T02:14:53.291639+0000', 'last_fullsized': '2024-09-21T02:14:53.291639+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:14:54.282109+0000', 'last_change': '2024-09-21T02:14:54.282109+0000', 'last_active': '2024-09-21T02:14:54.282109+0000', 'last_peered': '2024-09-21T02:14:54.282109+0000', 'last_clean': '2024-09-21T02:14:54.282109+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:54.282109+0000', 'last_undegraded': '2024-09-21T02:14:54.282109+0000', 'last_fullsized': '2024-09-21T02:14:54.282109+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:54.282109+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-21T02:15:00.339 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:14:53.290357+0000', 'last_change': '2024-09-21T02:14:53.290357+0000', 'last_active': '2024-09-21T02:14:53.271564+0000', 'last_peered': '2024-09-21T02:14:53.271564+0000', 'last_clean': '2024-09-21T02:14:53.271564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:14:53.290357+0000', 'last_undegraded': '2024-09-21T02:14:53.290357+0000', 'last_fullsized': '2024-09-21T02:14:53.290357+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'last_clean_scrub_stamp': '2024-09-21T02:14:53.271564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:15:00.340 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T02:15:00.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 -- 172.21.15.170:0/1066846077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700cb460 msgr2=0x7f68700cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 --2- 172.21.15.170:0/1066846077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700cb460 0x7f68700cb870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6860009900 tx=0x7f686001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 -- 172.21.15.170:0/1066846077 shutdown_connections 2024-09-21T02:15:00.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 --2- 172.21.15.170:0/1066846077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68700ce1c0 0x7f68700ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 --2- 172.21.15.170:0/1066846077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700cb460 0x7f68700cb870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 -- 172.21.15.170:0/1066846077 >> 172.21.15.170:0/1066846077 conn(0x7f68700b1c40 msgr2=0x7f68700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 -- 172.21.15.170:0/1066846077 shutdown_connections 2024-09-21T02:15:00.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 -- 172.21.15.170:0/1066846077 wait complete. 2024-09-21T02:15:00.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 Processor -- start 2024-09-21T02:15:00.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.510+0000 7f68777af640 1 -- start start 2024-09-21T02:15:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68700cb460 0x7f68701f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 0x7f68701494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68700cf7e0 0x7f68701fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68701690d0 con 0x7f68700ce1c0 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6870168f50 con 0x7f68700cf7e0 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6870169250 con 0x7f68700cb460 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 0x7f68701494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 0x7f68701494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59858/0 (socket says 172.21.15.170:59858) 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 -- 172.21.15.170:0/2487442954 learned_addr learned my addr 172.21.15.170:0/2487442954 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 -- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68700cb460 msgr2=0x7f68701f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68700cb460 0x7f68701f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 -- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68700cf7e0 msgr2=0x7f68701fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6875d4c640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68700cf7e0 0x7f68701fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68700cf7e0 0x7f68701fa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 -- 172.21.15.170:0/2487442954 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68700ae5b0 con 0x7f68700ce1c0 2024-09-21T02:15:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6874d4a640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 0x7f68701494f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6860009850 tx=0x7f68600041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600b2040 con 0x7f68700ce1c0 2024-09-21T02:15:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6860003600 con 0x7f68700ce1c0 2024-09-21T02:15:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6870149a30 con 0x7f68700ce1c0 2024-09-21T02:15:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860003a10 con 0x7f68700ce1c0 2024-09-21T02:15:00.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68701fafb0 con 0x7f68700ce1c0 2024-09-21T02:15:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68701faa40 con 0x7f68700ce1c0 2024-09-21T02:15:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.514+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f6860003bb0 con 0x7f68700ce1c0 2024-09-21T02:15:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.518+0000 7f6865ffb640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f68540733e0 0x7f68540758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.518+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f686012e530 con 0x7f68700ce1c0 2024-09-21T02:15:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.518+0000 7f687554b640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f68540733e0 0x7f68540758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.518+0000 7f687554b640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f68540733e0 0x7f68540758f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f686c009670 tx=0x7f686c006bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.522+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f68600be060 con 0x7f68700ce1c0 2024-09-21T02:15:00.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.750+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f68700cb870 con 0x7f68700ce1c0 2024-09-21T02:15:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.750+0000 7f6865ffb640 1 -- 172.21.15.170:0/2487442954 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1763 (secure 0 0 0) 0x7f68700ae5b0 con 0x7f68700ce1c0 2024-09-21T02:15:00.756 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:00.756 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":120,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1726884793,"num_in_osds":8,"osd_in_since":1726884783,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":57},{"state_name":"creating+peering","count":26},{"state_name":"active+clean","count":14}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591980,"bytes_used":252231680,"bytes_avail":772841881600,"bytes_total":773094113280,"unknown_pgs_ratio":0.58762884140014648,"inactive_pgs_ratio":0.2680412232875824,"write_bytes_sec":547,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":27,"btime":"2024-09-21T02:14:58:331015+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":4984}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-21T02:14:14.882101+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T02:15:00.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f68540733e0 msgr2=0x7f68540758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f68540733e0 0x7f68540758f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f686c009670 tx=0x7f686c006bd0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 msgr2=0x7f68701494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 0x7f68701494f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6860009850 tx=0x7f68600041a0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 shutdown_connections 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f68540733e0 0x7f68540758f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68700cf7e0 0x7f68701fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68700ce1c0 0x7f68701494f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 --2- 172.21.15.170:0/2487442954 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68700cb460 0x7f68701f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 >> 172.21.15.170:0/2487442954 conn(0x7f68700b1c40 msgr2=0x7f68700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 shutdown_connections 2024-09-21T02:15:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.754+0000 7f68777af640 1 -- 172.21.15.170:0/2487442954 wait complete. 2024-09-21T02:15:00.777 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T02:15:00.777 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 -- 172.21.15.170:0/3758048345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 msgr2=0x7febcc0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 --2- 172.21.15.170:0/3758048345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc0cb870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febc0009840 tx=0x7febc001c820 comp rx=0 tx=0).stop 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 -- 172.21.15.170:0/3758048345 shutdown_connections 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 --2- 172.21.15.170:0/3758048345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc0cb870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 --2- 172.21.15.170:0/3758048345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febcc0cf7e0 0x7febcc070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 -- 172.21.15.170:0/3758048345 >> 172.21.15.170:0/3758048345 conn(0x7febcc0b1c40 msgr2=0x7febcc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 -- 172.21.15.170:0/3758048345 shutdown_connections 2024-09-21T02:15:00.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.922+0000 7febd3094640 1 -- 172.21.15.170:0/3758048345 wait complete. 2024-09-21T02:15:00.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 Processor -- start 2024-09-21T02:15:00.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 -- start start 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc09d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febcc0cc8c0 0x7febcc1fa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 0x7febcc1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1690d0 con 0x7febcc0cf7e0 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7febcc168f50 con 0x7febcc0cb460 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7febcc169250 con 0x7febcc0cc8c0 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 0x7febcc1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 0x7febcc1f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59860/0 (socket says 172.21.15.170:59860) 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 -- 172.21.15.170:0/1081029987 learned_addr learned my addr 172.21.15.170:0/1081029987 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd0e30640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc09d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febcbfff640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febcc0cc8c0 0x7febcc1fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 -- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febcc0cc8c0 msgr2=0x7febcc1fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febcc0cc8c0 0x7febcc1fa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 -- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 msgr2=0x7febcc09d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc09d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 -- 172.21.15.170:0/1081029987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc09dda0 con 0x7febcc0cf7e0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd0e30640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc09d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd1631640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 0x7febcc1f7d10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7febc400cb10 tx=0x7febc400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4016020 con 0x7febcc0cf7e0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc400ce50 con 0x7febcc0cf7e0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc400ada0 con 0x7febcc0cf7e0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc1fae70 con 0x7febcc0cf7e0 2024-09-21T02:15:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.926+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1fb380 con 0x7febcc0cf7e0 2024-09-21T02:15:00.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.930+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febcc1fb0f0 con 0x7febcc0cf7e0 2024-09-21T02:15:00.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.930+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7febc4019020 con 0x7febcc0cf7e0 2024-09-21T02:15:00.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.930+0000 7febc97fa640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febb80733b0 0x7febb80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:00.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.930+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7febc4094660 con 0x7febcc0cf7e0 2024-09-21T02:15:00.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.930+0000 7febd0e30640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febb80733b0 0x7febb80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:00.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.930+0000 7febd0e30640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febb80733b0 0x7febb80758c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7febbc009b90 tx=0x7febbc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:00.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:00.934+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7febc4020030 con 0x7febcc0cf7e0 2024-09-21T02:15:01.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.098+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febcc1f7340 con 0x7febb80733b0 2024-09-21T02:15:01.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.110+0000 7febc97fa640 1 -- 172.21.15.170:0/1081029987 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7febcc1f7340 con 0x7febb80733b0 2024-09-21T02:15:01.116 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:01.119 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febb80733b0 msgr2=0x7febb80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febb80733b0 0x7febb80758c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7febbc009b90 tx=0x7febbc0091e0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 msgr2=0x7febcc1f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 0x7febcc1f7d10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7febc400cb10 tx=0x7febc400cc00 comp rx=0 tx=0).stop 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 shutdown_connections 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febb80733b0 0x7febb80758c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febcc0cc8c0 0x7febcc1fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febcc0cf7e0 0x7febcc1f7d10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 --2- 172.21.15.170:0/1081029987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febcc0cb460 0x7febcc09d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:01.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 >> 172.21.15.170:0/1081029987 conn(0x7febcc0b1c40 msgr2=0x7febcc15b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:01.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 shutdown_connections 2024-09-21T02:15:01.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:01.114+0000 7febd3094640 1 -- 172.21.15.170:0/1081029987 wait complete. 2024-09-21T02:15:01.137 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-09-21T02:15:00.899375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250776,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754723944,"statfs":{"total":773094113280,"available":772837318656,"internally_reserved":0,"allocated":5513216,"data_stored":2899180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.603229"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.550009+0000","last_change":"2024-09-21T02:14:55.327257+0000","last_active":"2024-09-21T02:14:56.550009+0000","last_peered":"2024-09-21T02:14:56.550009+0000","last_clean":"2024-09-21T02:14:56.550009+0000","last_became_active":"2024-09-21T02:14:55.326386+0000","last_became_peered":"2024-09-21T02:14:55.326386+0000","last_unstale":"2024-09-21T02:14:56.550009+0000","last_undegraded":"2024-09-21T02:14:56.550009+0000","last_fullsized":"2024-09-21T02:14:56.550009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:45.910986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327156+0000","last_change":"2024-09-21T02:14:55.329573+0000","last_active":"2024-09-21T02:14:56.327156+0000","last_peered":"2024-09-21T02:14:56.327156+0000","last_clean":"2024-09-21T02:14:56.327156+0000","last_became_active":"2024-09-21T02:14:55.329010+0000","last_became_peered":"2024-09-21T02:14:55.329010+0000","last_unstale":"2024-09-21T02:14:56.327156+0000","last_undegraded":"2024-09-21T02:14:56.327156+0000","last_fullsized":"2024-09-21T02:14:56.327156+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:37.356974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682545+0000","last_change":"2024-09-21T02:14:55.342163+0000","last_active":"2024-09-21T02:14:56.682545+0000","last_peered":"2024-09-21T02:14:56.682545+0000","last_clean":"2024-09-21T02:14:56.682545+0000","last_became_active":"2024-09-21T02:14:55.341739+0000","last_became_peered":"2024-09-21T02:14:55.341739+0000","last_unstale":"2024-09-21T02:14:56.682545+0000","last_undegraded":"2024-09-21T02:14:56.682545+0000","last_fullsized":"2024-09-21T02:14:56.682545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.022851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327283+0000","last_change":"2024-09-21T02:14:55.325061+0000","last_active":"2024-09-21T02:14:56.327283+0000","last_peered":"2024-09-21T02:14:56.327283+0000","last_clean":"2024-09-21T02:14:56.327283+0000","last_became_active":"2024-09-21T02:14:55.324760+0000","last_became_peered":"2024-09-21T02:14:55.324760+0000","last_unstale":"2024-09-21T02:14:56.327283+0000","last_undegraded":"2024-09-21T02:14:56.327283+0000","last_fullsized":"2024-09-21T02:14:56.327283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:20.425059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549798+0000","last_change":"2024-09-21T02:14:55.323783+0000","last_active":"2024-09-21T02:14:56.549798+0000","last_peered":"2024-09-21T02:14:56.549798+0000","last_clean":"2024-09-21T02:14:56.549798+0000","last_became_active":"2024-09-21T02:14:55.323418+0000","last_became_peered":"2024-09-21T02:14:55.323418+0000","last_unstale":"2024-09-21T02:14:56.549798+0000","last_undegraded":"2024-09-21T02:14:56.549798+0000","last_fullsized":"2024-09-21T02:14:56.549798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:29.087991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403160+0000","last_change":"2024-09-21T02:14:55.318482+0000","last_active":"2024-09-21T02:14:56.403160+0000","last_peered":"2024-09-21T02:14:56.403160+0000","last_clean":"2024-09-21T02:14:56.403160+0000","last_became_active":"2024-09-21T02:14:55.318010+0000","last_became_peered":"2024-09-21T02:14:55.318010+0000","last_unstale":"2024-09-21T02:14:56.403160+0000","last_undegraded":"2024-09-21T02:14:56.403160+0000","last_fullsized":"2024-09-21T02:14:56.403160+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:53.409184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.680557+0000","last_change":"2024-09-21T02:14:55.335794+0000","last_active":"2024-09-21T02:14:56.680557+0000","last_peered":"2024-09-21T02:14:56.680557+0000","last_clean":"2024-09-21T02:14:56.680557+0000","last_became_active":"2024-09-21T02:14:55.335485+0000","last_became_peered":"2024-09-21T02:14:55.335485+0000","last_unstale":"2024-09-21T02:14:56.680557+0000","last_undegraded":"2024-09-21T02:14:56.680557+0000","last_fullsized":"2024-09-21T02:14:56.680557+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:59.416901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326632+0000","last_change":"2024-09-21T02:14:55.331996+0000","last_active":"2024-09-21T02:14:56.326632+0000","last_peered":"2024-09-21T02:14:56.326632+0000","last_clean":"2024-09-21T02:14:56.326632+0000","last_became_active":"2024-09-21T02:14:55.331640+0000","last_became_peered":"2024-09-21T02:14:55.331640+0000","last_unstale":"2024-09-21T02:14:56.326632+0000","last_undegraded":"2024-09-21T02:14:56.326632+0000","last_fullsized":"2024-09-21T02:14:56.326632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:22.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403439+0000","last_change":"2024-09-21T02:14:55.324028+0000","last_active":"2024-09-21T02:14:56.403439+0000","last_peered":"2024-09-21T02:14:56.403439+0000","last_clean":"2024-09-21T02:14:56.403439+0000","last_became_active":"2024-09-21T02:14:55.323726+0000","last_became_peered":"2024-09-21T02:14:55.323726+0000","last_unstale":"2024-09-21T02:14:56.403439+0000","last_undegraded":"2024-09-21T02:14:56.403439+0000","last_fullsized":"2024-09-21T02:14:56.403439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.809074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329777+0000","last_change":"2024-09-21T02:14:55.329046+0000","last_active":"2024-09-21T02:14:56.329777+0000","last_peered":"2024-09-21T02:14:56.329777+0000","last_clean":"2024-09-21T02:14:56.329777+0000","last_became_active":"2024-09-21T02:14:55.328689+0000","last_became_peered":"2024-09-21T02:14:55.328689+0000","last_unstale":"2024-09-21T02:14:56.329777+0000","last_undegraded":"2024-09-21T02:14:56.329777+0000","last_fullsized":"2024-09-21T02:14:56.329777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:50.567278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328970+0000","last_change":"2024-09-21T02:14:55.327536+0000","last_active":"2024-09-21T02:14:56.328970+0000","last_peered":"2024-09-21T02:14:56.328970+0000","last_clean":"2024-09-21T02:14:56.328970+0000","last_became_active":"2024-09-21T02:14:55.327010+0000","last_became_peered":"2024-09-21T02:14:55.327010+0000","last_unstale":"2024-09-21T02:14:56.328970+0000","last_undegraded":"2024-09-21T02:14:56.328970+0000","last_fullsized":"2024-09-21T02:14:56.328970+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:18.272703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326399+0000","last_change":"2024-09-21T02:14:55.318421+0000","last_active":"2024-09-21T02:14:56.326399+0000","last_peered":"2024-09-21T02:14:56.326399+0000","last_clean":"2024-09-21T02:14:56.326399+0000","last_became_active":"2024-09-21T02:14:55.317551+0000","last_became_peered":"2024-09-21T02:14:55.317551+0000","last_unstale":"2024-09-21T02:14:56.326399+0000","last_undegraded":"2024-09-21T02:14:56.326399+0000","last_fullsized":"2024-09-21T02:14:56.326399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:00.680972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402663+0000","last_change":"2024-09-21T02:14:55.319633+0000","last_active":"2024-09-21T02:14:56.402663+0000","last_peered":"2024-09-21T02:14:56.402663+0000","last_clean":"2024-09-21T02:14:56.402663+0000","last_became_active":"2024-09-21T02:14:55.318890+0000","last_became_peered":"2024-09-21T02:14:55.318890+0000","last_unstale":"2024-09-21T02:14:56.402663+0000","last_undegraded":"2024-09-21T02:14:56.402663+0000","last_fullsized":"2024-09-21T02:14:56.402663+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:53.437794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328681+0000","last_change":"2024-09-21T02:14:55.334999+0000","last_active":"2024-09-21T02:14:56.328681+0000","last_peered":"2024-09-21T02:14:56.328681+0000","last_clean":"2024-09-21T02:14:56.328681+0000","last_became_active":"2024-09-21T02:14:55.334258+0000","last_became_peered":"2024-09-21T02:14:55.334258+0000","last_unstale":"2024-09-21T02:14:56.328681+0000","last_undegraded":"2024-09-21T02:14:56.328681+0000","last_fullsized":"2024-09-21T02:14:56.328681+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:48.439070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326754+0000","last_change":"2024-09-21T02:14:55.325404+0000","last_active":"2024-09-21T02:14:56.326754+0000","last_peered":"2024-09-21T02:14:56.326754+0000","last_clean":"2024-09-21T02:14:56.326754+0000","last_became_active":"2024-09-21T02:14:55.324421+0000","last_became_peered":"2024-09-21T02:14:55.324421+0000","last_unstale":"2024-09-21T02:14:56.326754+0000","last_undegraded":"2024-09-21T02:14:56.326754+0000","last_fullsized":"2024-09-21T02:14:56.326754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:29.710578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548631+0000","last_change":"2024-09-21T02:14:55.339049+0000","last_active":"2024-09-21T02:14:56.548631+0000","last_peered":"2024-09-21T02:14:56.548631+0000","last_clean":"2024-09-21T02:14:56.548631+0000","last_became_active":"2024-09-21T02:14:55.338619+0000","last_became_peered":"2024-09-21T02:14:55.338619+0000","last_unstale":"2024-09-21T02:14:56.548631+0000","last_undegraded":"2024-09-21T02:14:56.548631+0000","last_fullsized":"2024-09-21T02:14:56.548631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:37.356974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327520+0000","last_change":"2024-09-21T02:14:55.321172+0000","last_active":"2024-09-21T02:14:56.327520+0000","last_peered":"2024-09-21T02:14:56.327520+0000","last_clean":"2024-09-21T02:14:56.327520+0000","last_became_active":"2024-09-21T02:14:55.313203+0000","last_became_peered":"2024-09-21T02:14:55.313203+0000","last_unstale":"2024-09-21T02:14:56.327520+0000","last_undegraded":"2024-09-21T02:14:56.327520+0000","last_fullsized":"2024-09-21T02:14:56.327520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:14.197505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682821+0000","last_change":"2024-09-21T02:14:55.327950+0000","last_active":"2024-09-21T02:14:56.682821+0000","last_peered":"2024-09-21T02:14:56.682821+0000","last_clean":"2024-09-21T02:14:56.682821+0000","last_became_active":"2024-09-21T02:14:55.327317+0000","last_became_peered":"2024-09-21T02:14:55.327317+0000","last_unstale":"2024-09-21T02:14:56.682821+0000","last_undegraded":"2024-09-21T02:14:56.682821+0000","last_fullsized":"2024-09-21T02:14:56.682821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:23.054432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549367+0000","last_change":"2024-09-21T02:14:55.331527+0000","last_active":"2024-09-21T02:14:56.549367+0000","last_peered":"2024-09-21T02:14:56.549367+0000","last_clean":"2024-09-21T02:14:56.549367+0000","last_became_active":"2024-09-21T02:14:55.331060+0000","last_became_peered":"2024-09-21T02:14:55.331060+0000","last_unstale":"2024-09-21T02:14:56.549367+0000","last_undegraded":"2024-09-21T02:14:56.549367+0000","last_fullsized":"2024-09-21T02:14:56.549367+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:15.077374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329725+0000","last_change":"2024-09-21T02:14:55.333330+0000","last_active":"2024-09-21T02:14:56.329725+0000","last_peered":"2024-09-21T02:14:56.329725+0000","last_clean":"2024-09-21T02:14:56.329725+0000","last_became_active":"2024-09-21T02:14:55.332877+0000","last_became_peered":"2024-09-21T02:14:55.332877+0000","last_unstale":"2024-09-21T02:14:56.329725+0000","last_undegraded":"2024-09-21T02:14:56.329725+0000","last_fullsized":"2024-09-21T02:14:56.329725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:06.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327439+0000","last_change":"2024-09-21T02:14:55.325693+0000","last_active":"2024-09-21T02:14:56.327439+0000","last_peered":"2024-09-21T02:14:56.327439+0000","last_clean":"2024-09-21T02:14:56.327439+0000","last_became_active":"2024-09-21T02:14:55.319881+0000","last_became_peered":"2024-09-21T02:14:55.319881+0000","last_unstale":"2024-09-21T02:14:56.327439+0000","last_undegraded":"2024-09-21T02:14:56.327439+0000","last_fullsized":"2024-09-21T02:14:56.327439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:27.562478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681908+0000","last_change":"2024-09-21T02:14:55.323917+0000","last_active":"2024-09-21T02:14:56.681908+0000","last_peered":"2024-09-21T02:14:56.681908+0000","last_clean":"2024-09-21T02:14:56.681908+0000","last_became_active":"2024-09-21T02:14:55.323342+0000","last_became_peered":"2024-09-21T02:14:55.323342+0000","last_unstale":"2024-09-21T02:14:56.681908+0000","last_undegraded":"2024-09-21T02:14:56.681908+0000","last_fullsized":"2024-09-21T02:14:56.681908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:37.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681110+0000","last_change":"2024-09-21T02:14:55.331910+0000","last_active":"2024-09-21T02:14:56.681110+0000","last_peered":"2024-09-21T02:14:56.681110+0000","last_clean":"2024-09-21T02:14:56.681110+0000","last_became_active":"2024-09-21T02:14:55.331595+0000","last_became_peered":"2024-09-21T02:14:55.331595+0000","last_unstale":"2024-09-21T02:14:56.681110+0000","last_undegraded":"2024-09-21T02:14:56.681110+0000","last_fullsized":"2024-09-21T02:14:56.681110+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:09.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.325977+0000","last_change":"2024-09-21T02:14:55.326958+0000","last_active":"2024-09-21T02:14:56.325977+0000","last_peered":"2024-09-21T02:14:56.325977+0000","last_clean":"2024-09-21T02:14:56.325977+0000","last_became_active":"2024-09-21T02:14:55.326606+0000","last_became_peered":"2024-09-21T02:14:55.326606+0000","last_unstale":"2024-09-21T02:14:56.325977+0000","last_undegraded":"2024-09-21T02:14:56.325977+0000","last_fullsized":"2024-09-21T02:14:56.325977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:51.003999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.550355+0000","last_change":"2024-09-21T02:14:55.326778+0000","last_active":"2024-09-21T02:14:56.550355+0000","last_peered":"2024-09-21T02:14:56.550355+0000","last_clean":"2024-09-21T02:14:56.550355+0000","last_became_active":"2024-09-21T02:14:55.325840+0000","last_became_peered":"2024-09-21T02:14:55.325840+0000","last_unstale":"2024-09-21T02:14:56.550355+0000","last_undegraded":"2024-09-21T02:14:56.550355+0000","last_fullsized":"2024-09-21T02:14:56.550355+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:04.861616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327867+0000","last_change":"2024-09-21T02:14:55.316584+0000","last_active":"2024-09-21T02:14:56.327867+0000","last_peered":"2024-09-21T02:14:56.327867+0000","last_clean":"2024-09-21T02:14:56.327867+0000","last_became_active":"2024-09-21T02:14:55.316082+0000","last_became_peered":"2024-09-21T02:14:55.316082+0000","last_unstale":"2024-09-21T02:14:56.327867+0000","last_undegraded":"2024-09-21T02:14:56.327867+0000","last_fullsized":"2024-09-21T02:14:56.327867+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:22.367502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683490+0000","last_change":"2024-09-21T02:14:55.320184+0000","last_active":"2024-09-21T02:14:56.683490+0000","last_peered":"2024-09-21T02:14:56.683490+0000","last_clean":"2024-09-21T02:14:56.683490+0000","last_became_active":"2024-09-21T02:14:55.319858+0000","last_became_peered":"2024-09-21T02:14:55.319858+0000","last_unstale":"2024-09-21T02:14:56.683490+0000","last_undegraded":"2024-09-21T02:14:56.683490+0000","last_fullsized":"2024-09-21T02:14:56.683490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:44.385122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682126+0000","last_change":"2024-09-21T02:14:55.320086+0000","last_active":"2024-09-21T02:14:56.682126+0000","last_peered":"2024-09-21T02:14:56.682126+0000","last_clean":"2024-09-21T02:14:56.682126+0000","last_became_active":"2024-09-21T02:14:55.319752+0000","last_became_peered":"2024-09-21T02:14:55.319752+0000","last_unstale":"2024-09-21T02:14:56.682126+0000","last_undegraded":"2024-09-21T02:14:56.682126+0000","last_fullsized":"2024-09-21T02:14:56.682126+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:58.154497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549206+0000","last_change":"2024-09-21T02:14:55.339772+0000","last_active":"2024-09-21T02:14:56.549206+0000","last_peered":"2024-09-21T02:14:56.549206+0000","last_clean":"2024-09-21T02:14:56.549206+0000","last_became_active":"2024-09-21T02:14:55.339235+0000","last_became_peered":"2024-09-21T02:14:55.339235+0000","last_unstale":"2024-09-21T02:14:56.549206+0000","last_undegraded":"2024-09-21T02:14:56.549206+0000","last_fullsized":"2024-09-21T02:14:56.549206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:14.197505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326467+0000","last_change":"2024-09-21T02:14:55.320525+0000","last_active":"2024-09-21T02:14:56.326467+0000","last_peered":"2024-09-21T02:14:56.326467+0000","last_clean":"2024-09-21T02:14:56.326467+0000","last_became_active":"2024-09-21T02:14:55.320074+0000","last_became_peered":"2024-09-21T02:14:55.320074+0000","last_unstale":"2024-09-21T02:14:56.326467+0000","last_undegraded":"2024-09-21T02:14:56.326467+0000","last_fullsized":"2024-09-21T02:14:56.326467+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:54.734955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402271+0000","last_change":"2024-09-21T02:14:55.332430+0000","last_active":"2024-09-21T02:14:56.402271+0000","last_peered":"2024-09-21T02:14:56.402271+0000","last_clean":"2024-09-21T02:14:56.402271+0000","last_became_active":"2024-09-21T02:14:55.332005+0000","last_became_peered":"2024-09-21T02:14:55.332005+0000","last_unstale":"2024-09-21T02:14:56.402271+0000","last_undegraded":"2024-09-21T02:14:56.402271+0000","last_fullsized":"2024-09-21T02:14:56.402271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:53.575464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326582+0000","last_change":"2024-09-21T02:14:55.327609+0000","last_active":"2024-09-21T02:14:56.326582+0000","last_peered":"2024-09-21T02:14:56.326582+0000","last_clean":"2024-09-21T02:14:56.326582+0000","last_became_active":"2024-09-21T02:14:55.327089+0000","last_became_peered":"2024-09-21T02:14:55.327089+0000","last_unstale":"2024-09-21T02:14:56.326582+0000","last_undegraded":"2024-09-21T02:14:56.326582+0000","last_fullsized":"2024-09-21T02:14:56.326582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:17.740610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682075+0000","last_change":"2024-09-21T02:14:55.327561+0000","last_active":"2024-09-21T02:14:56.682075+0000","last_peered":"2024-09-21T02:14:56.682075+0000","last_clean":"2024-09-21T02:14:56.682075+0000","last_became_active":"2024-09-21T02:14:55.327119+0000","last_became_peered":"2024-09-21T02:14:55.327119+0000","last_unstale":"2024-09-21T02:14:56.682075+0000","last_undegraded":"2024-09-21T02:14:56.682075+0000","last_fullsized":"2024-09-21T02:14:56.682075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:20.856161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403547+0000","last_change":"2024-09-21T02:14:54.339187+0000","last_active":"2024-09-21T02:14:56.403547+0000","last_peered":"2024-09-21T02:14:56.403547+0000","last_clean":"2024-09-21T02:14:56.403547+0000","last_became_active":"2024-09-21T02:14:54.338577+0000","last_became_peered":"2024-09-21T02:14:54.338577+0000","last_unstale":"2024-09-21T02:14:56.403547+0000","last_undegraded":"2024-09-21T02:14:56.403547+0000","last_fullsized":"2024-09-21T02:14:56.403547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:46.922448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.550338+0000","last_change":"2024-09-21T02:14:55.312579+0000","last_active":"2024-09-21T02:14:56.550338+0000","last_peered":"2024-09-21T02:14:56.550338+0000","last_clean":"2024-09-21T02:14:56.550338+0000","last_became_active":"2024-09-21T02:14:55.312172+0000","last_became_peered":"2024-09-21T02:14:55.312172+0000","last_unstale":"2024-09-21T02:14:56.550338+0000","last_undegraded":"2024-09-21T02:14:56.550338+0000","last_fullsized":"2024-09-21T02:14:56.550338+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:22.129192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683647+0000","last_change":"2024-09-21T02:14:54.326175+0000","last_active":"2024-09-21T02:14:56.683647+0000","last_peered":"2024-09-21T02:14:56.683647+0000","last_clean":"2024-09-21T02:14:56.683647+0000","last_became_active":"2024-09-21T02:14:54.324027+0000","last_became_peered":"2024-09-21T02:14:54.324027+0000","last_unstale":"2024-09-21T02:14:56.683647+0000","last_undegraded":"2024-09-21T02:14:56.683647+0000","last_fullsized":"2024-09-21T02:14:56.683647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:21.118647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326680+0000","last_change":"2024-09-21T02:14:55.334182+0000","last_active":"2024-09-21T02:14:56.326680+0000","last_peered":"2024-09-21T02:14:56.326680+0000","last_clean":"2024-09-21T02:14:56.326680+0000","last_became_active":"2024-09-21T02:14:55.333902+0000","last_became_peered":"2024-09-21T02:14:55.333902+0000","last_unstale":"2024-09-21T02:14:56.326680+0000","last_undegraded":"2024-09-21T02:14:56.326680+0000","last_fullsized":"2024-09-21T02:14:56.326680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:19.457723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402973+0000","last_change":"2024-09-21T02:14:54.339702+0000","last_active":"2024-09-21T02:14:56.402973+0000","last_peered":"2024-09-21T02:14:56.402973+0000","last_clean":"2024-09-21T02:14:56.402973+0000","last_became_active":"2024-09-21T02:14:54.338812+0000","last_became_peered":"2024-09-21T02:14:54.338812+0000","last_unstale":"2024-09-21T02:14:56.402973+0000","last_undegraded":"2024-09-21T02:14:56.402973+0000","last_fullsized":"2024-09-21T02:14:56.402973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:01.599009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329586+0000","last_change":"2024-09-21T02:14:55.327662+0000","last_active":"2024-09-21T02:14:56.329586+0000","last_peered":"2024-09-21T02:14:56.329586+0000","last_clean":"2024-09-21T02:14:56.329586+0000","last_became_active":"2024-09-21T02:14:55.326518+0000","last_became_peered":"2024-09-21T02:14:55.326518+0000","last_unstale":"2024-09-21T02:14:56.329586+0000","last_undegraded":"2024-09-21T02:14:56.329586+0000","last_fullsized":"2024-09-21T02:14:56.329586+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:10.446600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682502+0000","last_change":"2024-09-21T02:14:54.326589+0000","last_active":"2024-09-21T02:14:56.682502+0000","last_peered":"2024-09-21T02:14:56.682502+0000","last_clean":"2024-09-21T02:14:56.682502+0000","last_became_active":"2024-09-21T02:14:54.325839+0000","last_became_peered":"2024-09-21T02:14:54.325839+0000","last_unstale":"2024-09-21T02:14:56.682502+0000","last_undegraded":"2024-09-21T02:14:56.682502+0000","last_fullsized":"2024-09-21T02:14:56.682502+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:19.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681388+0000","last_change":"2024-09-21T02:14:55.325001+0000","last_active":"2024-09-21T02:14:56.681388+0000","last_peered":"2024-09-21T02:14:56.681388+0000","last_clean":"2024-09-21T02:14:56.681388+0000","last_became_active":"2024-09-21T02:14:55.324690+0000","last_became_peered":"2024-09-21T02:14:55.324690+0000","last_unstale":"2024-09-21T02:14:56.681388+0000","last_undegraded":"2024-09-21T02:14:56.681388+0000","last_fullsized":"2024-09-21T02:14:56.681388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:37.953461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403047+0000","last_change":"2024-09-21T02:14:54.340918+0000","last_active":"2024-09-21T02:14:56.403047+0000","last_peered":"2024-09-21T02:14:56.403047+0000","last_clean":"2024-09-21T02:14:56.403047+0000","last_became_active":"2024-09-21T02:14:54.339864+0000","last_became_peered":"2024-09-21T02:14:54.339864+0000","last_unstale":"2024-09-21T02:14:56.403047+0000","last_undegraded":"2024-09-21T02:14:56.403047+0000","last_fullsized":"2024-09-21T02:14:56.403047+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:22.209018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329912+0000","last_change":"2024-09-21T02:14:55.339871+0000","last_active":"2024-09-21T02:14:56.329912+0000","last_peered":"2024-09-21T02:14:56.329912+0000","last_clean":"2024-09-21T02:14:56.329912+0000","last_became_active":"2024-09-21T02:14:55.339582+0000","last_became_peered":"2024-09-21T02:14:55.339582+0000","last_unstale":"2024-09-21T02:14:56.329912+0000","last_undegraded":"2024-09-21T02:14:56.329912+0000","last_fullsized":"2024-09-21T02:14:56.329912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:12.830203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681564+0000","last_change":"2024-09-21T02:14:54.312459+0000","last_active":"2024-09-21T02:14:56.681564+0000","last_peered":"2024-09-21T02:14:56.681564+0000","last_clean":"2024-09-21T02:14:56.681564+0000","last_became_active":"2024-09-21T02:14:54.312050+0000","last_became_peered":"2024-09-21T02:14:54.312050+0000","last_unstale":"2024-09-21T02:14:56.681564+0000","last_undegraded":"2024-09-21T02:14:56.681564+0000","last_fullsized":"2024-09-21T02:14:56.681564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:18.028470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328781+0000","last_change":"2024-09-21T02:14:55.336714+0000","last_active":"2024-09-21T02:14:56.328781+0000","last_peered":"2024-09-21T02:14:56.328781+0000","last_clean":"2024-09-21T02:14:56.328781+0000","last_became_active":"2024-09-21T02:14:55.336144+0000","last_became_peered":"2024-09-21T02:14:55.336144+0000","last_unstale":"2024-09-21T02:14:56.328781+0000","last_undegraded":"2024-09-21T02:14:56.328781+0000","last_fullsized":"2024-09-21T02:14:56.328781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:34.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548710+0000","last_change":"2024-09-21T02:14:54.323694+0000","last_active":"2024-09-21T02:14:56.548710+0000","last_peered":"2024-09-21T02:14:56.548710+0000","last_clean":"2024-09-21T02:14:56.548710+0000","last_became_active":"2024-09-21T02:14:54.323375+0000","last_became_peered":"2024-09-21T02:14:54.323375+0000","last_unstale":"2024-09-21T02:14:56.548710+0000","last_undegraded":"2024-09-21T02:14:56.548710+0000","last_fullsized":"2024-09-21T02:14:56.548710+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:36.113978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.680993+0000","last_change":"2024-09-21T02:14:55.314477+0000","last_active":"2024-09-21T02:14:56.680993+0000","last_peered":"2024-09-21T02:14:56.680993+0000","last_clean":"2024-09-21T02:14:56.680993+0000","last_became_active":"2024-09-21T02:14:55.314176+0000","last_became_peered":"2024-09-21T02:14:55.314176+0000","last_unstale":"2024-09-21T02:14:56.680993+0000","last_undegraded":"2024-09-21T02:14:56.680993+0000","last_fullsized":"2024-09-21T02:14:56.680993+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:44.961823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329124+0000","last_change":"2024-09-21T02:14:54.329228+0000","last_active":"2024-09-21T02:14:56.329124+0000","last_peered":"2024-09-21T02:14:56.329124+0000","last_clean":"2024-09-21T02:14:56.329124+0000","last_became_active":"2024-09-21T02:14:54.328430+0000","last_became_peered":"2024-09-21T02:14:54.328430+0000","last_unstale":"2024-09-21T02:14:56.329124+0000","last_undegraded":"2024-09-21T02:14:56.329124+0000","last_fullsized":"2024-09-21T02:14:56.329124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:46.117802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683396+0000","last_change":"2024-09-21T02:14:55.330082+0000","last_active":"2024-09-21T02:14:56.683396+0000","last_peered":"2024-09-21T02:14:56.683396+0000","last_clean":"2024-09-21T02:14:56.683396+0000","last_became_active":"2024-09-21T02:14:55.329669+0000","last_became_peered":"2024-09-21T02:14:55.329669+0000","last_unstale":"2024-09-21T02:14:56.683396+0000","last_undegraded":"2024-09-21T02:14:56.683396+0000","last_fullsized":"2024-09-21T02:14:56.683396+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:50.391928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682998+0000","last_change":"2024-09-21T02:14:54.337122+0000","last_active":"2024-09-21T02:14:56.682998+0000","last_peered":"2024-09-21T02:14:56.682998+0000","last_clean":"2024-09-21T02:14:56.682998+0000","last_became_active":"2024-09-21T02:14:54.336774+0000","last_became_peered":"2024-09-21T02:14:54.336774+0000","last_unstale":"2024-09-21T02:14:56.682998+0000","last_undegraded":"2024-09-21T02:14:56.682998+0000","last_fullsized":"2024-09-21T02:14:56.682998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:32.826706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683401+0000","last_change":"2024-09-21T02:14:55.332894+0000","last_active":"2024-09-21T02:14:56.683401+0000","last_peered":"2024-09-21T02:14:56.683401+0000","last_clean":"2024-09-21T02:14:56.683401+0000","last_became_active":"2024-09-21T02:14:55.332483+0000","last_became_peered":"2024-09-21T02:14:55.332483+0000","last_unstale":"2024-09-21T02:14:56.683401+0000","last_undegraded":"2024-09-21T02:14:56.683401+0000","last_fullsized":"2024-09-21T02:14:56.683401+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:48.301871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329961+0000","last_change":"2024-09-21T02:14:54.329034+0000","last_active":"2024-09-21T02:14:56.329961+0000","last_peered":"2024-09-21T02:14:56.329961+0000","last_clean":"2024-09-21T02:14:56.329961+0000","last_became_active":"2024-09-21T02:14:54.327414+0000","last_became_peered":"2024-09-21T02:14:54.327414+0000","last_unstale":"2024-09-21T02:14:56.329961+0000","last_undegraded":"2024-09-21T02:14:56.329961+0000","last_fullsized":"2024-09-21T02:14:56.329961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:29.369340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327551+0000","last_change":"2024-09-21T02:14:55.334653+0000","last_active":"2024-09-21T02:14:56.327551+0000","last_peered":"2024-09-21T02:14:56.327551+0000","last_clean":"2024-09-21T02:14:56.327551+0000","last_became_active":"2024-09-21T02:14:55.334280+0000","last_became_peered":"2024-09-21T02:14:55.334280+0000","last_unstale":"2024-09-21T02:14:56.327551+0000","last_undegraded":"2024-09-21T02:14:56.327551+0000","last_fullsized":"2024-09-21T02:14:56.327551+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:19.127968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.325827+0000","last_change":"2024-09-21T02:14:54.316458+0000","last_active":"2024-09-21T02:14:56.325827+0000","last_peered":"2024-09-21T02:14:56.325827+0000","last_clean":"2024-09-21T02:14:56.325827+0000","last_became_active":"2024-09-21T02:14:54.315604+0000","last_became_peered":"2024-09-21T02:14:54.315604+0000","last_unstale":"2024-09-21T02:14:56.325827+0000","last_undegraded":"2024-09-21T02:14:56.325827+0000","last_fullsized":"2024-09-21T02:14:56.325827+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:05.699325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683595+0000","last_change":"2024-09-21T02:14:55.339134+0000","last_active":"2024-09-21T02:14:56.683595+0000","last_peered":"2024-09-21T02:14:56.683595+0000","last_clean":"2024-09-21T02:14:56.683595+0000","last_became_active":"2024-09-21T02:14:55.338786+0000","last_became_peered":"2024-09-21T02:14:55.338786+0000","last_unstale":"2024-09-21T02:14:56.683595+0000","last_undegraded":"2024-09-21T02:14:56.683595+0000","last_fullsized":"2024-09-21T02:14:56.683595+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:24.396896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330095+0000","last_change":"2024-09-21T02:14:54.319009+0000","last_active":"2024-09-21T02:14:56.330095+0000","last_peered":"2024-09-21T02:14:56.330095+0000","last_clean":"2024-09-21T02:14:56.330095+0000","last_became_active":"2024-09-21T02:14:54.317796+0000","last_became_peered":"2024-09-21T02:14:54.317796+0000","last_unstale":"2024-09-21T02:14:56.330095+0000","last_undegraded":"2024-09-21T02:14:56.330095+0000","last_fullsized":"2024-09-21T02:14:56.330095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:16.231157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682424+0000","last_change":"2024-09-21T02:14:55.328400+0000","last_active":"2024-09-21T02:14:56.682424+0000","last_peered":"2024-09-21T02:14:56.682424+0000","last_clean":"2024-09-21T02:14:56.682424+0000","last_became_active":"2024-09-21T02:14:55.327810+0000","last_became_peered":"2024-09-21T02:14:55.327810+0000","last_unstale":"2024-09-21T02:14:56.682424+0000","last_undegraded":"2024-09-21T02:14:56.682424+0000","last_fullsized":"2024-09-21T02:14:56.682424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.604823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681481+0000","last_change":"2024-09-21T02:14:54.343831+0000","last_active":"2024-09-21T02:14:56.681481+0000","last_peered":"2024-09-21T02:14:56.681481+0000","last_clean":"2024-09-21T02:14:56.681481+0000","last_became_active":"2024-09-21T02:14:54.342819+0000","last_became_peered":"2024-09-21T02:14:54.342819+0000","last_unstale":"2024-09-21T02:14:56.681481+0000","last_undegraded":"2024-09-21T02:14:56.681481+0000","last_fullsized":"2024-09-21T02:14:56.681481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.247144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681950+0000","last_change":"2024-09-21T02:14:55.320585+0000","last_active":"2024-09-21T02:14:56.681950+0000","last_peered":"2024-09-21T02:14:56.681950+0000","last_clean":"2024-09-21T02:14:56.681950+0000","last_became_active":"2024-09-21T02:14:55.320194+0000","last_became_peered":"2024-09-21T02:14:55.320194+0000","last_unstale":"2024-09-21T02:14:56.681950+0000","last_undegraded":"2024-09-21T02:14:56.681950+0000","last_fullsized":"2024-09-21T02:14:56.681950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:00.906337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548402+0000","last_change":"2024-09-21T02:14:54.333776+0000","last_active":"2024-09-21T02:14:56.548402+0000","last_peered":"2024-09-21T02:14:56.548402+0000","last_clean":"2024-09-21T02:14:56.548402+0000","last_became_active":"2024-09-21T02:14:54.333330+0000","last_became_peered":"2024-09-21T02:14:54.333330+0000","last_unstale":"2024-09-21T02:14:56.548402+0000","last_undegraded":"2024-09-21T02:14:56.548402+0000","last_fullsized":"2024-09-21T02:14:56.548402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:56.898580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681657+0000","last_change":"2024-09-21T02:14:55.333564+0000","last_active":"2024-09-21T02:14:56.681657+0000","last_peered":"2024-09-21T02:14:56.681657+0000","last_clean":"2024-09-21T02:14:56.681657+0000","last_became_active":"2024-09-21T02:14:55.333161+0000","last_became_peered":"2024-09-21T02:14:55.333161+0000","last_unstale":"2024-09-21T02:14:56.681657+0000","last_undegraded":"2024-09-21T02:14:56.681657+0000","last_fullsized":"2024-09-21T02:14:56.681657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:28.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402147+0000","last_change":"2024-09-21T02:14:54.336941+0000","last_active":"2024-09-21T02:14:56.402147+0000","last_peered":"2024-09-21T02:14:56.402147+0000","last_clean":"2024-09-21T02:14:56.402147+0000","last_became_active":"2024-09-21T02:14:54.336121+0000","last_became_peered":"2024-09-21T02:14:54.336121+0000","last_unstale":"2024-09-21T02:14:56.402147+0000","last_undegraded":"2024-09-21T02:14:56.402147+0000","last_fullsized":"2024-09-21T02:14:56.402147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:45.583899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.325727+0000","last_change":"2024-09-21T02:14:55.331826+0000","last_active":"2024-09-21T02:14:56.325727+0000","last_peered":"2024-09-21T02:14:56.325727+0000","last_clean":"2024-09-21T02:14:56.325727+0000","last_became_active":"2024-09-21T02:14:55.331500+0000","last_became_peered":"2024-09-21T02:14:55.331500+0000","last_unstale":"2024-09-21T02:14:56.325727+0000","last_undegraded":"2024-09-21T02:14:56.325727+0000","last_fullsized":"2024-09-21T02:14:56.325727+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:08.349497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329396+0000","last_change":"2024-09-21T02:14:54.319185+0000","last_active":"2024-09-21T02:14:56.329396+0000","last_peered":"2024-09-21T02:14:56.329396+0000","last_clean":"2024-09-21T02:14:56.329396+0000","last_became_active":"2024-09-21T02:14:54.318313+0000","last_became_peered":"2024-09-21T02:14:54.318313+0000","last_unstale":"2024-09-21T02:14:56.329396+0000","last_undegraded":"2024-09-21T02:14:56.329396+0000","last_fullsized":"2024-09-21T02:14:56.329396+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:08.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682379+0000","last_change":"2024-09-21T02:14:55.322130+0000","last_active":"2024-09-21T02:14:56.682379+0000","last_peered":"2024-09-21T02:14:56.682379+0000","last_clean":"2024-09-21T02:14:56.682379+0000","last_became_active":"2024-09-21T02:14:55.321649+0000","last_became_peered":"2024-09-21T02:14:55.321649+0000","last_unstale":"2024-09-21T02:14:56.682379+0000","last_undegraded":"2024-09-21T02:14:56.682379+0000","last_fullsized":"2024-09-21T02:14:56.682379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:54.430744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327057+0000","last_change":"2024-09-21T02:14:54.343765+0000","last_active":"2024-09-21T02:14:56.327057+0000","last_peered":"2024-09-21T02:14:56.327057+0000","last_clean":"2024-09-21T02:14:56.327057+0000","last_became_active":"2024-09-21T02:14:54.343116+0000","last_became_peered":"2024-09-21T02:14:54.343116+0000","last_unstale":"2024-09-21T02:14:56.327057+0000","last_undegraded":"2024-09-21T02:14:56.327057+0000","last_fullsized":"2024-09-21T02:14:56.327057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:07.045536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682594+0000","last_change":"2024-09-21T02:14:55.327468+0000","last_active":"2024-09-21T02:14:56.682594+0000","last_peered":"2024-09-21T02:14:56.682594+0000","last_clean":"2024-09-21T02:14:56.682594+0000","last_became_active":"2024-09-21T02:14:55.327131+0000","last_became_peered":"2024-09-21T02:14:55.327131+0000","last_unstale":"2024-09-21T02:14:56.682594+0000","last_undegraded":"2024-09-21T02:14:56.682594+0000","last_fullsized":"2024-09-21T02:14:56.682594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:37.144365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683234+0000","last_change":"2024-09-21T02:14:54.334494+0000","last_active":"2024-09-21T02:14:56.683234+0000","last_peered":"2024-09-21T02:14:56.683234+0000","last_clean":"2024-09-21T02:14:56.683234+0000","last_became_active":"2024-09-21T02:14:54.333728+0000","last_became_peered":"2024-09-21T02:14:54.333728+0000","last_unstale":"2024-09-21T02:14:56.683234+0000","last_undegraded":"2024-09-21T02:14:56.683234+0000","last_fullsized":"2024-09-21T02:14:56.683234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:52.902706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326797+0000","last_change":"2024-09-21T02:14:55.317502+0000","last_active":"2024-09-21T02:14:56.326797+0000","last_peered":"2024-09-21T02:14:56.326797+0000","last_clean":"2024-09-21T02:14:56.326797+0000","last_became_active":"2024-09-21T02:14:55.317159+0000","last_became_peered":"2024-09-21T02:14:55.317159+0000","last_unstale":"2024-09-21T02:14:56.326797+0000","last_undegraded":"2024-09-21T02:14:56.326797+0000","last_fullsized":"2024-09-21T02:14:56.326797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:54.075840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682748+0000","last_change":"2024-09-21T02:14:54.322488+0000","last_active":"2024-09-21T02:14:56.682748+0000","last_peered":"2024-09-21T02:14:56.682748+0000","last_clean":"2024-09-21T02:14:56.682748+0000","last_became_active":"2024-09-21T02:14:54.322002+0000","last_became_peered":"2024-09-21T02:14:54.322002+0000","last_unstale":"2024-09-21T02:14:56.682748+0000","last_undegraded":"2024-09-21T02:14:56.682748+0000","last_fullsized":"2024-09-21T02:14:56.682748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:26.099198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682684+0000","last_change":"2024-09-21T02:14:55.341864+0000","last_active":"2024-09-21T02:14:56.682684+0000","last_peered":"2024-09-21T02:14:56.682684+0000","last_clean":"2024-09-21T02:14:56.682684+0000","last_became_active":"2024-09-21T02:14:55.341135+0000","last_became_peered":"2024-09-21T02:14:55.341135+0000","last_unstale":"2024-09-21T02:14:56.682684+0000","last_undegraded":"2024-09-21T02:14:56.682684+0000","last_fullsized":"2024-09-21T02:14:56.682684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:00.801062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402369+0000","last_change":"2024-09-21T02:14:54.332963+0000","last_active":"2024-09-21T02:14:56.402369+0000","last_peered":"2024-09-21T02:14:56.402369+0000","last_clean":"2024-09-21T02:14:56.402369+0000","last_became_active":"2024-09-21T02:14:54.332128+0000","last_became_peered":"2024-09-21T02:14:54.332128+0000","last_unstale":"2024-09-21T02:14:56.402369+0000","last_undegraded":"2024-09-21T02:14:56.402369+0000","last_fullsized":"2024-09-21T02:14:56.402369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328752+0000","last_change":"2024-09-21T02:14:55.328993+0000","last_active":"2024-09-21T02:14:56.328752+0000","last_peered":"2024-09-21T02:14:56.328752+0000","last_clean":"2024-09-21T02:14:56.328752+0000","last_became_active":"2024-09-21T02:14:55.328635+0000","last_became_peered":"2024-09-21T02:14:55.328635+0000","last_unstale":"2024-09-21T02:14:56.328752+0000","last_undegraded":"2024-09-21T02:14:56.328752+0000","last_fullsized":"2024-09-21T02:14:56.328752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:09.392936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328949+0000","last_change":"2024-09-21T02:14:54.323086+0000","last_active":"2024-09-21T02:14:56.328949+0000","last_peered":"2024-09-21T02:14:56.328949+0000","last_clean":"2024-09-21T02:14:56.328949+0000","last_became_active":"2024-09-21T02:14:54.321950+0000","last_became_peered":"2024-09-21T02:14:54.321950+0000","last_unstale":"2024-09-21T02:14:56.328949+0000","last_undegraded":"2024-09-21T02:14:56.328949+0000","last_fullsized":"2024-09-21T02:14:56.328949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.939679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403037+0000","last_change":"2024-09-21T02:14:55.315473+0000","last_active":"2024-09-21T02:14:56.403037+0000","last_peered":"2024-09-21T02:14:56.403037+0000","last_clean":"2024-09-21T02:14:56.403037+0000","last_became_active":"2024-09-21T02:14:55.314644+0000","last_became_peered":"2024-09-21T02:14:55.314644+0000","last_unstale":"2024-09-21T02:14:56.403037+0000","last_undegraded":"2024-09-21T02:14:56.403037+0000","last_fullsized":"2024-09-21T02:14:56.403037+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:38.236709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548573+0000","last_change":"2024-09-21T02:14:54.327284+0000","last_active":"2024-09-21T02:14:56.548573+0000","last_peered":"2024-09-21T02:14:56.548573+0000","last_clean":"2024-09-21T02:14:56.548573+0000","last_became_active":"2024-09-21T02:14:54.326598+0000","last_became_peered":"2024-09-21T02:14:54.326598+0000","last_unstale":"2024-09-21T02:14:56.548573+0000","last_undegraded":"2024-09-21T02:14:56.548573+0000","last_fullsized":"2024-09-21T02:14:56.548573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:36.997240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326891+0000","last_change":"2024-09-21T02:14:55.327865+0000","last_active":"2024-09-21T02:14:56.326891+0000","last_peered":"2024-09-21T02:14:56.326891+0000","last_clean":"2024-09-21T02:14:56.326891+0000","last_became_active":"2024-09-21T02:14:55.327136+0000","last_became_peered":"2024-09-21T02:14:55.327136+0000","last_unstale":"2024-09-21T02:14:56.326891+0000","last_undegraded":"2024-09-21T02:14:56.326891+0000","last_fullsized":"2024-09-21T02:14:56.326891+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:22.129192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328456+0000","last_change":"2024-09-21T02:14:54.328647+0000","last_active":"2024-09-21T02:14:56.328456+0000","last_peered":"2024-09-21T02:14:56.328456+0000","last_clean":"2024-09-21T02:14:56.328456+0000","last_became_active":"2024-09-21T02:14:54.327622+0000","last_became_peered":"2024-09-21T02:14:54.327622+0000","last_unstale":"2024-09-21T02:14:56.328456+0000","last_undegraded":"2024-09-21T02:14:56.328456+0000","last_fullsized":"2024-09-21T02:14:56.328456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:34.074515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326445+0000","last_change":"2024-09-21T02:14:55.329200+0000","last_active":"2024-09-21T02:14:56.326445+0000","last_peered":"2024-09-21T02:14:56.326445+0000","last_clean":"2024-09-21T02:14:56.326445+0000","last_became_active":"2024-09-21T02:14:55.328846+0000","last_became_peered":"2024-09-21T02:14:55.328846+0000","last_unstale":"2024-09-21T02:14:56.326445+0000","last_undegraded":"2024-09-21T02:14:56.326445+0000","last_fullsized":"2024-09-21T02:14:56.326445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:25.460954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.680151+0000","last_change":"2024-09-21T02:14:54.337839+0000","last_active":"2024-09-21T02:14:56.680151+0000","last_peered":"2024-09-21T02:14:56.680151+0000","last_clean":"2024-09-21T02:14:56.680151+0000","last_became_active":"2024-09-21T02:14:54.337425+0000","last_became_peered":"2024-09-21T02:14:54.337425+0000","last_unstale":"2024-09-21T02:14:56.680151+0000","last_undegraded":"2024-09-21T02:14:56.680151+0000","last_fullsized":"2024-09-21T02:14:56.680151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:28.700032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.684178+0000","last_change":"2024-09-21T02:14:55.313512+0000","last_active":"2024-09-21T02:14:56.684178+0000","last_peered":"2024-09-21T02:14:56.684178+0000","last_clean":"2024-09-21T02:14:56.684178+0000","last_became_active":"2024-09-21T02:14:55.313169+0000","last_became_peered":"2024-09-21T02:14:55.313169+0000","last_unstale":"2024-09-21T02:14:56.684178+0000","last_undegraded":"2024-09-21T02:14:56.684178+0000","last_fullsized":"2024-09-21T02:14:56.684178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:08.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327598+0000","last_change":"2024-09-21T02:14:54.332066+0000","last_active":"2024-09-21T02:14:56.327598+0000","last_peered":"2024-09-21T02:14:56.327598+0000","last_clean":"2024-09-21T02:14:56.327598+0000","last_became_active":"2024-09-21T02:14:54.330849+0000","last_became_peered":"2024-09-21T02:14:54.330849+0000","last_unstale":"2024-09-21T02:14:56.327598+0000","last_undegraded":"2024-09-21T02:14:56.327598+0000","last_fullsized":"2024-09-21T02:14:56.327598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:15.226566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330281+0000","last_change":"2024-09-21T02:14:55.332453+0000","last_active":"2024-09-21T02:14:56.330281+0000","last_peered":"2024-09-21T02:14:56.330281+0000","last_clean":"2024-09-21T02:14:56.330281+0000","last_became_active":"2024-09-21T02:14:55.332018+0000","last_became_peered":"2024-09-21T02:14:55.332018+0000","last_unstale":"2024-09-21T02:14:56.330281+0000","last_undegraded":"2024-09-21T02:14:56.330281+0000","last_fullsized":"2024-09-21T02:14:56.330281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.636901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549022+0000","last_change":"2024-09-21T02:14:54.325141+0000","last_active":"2024-09-21T02:14:56.549022+0000","last_peered":"2024-09-21T02:14:56.549022+0000","last_clean":"2024-09-21T02:14:56.549022+0000","last_became_active":"2024-09-21T02:14:54.324868+0000","last_became_peered":"2024-09-21T02:14:54.324868+0000","last_unstale":"2024-09-21T02:14:56.549022+0000","last_undegraded":"2024-09-21T02:14:56.549022+0000","last_fullsized":"2024-09-21T02:14:56.549022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:35.160348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.684095+0000","last_change":"2024-09-21T02:14:55.339447+0000","last_active":"2024-09-21T02:14:56.684095+0000","last_peered":"2024-09-21T02:14:56.684095+0000","last_clean":"2024-09-21T02:14:56.684095+0000","last_became_active":"2024-09-21T02:14:55.339020+0000","last_became_peered":"2024-09-21T02:14:55.339020+0000","last_unstale":"2024-09-21T02:14:56.684095+0000","last_undegraded":"2024-09-21T02:14:56.684095+0000","last_fullsized":"2024-09-21T02:14:56.684095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:46.446804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682093+0000","last_change":"2024-09-21T02:14:54.337500+0000","last_active":"2024-09-21T02:14:56.682093+0000","last_peered":"2024-09-21T02:14:56.682093+0000","last_clean":"2024-09-21T02:14:56.682093+0000","last_became_active":"2024-09-21T02:14:54.336706+0000","last_became_peered":"2024-09-21T02:14:54.336706+0000","last_unstale":"2024-09-21T02:14:56.682093+0000","last_undegraded":"2024-09-21T02:14:56.682093+0000","last_fullsized":"2024-09-21T02:14:56.682093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:25.688888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328821+0000","last_change":"2024-09-21T02:14:55.334564+0000","last_active":"2024-09-21T02:14:56.328821+0000","last_peered":"2024-09-21T02:14:56.328821+0000","last_clean":"2024-09-21T02:14:56.328821+0000","last_became_active":"2024-09-21T02:14:55.333693+0000","last_became_peered":"2024-09-21T02:14:55.333693+0000","last_unstale":"2024-09-21T02:14:56.328821+0000","last_undegraded":"2024-09-21T02:14:56.328821+0000","last_fullsized":"2024-09-21T02:14:56.328821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.259862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549375+0000","last_change":"2024-09-21T02:14:54.327226+0000","last_active":"2024-09-21T02:14:56.549375+0000","last_peered":"2024-09-21T02:14:56.549375+0000","last_clean":"2024-09-21T02:14:56.549375+0000","last_became_active":"2024-09-21T02:14:54.326898+0000","last_became_peered":"2024-09-21T02:14:54.326898+0000","last_unstale":"2024-09-21T02:14:56.549375+0000","last_undegraded":"2024-09-21T02:14:56.549375+0000","last_fullsized":"2024-09-21T02:14:56.549375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:21.441939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326897+0000","last_change":"2024-09-21T02:14:55.320066+0000","last_active":"2024-09-21T02:14:56.326897+0000","last_peered":"2024-09-21T02:14:56.326897+0000","last_clean":"2024-09-21T02:14:56.326897+0000","last_became_active":"2024-09-21T02:14:55.319396+0000","last_became_peered":"2024-09-21T02:14:55.319396+0000","last_unstale":"2024-09-21T02:14:56.326897+0000","last_undegraded":"2024-09-21T02:14:56.326897+0000","last_fullsized":"2024-09-21T02:14:56.326897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:13.436234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681875+0000","last_change":"2024-09-21T02:14:54.320490+0000","last_active":"2024-09-21T02:14:56.681875+0000","last_peered":"2024-09-21T02:14:56.681875+0000","last_clean":"2024-09-21T02:14:56.681875+0000","last_became_active":"2024-09-21T02:14:54.319598+0000","last_became_peered":"2024-09-21T02:14:54.319598+0000","last_unstale":"2024-09-21T02:14:56.681875+0000","last_undegraded":"2024-09-21T02:14:56.681875+0000","last_fullsized":"2024-09-21T02:14:56.681875+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:47.859756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681383+0000","last_change":"2024-09-21T02:14:55.338009+0000","last_active":"2024-09-21T02:14:56.681383+0000","last_peered":"2024-09-21T02:14:56.681383+0000","last_clean":"2024-09-21T02:14:56.681383+0000","last_became_active":"2024-09-21T02:14:55.337706+0000","last_became_peered":"2024-09-21T02:14:55.337706+0000","last_unstale":"2024-09-21T02:14:56.681383+0000","last_undegraded":"2024-09-21T02:14:56.681383+0000","last_fullsized":"2024-09-21T02:14:56.681383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:33.085610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402470+0000","last_change":"2024-09-21T02:14:54.338746+0000","last_active":"2024-09-21T02:14:56.402470+0000","last_peered":"2024-09-21T02:14:56.402470+0000","last_clean":"2024-09-21T02:14:56.402470+0000","last_became_active":"2024-09-21T02:14:54.337797+0000","last_became_peered":"2024-09-21T02:14:54.337797+0000","last_unstale":"2024-09-21T02:14:56.402470+0000","last_undegraded":"2024-09-21T02:14:56.402470+0000","last_fullsized":"2024-09-21T02:14:56.402470+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:14.109286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326158+0000","last_change":"2024-09-21T02:14:55.319375+0000","last_active":"2024-09-21T02:14:56.326158+0000","last_peered":"2024-09-21T02:14:56.326158+0000","last_clean":"2024-09-21T02:14:56.326158+0000","last_became_active":"2024-09-21T02:14:55.318818+0000","last_became_peered":"2024-09-21T02:14:55.318818+0000","last_unstale":"2024-09-21T02:14:56.326158+0000","last_undegraded":"2024-09-21T02:14:56.326158+0000","last_fullsized":"2024-09-21T02:14:56.326158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:45.847619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681417+0000","last_change":"2024-09-21T02:14:54.339747+0000","last_active":"2024-09-21T02:14:56.681417+0000","last_peered":"2024-09-21T02:14:56.681417+0000","last_clean":"2024-09-21T02:14:56.681417+0000","last_became_active":"2024-09-21T02:14:54.338954+0000","last_became_peered":"2024-09-21T02:14:54.338954+0000","last_unstale":"2024-09-21T02:14:56.681417+0000","last_undegraded":"2024-09-21T02:14:56.681417+0000","last_fullsized":"2024-09-21T02:14:56.681417+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:05.874151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327402+0000","last_change":"2024-09-21T02:14:55.316820+0000","last_active":"2024-09-21T02:14:56.327402+0000","last_peered":"2024-09-21T02:14:56.327402+0000","last_clean":"2024-09-21T02:14:56.327402+0000","last_became_active":"2024-09-21T02:14:55.316430+0000","last_became_peered":"2024-09-21T02:14:55.316430+0000","last_unstale":"2024-09-21T02:14:56.327402+0000","last_undegraded":"2024-09-21T02:14:56.327402+0000","last_fullsized":"2024-09-21T02:14:56.327402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:19.039015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683715+0000","last_change":"2024-09-21T02:14:54.335375+0000","last_active":"2024-09-21T02:14:56.683715+0000","last_peered":"2024-09-21T02:14:56.683715+0000","last_clean":"2024-09-21T02:14:56.683715+0000","last_became_active":"2024-09-21T02:14:54.334757+0000","last_became_peered":"2024-09-21T02:14:54.334757+0000","last_unstale":"2024-09-21T02:14:56.683715+0000","last_undegraded":"2024-09-21T02:14:56.683715+0000","last_fullsized":"2024-09-21T02:14:56.683715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:44.207462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":152,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681081+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:56.681081+0000","last_peered":"2024-09-21T02:14:56.681081+0000","last_clean":"2024-09-21T02:14:56.681081+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:56.681081+0000","last_undegraded":"2024-09-21T02:14:56.681081+0000","last_fullsized":"2024-09-21T02:14:56.681081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":214237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":214819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1130496,"data_stored":804593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":215815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1130496,"data_stored":804641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":552960,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:15:04.185 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:15:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9ad448640 1 -- 172.21.15.170:0/1776074414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004d50 msgr2=0x7fa994005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9ad448640 1 --2- 172.21.15.170:0/1776074414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004d50 0x7fa994005160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80b5250 tx=0x7fa9a001f760 comp rx=0 tx=0).stop 2024-09-21T02:15:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9a77fe640 1 -- 172.21.15.170:0/1776074414 reap_dead start 2024-09-21T02:15:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9ad448640 1 -- 172.21.15.170:0/1776074414 shutdown_connections 2024-09-21T02:15:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9ad448640 1 -- 172.21.15.170:0/1776074414 >> 172.21.15.170:0/1776074414 conn(0x7fa9a80b1c40 msgr2=0x7fa9a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:04.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9ad448640 1 -- 172.21.15.170:0/1776074414 shutdown_connections 2024-09-21T02:15:04.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.350+0000 7fa9ad448640 1 -- 172.21.15.170:0/1776074414 wait complete. 2024-09-21T02:15:04.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 Processor -- start 2024-09-21T02:15:04.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 -- start start 2024-09-21T02:15:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a815c2e0 0x7fa9a815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8150e50 0x7fa9a8151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 0x7fa9a814f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80ce750 con 0x7fa9a81517a0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80ce5d0 con 0x7fa9a8150e50 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a80ce8d0 con 0x7fa9a815c2e0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 0x7fa9a814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 0x7fa9a814f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59880/0 (socket says 172.21.15.170:59880) 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 -- 172.21.15.170:0/2978378612 learned_addr learned my addr 172.21.15.170:0/2978378612 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a6ffd640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a815c2e0 0x7fa9a815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 -- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a815c2e0 msgr2=0x7fa9a815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a67fc640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8150e50 0x7fa9a8151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a815c2e0 0x7fa9a815c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 -- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8150e50 msgr2=0x7fa9a8151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8150e50 0x7fa9a8151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 -- 172.21.15.170:0/2978378612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a815d0a0 con 0x7fa9a81517a0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9a77fe640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 0x7fa9a814f100 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b2040 tx=0x7fa9a00a9f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00bc020 con 0x7fa9a81517a0 2024-09-21T02:15:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9a00a8630 con 0x7fa9a81517a0 2024-09-21T02:15:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a815d380 con 0x7fa9a81517a0 2024-09-21T02:15:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0009a90 con 0x7fa9a81517a0 2024-09-21T02:15:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.354+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a815d940 con 0x7fa9a81517a0 2024-09-21T02:15:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.358+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a80cee10 con 0x7fa9a81517a0 2024-09-21T02:15:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.358+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fa9a00bf070 con 0x7fa9a81517a0 2024-09-21T02:15:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.358+0000 7fa99f7fe640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9880733b0 0x7fa9880758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.358+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a012f470 con 0x7fa9a81517a0 2024-09-21T02:15:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.358+0000 7fa9a6ffd640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9880733b0 0x7fa9880758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.358+0000 7fa9a6ffd640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9880733b0 0x7fa9880758c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8155530 tx=0x7fa9980015a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:04.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.362+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9a00194b0 con 0x7fa9a81517a0 2024-09-21T02:15:04.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.530+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9a80b4160 con 0x7fa9880733b0 2024-09-21T02:15:04.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.542+0000 7fa99f7fe640 1 -- 172.21.15.170:0/2978378612 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fa9a80b4160 con 0x7fa9880733b0 2024-09-21T02:15:04.546 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:04.548 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9880733b0 msgr2=0x7fa9880758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9880733b0 0x7fa9880758c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8155530 tx=0x7fa9980015a0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 msgr2=0x7fa9a814f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 0x7fa9a814f100 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b2040 tx=0x7fa9a00a9f60 comp rx=0 tx=0).stop 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 shutdown_connections 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9880733b0 0x7fa9880758c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a81517a0 0x7fa9a814f100 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a8150e50 0x7fa9a8151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 --2- 172.21.15.170:0/2978378612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a815c2e0 0x7fa9a815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 >> 172.21.15.170:0/2978378612 conn(0x7fa9a80b1c40 msgr2=0x7fa9a80b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 shutdown_connections 2024-09-21T02:15:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.546+0000 7fa9ad448640 1 -- 172.21.15.170:0/2978378612 wait complete. 2024-09-21T02:15:04.566 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-21T02:15:02.900356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250776,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754723944,"statfs":{"total":773094113280,"available":772837318656,"internally_reserved":0,"allocated":5513216,"data_stored":2899180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003005"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.550009+0000","last_change":"2024-09-21T02:14:55.327257+0000","last_active":"2024-09-21T02:14:56.550009+0000","last_peered":"2024-09-21T02:14:56.550009+0000","last_clean":"2024-09-21T02:14:56.550009+0000","last_became_active":"2024-09-21T02:14:55.326386+0000","last_became_peered":"2024-09-21T02:14:55.326386+0000","last_unstale":"2024-09-21T02:14:56.550009+0000","last_undegraded":"2024-09-21T02:14:56.550009+0000","last_fullsized":"2024-09-21T02:14:56.550009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:45.910986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327156+0000","last_change":"2024-09-21T02:14:55.329573+0000","last_active":"2024-09-21T02:14:56.327156+0000","last_peered":"2024-09-21T02:14:56.327156+0000","last_clean":"2024-09-21T02:14:56.327156+0000","last_became_active":"2024-09-21T02:14:55.329010+0000","last_became_peered":"2024-09-21T02:14:55.329010+0000","last_unstale":"2024-09-21T02:14:56.327156+0000","last_undegraded":"2024-09-21T02:14:56.327156+0000","last_fullsized":"2024-09-21T02:14:56.327156+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:37.356974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682545+0000","last_change":"2024-09-21T02:14:55.342163+0000","last_active":"2024-09-21T02:14:56.682545+0000","last_peered":"2024-09-21T02:14:56.682545+0000","last_clean":"2024-09-21T02:14:56.682545+0000","last_became_active":"2024-09-21T02:14:55.341739+0000","last_became_peered":"2024-09-21T02:14:55.341739+0000","last_unstale":"2024-09-21T02:14:56.682545+0000","last_undegraded":"2024-09-21T02:14:56.682545+0000","last_fullsized":"2024-09-21T02:14:56.682545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.022851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327283+0000","last_change":"2024-09-21T02:14:55.325061+0000","last_active":"2024-09-21T02:14:56.327283+0000","last_peered":"2024-09-21T02:14:56.327283+0000","last_clean":"2024-09-21T02:14:56.327283+0000","last_became_active":"2024-09-21T02:14:55.324760+0000","last_became_peered":"2024-09-21T02:14:55.324760+0000","last_unstale":"2024-09-21T02:14:56.327283+0000","last_undegraded":"2024-09-21T02:14:56.327283+0000","last_fullsized":"2024-09-21T02:14:56.327283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:20.425059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549798+0000","last_change":"2024-09-21T02:14:55.323783+0000","last_active":"2024-09-21T02:14:56.549798+0000","last_peered":"2024-09-21T02:14:56.549798+0000","last_clean":"2024-09-21T02:14:56.549798+0000","last_became_active":"2024-09-21T02:14:55.323418+0000","last_became_peered":"2024-09-21T02:14:55.323418+0000","last_unstale":"2024-09-21T02:14:56.549798+0000","last_undegraded":"2024-09-21T02:14:56.549798+0000","last_fullsized":"2024-09-21T02:14:56.549798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:29.087991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403160+0000","last_change":"2024-09-21T02:14:55.318482+0000","last_active":"2024-09-21T02:14:56.403160+0000","last_peered":"2024-09-21T02:14:56.403160+0000","last_clean":"2024-09-21T02:14:56.403160+0000","last_became_active":"2024-09-21T02:14:55.318010+0000","last_became_peered":"2024-09-21T02:14:55.318010+0000","last_unstale":"2024-09-21T02:14:56.403160+0000","last_undegraded":"2024-09-21T02:14:56.403160+0000","last_fullsized":"2024-09-21T02:14:56.403160+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:53.409184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.680557+0000","last_change":"2024-09-21T02:14:55.335794+0000","last_active":"2024-09-21T02:14:56.680557+0000","last_peered":"2024-09-21T02:14:56.680557+0000","last_clean":"2024-09-21T02:14:56.680557+0000","last_became_active":"2024-09-21T02:14:55.335485+0000","last_became_peered":"2024-09-21T02:14:55.335485+0000","last_unstale":"2024-09-21T02:14:56.680557+0000","last_undegraded":"2024-09-21T02:14:56.680557+0000","last_fullsized":"2024-09-21T02:14:56.680557+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:59.416901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326632+0000","last_change":"2024-09-21T02:14:55.331996+0000","last_active":"2024-09-21T02:14:56.326632+0000","last_peered":"2024-09-21T02:14:56.326632+0000","last_clean":"2024-09-21T02:14:56.326632+0000","last_became_active":"2024-09-21T02:14:55.331640+0000","last_became_peered":"2024-09-21T02:14:55.331640+0000","last_unstale":"2024-09-21T02:14:56.326632+0000","last_undegraded":"2024-09-21T02:14:56.326632+0000","last_fullsized":"2024-09-21T02:14:56.326632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:22.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403439+0000","last_change":"2024-09-21T02:14:55.324028+0000","last_active":"2024-09-21T02:14:56.403439+0000","last_peered":"2024-09-21T02:14:56.403439+0000","last_clean":"2024-09-21T02:14:56.403439+0000","last_became_active":"2024-09-21T02:14:55.323726+0000","last_became_peered":"2024-09-21T02:14:55.323726+0000","last_unstale":"2024-09-21T02:14:56.403439+0000","last_undegraded":"2024-09-21T02:14:56.403439+0000","last_fullsized":"2024-09-21T02:14:56.403439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.809074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329777+0000","last_change":"2024-09-21T02:14:55.329046+0000","last_active":"2024-09-21T02:14:56.329777+0000","last_peered":"2024-09-21T02:14:56.329777+0000","last_clean":"2024-09-21T02:14:56.329777+0000","last_became_active":"2024-09-21T02:14:55.328689+0000","last_became_peered":"2024-09-21T02:14:55.328689+0000","last_unstale":"2024-09-21T02:14:56.329777+0000","last_undegraded":"2024-09-21T02:14:56.329777+0000","last_fullsized":"2024-09-21T02:14:56.329777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:50.567278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328970+0000","last_change":"2024-09-21T02:14:55.327536+0000","last_active":"2024-09-21T02:14:56.328970+0000","last_peered":"2024-09-21T02:14:56.328970+0000","last_clean":"2024-09-21T02:14:56.328970+0000","last_became_active":"2024-09-21T02:14:55.327010+0000","last_became_peered":"2024-09-21T02:14:55.327010+0000","last_unstale":"2024-09-21T02:14:56.328970+0000","last_undegraded":"2024-09-21T02:14:56.328970+0000","last_fullsized":"2024-09-21T02:14:56.328970+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:18.272703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326399+0000","last_change":"2024-09-21T02:14:55.318421+0000","last_active":"2024-09-21T02:14:56.326399+0000","last_peered":"2024-09-21T02:14:56.326399+0000","last_clean":"2024-09-21T02:14:56.326399+0000","last_became_active":"2024-09-21T02:14:55.317551+0000","last_became_peered":"2024-09-21T02:14:55.317551+0000","last_unstale":"2024-09-21T02:14:56.326399+0000","last_undegraded":"2024-09-21T02:14:56.326399+0000","last_fullsized":"2024-09-21T02:14:56.326399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:00.680972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402663+0000","last_change":"2024-09-21T02:14:55.319633+0000","last_active":"2024-09-21T02:14:56.402663+0000","last_peered":"2024-09-21T02:14:56.402663+0000","last_clean":"2024-09-21T02:14:56.402663+0000","last_became_active":"2024-09-21T02:14:55.318890+0000","last_became_peered":"2024-09-21T02:14:55.318890+0000","last_unstale":"2024-09-21T02:14:56.402663+0000","last_undegraded":"2024-09-21T02:14:56.402663+0000","last_fullsized":"2024-09-21T02:14:56.402663+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:53.437794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328681+0000","last_change":"2024-09-21T02:14:55.334999+0000","last_active":"2024-09-21T02:14:56.328681+0000","last_peered":"2024-09-21T02:14:56.328681+0000","last_clean":"2024-09-21T02:14:56.328681+0000","last_became_active":"2024-09-21T02:14:55.334258+0000","last_became_peered":"2024-09-21T02:14:55.334258+0000","last_unstale":"2024-09-21T02:14:56.328681+0000","last_undegraded":"2024-09-21T02:14:56.328681+0000","last_fullsized":"2024-09-21T02:14:56.328681+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:48.439070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326754+0000","last_change":"2024-09-21T02:14:55.325404+0000","last_active":"2024-09-21T02:14:56.326754+0000","last_peered":"2024-09-21T02:14:56.326754+0000","last_clean":"2024-09-21T02:14:56.326754+0000","last_became_active":"2024-09-21T02:14:55.324421+0000","last_became_peered":"2024-09-21T02:14:55.324421+0000","last_unstale":"2024-09-21T02:14:56.326754+0000","last_undegraded":"2024-09-21T02:14:56.326754+0000","last_fullsized":"2024-09-21T02:14:56.326754+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:29.710578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548631+0000","last_change":"2024-09-21T02:14:55.339049+0000","last_active":"2024-09-21T02:14:56.548631+0000","last_peered":"2024-09-21T02:14:56.548631+0000","last_clean":"2024-09-21T02:14:56.548631+0000","last_became_active":"2024-09-21T02:14:55.338619+0000","last_became_peered":"2024-09-21T02:14:55.338619+0000","last_unstale":"2024-09-21T02:14:56.548631+0000","last_undegraded":"2024-09-21T02:14:56.548631+0000","last_fullsized":"2024-09-21T02:14:56.548631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:37.356974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327520+0000","last_change":"2024-09-21T02:14:55.321172+0000","last_active":"2024-09-21T02:14:56.327520+0000","last_peered":"2024-09-21T02:14:56.327520+0000","last_clean":"2024-09-21T02:14:56.327520+0000","last_became_active":"2024-09-21T02:14:55.313203+0000","last_became_peered":"2024-09-21T02:14:55.313203+0000","last_unstale":"2024-09-21T02:14:56.327520+0000","last_undegraded":"2024-09-21T02:14:56.327520+0000","last_fullsized":"2024-09-21T02:14:56.327520+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:14.197505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682821+0000","last_change":"2024-09-21T02:14:55.327950+0000","last_active":"2024-09-21T02:14:56.682821+0000","last_peered":"2024-09-21T02:14:56.682821+0000","last_clean":"2024-09-21T02:14:56.682821+0000","last_became_active":"2024-09-21T02:14:55.327317+0000","last_became_peered":"2024-09-21T02:14:55.327317+0000","last_unstale":"2024-09-21T02:14:56.682821+0000","last_undegraded":"2024-09-21T02:14:56.682821+0000","last_fullsized":"2024-09-21T02:14:56.682821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:23.054432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549367+0000","last_change":"2024-09-21T02:14:55.331527+0000","last_active":"2024-09-21T02:14:56.549367+0000","last_peered":"2024-09-21T02:14:56.549367+0000","last_clean":"2024-09-21T02:14:56.549367+0000","last_became_active":"2024-09-21T02:14:55.331060+0000","last_became_peered":"2024-09-21T02:14:55.331060+0000","last_unstale":"2024-09-21T02:14:56.549367+0000","last_undegraded":"2024-09-21T02:14:56.549367+0000","last_fullsized":"2024-09-21T02:14:56.549367+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:15.077374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329725+0000","last_change":"2024-09-21T02:14:55.333330+0000","last_active":"2024-09-21T02:14:56.329725+0000","last_peered":"2024-09-21T02:14:56.329725+0000","last_clean":"2024-09-21T02:14:56.329725+0000","last_became_active":"2024-09-21T02:14:55.332877+0000","last_became_peered":"2024-09-21T02:14:55.332877+0000","last_unstale":"2024-09-21T02:14:56.329725+0000","last_undegraded":"2024-09-21T02:14:56.329725+0000","last_fullsized":"2024-09-21T02:14:56.329725+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:06.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327439+0000","last_change":"2024-09-21T02:14:55.325693+0000","last_active":"2024-09-21T02:14:56.327439+0000","last_peered":"2024-09-21T02:14:56.327439+0000","last_clean":"2024-09-21T02:14:56.327439+0000","last_became_active":"2024-09-21T02:14:55.319881+0000","last_became_peered":"2024-09-21T02:14:55.319881+0000","last_unstale":"2024-09-21T02:14:56.327439+0000","last_undegraded":"2024-09-21T02:14:56.327439+0000","last_fullsized":"2024-09-21T02:14:56.327439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:27.562478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681908+0000","last_change":"2024-09-21T02:14:55.323917+0000","last_active":"2024-09-21T02:14:56.681908+0000","last_peered":"2024-09-21T02:14:56.681908+0000","last_clean":"2024-09-21T02:14:56.681908+0000","last_became_active":"2024-09-21T02:14:55.323342+0000","last_became_peered":"2024-09-21T02:14:55.323342+0000","last_unstale":"2024-09-21T02:14:56.681908+0000","last_undegraded":"2024-09-21T02:14:56.681908+0000","last_fullsized":"2024-09-21T02:14:56.681908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:37.385178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681110+0000","last_change":"2024-09-21T02:14:55.331910+0000","last_active":"2024-09-21T02:14:56.681110+0000","last_peered":"2024-09-21T02:14:56.681110+0000","last_clean":"2024-09-21T02:14:56.681110+0000","last_became_active":"2024-09-21T02:14:55.331595+0000","last_became_peered":"2024-09-21T02:14:55.331595+0000","last_unstale":"2024-09-21T02:14:56.681110+0000","last_undegraded":"2024-09-21T02:14:56.681110+0000","last_fullsized":"2024-09-21T02:14:56.681110+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:09.352297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.325977+0000","last_change":"2024-09-21T02:14:55.326958+0000","last_active":"2024-09-21T02:14:56.325977+0000","last_peered":"2024-09-21T02:14:56.325977+0000","last_clean":"2024-09-21T02:14:56.325977+0000","last_became_active":"2024-09-21T02:14:55.326606+0000","last_became_peered":"2024-09-21T02:14:55.326606+0000","last_unstale":"2024-09-21T02:14:56.325977+0000","last_undegraded":"2024-09-21T02:14:56.325977+0000","last_fullsized":"2024-09-21T02:14:56.325977+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:51.003999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.550355+0000","last_change":"2024-09-21T02:14:55.326778+0000","last_active":"2024-09-21T02:14:56.550355+0000","last_peered":"2024-09-21T02:14:56.550355+0000","last_clean":"2024-09-21T02:14:56.550355+0000","last_became_active":"2024-09-21T02:14:55.325840+0000","last_became_peered":"2024-09-21T02:14:55.325840+0000","last_unstale":"2024-09-21T02:14:56.550355+0000","last_undegraded":"2024-09-21T02:14:56.550355+0000","last_fullsized":"2024-09-21T02:14:56.550355+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:04.861616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327867+0000","last_change":"2024-09-21T02:14:55.316584+0000","last_active":"2024-09-21T02:14:56.327867+0000","last_peered":"2024-09-21T02:14:56.327867+0000","last_clean":"2024-09-21T02:14:56.327867+0000","last_became_active":"2024-09-21T02:14:55.316082+0000","last_became_peered":"2024-09-21T02:14:55.316082+0000","last_unstale":"2024-09-21T02:14:56.327867+0000","last_undegraded":"2024-09-21T02:14:56.327867+0000","last_fullsized":"2024-09-21T02:14:56.327867+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:22.367502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683490+0000","last_change":"2024-09-21T02:14:55.320184+0000","last_active":"2024-09-21T02:14:56.683490+0000","last_peered":"2024-09-21T02:14:56.683490+0000","last_clean":"2024-09-21T02:14:56.683490+0000","last_became_active":"2024-09-21T02:14:55.319858+0000","last_became_peered":"2024-09-21T02:14:55.319858+0000","last_unstale":"2024-09-21T02:14:56.683490+0000","last_undegraded":"2024-09-21T02:14:56.683490+0000","last_fullsized":"2024-09-21T02:14:56.683490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:44.385122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682126+0000","last_change":"2024-09-21T02:14:55.320086+0000","last_active":"2024-09-21T02:14:56.682126+0000","last_peered":"2024-09-21T02:14:56.682126+0000","last_clean":"2024-09-21T02:14:56.682126+0000","last_became_active":"2024-09-21T02:14:55.319752+0000","last_became_peered":"2024-09-21T02:14:55.319752+0000","last_unstale":"2024-09-21T02:14:56.682126+0000","last_undegraded":"2024-09-21T02:14:56.682126+0000","last_fullsized":"2024-09-21T02:14:56.682126+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:58.154497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549206+0000","last_change":"2024-09-21T02:14:55.339772+0000","last_active":"2024-09-21T02:14:56.549206+0000","last_peered":"2024-09-21T02:14:56.549206+0000","last_clean":"2024-09-21T02:14:56.549206+0000","last_became_active":"2024-09-21T02:14:55.339235+0000","last_became_peered":"2024-09-21T02:14:55.339235+0000","last_unstale":"2024-09-21T02:14:56.549206+0000","last_undegraded":"2024-09-21T02:14:56.549206+0000","last_fullsized":"2024-09-21T02:14:56.549206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:14.197505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326467+0000","last_change":"2024-09-21T02:14:55.320525+0000","last_active":"2024-09-21T02:14:56.326467+0000","last_peered":"2024-09-21T02:14:56.326467+0000","last_clean":"2024-09-21T02:14:56.326467+0000","last_became_active":"2024-09-21T02:14:55.320074+0000","last_became_peered":"2024-09-21T02:14:55.320074+0000","last_unstale":"2024-09-21T02:14:56.326467+0000","last_undegraded":"2024-09-21T02:14:56.326467+0000","last_fullsized":"2024-09-21T02:14:56.326467+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:54.734955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402271+0000","last_change":"2024-09-21T02:14:55.332430+0000","last_active":"2024-09-21T02:14:56.402271+0000","last_peered":"2024-09-21T02:14:56.402271+0000","last_clean":"2024-09-21T02:14:56.402271+0000","last_became_active":"2024-09-21T02:14:55.332005+0000","last_became_peered":"2024-09-21T02:14:55.332005+0000","last_unstale":"2024-09-21T02:14:56.402271+0000","last_undegraded":"2024-09-21T02:14:56.402271+0000","last_fullsized":"2024-09-21T02:14:56.402271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:53.575464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326582+0000","last_change":"2024-09-21T02:14:55.327609+0000","last_active":"2024-09-21T02:14:56.326582+0000","last_peered":"2024-09-21T02:14:56.326582+0000","last_clean":"2024-09-21T02:14:56.326582+0000","last_became_active":"2024-09-21T02:14:55.327089+0000","last_became_peered":"2024-09-21T02:14:55.327089+0000","last_unstale":"2024-09-21T02:14:56.326582+0000","last_undegraded":"2024-09-21T02:14:56.326582+0000","last_fullsized":"2024-09-21T02:14:56.326582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:17.740610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682075+0000","last_change":"2024-09-21T02:14:55.327561+0000","last_active":"2024-09-21T02:14:56.682075+0000","last_peered":"2024-09-21T02:14:56.682075+0000","last_clean":"2024-09-21T02:14:56.682075+0000","last_became_active":"2024-09-21T02:14:55.327119+0000","last_became_peered":"2024-09-21T02:14:55.327119+0000","last_unstale":"2024-09-21T02:14:56.682075+0000","last_undegraded":"2024-09-21T02:14:56.682075+0000","last_fullsized":"2024-09-21T02:14:56.682075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:20.856161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403547+0000","last_change":"2024-09-21T02:14:54.339187+0000","last_active":"2024-09-21T02:14:56.403547+0000","last_peered":"2024-09-21T02:14:56.403547+0000","last_clean":"2024-09-21T02:14:56.403547+0000","last_became_active":"2024-09-21T02:14:54.338577+0000","last_became_peered":"2024-09-21T02:14:54.338577+0000","last_unstale":"2024-09-21T02:14:56.403547+0000","last_undegraded":"2024-09-21T02:14:56.403547+0000","last_fullsized":"2024-09-21T02:14:56.403547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:46.922448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.550338+0000","last_change":"2024-09-21T02:14:55.312579+0000","last_active":"2024-09-21T02:14:56.550338+0000","last_peered":"2024-09-21T02:14:56.550338+0000","last_clean":"2024-09-21T02:14:56.550338+0000","last_became_active":"2024-09-21T02:14:55.312172+0000","last_became_peered":"2024-09-21T02:14:55.312172+0000","last_unstale":"2024-09-21T02:14:56.550338+0000","last_undegraded":"2024-09-21T02:14:56.550338+0000","last_fullsized":"2024-09-21T02:14:56.550338+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:22.129192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683647+0000","last_change":"2024-09-21T02:14:54.326175+0000","last_active":"2024-09-21T02:14:56.683647+0000","last_peered":"2024-09-21T02:14:56.683647+0000","last_clean":"2024-09-21T02:14:56.683647+0000","last_became_active":"2024-09-21T02:14:54.324027+0000","last_became_peered":"2024-09-21T02:14:54.324027+0000","last_unstale":"2024-09-21T02:14:56.683647+0000","last_undegraded":"2024-09-21T02:14:56.683647+0000","last_fullsized":"2024-09-21T02:14:56.683647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:21.118647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326680+0000","last_change":"2024-09-21T02:14:55.334182+0000","last_active":"2024-09-21T02:14:56.326680+0000","last_peered":"2024-09-21T02:14:56.326680+0000","last_clean":"2024-09-21T02:14:56.326680+0000","last_became_active":"2024-09-21T02:14:55.333902+0000","last_became_peered":"2024-09-21T02:14:55.333902+0000","last_unstale":"2024-09-21T02:14:56.326680+0000","last_undegraded":"2024-09-21T02:14:56.326680+0000","last_fullsized":"2024-09-21T02:14:56.326680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:19.457723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402973+0000","last_change":"2024-09-21T02:14:54.339702+0000","last_active":"2024-09-21T02:14:56.402973+0000","last_peered":"2024-09-21T02:14:56.402973+0000","last_clean":"2024-09-21T02:14:56.402973+0000","last_became_active":"2024-09-21T02:14:54.338812+0000","last_became_peered":"2024-09-21T02:14:54.338812+0000","last_unstale":"2024-09-21T02:14:56.402973+0000","last_undegraded":"2024-09-21T02:14:56.402973+0000","last_fullsized":"2024-09-21T02:14:56.402973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:01.599009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329586+0000","last_change":"2024-09-21T02:14:55.327662+0000","last_active":"2024-09-21T02:14:56.329586+0000","last_peered":"2024-09-21T02:14:56.329586+0000","last_clean":"2024-09-21T02:14:56.329586+0000","last_became_active":"2024-09-21T02:14:55.326518+0000","last_became_peered":"2024-09-21T02:14:55.326518+0000","last_unstale":"2024-09-21T02:14:56.329586+0000","last_undegraded":"2024-09-21T02:14:56.329586+0000","last_fullsized":"2024-09-21T02:14:56.329586+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:10.446600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682502+0000","last_change":"2024-09-21T02:14:54.326589+0000","last_active":"2024-09-21T02:14:56.682502+0000","last_peered":"2024-09-21T02:14:56.682502+0000","last_clean":"2024-09-21T02:14:56.682502+0000","last_became_active":"2024-09-21T02:14:54.325839+0000","last_became_peered":"2024-09-21T02:14:54.325839+0000","last_unstale":"2024-09-21T02:14:56.682502+0000","last_undegraded":"2024-09-21T02:14:56.682502+0000","last_fullsized":"2024-09-21T02:14:56.682502+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:19.414513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681388+0000","last_change":"2024-09-21T02:14:55.325001+0000","last_active":"2024-09-21T02:14:56.681388+0000","last_peered":"2024-09-21T02:14:56.681388+0000","last_clean":"2024-09-21T02:14:56.681388+0000","last_became_active":"2024-09-21T02:14:55.324690+0000","last_became_peered":"2024-09-21T02:14:55.324690+0000","last_unstale":"2024-09-21T02:14:56.681388+0000","last_undegraded":"2024-09-21T02:14:56.681388+0000","last_fullsized":"2024-09-21T02:14:56.681388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:37.953461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403047+0000","last_change":"2024-09-21T02:14:54.340918+0000","last_active":"2024-09-21T02:14:56.403047+0000","last_peered":"2024-09-21T02:14:56.403047+0000","last_clean":"2024-09-21T02:14:56.403047+0000","last_became_active":"2024-09-21T02:14:54.339864+0000","last_became_peered":"2024-09-21T02:14:54.339864+0000","last_unstale":"2024-09-21T02:14:56.403047+0000","last_undegraded":"2024-09-21T02:14:56.403047+0000","last_fullsized":"2024-09-21T02:14:56.403047+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:22.209018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329912+0000","last_change":"2024-09-21T02:14:55.339871+0000","last_active":"2024-09-21T02:14:56.329912+0000","last_peered":"2024-09-21T02:14:56.329912+0000","last_clean":"2024-09-21T02:14:56.329912+0000","last_became_active":"2024-09-21T02:14:55.339582+0000","last_became_peered":"2024-09-21T02:14:55.339582+0000","last_unstale":"2024-09-21T02:14:56.329912+0000","last_undegraded":"2024-09-21T02:14:56.329912+0000","last_fullsized":"2024-09-21T02:14:56.329912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:12.830203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681564+0000","last_change":"2024-09-21T02:14:54.312459+0000","last_active":"2024-09-21T02:14:56.681564+0000","last_peered":"2024-09-21T02:14:56.681564+0000","last_clean":"2024-09-21T02:14:56.681564+0000","last_became_active":"2024-09-21T02:14:54.312050+0000","last_became_peered":"2024-09-21T02:14:54.312050+0000","last_unstale":"2024-09-21T02:14:56.681564+0000","last_undegraded":"2024-09-21T02:14:56.681564+0000","last_fullsized":"2024-09-21T02:14:56.681564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:18.028470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328781+0000","last_change":"2024-09-21T02:14:55.336714+0000","last_active":"2024-09-21T02:14:56.328781+0000","last_peered":"2024-09-21T02:14:56.328781+0000","last_clean":"2024-09-21T02:14:56.328781+0000","last_became_active":"2024-09-21T02:14:55.336144+0000","last_became_peered":"2024-09-21T02:14:55.336144+0000","last_unstale":"2024-09-21T02:14:56.328781+0000","last_undegraded":"2024-09-21T02:14:56.328781+0000","last_fullsized":"2024-09-21T02:14:56.328781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:34.313582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548710+0000","last_change":"2024-09-21T02:14:54.323694+0000","last_active":"2024-09-21T02:14:56.548710+0000","last_peered":"2024-09-21T02:14:56.548710+0000","last_clean":"2024-09-21T02:14:56.548710+0000","last_became_active":"2024-09-21T02:14:54.323375+0000","last_became_peered":"2024-09-21T02:14:54.323375+0000","last_unstale":"2024-09-21T02:14:56.548710+0000","last_undegraded":"2024-09-21T02:14:56.548710+0000","last_fullsized":"2024-09-21T02:14:56.548710+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:36.113978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.680993+0000","last_change":"2024-09-21T02:14:55.314477+0000","last_active":"2024-09-21T02:14:56.680993+0000","last_peered":"2024-09-21T02:14:56.680993+0000","last_clean":"2024-09-21T02:14:56.680993+0000","last_became_active":"2024-09-21T02:14:55.314176+0000","last_became_peered":"2024-09-21T02:14:55.314176+0000","last_unstale":"2024-09-21T02:14:56.680993+0000","last_undegraded":"2024-09-21T02:14:56.680993+0000","last_fullsized":"2024-09-21T02:14:56.680993+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:44.961823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329124+0000","last_change":"2024-09-21T02:14:54.329228+0000","last_active":"2024-09-21T02:14:56.329124+0000","last_peered":"2024-09-21T02:14:56.329124+0000","last_clean":"2024-09-21T02:14:56.329124+0000","last_became_active":"2024-09-21T02:14:54.328430+0000","last_became_peered":"2024-09-21T02:14:54.328430+0000","last_unstale":"2024-09-21T02:14:56.329124+0000","last_undegraded":"2024-09-21T02:14:56.329124+0000","last_fullsized":"2024-09-21T02:14:56.329124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:46.117802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683396+0000","last_change":"2024-09-21T02:14:55.330082+0000","last_active":"2024-09-21T02:14:56.683396+0000","last_peered":"2024-09-21T02:14:56.683396+0000","last_clean":"2024-09-21T02:14:56.683396+0000","last_became_active":"2024-09-21T02:14:55.329669+0000","last_became_peered":"2024-09-21T02:14:55.329669+0000","last_unstale":"2024-09-21T02:14:56.683396+0000","last_undegraded":"2024-09-21T02:14:56.683396+0000","last_fullsized":"2024-09-21T02:14:56.683396+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:50.391928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682998+0000","last_change":"2024-09-21T02:14:54.337122+0000","last_active":"2024-09-21T02:14:56.682998+0000","last_peered":"2024-09-21T02:14:56.682998+0000","last_clean":"2024-09-21T02:14:56.682998+0000","last_became_active":"2024-09-21T02:14:54.336774+0000","last_became_peered":"2024-09-21T02:14:54.336774+0000","last_unstale":"2024-09-21T02:14:56.682998+0000","last_undegraded":"2024-09-21T02:14:56.682998+0000","last_fullsized":"2024-09-21T02:14:56.682998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:32.826706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683401+0000","last_change":"2024-09-21T02:14:55.332894+0000","last_active":"2024-09-21T02:14:56.683401+0000","last_peered":"2024-09-21T02:14:56.683401+0000","last_clean":"2024-09-21T02:14:56.683401+0000","last_became_active":"2024-09-21T02:14:55.332483+0000","last_became_peered":"2024-09-21T02:14:55.332483+0000","last_unstale":"2024-09-21T02:14:56.683401+0000","last_undegraded":"2024-09-21T02:14:56.683401+0000","last_fullsized":"2024-09-21T02:14:56.683401+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:48.301871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329961+0000","last_change":"2024-09-21T02:14:54.329034+0000","last_active":"2024-09-21T02:14:56.329961+0000","last_peered":"2024-09-21T02:14:56.329961+0000","last_clean":"2024-09-21T02:14:56.329961+0000","last_became_active":"2024-09-21T02:14:54.327414+0000","last_became_peered":"2024-09-21T02:14:54.327414+0000","last_unstale":"2024-09-21T02:14:56.329961+0000","last_undegraded":"2024-09-21T02:14:56.329961+0000","last_fullsized":"2024-09-21T02:14:56.329961+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:29.369340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327551+0000","last_change":"2024-09-21T02:14:55.334653+0000","last_active":"2024-09-21T02:14:56.327551+0000","last_peered":"2024-09-21T02:14:56.327551+0000","last_clean":"2024-09-21T02:14:56.327551+0000","last_became_active":"2024-09-21T02:14:55.334280+0000","last_became_peered":"2024-09-21T02:14:55.334280+0000","last_unstale":"2024-09-21T02:14:56.327551+0000","last_undegraded":"2024-09-21T02:14:56.327551+0000","last_fullsized":"2024-09-21T02:14:56.327551+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:19.127968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.325827+0000","last_change":"2024-09-21T02:14:54.316458+0000","last_active":"2024-09-21T02:14:56.325827+0000","last_peered":"2024-09-21T02:14:56.325827+0000","last_clean":"2024-09-21T02:14:56.325827+0000","last_became_active":"2024-09-21T02:14:54.315604+0000","last_became_peered":"2024-09-21T02:14:54.315604+0000","last_unstale":"2024-09-21T02:14:56.325827+0000","last_undegraded":"2024-09-21T02:14:56.325827+0000","last_fullsized":"2024-09-21T02:14:56.325827+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:05.699325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683595+0000","last_change":"2024-09-21T02:14:55.339134+0000","last_active":"2024-09-21T02:14:56.683595+0000","last_peered":"2024-09-21T02:14:56.683595+0000","last_clean":"2024-09-21T02:14:56.683595+0000","last_became_active":"2024-09-21T02:14:55.338786+0000","last_became_peered":"2024-09-21T02:14:55.338786+0000","last_unstale":"2024-09-21T02:14:56.683595+0000","last_undegraded":"2024-09-21T02:14:56.683595+0000","last_fullsized":"2024-09-21T02:14:56.683595+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:24.396896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330095+0000","last_change":"2024-09-21T02:14:54.319009+0000","last_active":"2024-09-21T02:14:56.330095+0000","last_peered":"2024-09-21T02:14:56.330095+0000","last_clean":"2024-09-21T02:14:56.330095+0000","last_became_active":"2024-09-21T02:14:54.317796+0000","last_became_peered":"2024-09-21T02:14:54.317796+0000","last_unstale":"2024-09-21T02:14:56.330095+0000","last_undegraded":"2024-09-21T02:14:56.330095+0000","last_fullsized":"2024-09-21T02:14:56.330095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:16.231157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682424+0000","last_change":"2024-09-21T02:14:55.328400+0000","last_active":"2024-09-21T02:14:56.682424+0000","last_peered":"2024-09-21T02:14:56.682424+0000","last_clean":"2024-09-21T02:14:56.682424+0000","last_became_active":"2024-09-21T02:14:55.327810+0000","last_became_peered":"2024-09-21T02:14:55.327810+0000","last_unstale":"2024-09-21T02:14:56.682424+0000","last_undegraded":"2024-09-21T02:14:56.682424+0000","last_fullsized":"2024-09-21T02:14:56.682424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.604823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681481+0000","last_change":"2024-09-21T02:14:54.343831+0000","last_active":"2024-09-21T02:14:56.681481+0000","last_peered":"2024-09-21T02:14:56.681481+0000","last_clean":"2024-09-21T02:14:56.681481+0000","last_became_active":"2024-09-21T02:14:54.342819+0000","last_became_peered":"2024-09-21T02:14:54.342819+0000","last_unstale":"2024-09-21T02:14:56.681481+0000","last_undegraded":"2024-09-21T02:14:56.681481+0000","last_fullsized":"2024-09-21T02:14:56.681481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:51.247144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681950+0000","last_change":"2024-09-21T02:14:55.320585+0000","last_active":"2024-09-21T02:14:56.681950+0000","last_peered":"2024-09-21T02:14:56.681950+0000","last_clean":"2024-09-21T02:14:56.681950+0000","last_became_active":"2024-09-21T02:14:55.320194+0000","last_became_peered":"2024-09-21T02:14:55.320194+0000","last_unstale":"2024-09-21T02:14:56.681950+0000","last_undegraded":"2024-09-21T02:14:56.681950+0000","last_fullsized":"2024-09-21T02:14:56.681950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:00.906337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548402+0000","last_change":"2024-09-21T02:14:54.333776+0000","last_active":"2024-09-21T02:14:56.548402+0000","last_peered":"2024-09-21T02:14:56.548402+0000","last_clean":"2024-09-21T02:14:56.548402+0000","last_became_active":"2024-09-21T02:14:54.333330+0000","last_became_peered":"2024-09-21T02:14:54.333330+0000","last_unstale":"2024-09-21T02:14:56.548402+0000","last_undegraded":"2024-09-21T02:14:56.548402+0000","last_fullsized":"2024-09-21T02:14:56.548402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:56.898580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681657+0000","last_change":"2024-09-21T02:14:55.333564+0000","last_active":"2024-09-21T02:14:56.681657+0000","last_peered":"2024-09-21T02:14:56.681657+0000","last_clean":"2024-09-21T02:14:56.681657+0000","last_became_active":"2024-09-21T02:14:55.333161+0000","last_became_peered":"2024-09-21T02:14:55.333161+0000","last_unstale":"2024-09-21T02:14:56.681657+0000","last_undegraded":"2024-09-21T02:14:56.681657+0000","last_fullsized":"2024-09-21T02:14:56.681657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:28.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402147+0000","last_change":"2024-09-21T02:14:54.336941+0000","last_active":"2024-09-21T02:14:56.402147+0000","last_peered":"2024-09-21T02:14:56.402147+0000","last_clean":"2024-09-21T02:14:56.402147+0000","last_became_active":"2024-09-21T02:14:54.336121+0000","last_became_peered":"2024-09-21T02:14:54.336121+0000","last_unstale":"2024-09-21T02:14:56.402147+0000","last_undegraded":"2024-09-21T02:14:56.402147+0000","last_fullsized":"2024-09-21T02:14:56.402147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:45.583899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.325727+0000","last_change":"2024-09-21T02:14:55.331826+0000","last_active":"2024-09-21T02:14:56.325727+0000","last_peered":"2024-09-21T02:14:56.325727+0000","last_clean":"2024-09-21T02:14:56.325727+0000","last_became_active":"2024-09-21T02:14:55.331500+0000","last_became_peered":"2024-09-21T02:14:55.331500+0000","last_unstale":"2024-09-21T02:14:56.325727+0000","last_undegraded":"2024-09-21T02:14:56.325727+0000","last_fullsized":"2024-09-21T02:14:56.325727+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:08.349497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.329396+0000","last_change":"2024-09-21T02:14:54.319185+0000","last_active":"2024-09-21T02:14:56.329396+0000","last_peered":"2024-09-21T02:14:56.329396+0000","last_clean":"2024-09-21T02:14:56.329396+0000","last_became_active":"2024-09-21T02:14:54.318313+0000","last_became_peered":"2024-09-21T02:14:54.318313+0000","last_unstale":"2024-09-21T02:14:56.329396+0000","last_undegraded":"2024-09-21T02:14:56.329396+0000","last_fullsized":"2024-09-21T02:14:56.329396+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:36:08.133172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682379+0000","last_change":"2024-09-21T02:14:55.322130+0000","last_active":"2024-09-21T02:14:56.682379+0000","last_peered":"2024-09-21T02:14:56.682379+0000","last_clean":"2024-09-21T02:14:56.682379+0000","last_became_active":"2024-09-21T02:14:55.321649+0000","last_became_peered":"2024-09-21T02:14:55.321649+0000","last_unstale":"2024-09-21T02:14:56.682379+0000","last_undegraded":"2024-09-21T02:14:56.682379+0000","last_fullsized":"2024-09-21T02:14:56.682379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:54.430744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327057+0000","last_change":"2024-09-21T02:14:54.343765+0000","last_active":"2024-09-21T02:14:56.327057+0000","last_peered":"2024-09-21T02:14:56.327057+0000","last_clean":"2024-09-21T02:14:56.327057+0000","last_became_active":"2024-09-21T02:14:54.343116+0000","last_became_peered":"2024-09-21T02:14:54.343116+0000","last_unstale":"2024-09-21T02:14:56.327057+0000","last_undegraded":"2024-09-21T02:14:56.327057+0000","last_fullsized":"2024-09-21T02:14:56.327057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:07.045536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682594+0000","last_change":"2024-09-21T02:14:55.327468+0000","last_active":"2024-09-21T02:14:56.682594+0000","last_peered":"2024-09-21T02:14:56.682594+0000","last_clean":"2024-09-21T02:14:56.682594+0000","last_became_active":"2024-09-21T02:14:55.327131+0000","last_became_peered":"2024-09-21T02:14:55.327131+0000","last_unstale":"2024-09-21T02:14:56.682594+0000","last_undegraded":"2024-09-21T02:14:56.682594+0000","last_fullsized":"2024-09-21T02:14:56.682594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:37.144365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683234+0000","last_change":"2024-09-21T02:14:54.334494+0000","last_active":"2024-09-21T02:14:56.683234+0000","last_peered":"2024-09-21T02:14:56.683234+0000","last_clean":"2024-09-21T02:14:56.683234+0000","last_became_active":"2024-09-21T02:14:54.333728+0000","last_became_peered":"2024-09-21T02:14:54.333728+0000","last_unstale":"2024-09-21T02:14:56.683234+0000","last_undegraded":"2024-09-21T02:14:56.683234+0000","last_fullsized":"2024-09-21T02:14:56.683234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:52.902706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326797+0000","last_change":"2024-09-21T02:14:55.317502+0000","last_active":"2024-09-21T02:14:56.326797+0000","last_peered":"2024-09-21T02:14:56.326797+0000","last_clean":"2024-09-21T02:14:56.326797+0000","last_became_active":"2024-09-21T02:14:55.317159+0000","last_became_peered":"2024-09-21T02:14:55.317159+0000","last_unstale":"2024-09-21T02:14:56.326797+0000","last_undegraded":"2024-09-21T02:14:56.326797+0000","last_fullsized":"2024-09-21T02:14:56.326797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:54.075840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682748+0000","last_change":"2024-09-21T02:14:54.322488+0000","last_active":"2024-09-21T02:14:56.682748+0000","last_peered":"2024-09-21T02:14:56.682748+0000","last_clean":"2024-09-21T02:14:56.682748+0000","last_became_active":"2024-09-21T02:14:54.322002+0000","last_became_peered":"2024-09-21T02:14:54.322002+0000","last_unstale":"2024-09-21T02:14:56.682748+0000","last_undegraded":"2024-09-21T02:14:56.682748+0000","last_fullsized":"2024-09-21T02:14:56.682748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:26.099198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682684+0000","last_change":"2024-09-21T02:14:55.341864+0000","last_active":"2024-09-21T02:14:56.682684+0000","last_peered":"2024-09-21T02:14:56.682684+0000","last_clean":"2024-09-21T02:14:56.682684+0000","last_became_active":"2024-09-21T02:14:55.341135+0000","last_became_peered":"2024-09-21T02:14:55.341135+0000","last_unstale":"2024-09-21T02:14:56.682684+0000","last_undegraded":"2024-09-21T02:14:56.682684+0000","last_fullsized":"2024-09-21T02:14:56.682684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:00.801062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402369+0000","last_change":"2024-09-21T02:14:54.332963+0000","last_active":"2024-09-21T02:14:56.402369+0000","last_peered":"2024-09-21T02:14:56.402369+0000","last_clean":"2024-09-21T02:14:56.402369+0000","last_became_active":"2024-09-21T02:14:54.332128+0000","last_became_peered":"2024-09-21T02:14:54.332128+0000","last_unstale":"2024-09-21T02:14:56.402369+0000","last_undegraded":"2024-09-21T02:14:56.402369+0000","last_fullsized":"2024-09-21T02:14:56.402369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.123890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328752+0000","last_change":"2024-09-21T02:14:55.328993+0000","last_active":"2024-09-21T02:14:56.328752+0000","last_peered":"2024-09-21T02:14:56.328752+0000","last_clean":"2024-09-21T02:14:56.328752+0000","last_became_active":"2024-09-21T02:14:55.328635+0000","last_became_peered":"2024-09-21T02:14:55.328635+0000","last_unstale":"2024-09-21T02:14:56.328752+0000","last_undegraded":"2024-09-21T02:14:56.328752+0000","last_fullsized":"2024-09-21T02:14:56.328752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:09.392936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328949+0000","last_change":"2024-09-21T02:14:54.323086+0000","last_active":"2024-09-21T02:14:56.328949+0000","last_peered":"2024-09-21T02:14:56.328949+0000","last_clean":"2024-09-21T02:14:56.328949+0000","last_became_active":"2024-09-21T02:14:54.321950+0000","last_became_peered":"2024-09-21T02:14:54.321950+0000","last_unstale":"2024-09-21T02:14:56.328949+0000","last_undegraded":"2024-09-21T02:14:56.328949+0000","last_fullsized":"2024-09-21T02:14:56.328949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:23.939679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.403037+0000","last_change":"2024-09-21T02:14:55.315473+0000","last_active":"2024-09-21T02:14:56.403037+0000","last_peered":"2024-09-21T02:14:56.403037+0000","last_clean":"2024-09-21T02:14:56.403037+0000","last_became_active":"2024-09-21T02:14:55.314644+0000","last_became_peered":"2024-09-21T02:14:55.314644+0000","last_unstale":"2024-09-21T02:14:56.403037+0000","last_undegraded":"2024-09-21T02:14:56.403037+0000","last_fullsized":"2024-09-21T02:14:56.403037+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:38.236709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.548573+0000","last_change":"2024-09-21T02:14:54.327284+0000","last_active":"2024-09-21T02:14:56.548573+0000","last_peered":"2024-09-21T02:14:56.548573+0000","last_clean":"2024-09-21T02:14:56.548573+0000","last_became_active":"2024-09-21T02:14:54.326598+0000","last_became_peered":"2024-09-21T02:14:54.326598+0000","last_unstale":"2024-09-21T02:14:56.548573+0000","last_undegraded":"2024-09-21T02:14:56.548573+0000","last_fullsized":"2024-09-21T02:14:56.548573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:36.997240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326891+0000","last_change":"2024-09-21T02:14:55.327865+0000","last_active":"2024-09-21T02:14:56.326891+0000","last_peered":"2024-09-21T02:14:56.326891+0000","last_clean":"2024-09-21T02:14:56.326891+0000","last_became_active":"2024-09-21T02:14:55.327136+0000","last_became_peered":"2024-09-21T02:14:55.327136+0000","last_unstale":"2024-09-21T02:14:56.326891+0000","last_undegraded":"2024-09-21T02:14:56.326891+0000","last_fullsized":"2024-09-21T02:14:56.326891+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:22.129192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328456+0000","last_change":"2024-09-21T02:14:54.328647+0000","last_active":"2024-09-21T02:14:56.328456+0000","last_peered":"2024-09-21T02:14:56.328456+0000","last_clean":"2024-09-21T02:14:56.328456+0000","last_became_active":"2024-09-21T02:14:54.327622+0000","last_became_peered":"2024-09-21T02:14:54.327622+0000","last_unstale":"2024-09-21T02:14:56.328456+0000","last_undegraded":"2024-09-21T02:14:56.328456+0000","last_fullsized":"2024-09-21T02:14:56.328456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:34.074515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326445+0000","last_change":"2024-09-21T02:14:55.329200+0000","last_active":"2024-09-21T02:14:56.326445+0000","last_peered":"2024-09-21T02:14:56.326445+0000","last_clean":"2024-09-21T02:14:56.326445+0000","last_became_active":"2024-09-21T02:14:55.328846+0000","last_became_peered":"2024-09-21T02:14:55.328846+0000","last_unstale":"2024-09-21T02:14:56.326445+0000","last_undegraded":"2024-09-21T02:14:56.326445+0000","last_fullsized":"2024-09-21T02:14:56.326445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:25.460954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.680151+0000","last_change":"2024-09-21T02:14:54.337839+0000","last_active":"2024-09-21T02:14:56.680151+0000","last_peered":"2024-09-21T02:14:56.680151+0000","last_clean":"2024-09-21T02:14:56.680151+0000","last_became_active":"2024-09-21T02:14:54.337425+0000","last_became_peered":"2024-09-21T02:14:54.337425+0000","last_unstale":"2024-09-21T02:14:56.680151+0000","last_undegraded":"2024-09-21T02:14:56.680151+0000","last_fullsized":"2024-09-21T02:14:56.680151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:28.700032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.684178+0000","last_change":"2024-09-21T02:14:55.313512+0000","last_active":"2024-09-21T02:14:56.684178+0000","last_peered":"2024-09-21T02:14:56.684178+0000","last_clean":"2024-09-21T02:14:56.684178+0000","last_became_active":"2024-09-21T02:14:55.313169+0000","last_became_peered":"2024-09-21T02:14:55.313169+0000","last_unstale":"2024-09-21T02:14:56.684178+0000","last_undegraded":"2024-09-21T02:14:56.684178+0000","last_fullsized":"2024-09-21T02:14:56.684178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:08.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327598+0000","last_change":"2024-09-21T02:14:54.332066+0000","last_active":"2024-09-21T02:14:56.327598+0000","last_peered":"2024-09-21T02:14:56.327598+0000","last_clean":"2024-09-21T02:14:56.327598+0000","last_became_active":"2024-09-21T02:14:54.330849+0000","last_became_peered":"2024-09-21T02:14:54.330849+0000","last_unstale":"2024-09-21T02:14:56.327598+0000","last_undegraded":"2024-09-21T02:14:56.327598+0000","last_fullsized":"2024-09-21T02:14:56.327598+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:15.226566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.330281+0000","last_change":"2024-09-21T02:14:55.332453+0000","last_active":"2024-09-21T02:14:56.330281+0000","last_peered":"2024-09-21T02:14:56.330281+0000","last_clean":"2024-09-21T02:14:56.330281+0000","last_became_active":"2024-09-21T02:14:55.332018+0000","last_became_peered":"2024-09-21T02:14:55.332018+0000","last_unstale":"2024-09-21T02:14:56.330281+0000","last_undegraded":"2024-09-21T02:14:56.330281+0000","last_fullsized":"2024-09-21T02:14:56.330281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:23.636901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549022+0000","last_change":"2024-09-21T02:14:54.325141+0000","last_active":"2024-09-21T02:14:56.549022+0000","last_peered":"2024-09-21T02:14:56.549022+0000","last_clean":"2024-09-21T02:14:56.549022+0000","last_became_active":"2024-09-21T02:14:54.324868+0000","last_became_peered":"2024-09-21T02:14:54.324868+0000","last_unstale":"2024-09-21T02:14:56.549022+0000","last_undegraded":"2024-09-21T02:14:56.549022+0000","last_fullsized":"2024-09-21T02:14:56.549022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:35.160348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.684095+0000","last_change":"2024-09-21T02:14:55.339447+0000","last_active":"2024-09-21T02:14:56.684095+0000","last_peered":"2024-09-21T02:14:56.684095+0000","last_clean":"2024-09-21T02:14:56.684095+0000","last_became_active":"2024-09-21T02:14:55.339020+0000","last_became_peered":"2024-09-21T02:14:55.339020+0000","last_unstale":"2024-09-21T02:14:56.684095+0000","last_undegraded":"2024-09-21T02:14:56.684095+0000","last_fullsized":"2024-09-21T02:14:56.684095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:46.446804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.682093+0000","last_change":"2024-09-21T02:14:54.337500+0000","last_active":"2024-09-21T02:14:56.682093+0000","last_peered":"2024-09-21T02:14:56.682093+0000","last_clean":"2024-09-21T02:14:56.682093+0000","last_became_active":"2024-09-21T02:14:54.336706+0000","last_became_peered":"2024-09-21T02:14:54.336706+0000","last_unstale":"2024-09-21T02:14:56.682093+0000","last_undegraded":"2024-09-21T02:14:56.682093+0000","last_fullsized":"2024-09-21T02:14:56.682093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:25.688888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.328821+0000","last_change":"2024-09-21T02:14:55.334564+0000","last_active":"2024-09-21T02:14:56.328821+0000","last_peered":"2024-09-21T02:14:56.328821+0000","last_clean":"2024-09-21T02:14:56.328821+0000","last_became_active":"2024-09-21T02:14:55.333693+0000","last_became_peered":"2024-09-21T02:14:55.333693+0000","last_unstale":"2024-09-21T02:14:56.328821+0000","last_undegraded":"2024-09-21T02:14:56.328821+0000","last_fullsized":"2024-09-21T02:14:56.328821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.259862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.549375+0000","last_change":"2024-09-21T02:14:54.327226+0000","last_active":"2024-09-21T02:14:56.549375+0000","last_peered":"2024-09-21T02:14:56.549375+0000","last_clean":"2024-09-21T02:14:56.549375+0000","last_became_active":"2024-09-21T02:14:54.326898+0000","last_became_peered":"2024-09-21T02:14:54.326898+0000","last_unstale":"2024-09-21T02:14:56.549375+0000","last_undegraded":"2024-09-21T02:14:56.549375+0000","last_fullsized":"2024-09-21T02:14:56.549375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:21.441939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326897+0000","last_change":"2024-09-21T02:14:55.320066+0000","last_active":"2024-09-21T02:14:56.326897+0000","last_peered":"2024-09-21T02:14:56.326897+0000","last_clean":"2024-09-21T02:14:56.326897+0000","last_became_active":"2024-09-21T02:14:55.319396+0000","last_became_peered":"2024-09-21T02:14:55.319396+0000","last_unstale":"2024-09-21T02:14:56.326897+0000","last_undegraded":"2024-09-21T02:14:56.326897+0000","last_fullsized":"2024-09-21T02:14:56.326897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:13.436234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681875+0000","last_change":"2024-09-21T02:14:54.320490+0000","last_active":"2024-09-21T02:14:56.681875+0000","last_peered":"2024-09-21T02:14:56.681875+0000","last_clean":"2024-09-21T02:14:56.681875+0000","last_became_active":"2024-09-21T02:14:54.319598+0000","last_became_peered":"2024-09-21T02:14:54.319598+0000","last_unstale":"2024-09-21T02:14:56.681875+0000","last_undegraded":"2024-09-21T02:14:56.681875+0000","last_fullsized":"2024-09-21T02:14:56.681875+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:47.859756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681383+0000","last_change":"2024-09-21T02:14:55.338009+0000","last_active":"2024-09-21T02:14:56.681383+0000","last_peered":"2024-09-21T02:14:56.681383+0000","last_clean":"2024-09-21T02:14:56.681383+0000","last_became_active":"2024-09-21T02:14:55.337706+0000","last_became_peered":"2024-09-21T02:14:55.337706+0000","last_unstale":"2024-09-21T02:14:56.681383+0000","last_undegraded":"2024-09-21T02:14:56.681383+0000","last_fullsized":"2024-09-21T02:14:56.681383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:33.085610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.402470+0000","last_change":"2024-09-21T02:14:54.338746+0000","last_active":"2024-09-21T02:14:56.402470+0000","last_peered":"2024-09-21T02:14:56.402470+0000","last_clean":"2024-09-21T02:14:56.402470+0000","last_became_active":"2024-09-21T02:14:54.337797+0000","last_became_peered":"2024-09-21T02:14:54.337797+0000","last_unstale":"2024-09-21T02:14:56.402470+0000","last_undegraded":"2024-09-21T02:14:56.402470+0000","last_fullsized":"2024-09-21T02:14:56.402470+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:14.109286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.326158+0000","last_change":"2024-09-21T02:14:55.319375+0000","last_active":"2024-09-21T02:14:56.326158+0000","last_peered":"2024-09-21T02:14:56.326158+0000","last_clean":"2024-09-21T02:14:56.326158+0000","last_became_active":"2024-09-21T02:14:55.318818+0000","last_became_peered":"2024-09-21T02:14:55.318818+0000","last_unstale":"2024-09-21T02:14:56.326158+0000","last_undegraded":"2024-09-21T02:14:56.326158+0000","last_fullsized":"2024-09-21T02:14:56.326158+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:45.847619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681417+0000","last_change":"2024-09-21T02:14:54.339747+0000","last_active":"2024-09-21T02:14:56.681417+0000","last_peered":"2024-09-21T02:14:56.681417+0000","last_clean":"2024-09-21T02:14:56.681417+0000","last_became_active":"2024-09-21T02:14:54.338954+0000","last_became_peered":"2024-09-21T02:14:54.338954+0000","last_unstale":"2024-09-21T02:14:56.681417+0000","last_undegraded":"2024-09-21T02:14:56.681417+0000","last_fullsized":"2024-09-21T02:14:56.681417+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:05.874151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.327402+0000","last_change":"2024-09-21T02:14:55.316820+0000","last_active":"2024-09-21T02:14:56.327402+0000","last_peered":"2024-09-21T02:14:56.327402+0000","last_clean":"2024-09-21T02:14:56.327402+0000","last_became_active":"2024-09-21T02:14:55.316430+0000","last_became_peered":"2024-09-21T02:14:55.316430+0000","last_unstale":"2024-09-21T02:14:56.327402+0000","last_undegraded":"2024-09-21T02:14:56.327402+0000","last_fullsized":"2024-09-21T02:14:56.327402+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:54.282109+0000","last_clean_scrub_stamp":"2024-09-21T02:14:54.282109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:19.039015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.683715+0000","last_change":"2024-09-21T02:14:54.335375+0000","last_active":"2024-09-21T02:14:56.683715+0000","last_peered":"2024-09-21T02:14:56.683715+0000","last_clean":"2024-09-21T02:14:56.683715+0000","last_became_active":"2024-09-21T02:14:54.334757+0000","last_became_peered":"2024-09-21T02:14:54.334757+0000","last_unstale":"2024-09-21T02:14:56.683715+0000","last_undegraded":"2024-09-21T02:14:56.683715+0000","last_fullsized":"2024-09-21T02:14:56.683715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:14:53.271564+0000","last_clean_scrub_stamp":"2024-09-21T02:14:53.271564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:44.207462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":152,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T02:14:56.681081+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:14:56.681081+0000","last_peered":"2024-09-21T02:14:56.681081+0000","last_clean":"2024-09-21T02:14:56.681081+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:14:56.681081+0000","last_undegraded":"2024-09-21T02:14:56.681081+0000","last_fullsized":"2024-09-21T02:14:56.681081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":214237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":214819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1130496,"data_stored":804593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":215815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":214293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36456,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335384,"statfs":{"total":96636764160,"available":96599433216,"internally_reserved":0,"allocated":1130496,"data_stored":804641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":214875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":552960,"data_stored":215907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:15:04.570 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:15:04.570 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:15:04.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a1036d640 1 -- 172.21.15.170:0/2665341617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69f0004810 msgr2=0x7f69f0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a1036d640 1 --2- 172.21.15.170:0/2665341617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69f0004810 0x7f69f0004c20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00ac70 tx=0x7f69fc01e0d0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a0e90a640 1 -- 172.21.15.170:0/2665341617 reap_dead start 2024-09-21T02:15:04.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a1036d640 1 -- 172.21.15.170:0/2665341617 shutdown_connections 2024-09-21T02:15:04.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a1036d640 1 -- 172.21.15.170:0/2665341617 >> 172.21.15.170:0/2665341617 conn(0x7f6a080d0170 msgr2=0x7f6a080d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:04.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a1036d640 1 -- 172.21.15.170:0/2665341617 shutdown_connections 2024-09-21T02:15:04.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.774+0000 7f6a1036d640 1 -- 172.21.15.170:0/2665341617 wait complete. 2024-09-21T02:15:04.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 Processor -- start 2024-09-21T02:15:04.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- start start 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 0x7f6a0809bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a0809c080 0x7f6a080c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a080c9740 0x7f6a081f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08073d50 con 0x7f6a080c68f0 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08073bd0 con 0x7f6a080c9740 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6a08073ed0 con 0x7f6a0809c080 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 0x7f6a0809bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 0x7f6a0809bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59900/0 (socket says 172.21.15.170:59900) 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 -- 172.21.15.170:0/1539880285 learned_addr learned my addr 172.21.15.170:0/1539880285 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 -- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a0809c080 msgr2=0x7f6a080c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a0809c080 0x7f6a080c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e90a640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a080c9740 0x7f6a081f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 -- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a080c9740 msgr2=0x7f6a081f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a080c9740 0x7f6a081f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 -- 172.21.15.170:0/1539880285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a081f7260 con 0x7f6a080c68f0 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a0e109640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 0x7f6a0809bb40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f69f800fbd0 tx=0x7f69f800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8011640 con 0x7f6a080c68f0 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f80117e0 con 0x7f6a080c68f0 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a080c8440 con 0x7f6a080c68f0 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8011c30 con 0x7f6a080c68f0 2024-09-21T02:15:04.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a080c8a00 con 0x7f6a080c68f0 2024-09-21T02:15:04.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.778+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a08074430 con 0x7f6a080c68f0 2024-09-21T02:15:04.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.782+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f69f8024440 con 0x7f6a080c68f0 2024-09-21T02:15:04.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.782+0000 7f6a06ffd640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f69f40733b0 0x7f69f40758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:04.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.782+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f69f8095450 con 0x7f6a080c68f0 2024-09-21T02:15:04.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.782+0000 7f6a0d908640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f69f40733b0 0x7f69f40758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:04.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.782+0000 7f6a0d908640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f69f40733b0 0x7f69f40758c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00ac70 tx=0x7f69f0005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:04.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:04.786+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f69f8060200 con 0x7f6a080c68f0 2024-09-21T02:15:05.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.006+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6a0806dec0 con 0x7f6a080c68f0 2024-09-21T02:15:05.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.010+0000 7f6a06ffd640 1 -- 172.21.15.170:0/1539880285 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f69f805fb00 con 0x7f6a080c68f0 2024-09-21T02:15:05.014 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:15:05.014 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:15:05.014 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:15:05.014 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:15:05.014 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:15:05.015 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:15:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f69f40733b0 msgr2=0x7f69f40758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f69f40733b0 0x7f69f40758c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00ac70 tx=0x7f69f0005840 comp rx=0 tx=0).stop 2024-09-21T02:15:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 msgr2=0x7f6a0809bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 0x7f6a0809bb40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f69f800fbd0 tx=0x7f69f800d6b0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 shutdown_connections 2024-09-21T02:15:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f69f40733b0 0x7f69f40758c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a0809c080 0x7f6a080c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a080c9740 0x7f6a081f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 --2- 172.21.15.170:0/1539880285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a080c68f0 0x7f6a0809bb40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 >> 172.21.15.170:0/1539880285 conn(0x7f6a080d0170 msgr2=0x7f6a080b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 shutdown_connections 2024-09-21T02:15:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.014+0000 7f6a1036d640 1 -- 172.21.15.170:0/1539880285 wait complete. 2024-09-21T02:15:05.036 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:05.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 -- 172.21.15.170:0/3085435549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f80cea00 msgr2=0x7f87f80cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 --2- 172.21.15.170:0/3085435549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f80cea00 0x7f87f80cadf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f87f80b2d00 tx=0x7f87f001cd50 comp rx=0 tx=0).stop 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 -- 172.21.15.170:0/3085435549 shutdown_connections 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 --2- 172.21.15.170:0/3085435549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f80cea00 0x7f87f80cadf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 --2- 172.21.15.170:0/3085435549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f8070570 0x7f87f80ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 --2- 172.21.15.170:0/3085435549 >> v2:172.21.15.191:3300/0 conn(0x7f87f815f990 0x7f87f8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 -- 172.21.15.170:0/3085435549 >> 172.21.15.170:0/3085435549 conn(0x7f87f80b1c40 msgr2=0x7f87f80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 -- 172.21.15.170:0/3085435549 shutdown_connections 2024-09-21T02:15:05.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.174+0000 7f87fd242640 1 -- 172.21.15.170:0/3085435549 wait complete. 2024-09-21T02:15:05.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 Processor -- start 2024-09-21T02:15:05.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- start start 2024-09-21T02:15:05.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 0x7f87f81548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:05.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f80cea00 0x7f87f80b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:05.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f815f990 0x7f87f80b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87f80732d0 con 0x7f87f8070570 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8073150 con 0x7f87f815f990 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f87f8073450 con 0x7f87f80cea00 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 0x7f87f81548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 0x7f87f81548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59912/0 (socket says 172.21.15.170:59912) 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 -- 172.21.15.170:0/3394145802 learned_addr learned my addr 172.21.15.170:0/3394145802 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 -- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f80cea00 msgr2=0x7f87f80b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f80cea00 0x7f87f80b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 -- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f815f990 msgr2=0x7f87f80b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f815f990 0x7f87f80b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 -- 172.21.15.170:0/3394145802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f8158140 con 0x7f87f8070570 2024-09-21T02:15:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87f6d9d640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 0x7f87f81548e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0058d0 tx=0x7f87ec00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec017070 con 0x7f87f8070570 2024-09-21T02:15:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ec009d20 con 0x7f87f8070570 2024-09-21T02:15:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0106c0 con 0x7f87f8070570 2024-09-21T02:15:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f809a9c0 con 0x7f87f8070570 2024-09-21T02:15:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f809af80 con 0x7f87f8070570 2024-09-21T02:15:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.178+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f80cadf0 con 0x7f87f8070570 2024-09-21T02:15:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.182+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f87ec01a070 con 0x7f87f8070570 2024-09-21T02:15:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.182+0000 7f87df7fe640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87e40733b0 0x7f87e40758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.182+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f87ec094ca0 con 0x7f87f8070570 2024-09-21T02:15:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.182+0000 7f87f659c640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87e40733b0 0x7f87e40758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:05.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.182+0000 7f87f659c640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87e40733b0 0x7f87e40758c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f87f80b23a0 tx=0x7f87e8005b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.186+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f87ec05f910 con 0x7f87f8070570 2024-09-21T02:15:05.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.398+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87f80d0ee0 con 0x7f87f8070570 2024-09-21T02:15:05.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.398+0000 7f87df7fe640 1 -- 172.21.15.170:0/3394145802 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5235 (secure 0 0 0) 0x7f87ec05f210 con 0x7f87f8070570 2024-09-21T02:15:05.401 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:05.401 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":27,"btime":"2024-09-21T02:14:58:331015+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:05.402 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 27 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87e40733b0 msgr2=0x7f87e40758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87e40733b0 0x7f87e40758c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f87f80b23a0 tx=0x7f87e8005b70 comp rx=0 tx=0).stop 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 msgr2=0x7f87f81548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 0x7f87f81548e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0058d0 tx=0x7f87ec00dde0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 shutdown_connections 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87e40733b0 0x7f87e40758c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f80cea00 0x7f87f80b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f815f990 0x7f87f80b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 --2- 172.21.15.170:0/3394145802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8070570 0x7f87f81548e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 >> 172.21.15.170:0/3394145802 conn(0x7f87f80b1c40 msgr2=0x7f87f8159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:05.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 shutdown_connections 2024-09-21T02:15:05.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.402+0000 7f87fd242640 1 -- 172.21.15.170:0/3394145802 wait complete. 2024-09-21T02:15:05.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:14:54.692519+0000', 'modified': '2024-09-21T02:14:57.742100+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4984}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4984': {'gid': 4984, 'name': 'b', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6837/2965172270', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 2965172270}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 2965172270}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4984, 'qdb_cluster': [4984]} 2024-09-21T02:15:05.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:15:05.826 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 -- 172.21.15.170:0/1327126197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c004d50 msgr2=0x7f286c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 --2- 172.21.15.170:0/1327126197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f286c004d50 0x7f286c0051c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f28840cdc20 tx=0x7f287c01d2c0 comp rx=0 tx=0).stop 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2882d9d640 1 -- 172.21.15.170:0/1327126197 reap_dead start 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 -- 172.21.15.170:0/1327126197 shutdown_connections 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 -- 172.21.15.170:0/1327126197 >> 172.21.15.170:0/1327126197 conn(0x7f28840b1c40 msgr2=0x7f28840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 -- 172.21.15.170:0/1327126197 shutdown_connections 2024-09-21T02:15:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 -- 172.21.15.170:0/1327126197 wait complete. 2024-09-21T02:15:05.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 Processor -- start 2024-09-21T02:15:05.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.994+0000 7f2888d37640 1 -- start start 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288414e380 0x7f288415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f288415cdb0 0x7f28841f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 0x7f28841e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28840cb500 con 0x7f28841f3fd0 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2884074f00 con 0x7f288414e380 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f28840cb680 con 0x7f288415cdb0 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 0x7f28841e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 0x7f28841e6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57640/0 (socket says 172.21.15.170:57640) 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 -- 172.21.15.170:0/3983120357 learned_addr learned my addr 172.21.15.170:0/3983120357 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f288259c640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288414e380 0x7f288415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2881d9b640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f288415cdb0 0x7f28841f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 -- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f288415cdb0 msgr2=0x7f28841f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f288415cdb0 0x7f28841f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 -- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288414e380 msgr2=0x7f288415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288414e380 0x7f288415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 -- 172.21.15.170:0/3983120357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28841f4540 con 0x7f28841f3fd0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f288259c640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288414e380 0x7f288415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2882d9d640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 0x7f28841e6ec0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f287c01cd80 tx=0x7f287c01d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2872ffd640 1 -- 172.21.15.170:0/3983120357 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c003b90 con 0x7f28841f3fd0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2872ffd640 1 -- 172.21.15.170:0/3983120357 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f287c003d30 con 0x7f28841f3fd0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28841f4820 con 0x7f28841f3fd0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2872ffd640 1 -- 172.21.15.170:0/3983120357 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0062d0 con 0x7f28841f3fd0 2024-09-21T02:15:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288415b7a0 con 0x7f28841f3fd0 2024-09-21T02:15:06.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2872ffd640 1 -- 172.21.15.170:0/3983120357 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f287c006470 con 0x7f28841f3fd0 2024-09-21T02:15:06.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2872ffd640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28640733b0 0x7f28640758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:05.998+0000 7f2872ffd640 1 -- 172.21.15.170:0/3983120357 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5854+0+0 (secure 0 0 0) 0x7f287c0b5020 con 0x7f28841f3fd0 2024-09-21T02:15:06.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28640733b0 0x7f28640758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 Processor -- start 2024-09-21T02:15:06.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- start start 2024-09-21T02:15:06.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28640733b0 0x7f28640758c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28740057b0 tx=0x7f2874005f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d23e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d23ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d23f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c16d113410 con 0x55c16ccfd460 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c16d050b40 con 0x55c16d0d6ca0 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55c16d03ff70 con 0x55c16d0ed950 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d23e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d23e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57654/0 (socket says 172.21.15.170:57654) 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 -- 172.21.15.170:0/2309853439 learned_addr learned my addr 172.21.15.170:0/2309853439 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:06.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 -- 172.21.15.170:0/2309853439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 msgr2=0x55c16d23ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d23ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 -- 172.21.15.170:0/2309853439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 msgr2=0x55c16d23f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d23f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 -- 172.21.15.170:0/2309853439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c16d0075b0 con 0x55c16ccfd460 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f288259c640 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d23e6a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f287400bfd0 tx=0x7f2874012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff419948da6a8211 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2863fff640 1 -- 172.21.15.170:0/2309853439 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874022070 con 0x55c16ccfd460 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2863fff640 1 -- 172.21.15.170:0/2309853439 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2874011e50 con 0x55c16ccfd460 2024-09-21T02:15:06.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2863fff640 1 -- 172.21.15.170:0/2309853439 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287401e960 con 0x55c16ccfd460 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- 172.21.15.170:0/2309853439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 msgr2=0x55c16d23e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d23e6a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f287400bfd0 tx=0x7f2874012000 comp rx=0 tx=0).stop 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- 172.21.15.170:0/2309853439 shutdown_connections 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d23ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d23f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 --2- 172.21.15.170:0/2309853439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d23e6a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- 172.21.15.170:0/2309853439 >> 172.21.15.170:0/2309853439 conn(0x55c16d1c3d90 msgr2=0x55c16d1f3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- 172.21.15.170:0/2309853439 shutdown_connections 2024-09-21T02:15:06.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.002+0000 7f2890002000 1 -- 172.21.15.170:0/2309853439 wait complete. 2024-09-21T02:15:06.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- start start 2024-09-21T02:15:06.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 Processor -- start 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16ccfd460 0x55c16d245920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d245e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 0x55c16d2463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c16cfda5f0 con 0x55c16d0ed950 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c16d043d80 con 0x55c16d0d6ca0 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55c16cfe6520 con 0x55c16ccfd460 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 10 client.0 _ll_register_callbacks cb 0x55c16cfea830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 0x55c16d2463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 0x55c16d2463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57662/0 (socket says 172.21.15.170:57662) 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 -- 172.21.15.170:0/1475840934 learned_addr learned my addr 172.21.15.170:0/1475840934 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f288259c640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16ccfd460 0x55c16d245920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2881d9b640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d245e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 -- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16ccfd460 msgr2=0x55c16d245920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16ccfd460 0x55c16d245920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 -- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 msgr2=0x55c16d245e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d245e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 -- 172.21.15.170:0/1475840934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16cfde190 con 0x55c16d0ed950 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2882d9d640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 0x55c16d2463a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f287c01d7a0 tx=0x7f287c0f8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad329eaef7e0e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f28617fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f28617fa640 1 -- 172.21.15.170:0/1475840934 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0bf040 con 0x55c16d0ed950 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f28617fa640 1 -- 172.21.15.170:0/1475840934 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5854+0+0 (secure 0 0 0) 0x7f287c0f8570 con 0x55c16d0ed950 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c16cfd8110 con 0x55c16d0ed950 2024-09-21T02:15:06.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f28617fa640 1 -- 172.21.15.170:0/1475840934 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f287c0f9a10 con 0x55c16d0ed950 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 10 client.5529 fetch_fsmap learned FSMap version 27 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55c16d0c4f10 con 0x55c16d0ed950 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f28617fa640 1 -- 172.21.15.170:0/1475840934 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c16cfd8110 con 0x55c16d0ed950 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f28617fa640 1 -- 172.21.15.170:0/1475840934 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f287c100020 con 0x55c16d0ed950 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 10 client.5529 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 10 client.5529 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 20 client.5529 populate_metadata read hostname 'smithi170' 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16cce27d0 0x55c16d24fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 4 client.5529 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 --> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c16d052a90 con 0x55c16cce27d0 2024-09-21T02:15:06.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.006+0000 7f288259c640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16cce27d0 0x55c16d24fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f28617fa640 10 client.5529 ms_handle_connect on v2:172.21.15.191:6836/2965172270 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f288259c640 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16cce27d0 0x55c16d24fd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b998acfb46a97ec4 server_cookie=eee6fd1f424fcf26 in_seq=0 out_seq=0 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f28617fa640 1 -- 172.21.15.170:0/1475840934 <== mds.0 v2:172.21.15.191:6836/2965172270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c16d052a90 con 0x55c16cce27d0 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f28617fa640 10 client.5529 handle_command_reply: tid=0 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 client.5529 shutdown 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 msgr2=0x55c16d2463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 0x55c16d2463a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f287c01d7a0 tx=0x7f287c0f8c60 comp rx=0 tx=0).stop 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 shutdown_connections 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16cce27d0 0x55c16d24fd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0ed950 0x55c16d2463a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0d6ca0 0x55c16d245e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f28617fa640 0 client.5529 ms_handle_reset on v2:172.21.15.191:6836/2965172270 2024-09-21T02:15:06.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 --2- 172.21.15.170:0/1475840934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16ccfd460 0x55c16d245920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 >> 172.21.15.170:0/1475840934 conn(0x55c16d1c3d90 msgr2=0x55c16d1fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 shutdown_connections 2024-09-21T02:15:06.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 1 -- 172.21.15.170:0/1475840934 wait complete. 2024-09-21T02:15:06.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.010+0000 7f2890002000 20 client.5529 trim_cache size 0 max 16384 2024-09-21T02:15:06.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 Processor -- start 2024-09-21T02:15:06.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- start start 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d2b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0d6ca0 0x55c16d2b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0ed950 0x55c16d2b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2884200c50 con 0x55c16ccfd460 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f28841ffcb0 con 0x55c16d0ed950 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f288415e4f0 con 0x55c16d0d6ca0 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d2b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d2b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57672/0 (socket says 172.21.15.170:57672) 2024-09-21T02:15:06.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 -- 172.21.15.170:0/873029319 learned_addr learned my addr 172.21.15.170:0/873029319 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 -- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0d6ca0 msgr2=0x55c16d2b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0d6ca0 0x55c16d2b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2882d9d640 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0ed950 0x55c16d2b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 -- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0ed950 msgr2=0x55c16d2b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0ed950 0x55c16d2b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 -- 172.21.15.170:0/873029319 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c16d09bdb0 con 0x55c16ccfd460 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f288259c640 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d2b4a30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f287400ba60 tx=0x7f2874005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71cced96c308483 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f28637fe640 1 -- 172.21.15.170:0/873029319 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874015040 con 0x55c16ccfd460 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f28637fe640 1 -- 172.21.15.170:0/873029319 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2874028910 con 0x55c16ccfd460 2024-09-21T02:15:06.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f28637fe640 1 -- 172.21.15.170:0/873029319 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874037040 con 0x55c16ccfd460 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- 172.21.15.170:0/873029319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 msgr2=0x55c16d2b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d2b4a30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f287400ba60 tx=0x7f2874005860 comp rx=0 tx=0).stop 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- 172.21.15.170:0/873029319 shutdown_connections 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0d6ca0 0x55c16d2b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16d0ed950 0x55c16d2b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 --2- 172.21.15.170:0/873029319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16ccfd460 0x55c16d2b4a30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- 172.21.15.170:0/873029319 >> 172.21.15.170:0/873029319 conn(0x55c16d1c3d90 msgr2=0x55c16d23c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- 172.21.15.170:0/873029319 shutdown_connections 2024-09-21T02:15:06.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.030+0000 7f2890002000 1 -- 172.21.15.170:0/873029319 wait complete. 2024-09-21T02:15:06.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- start start 2024-09-21T02:15:06.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 Processor -- start 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16ccfd460 0x55c16d2b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 0x55c16d2b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d2b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c16d043d80 con 0x55c16d0d6ca0 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f28841ff660 con 0x55c16ccfd460 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55c16cfda5f0 con 0x55c16d0ed950 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 10 client.0 _ll_register_callbacks cb 0x7f287c12cce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 0x55c16d2b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 0x55c16d2b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57684/0 (socket says 172.21.15.170:57684) 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 -- 172.21.15.170:0/719564894 learned_addr learned my addr 172.21.15.170:0/719564894 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:06.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2882d9d640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d2b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f288259c640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16ccfd460 0x55c16d2b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 -- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 msgr2=0x55c16d2b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d2b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 -- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16ccfd460 msgr2=0x55c16d2b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16ccfd460 0x55c16d2b87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 -- 172.21.15.170:0/719564894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16d006ab0 con 0x55c16d0d6ca0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2882d9d640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d2b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2881d9b640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 0x55c16d2b8ce0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f287800c680 tx=0x7f287800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517ee7ebff5e2869 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f28617fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f28617fa640 1 -- 172.21.15.170:0/719564894 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287801d060 con 0x55c16d0d6ca0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f28617fa640 1 -- 172.21.15.170:0/719564894 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5854+0+0 (secure 0 0 0) 0x7f287800a890 con 0x55c16d0d6ca0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2874028910 con 0x55c16d0d6ca0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f28617fa640 1 -- 172.21.15.170:0/719564894 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2878023050 con 0x55c16d0d6ca0 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 10 client.5535 fetch_fsmap learned FSMap version 27 2024-09-21T02:15:06.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55c16cfe5070 con 0x55c16d0d6ca0 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f28617fa640 1 -- 172.21.15.170:0/719564894 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2874028910 con 0x55c16d0d6ca0 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f28617fa640 1 -- 172.21.15.170:0/719564894 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f2878026020 con 0x55c16d0d6ca0 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 10 client.5535 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 10 client.5535 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 20 client.5535 populate_metadata read hostname 'smithi170' 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16ccd0830 0x55c16d274c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 4 client.5535 mds_command: new command op to 4984 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 --> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] -- command(tid 0: {"prefix": "status"}) -- 0x55c16d04e540 con 0x55c16ccd0830 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.034+0000 7f288259c640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16ccd0830 0x55c16d274c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f28617fa640 10 client.5535 ms_handle_connect on v2:172.21.15.191:6836/2965172270 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f288259c640 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16ccd0830 0x55c16d274c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d888d5b49e1b6e server_cookie=d7ca8a630560178 in_seq=0 out_seq=0 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f28617fa640 1 -- 172.21.15.170:0/719564894 <== mds.0 v2:172.21.15.191:6836/2965172270 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c16d04e540 con 0x55c16ccd0830 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f28617fa640 10 client.5535 handle_command_reply: tid=0 2024-09-21T02:15:06.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 client.5535 shutdown 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 msgr2=0x55c16d2b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 0x55c16d2b8ce0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f287800c680 tx=0x7f287800cb40 comp rx=0 tx=0).stop 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 shutdown_connections 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:6836/2965172270,v1:172.21.15.191:6837/2965172270] conn(0x55c16ccd0830 0x55c16d274c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c16d0ed950 0x55c16d2b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c16d0d6ca0 0x55c16d2b8ce0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f28617fa640 0 client.5535 ms_handle_reset on v2:172.21.15.191:6836/2965172270 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 --2- 172.21.15.170:0/719564894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c16ccfd460 0x55c16d2b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 >> 172.21.15.170:0/719564894 conn(0x55c16d1c3d90 msgr2=0x55c16d23cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 shutdown_connections 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 1 -- 172.21.15.170:0/719564894 wait complete. 2024-09-21T02:15:06.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2890002000 20 client.5535 trim_cache size 0 max 16384 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4984, 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.739643539999999, 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 27, 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 47, 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 46, 2024-09-21T02:15:06.044 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 15.270869447999999 2024-09-21T02:15:06.045 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:15:06.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28640733b0 msgr2=0x7f28640758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28640733b0 0x7f28640758c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28740057b0 tx=0x7f2874005f40 comp rx=0 tx=0).stop 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 msgr2=0x7f28841e6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 0x7f28841e6ec0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f287c01cd80 tx=0x7f287c01d510 comp rx=0 tx=0).stop 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 shutdown_connections 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f28640733b0 0x7f28640758c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f288415cdb0 0x7f28841f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28841f3fd0 0x7f28841e6ec0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 --2- 172.21.15.170:0/3983120357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288414e380 0x7f288415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 >> 172.21.15.170:0/3983120357 conn(0x7f28840b1c40 msgr2=0x7f28840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 shutdown_connections 2024-09-21T02:15:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.038+0000 7f2888d37640 1 -- 172.21.15.170:0/3983120357 wait complete. 2024-09-21T02:15:06.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:15:06.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 -- 172.21.15.170:0/3212705634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 msgr2=0x7fa110003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3212705634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa110003eb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa1200016f0 tx=0x7fa12001c7b0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa12ea67640 1 -- 172.21.15.170:0/3212705634 reap_dead start 2024-09-21T02:15:06.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 -- 172.21.15.170:0/3212705634 shutdown_connections 2024-09-21T02:15:06.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 -- 172.21.15.170:0/3212705634 >> 172.21.15.170:0/3212705634 conn(0x7fa1280b1c40 msgr2=0x7fa1280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 -- 172.21.15.170:0/3212705634 shutdown_connections 2024-09-21T02:15:06.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 -- 172.21.15.170:0/3212705634 wait complete. 2024-09-21T02:15:06.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 Processor -- start 2024-09-21T02:15:06.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.202+0000 7fa1304ca640 1 -- start start 2024-09-21T02:15:06.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa1281f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa128159430 0x7fa128152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa128159840 0x7fa12809c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa1280cf9a0 con 0x7fa110003a40 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1280cd420 con 0x7fa128159430 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa1280b3350 con 0x7fa128159840 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa1281f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa1281f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57698/0 (socket says 172.21.15.170:57698) 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 -- 172.21.15.170:0/3254127016 learned_addr learned my addr 172.21.15.170:0/3254127016 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12ea67640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa128159840 0x7fa12809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12da65640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa128159430 0x7fa128152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 -- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa128159840 msgr2=0x7fa12809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa128159840 0x7fa12809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 -- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa128159430 msgr2=0x7fa128152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa128159430 0x7fa128152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 -- 172.21.15.170:0/3254127016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa128152e50 con 0x7fa110003a40 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12da65640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa128159430 0x7fa128152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12ea67640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa128159840 0x7fa12809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa12e266640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa1281f8320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa120000c10 tx=0x7fa120003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1200bd070 con 0x7fa110003a40 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa120006500 con 0x7fa110003a40 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa128074320 con 0x7fa110003a40 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120006930 con 0x7fa110003a40 2024-09-21T02:15:06.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1280748e0 con 0x7fa110003a40 2024-09-21T02:15:06.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1280ccb00 con 0x7fa110003a40 2024-09-21T02:15:06.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.206+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fa1200a64f0 con 0x7fa110003a40 2024-09-21T02:15:06.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.210+0000 7fa126ffd640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa1140733b0 0x7fa1140758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:06.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.210+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5854+0+0 (secure 0 0 0) 0x7fa12012eb70 con 0x7fa110003a40 2024-09-21T02:15:06.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.210+0000 7fa12da65640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa1140733b0 0x7fa1140758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:06.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.210+0000 7fa12da65640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa1140733b0 0x7fa1140758c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000fe0 tx=0x7fa11c0015f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:06.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.214+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa128152e50 con 0x7fa110003a40 2024-09-21T02:15:06.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.410+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1281f8860 con 0x7fa110003a40 2024-09-21T02:15:06.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.410+0000 7fa126ffd640 1 -- 172.21.15.170:0/3254127016 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fa1200aa050 con 0x7fa110003a40 2024-09-21T02:15:06.414 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa1140733b0 msgr2=0x7fa1140758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa1140733b0 0x7fa1140758c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000fe0 tx=0x7fa11c0015f0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 msgr2=0x7fa1281f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa1281f8320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa120000c10 tx=0x7fa120003b40 comp rx=0 tx=0).stop 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 shutdown_connections 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa1140733b0 0x7fa1140758c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa128159840 0x7fa12809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa128159430 0x7fa128152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 --2- 172.21.15.170:0/3254127016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa110003a40 0x7fa1281f8320 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:06.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 >> 172.21.15.170:0/3254127016 conn(0x7fa1280b1c40 msgr2=0x7fa1280cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:06.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 shutdown_connections 2024-09-21T02:15:06.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06.414+0000 7fa1304ca640 1 -- 172.21.15.170:0/3254127016 wait complete. 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:15:06.434 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:15:06.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:06.435 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:15:06.440 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:15:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:15:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:15:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:15:06.440 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42409sec preferred_lft 42409sec 2024-09-21T02:15:06.441 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:15:06.442 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:06.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:06.443 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:15:06.492 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:15:06.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:06.494 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:15:06.548 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:15:06.549 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:15:06.549 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:15:06.549 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:15:06.549 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:15:06.549 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:15:06.549 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:15:06.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:15:06.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:15:06.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:06.667 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:15:06.714 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:15:06.714 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:15:06.714 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:15:06.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:06.715 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:15:06.769 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:15:06.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:15:06.769 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:06.814 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:15:06.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:15:06.814 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:06.862 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:15:06.920 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:15:06.971 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:15:06.971 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:15:06.971 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:15:06.971 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:15:06.971 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:15:06.971 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:15:06.972 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:06.973 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:06.974 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:06.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:06.974 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:15:07.023 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:15:07.023 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:15:07.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:15:07.081 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:15:07.127 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:15:07.127 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:15:07.127 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:15:07.127 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:15:07.127 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:15:07.128 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:07.129 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:07.130 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:07.130 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:07.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:07.131 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:15:07.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:15:07.174+0000 7f258bdac580 -1 init, newargv = 0x7f257c004370 newargc=15 2024-09-21T02:15:07.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[38466]: starting ceph client 2024-09-21T02:15:07.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[38466]: starting fuse 2024-09-21T02:15:08.279 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:15:08.295 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:15:08.344 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:15:08.345 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:15:08.346 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:08.346 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:15:08.346 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:08.346 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:15:08.346 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:08.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:08.347 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:15:08.395 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:15:08.395 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:15:08.395 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:15:08.395 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:15:08.396 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:15:08.397 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:15:08.519 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.38466.asok 2024-09-21T02:15:08.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:15:08.531 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38466.asok 2024-09-21T02:15:08.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:08.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38466.asok status 2024-09-21T02:15:08.714 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:15:08.714 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:15:08.714 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:15:08.714 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:15:08.714 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "38466", 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 5550, 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 5550 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:15:08.715 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 1247244934 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 1247244934 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.5550 192.168.0.1:0/1247244934", 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/1247244934", 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 27, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 48, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:15:08.716 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:15:08.717 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:15:08.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:08.732 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:15:08.732 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:08.738 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:15:08.738 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:08.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:08.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:08.795 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 -- 172.21.15.170:0/3249998947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8d0005260 msgr2=0x7fc8d000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/3249998947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8d0005260 0x7fc8d000b9d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009ee0 tx=0x7fc8dc01d630 comp rx=0 tx=0).stop 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 -- 172.21.15.170:0/3249998947 shutdown_connections 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/3249998947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8d0005260 0x7fc8d000b9d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 -- 172.21.15.170:0/3249998947 >> 172.21.15.170:0/3249998947 conn(0x7fc8e40cd3c0 msgr2=0x7fc8e40cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 -- 172.21.15.170:0/3249998947 shutdown_connections 2024-09-21T02:15:09.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 -- 172.21.15.170:0/3249998947 wait complete. 2024-09-21T02:15:09.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 Processor -- start 2024-09-21T02:15:09.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:08.998+0000 7fc8ebd33640 1 -- start start 2024-09-21T02:15:09.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d0005260 0x7fc8e4159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8e40aac30 0x7fc8e40ab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 0x7fc8e40b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40cb320 con 0x7fc8e40b02e0 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40cb1a0 con 0x7fc8d0005260 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e40cb4a0 con 0x7fc8e40aac30 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 0x7fc8e40b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 0x7fc8e40b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57726/0 (socket says 172.21.15.170:57726) 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 -- 172.21.15.170:0/364632370 learned_addr learned my addr 172.21.15.170:0/364632370 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8e9acf640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d0005260 0x7fc8e4159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8e92ce640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8e40aac30 0x7fc8e40ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 -- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8e40aac30 msgr2=0x7fc8e40ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8e40aac30 0x7fc8e40ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 -- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d0005260 msgr2=0x7fc8e4159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d0005260 0x7fc8e4159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 -- 172.21.15.170:0/364632370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e40b0c30 con 0x7fc8e40b02e0 2024-09-21T02:15:09.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ea2d0640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 0x7fc8e40b06f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc01ded0 tx=0x7fc8dc0a7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:09.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc009200 con 0x7fc8e40b02e0 2024-09-21T02:15:09.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8dc004a50 con 0x7fc8e40b02e0 2024-09-21T02:15:09.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc004e20 con 0x7fc8e40b02e0 2024-09-21T02:15:09.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e40a9760 con 0x7fc8e40b02e0 2024-09-21T02:15:09.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e40a9d20 con 0x7fc8e40b02e0 2024-09-21T02:15:09.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e40cb990 con 0x7fc8e40b02e0 2024-09-21T02:15:09.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.002+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fc8dc005030 con 0x7fc8e40b02e0 2024-09-21T02:15:09.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.006+0000 7fc8da7fc640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8c80733b0 0x7fc8c80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:09.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.006+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fc8dc12ed00 con 0x7fc8e40b02e0 2024-09-21T02:15:09.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.006+0000 7fc8e9acf640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8c80733b0 0x7fc8c80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:09.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.006+0000 7fc8e9acf640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8c80733b0 0x7fc8c80758c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000f70 tx=0x7fc8e0006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:09.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.010+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc8e40b0c30 con 0x7fc8e40b02e0 2024-09-21T02:15:09.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.222+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8e40cc520 con 0x7fc8e40b02e0 2024-09-21T02:15:09.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.222+0000 7fc8da7fc640 1 -- 172.21.15.170:0/364632370 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fc8dc017340 con 0x7fc8e40b02e0 2024-09-21T02:15:09.224 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8c80733b0 msgr2=0x7fc8c80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8c80733b0 0x7fc8c80758c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000f70 tx=0x7fc8e0006b70 comp rx=0 tx=0).stop 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 msgr2=0x7fc8e40b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 0x7fc8e40b06f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc01ded0 tx=0x7fc8dc0a7a40 comp rx=0 tx=0).stop 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 shutdown_connections 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8c80733b0 0x7fc8c80758c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8e40aac30 0x7fc8e40ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e40b02e0 0x7fc8e40b06f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 --2- 172.21.15.170:0/364632370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d0005260 0x7fc8e4159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 >> 172.21.15.170:0/364632370 conn(0x7fc8e40cd3c0 msgr2=0x7fc8e40b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:09.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 shutdown_connections 2024-09-21T02:15:09.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:09.226+0000 7fc8ebd33640 1 -- 172.21.15.170:0/364632370 wait complete. 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:15:09.246 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:15:09.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.247 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:09.254 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.255 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:15:09.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.256 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:15:09.307 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:15:09.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.309 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:15:09.360 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:15:09.361 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:15:09.361 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:15:09.361 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:15:09.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:15:09.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:15:09.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:15:09.362 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:15:09.449 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:15:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:15:09.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:15:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:15:09.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.487 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:15:09.493 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:15:09.493 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:15:09.493 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:15:09.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:15:09.547 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:15:09.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:15:09.547 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:09.595 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:15:09.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:15:09.596 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:09.643 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:15:09.699 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:15:09.749 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:15:09.750 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:15:09.751 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:09.752 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:09.752 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:09.752 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:09.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.753 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:15:09.800 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:15:09.800 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:15:09.845 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:15:09.859 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:15:09.908 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:15:09.909 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:09.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:09.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:09.911 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:15:09.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:15:09.953+0000 7f5ac3efb580 -1 init, newargv = 0x7f5ab4001280 newargc=15 2024-09-21T02:15:09.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34318]: starting ceph client 2024-09-21T02:15:09.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34318]: starting fuse 2024-09-21T02:15:11.064 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:15:11.080 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:15:11.130 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:15:11.130 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:15:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:15:11.132 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:15:11.133 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:11.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:11.134 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:15:11.180 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:15:11.180 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:15:11.180 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:15:11.180 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:15:11.180 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:15:11.180 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:15:11.181 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:15:11.182 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:15:11.308 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.34318.asok 2024-09-21T02:15:11.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:15:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:15:11.321 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34318.asok 2024-09-21T02:15:11.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:11.321 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34318.asok status 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:15:11.504 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "34318", 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 4892, 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 4892 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1948358558 2024-09-21T02:15:11.505 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1948358558 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.4892 192.168.0.1:0/1948358558", 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/1948358558", 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 27, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 48, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:15:11.506 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:15:11.507 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:15:11.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:11.520 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:15:11.520 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:11.525 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:15:11.525 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:11.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:11.525 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:11.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:15:11.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 -- 172.21.15.170:0/3823380167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d40ca720 msgr2=0x7f95d40cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:11.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 --2- 172.21.15.170:0/3823380167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d40ca720 0x7f95d40cab30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95d4071d80 tx=0x7f95c401d320 comp rx=0 tx=0).stop 2024-09-21T02:15:11.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 -- 172.21.15.170:0/3823380167 shutdown_connections 2024-09-21T02:15:11.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 --2- 172.21.15.170:0/3823380167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95d415e9f0 0x7f95d415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 --2- 172.21.15.170:0/3823380167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d40ca720 0x7f95d40cab30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 --2- 172.21.15.170:0/3823380167 >> v2:172.21.15.191:3300/0 conn(0x7f95d409dc00 0x7f95d415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 -- 172.21.15.170:0/3823380167 >> 172.21.15.170:0/3823380167 conn(0x7f95d40b1c40 msgr2=0x7f95d40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:11.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 -- 172.21.15.170:0/3823380167 shutdown_connections 2024-09-21T02:15:11.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 -- 172.21.15.170:0/3823380167 wait complete. 2024-09-21T02:15:11.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 Processor -- start 2024-09-21T02:15:11.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.750+0000 7f95dc278640 1 -- start start 2024-09-21T02:15:11.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95d409dc00 0x7f95d414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:11.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95d40ca720 0x7f95d415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:11.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 0x7f95d415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95d406f8e0 con 0x7f95d415e9f0 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f95d406f760 con 0x7f95d40ca720 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f95d406fa60 con 0x7f95d409dc00 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 0x7f95d415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 0x7f95d415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57758/0 (socket says 172.21.15.170:57758) 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 -- 172.21.15.170:0/3940656542 learned_addr learned my addr 172.21.15.170:0/3940656542 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da014640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95d409dc00 0x7f95d414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95d9813640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95d40ca720 0x7f95d415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 -- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95d409dc00 msgr2=0x7f95d414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:11.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95d409dc00 0x7f95d414b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 -- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95d40ca720 msgr2=0x7f95d415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95d40ca720 0x7f95d415cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 -- 172.21.15.170:0/3940656542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d41fb0c0 con 0x7f95d415e9f0 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da815640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 0x7f95d415d020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00e970 tx=0x7f95cc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004d30 con 0x7f95d415e9f0 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95cc00cb20 con 0x7f95d415e9f0 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d41f7020 con 0x7f95d415e9f0 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc005020 con 0x7f95d415e9f0 2024-09-21T02:15:11.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d409d6b0 con 0x7f95d415e9f0 2024-09-21T02:15:11.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d409d880 con 0x7f95d415e9f0 2024-09-21T02:15:11.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f95cc0051c0 con 0x7f95d415e9f0 2024-09-21T02:15:11.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95caffd640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95b80733b0 0x7f95b80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:11.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f95cc0950c0 con 0x7f95d415e9f0 2024-09-21T02:15:11.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.754+0000 7f95da014640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95b80733b0 0x7f95b80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:11.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.758+0000 7f95da014640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95b80733b0 0x7f95b80758c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f95d000bb00 tx=0x7f95d0005dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:11.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.762+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f95cc016030 con 0x7f95d415e9f0 2024-09-21T02:15:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.970+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f95d40cab30 con 0x7f95d415e9f0 2024-09-21T02:15:11.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95caffd640 1 -- 172.21.15.170:0/3940656542 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) ==== 136+0+0 (secure 0 0 0) 0x7f95cc05fc40 con 0x7f95d415e9f0 2024-09-21T02:15:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95b80733b0 msgr2=0x7f95b80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95b80733b0 0x7f95b80758c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f95d000bb00 tx=0x7f95d0005dd0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 msgr2=0x7f95d415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 0x7f95d415d020 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00e970 tx=0x7f95cc00ee30 comp rx=0 tx=0).stop 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 shutdown_connections 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f95b80733b0 0x7f95b80758c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d415e9f0 0x7f95d415d020 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95d40ca720 0x7f95d415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.994+0000 7f95dc278640 1 --2- 172.21.15.170:0/3940656542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95d409dc00 0x7f95d414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.998+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 >> 172.21.15.170:0/3940656542 conn(0x7f95d40b1c40 msgr2=0x7f95d40b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.998+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 shutdown_connections 2024-09-21T02:15:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:11.998+0000 7f95dc278640 1 -- 172.21.15.170:0/3940656542 wait complete. 2024-09-21T02:15:12.018 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:15:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 -- 172.21.15.170:0/2003784636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc164510 msgr2=0x7fa7dc166930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 --2- 172.21.15.170:0/2003784636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc164510 0x7fa7dc166930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009bc0 tx=0x7fa7d401cba0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 -- 172.21.15.170:0/2003784636 shutdown_connections 2024-09-21T02:15:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 --2- 172.21.15.170:0/2003784636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc164510 0x7fa7dc166930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 -- 172.21.15.170:0/2003784636 >> 172.21.15.170:0/2003784636 conn(0x7fa7dc0b1c40 msgr2=0x7fa7dc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 -- 172.21.15.170:0/2003784636 shutdown_connections 2024-09-21T02:15:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.170+0000 7fa7e1118640 1 -- 172.21.15.170:0/2003784636 wait complete. 2024-09-21T02:15:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 Processor -- start 2024-09-21T02:15:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- start start 2024-09-21T02:15:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7dc0cb460 0x7fa7dc09dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 0x7fa7dc09cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7dc164510 0x7fa7dc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc169e90 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc169d10 con 0x7fa7dc164510 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa7dc16a010 con 0x7fa7dc0cb460 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 0x7fa7dc09cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 0x7fa7dc09cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57766/0 (socket says 172.21.15.170:57766) 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 -- 172.21.15.170:0/4114069435 learned_addr learned my addr 172.21.15.170:0/4114069435 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7dad9d640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7dc0cb460 0x7fa7dc09dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7db59e640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7dc164510 0x7fa7dc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 -- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7dc0cb460 msgr2=0x7fa7dc09dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7dc0cb460 0x7fa7dc09dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 -- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7dc164510 msgr2=0x7fa7dc0b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7dc164510 0x7fa7dc0b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 -- 172.21.15.170:0/4114069435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc15c380 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7db59e640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7dc164510 0x7fa7dc0b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:12.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7da59c640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 0x7fa7dc09cde0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c3c0 tx=0x7fa7d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0017070 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7d0003600 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc15cae0 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0003a50 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc1f9020 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.174+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7dc0cb870 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.178+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fa7d001a070 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.178+0000 7fa7c37fe640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c80733b0 0x7fa7c80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.178+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fa7d0093910 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.178+0000 7fa7dad9d640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c80733b0 0x7fa7c80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.178+0000 7fa7dad9d640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c80733b0 0x7fa7c80758c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00b810 tx=0x7fa7cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:12.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.182+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa7d005e5a0 con 0x7fa7dc0cf7e0 2024-09-21T02:15:12.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.378+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa7dc15ef90 con 0x7fa7c80733b0 2024-09-21T02:15:12.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.382+0000 7fa7c37fe640 1 -- 172.21.15.170:0/4114069435 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa7dc15ef90 con 0x7fa7c80733b0 2024-09-21T02:15:12.384 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:15:12.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:15:12.384 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:15:12.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:15:12.385 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c80733b0 msgr2=0x7fa7c80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c80733b0 0x7fa7c80758c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00b810 tx=0x7fa7cc008040 comp rx=0 tx=0).stop 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 msgr2=0x7fa7dc09cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 0x7fa7dc09cde0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000c3c0 tx=0x7fa7d000c880 comp rx=0 tx=0).stop 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 shutdown_connections 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa7c80733b0 0x7fa7c80758c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7dc164510 0x7fa7dc0b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7dc0cf7e0 0x7fa7dc09cde0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 --2- 172.21.15.170:0/4114069435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7dc0cb460 0x7fa7dc09dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 >> 172.21.15.170:0/4114069435 conn(0x7fa7dc0b1c40 msgr2=0x7fa7dc0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 shutdown_connections 2024-09-21T02:15:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.386+0000 7fa7e1118640 1 -- 172.21.15.170:0/4114069435 wait complete. 2024-09-21T02:15:12.406 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:15:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 -- 172.21.15.170:0/80720703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c0d0b10 msgr2=0x7fea9c071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 --2- 172.21.15.170:0/80720703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c0d0b10 0x7fea9c071110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea94009c80 tx=0x7fea9401cce0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 -- 172.21.15.170:0/80720703 shutdown_connections 2024-09-21T02:15:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 --2- 172.21.15.170:0/80720703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c0d0b10 0x7fea9c071110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 --2- 172.21.15.170:0/80720703 >> v2:172.21.15.191:3300/0 conn(0x7fea9c1606e0 0x7fea9c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 -- 172.21.15.170:0/80720703 >> 172.21.15.170:0/80720703 conn(0x7fea9c0d2a30 msgr2=0x7fea9c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 -- 172.21.15.170:0/80720703 shutdown_connections 2024-09-21T02:15:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.562+0000 7feaa3da9640 1 -- 172.21.15.170:0/80720703 wait complete. 2024-09-21T02:15:12.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 Processor -- start 2024-09-21T02:15:12.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 -- start start 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 0x7fea9c159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea9c0d0b10 0x7fea9c1451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c1606e0 0x7fea9c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c16a090 con 0x7fea9c0d0700 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c169f10 con 0x7fea9c0d0b10 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c16a210 con 0x7fea9c1606e0 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 0x7fea9c159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 0x7fea9c159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57780/0 (socket says 172.21.15.170:57780) 2024-09-21T02:15:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 -- 172.21.15.170:0/310038794 learned_addr learned my addr 172.21.15.170:0/310038794 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa2346640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c1606e0 0x7fea9c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1344640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea9c0d0b10 0x7fea9c1451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 -- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c1606e0 msgr2=0x7fea9c1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c1606e0 0x7fea9c1573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 -- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea9c0d0b10 msgr2=0x7fea9c1451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea9c0d0b10 0x7fea9c1451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 -- 172.21.15.170:0/310038794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c154ca0 con 0x7fea9c0d0700 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa1b45640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 0x7fea9c159eb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00e9c0 tx=0x7fea8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00cd80 con 0x7fea9c0d0700 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c145720 con 0x7fea9c0d0700 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea8c010520 con 0x7fea9c0d0700 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c010970 con 0x7fea9c0d0700 2024-09-21T02:15:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.566+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c154400 con 0x7fea9c0d0700 2024-09-21T02:15:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.570+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c071110 con 0x7fea9c0d0700 2024-09-21T02:15:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.570+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fea8c01f020 con 0x7fea9c0d0700 2024-09-21T02:15:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.570+0000 7fea927fc640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fea800733b0 0x7fea800758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.570+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fea8c094ef0 con 0x7fea9c0d0700 2024-09-21T02:15:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.570+0000 7feaa1344640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fea800733b0 0x7fea800758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.570+0000 7feaa1344640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fea800733b0 0x7fea800758c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea9c145fb0 tx=0x7fea98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:12.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.574+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fea8c05fb00 con 0x7fea9c0d0700 2024-09-21T02:15:12.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.734+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fea9c0b8330 con 0x7fea9c0d0700 2024-09-21T02:15:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.734+0000 7fea927fc640 1 -- 172.21.15.170:0/310038794 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fea8c05f400 con 0x7fea9c0d0700 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fea800733b0 msgr2=0x7fea800758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fea800733b0 0x7fea800758c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea9c145fb0 tx=0x7fea98009040 comp rx=0 tx=0).stop 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 msgr2=0x7fea9c159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 0x7fea9c159eb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00e9c0 tx=0x7fea8c00ee80 comp rx=0 tx=0).stop 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 shutdown_connections 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fea800733b0 0x7fea800758c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea9c1606e0 0x7fea9c1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea9c0d0b10 0x7fea9c1451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 --2- 172.21.15.170:0/310038794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea9c0d0700 0x7fea9c159eb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 >> 172.21.15.170:0/310038794 conn(0x7fea9c0d2a30 msgr2=0x7fea9c15a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 shutdown_connections 2024-09-21T02:15:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.738+0000 7feaa3da9640 1 -- 172.21.15.170:0/310038794 wait complete. 2024-09-21T02:15:12.757 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:15:12.757 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:15:12.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/2359320436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c88004d50 msgr2=0x7f3c88005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/2359320436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c88004d50 0x7f3c88005200 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b5250 tx=0x7f3c9401d5d0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3c9bfff640 1 -- 172.21.15.170:0/2359320436 reap_dead start 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/2359320436 shutdown_connections 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/2359320436 >> 172.21.15.170:0/2359320436 conn(0x7f3c9c0b1c40 msgr2=0x7f3c9c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/2359320436 shutdown_connections 2024-09-21T02:15:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.946+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/2359320436 wait complete. 2024-09-21T02:15:12.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 Processor -- start 2024-09-21T02:15:12.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 -- start start 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c88004d50 0x7f3c9c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c1f7980 0x7f3c9c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 0x7f3c9c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c164cb0 con 0x7f3c9c0c9800 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c164b30 con 0x7f3c9c1f7980 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c164e30 con 0x7f3c88004d50 2024-09-21T02:15:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 0x7f3c9c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 0x7f3c9c0c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57802/0 (socket says 172.21.15.170:57802) 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 -- 172.21.15.170:0/3493165119 learned_addr learned my addr 172.21.15.170:0/3493165119 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c92ffd640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c1f7980 0x7f3c9c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9b7fe640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c88004d50 0x7f3c9c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 -- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c88004d50 msgr2=0x7f3c9c0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c88004d50 0x7f3c9c0c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 -- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c1f7980 msgr2=0x7f3c9c0c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c1f7980 0x7f3c9c0c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 -- 172.21.15.170:0/3493165119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c0c3f50 con 0x7f3c9c0c9800 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9b7fe640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c88004d50 0x7f3c9c0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 0x7f3c9c0c9c10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3c94002240 tx=0x7f3c940abde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94009ab0 con 0x7f3c9c0c9800 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c940a9510 con 0x7f3c9c0c9800 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c0c8760 con 0x7f3c9c0c9800 2024-09-21T02:15:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c940abed0 con 0x7f3c9c0c9800 2024-09-21T02:15:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.950+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c0c8d20 con 0x7f3c9c0c9800 2024-09-21T02:15:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.954+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9c0cb870 con 0x7f3c9c0c9800 2024-09-21T02:15:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.954+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f3c940b9030 con 0x7f3c9c0c9800 2024-09-21T02:15:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.954+0000 7f3c98ff9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3c7c0733d0 0x7f3c7c0758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.954+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f3c9412f890 con 0x7f3c9c0c9800 2024-09-21T02:15:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.954+0000 7f3c9b7fe640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3c7c0733d0 0x7f3c7c0758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:12.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.954+0000 7f3c9b7fe640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3c7c0733d0 0x7f3c7c0758e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c10 tx=0x7f3c8c006a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:12.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:12.958+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3c9c0c3f50 con 0x7f3c9c0c9800 2024-09-21T02:15:13.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.142+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3c9c15b3f0 con 0x7f3c9c0c9800 2024-09-21T02:15:13.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.142+0000 7f3c98ff9640 1 -- 172.21.15.170:0/3493165119 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f3c940ad030 con 0x7f3c9c0c9800 2024-09-21T02:15:13.144 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:15:13.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.142+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3c7c0733d0 msgr2=0x7f3c7c0758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.142+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3c7c0733d0 0x7f3c7c0758e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c10 tx=0x7f3c8c006a50 comp rx=0 tx=0).stop 2024-09-21T02:15:13.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.142+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 msgr2=0x7f3c9c0c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.142+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 0x7f3c9c0c9c10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3c94002240 tx=0x7f3c940abde0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 shutdown_connections 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3c7c0733d0 0x7f3c7c0758e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c0c9800 0x7f3c9c0c9c10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c1f7980 0x7f3c9c0c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 --2- 172.21.15.170:0/3493165119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c88004d50 0x7f3c9c0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 >> 172.21.15.170:0/3493165119 conn(0x7f3c9c0b1c40 msgr2=0x7f3c9c0aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 shutdown_connections 2024-09-21T02:15:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.146+0000 7f3ca1ac9640 1 -- 172.21.15.170:0/3493165119 wait complete. 2024-09-21T02:15:13.165 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:15:13.165 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:15:13.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/3922536469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a280cd210 msgr2=0x7f2a280b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/3922536469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a280cd210 0x7f2a280b3350 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2a28071d80 tx=0x7f2a2001cb80 comp rx=0 tx=0).stop 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/3922536469 shutdown_connections 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/3922536469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a280cd210 0x7f2a280b3350 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/3922536469 >> v2:172.21.15.191:3300/0 conn(0x7f2a280b5410 0x7f2a2806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/3922536469 >> 172.21.15.170:0/3922536469 conn(0x7f2a280b1c40 msgr2=0x7f2a280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/3922536469 shutdown_connections 2024-09-21T02:15:13.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/3922536469 wait complete. 2024-09-21T02:15:13.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 Processor -- start 2024-09-21T02:15:13.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.370+0000 7f2a2c9d1640 1 -- start start 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a280b2f40 0x7f2a281567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a280b5410 0x7f2a281f9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 0x7f2a281f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28169fd0 con 0x7f2a281584e0 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2a28169e50 con 0x7f2a280b5410 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2816a150 con 0x7f2a280b2f40 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 0x7f2a281f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 0x7f2a281f9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57814/0 (socket says 172.21.15.170:57814) 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 -- 172.21.15.170:0/983744392 learned_addr learned my addr 172.21.15.170:0/983744392 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a1dd9b640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a280b5410 0x7f2a281f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2659c640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a280b2f40 0x7f2a281567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 -- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a280b2f40 msgr2=0x7f2a281567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a280b2f40 0x7f2a281567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 -- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a280b5410 msgr2=0x7f2a281f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a280b5410 0x7f2a281f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 -- 172.21.15.170:0/983744392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a281fa530 con 0x7f2a281584e0 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a26d9d640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 0x7f2a281f9dc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009b10 tx=0x7f2a20003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a200bb070 con 0x7f2a281584e0 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a20004e00 con 0x7f2a281584e0 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a280c8030 con 0x7f2a281584e0 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a200b2040 con 0x7f2a281584e0 2024-09-21T02:15:13.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a280c85f0 con 0x7f2a281584e0 2024-09-21T02:15:13.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2806fa30 con 0x7f2a281584e0 2024-09-21T02:15:13.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.374+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f2a200be070 con 0x7f2a281584e0 2024-09-21T02:15:13.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.378+0000 7f2a1f7fe640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2a080733e0 0x7f2a080758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:13.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.378+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2a2012e2e0 con 0x7f2a281584e0 2024-09-21T02:15:13.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.378+0000 7f2a2659c640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2a080733e0 0x7f2a080758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:13.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.378+0000 7f2a2659c640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2a080733e0 0x7f2a080758f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2a18005fd0 tx=0x7f2a1800c460 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.382+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2a20016770 con 0x7f2a281584e0 2024-09-21T02:15:13.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.550+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a280b45c0 con 0x7f2a281584e0 2024-09-21T02:15:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.550+0000 7f2a1f7fe640 1 -- 172.21.15.170:0/983744392 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f2a200f8ee0 con 0x7f2a281584e0 2024-09-21T02:15:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:15:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2a080733e0 msgr2=0x7f2a080758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2a080733e0 0x7f2a080758f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2a18005fd0 tx=0x7f2a1800c460 comp rx=0 tx=0).stop 2024-09-21T02:15:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 msgr2=0x7f2a281f9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 0x7f2a281f9dc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009b10 tx=0x7f2a20003ae0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 shutdown_connections 2024-09-21T02:15:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2a080733e0 0x7f2a080758f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a281584e0 0x7f2a281f9dc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a280b5410 0x7f2a281f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 --2- 172.21.15.170:0/983744392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a280b2f40 0x7f2a281567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:13.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 >> 172.21.15.170:0/983744392 conn(0x7f2a280b1c40 msgr2=0x7f2a2815c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:13.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 shutdown_connections 2024-09-21T02:15:13.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:13.554+0000 7f2a2c9d1640 1 -- 172.21.15.170:0/983744392 wait complete. 2024-09-21T02:15:13.574 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:15:13.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:13.575 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:15:13.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[38466]: fuse finished with error 0 and tester_r 0 2024-09-21T02:15:13.586 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:15:19.687 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:15:19.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:19.687 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:15:19.695 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:15:19.695 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:15:19.702 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 -- 172.21.15.170:0/3623108370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 msgr2=0x7f0ce40cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/3623108370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40cf950 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40d0260 tx=0x7f0cd401c7e0 comp rx=0 tx=0).stop 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 -- 172.21.15.170:0/3623108370 shutdown_connections 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/3623108370 >> v2:172.21.15.191:3300/0 conn(0x7f0ce4160610 0x7f0ce40b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/3623108370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40cf950 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 -- 172.21.15.170:0/3623108370 >> 172.21.15.170:0/3623108370 conn(0x7f0ce409a9c0 msgr2=0x7f0ce40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 -- 172.21.15.170:0/3623108370 shutdown_connections 2024-09-21T02:15:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.902+0000 7f0ce8e03640 1 -- 172.21.15.170:0/3623108370 wait complete. 2024-09-21T02:15:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 Processor -- start 2024-09-21T02:15:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 -- start start 2024-09-21T02:15:19.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:19.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce415faf0 0x7f0ce4153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:19.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce4160610 0x7f0ce4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:19.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4168f40 con 0x7f0ce40b25f0 2024-09-21T02:15:19.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4168dc0 con 0x7f0ce4160610 2024-09-21T02:15:19.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce41690c0 con 0x7f0ce415faf0 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36632/0 (socket says 172.21.15.170:36632) 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 -- 172.21.15.170:0/2612476321 learned_addr learned my addr 172.21.15.170:0/2612476321 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce2d9d640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce4160610 0x7f0ce4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce1d9b640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce415faf0 0x7f0ce4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 -- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce415faf0 msgr2=0x7f0ce4153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce415faf0 0x7f0ce4153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 -- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce4160610 msgr2=0x7f0ce4153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce4160610 0x7f0ce4153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 -- 172.21.15.170:0/2612476321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce415f170 con 0x7f0ce40b25f0 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce2d9d640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce4160610 0x7f0ce4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce1d9b640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce415faf0 0x7f0ce4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce259c640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40ca770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009790 tx=0x7f0cd4004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40048d0 con 0x7f0ce40b25f0 2024-09-21T02:15:19.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cd40bd070 con 0x7f0ce40b25f0 2024-09-21T02:15:19.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce415f420 con 0x7f0ce40b25f0 2024-09-21T02:15:19.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4004da0 con 0x7f0ce40b25f0 2024-09-21T02:15:19.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.906+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce415ca00 con 0x7f0ce40b25f0 2024-09-21T02:15:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.910+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce40ce9d0 con 0x7f0ce40b25f0 2024-09-21T02:15:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.910+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f0cd4003600 con 0x7f0ce40b25f0 2024-09-21T02:15:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.910+0000 7f0cd2ffd640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0cc40733e0 0x7f0cc40758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.910+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f0cd412f440 con 0x7f0ce40b25f0 2024-09-21T02:15:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.910+0000 7f0ce1d9b640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0cc40733e0 0x7f0cc40758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.910+0000 7f0ce1d9b640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0cc40733e0 0x7f0cc40758f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4159980 tx=0x7f0cd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:19.914+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0cd40163d0 con 0x7f0ce40b25f0 2024-09-21T02:15:20.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.098+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ce406d050 con 0x7f0ce40b25f0 2024-09-21T02:15:20.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.098+0000 7f0cd2ffd640 1 -- 172.21.15.170:0/2612476321 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f0cd40fa040 con 0x7f0ce40b25f0 2024-09-21T02:15:20.102 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:15:20.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0cc40733e0 msgr2=0x7f0cc40758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0cc40733e0 0x7f0cc40758f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4159980 tx=0x7f0cd8009040 comp rx=0 tx=0).stop 2024-09-21T02:15:20.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 msgr2=0x7f0ce40ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40ca770 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009790 tx=0x7f0cd4004140 comp rx=0 tx=0).stop 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 shutdown_connections 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0cc40733e0 0x7f0cc40758f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ce415faf0 0x7f0ce4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ce4160610 0x7f0ce4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 --2- 172.21.15.170:0/2612476321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ce40b25f0 0x7f0ce40ca770 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 >> 172.21.15.170:0/2612476321 conn(0x7f0ce409a9c0 msgr2=0x7f0ce41589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 shutdown_connections 2024-09-21T02:15:20.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.102+0000 7f0ce8e03640 1 -- 172.21.15.170:0/2612476321 wait complete. 2024-09-21T02:15:20.124 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:15:20.125 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:15:20.132 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:15:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 -- 172.21.15.170:0/182939669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40ae500 msgr2=0x7f2be4169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 --2- 172.21.15.170:0/182939669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40ae500 0x7f2be4169870 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2be4073f30 tx=0x7f2bd401c770 comp rx=0 tx=0).stop 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 -- 172.21.15.170:0/182939669 shutdown_connections 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 --2- 172.21.15.170:0/182939669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40ae500 0x7f2be4169870 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 --2- 172.21.15.170:0/182939669 >> v2:172.21.15.191:3300/0 conn(0x7f2be406d510 0x7f2be40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 -- 172.21.15.170:0/182939669 >> 172.21.15.170:0/182939669 conn(0x7f2be409a9c0 msgr2=0x7f2be40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 -- 172.21.15.170:0/182939669 shutdown_connections 2024-09-21T02:15:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.282+0000 7f2beaa94640 1 -- 172.21.15.170:0/182939669 wait complete. 2024-09-21T02:15:20.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 Processor -- start 2024-09-21T02:15:20.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 -- start start 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2be40ae500 0x7f2be4151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2be4162e80 0x7f2be4152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 0x7f2be4152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2be416c5d0 con 0x7f2be40b4810 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2be415ee30 con 0x7f2be40ae500 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2be416c750 con 0x7f2be4162e80 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 0x7f2be4152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 0x7f2be4152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36656/0 (socket says 172.21.15.170:36656) 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 -- 172.21.15.170:0/2106562304 learned_addr learned my addr 172.21.15.170:0/2106562304 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be3fff640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2be4162e80 0x7f2be4152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 -- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2be4162e80 msgr2=0x7f2be4152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2be4162e80 0x7f2be4152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 -- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2be40ae500 msgr2=0x7f2be4151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2be40ae500 0x7f2be4151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 -- 172.21.15.170:0/2106562304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be40b4f80 con 0x7f2be40b4810 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be9031640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 0x7f2be4152b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2bd401cc50 tx=0x7f2bd4005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40b2040 con 0x7f2be40b4810 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd40bb070 con 0x7f2be40b4810 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be40b5260 con 0x7f2be40b4810 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4003600 con 0x7f2be40b4810 2024-09-21T02:15:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.286+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be40b5820 con 0x7f2be40b4810 2024-09-21T02:15:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.290+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be40b4cd0 con 0x7f2be40b4810 2024-09-21T02:15:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.290+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f2bd40037a0 con 0x7f2be40b4810 2024-09-21T02:15:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.290+0000 7f2be17fa640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2bd00733b0 0x7f2bd00758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.290+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2bd412ec10 con 0x7f2be40b4810 2024-09-21T02:15:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.290+0000 7f2be8830640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2bd00733b0 0x7f2bd00758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.290+0000 7f2be8830640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2bd00733b0 0x7f2bd00758c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc001be0 tx=0x7f2bdc007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:20.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.294+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2bd40163f0 con 0x7f2be40b4810 2024-09-21T02:15:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.474+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2be40cee40 con 0x7f2be40b4810 2024-09-21T02:15:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.474+0000 7f2be17fa640 1 -- 172.21.15.170:0/2106562304 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f2bd40f98a0 con 0x7f2be40b4810 2024-09-21T02:15:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.474+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2bd00733b0 msgr2=0x7f2bd00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.474+0000 7f2beaa94640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2bd00733b0 0x7f2bd00758c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc001be0 tx=0x7f2bdc007ca0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.474+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 msgr2=0x7f2be4152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.474+0000 7f2beaa94640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 0x7f2be4152b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2bd401cc50 tx=0x7f2bd4005d20 comp rx=0 tx=0).stop 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 shutdown_connections 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2bd00733b0 0x7f2bd00758c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2be4162e80 0x7f2be4152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2be40b4810 0x7f2be4152b50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 --2- 172.21.15.170:0/2106562304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2be40ae500 0x7f2be4151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 >> 172.21.15.170:0/2106562304 conn(0x7f2be409a9c0 msgr2=0x7f2be4071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:20.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 shutdown_connections 2024-09-21T02:15:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:20.478+0000 7f2beaa94640 1 -- 172.21.15.170:0/2106562304 wait complete. 2024-09-21T02:15:20.497 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:15:20.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:20.497 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:15:20.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34318]: fuse finished with error 0 and tester_r 0 2024-09-21T02:15:20.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:15:23.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:15:23.756 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:15:26.612 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:15:26.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:15:26.613 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:15:26.620 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- 172.21.15.170:0/193916594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabdc003a10 msgr2=0x7fabdc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 --2- 172.21.15.170:0/193916594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabdc003a10 0x7fabdc003e80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009ed0 tx=0x7fabe801d630 comp rx=0 tx=0).stop 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabef59e640 1 -- 172.21.15.170:0/193916594 reap_dead start 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- 172.21.15.170:0/193916594 shutdown_connections 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- 172.21.15.170:0/193916594 >> 172.21.15.170:0/193916594 conn(0x7fabf009a9c0 msgr2=0x7fabf00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- 172.21.15.170:0/193916594 shutdown_connections 2024-09-21T02:15:26.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- 172.21.15.170:0/193916594 wait complete. 2024-09-21T02:15:26.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 Processor -- start 2024-09-21T02:15:26.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- start start 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabdc003a10 0x7fabf00c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf015f100 0x7fabf015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 0x7fabf0150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fabf015ec20 con 0x7fabf014dbc0 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fabf00d1000 con 0x7fabdc003a10 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabf50b7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fabf015eda0 con 0x7fabf015f100 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabef59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 0x7fabf0150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabef59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 0x7fabf0150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47084/0 (socket says 172.21.15.170:47084) 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabef59e640 1 -- 172.21.15.170:0/3672855977 learned_addr learned my addr 172.21.15.170:0/3672855977 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabee59c640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf015f100 0x7fabf015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.786+0000 7fabeed9d640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabdc003a10 0x7fabf00c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabef59e640 1 -- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf015f100 msgr2=0x7fabf015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabef59e640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf015f100 0x7fabf015f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabef59e640 1 -- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabdc003a10 msgr2=0x7fabf00c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabef59e640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabdc003a10 0x7fabf00c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabef59e640 1 -- 172.21.15.170:0/3672855977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf00b1100 con 0x7fabf014dbc0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabef59e640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 0x7fabf0150650 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fabf00726f0 tx=0x7fabe0010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0004030 con 0x7fabf014dbc0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabe00042e0 con 0x7fabf014dbc0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf00b13e0 con 0x7fabf014dbc0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0005280 con 0x7fabf014dbc0 2024-09-21T02:15:26.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf015fe10 con 0x7fabf014dbc0 2024-09-21T02:15:26.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf0160070 con 0x7fabf014dbc0 2024-09-21T02:15:26.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7fabe001c020 con 0x7fabf014dbc0 2024-09-21T02:15:26.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabe77fe640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd00733b0 0x7fabd00758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:26.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fabe00943e0 con 0x7fabf014dbc0 2024-09-21T02:15:26.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.790+0000 7fabeed9d640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd00733b0 0x7fabd00758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:26.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.794+0000 7fabeed9d640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd00733b0 0x7fabd00758c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fabe80af000 tx=0x7fabe801a3d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:26.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.794+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fabe005ea20 con 0x7fabf014dbc0 2024-09-21T02:15:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:26.998+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabf006ce00 con 0x7fabf014dbc0 2024-09-21T02:15:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabe77fe640 1 -- 172.21.15.170:0/3672855977 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7fabe0063450 con 0x7fabf014dbc0 2024-09-21T02:15:27.003 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:27.003 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:27.003 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd00733b0 msgr2=0x7fabd00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd00733b0 0x7fabd00758c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fabe80af000 tx=0x7fabe801a3d0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 msgr2=0x7fabf0150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 0x7fabf0150650 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fabf00726f0 tx=0x7fabe0010b50 comp rx=0 tx=0).stop 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 shutdown_connections 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd00733b0 0x7fabd00758c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf015f100 0x7fabf015f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf014dbc0 0x7fabf0150650 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 --2- 172.21.15.170:0/3672855977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabdc003a10 0x7fabf00c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 >> 172.21.15.170:0/3672855977 conn(0x7fabf009a9c0 msgr2=0x7fabf00b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 shutdown_connections 2024-09-21T02:15:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.002+0000 7fabf50b7640 1 -- 172.21.15.170:0/3672855977 wait complete. 2024-09-21T02:15:27.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:27.201 INFO:teuthology.orchestra.run.smithi191.stdout:[] 2024-09-21T02:15:27.201 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T02:15:27.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol950 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 -- 172.21.15.170:0/2324413925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40cea80 msgr2=0x7f5ae40cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/2324413925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40cea80 0x7f5ae40cee90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009870 tx=0x7f5ad401c820 comp rx=0 tx=0).stop 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 -- 172.21.15.170:0/2324413925 shutdown_connections 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/2324413925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40cea80 0x7f5ae40cee90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/2324413925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ae40b2fc0 0x7f5ae40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 -- 172.21.15.170:0/2324413925 >> 172.21.15.170:0/2324413925 conn(0x7f5ae40d0170 msgr2=0x7f5ae40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 -- 172.21.15.170:0/2324413925 shutdown_connections 2024-09-21T02:15:27.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 -- 172.21.15.170:0/2324413925 wait complete. 2024-09-21T02:15:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 Processor -- start 2024-09-21T02:15:27.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.366+0000 7f5aeb95b640 1 -- start start 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ae406ef40 0x7f5ae4166320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 0x7f5ae4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ae40cea80 0x7f5ae4140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4169fe0 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae40b35f0 con 0x7f5ae406ef40 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae416a160 con 0x7f5ae40cea80 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 0x7f5ae4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:27.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 0x7f5ae4152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47114/0 (socket says 172.21.15.170:47114) 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 -- 172.21.15.170:0/291568779 learned_addr learned my addr 172.21.15.170:0/291568779 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 -- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ae40cea80 msgr2=0x7f5ae4140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ae40cea80 0x7f5ae4140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 -- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ae406ef40 msgr2=0x7f5ae4166320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ae406ef40 0x7f5ae4166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 -- 172.21.15.170:0/291568779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae40cbc40 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ae8ef6640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 0x7f5ae4152a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5ad401d0c0 tx=0x7f5ad40a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40040c0 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ad4004350 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4003800 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae40cbf20 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae413ed10 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae40ca3b0 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f5ad40c1020 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ad9ffb640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5ac80733e0 0x7f5ac80758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.370+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f5ad412fae0 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.374+0000 7f5ae96f7640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5ac80733e0 0x7f5ac80758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:27.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.374+0000 7f5ae96f7640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5ac80733e0 0x7f5ac80758f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000a860 tx=0x7f5ae0005b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.374+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5ad40c8020 con 0x7f5ae40b2fc0 2024-09-21T02:15:27.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.574+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol950", "target": ["mon-mgr", ""]}) -- 0x7f5ae40ca0d0 con 0x7f5ac80733e0 2024-09-21T02:15:27.588 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:15:27.584+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:15:27.588 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:15:27.584+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:15:27.588 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:15:27.584+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:15:27.588 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:15:27.584+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:15:27.588 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:15:27.584+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:15:27.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.634+0000 7f5ad9ffb640 1 -- 172.21.15.170:0/291568779 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ae40ca0d0 con 0x7f5ac80733e0 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5ac80733e0 msgr2=0x7f5ac80758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5ac80733e0 0x7f5ac80758f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000a860 tx=0x7f5ae0005b60 comp rx=0 tx=0).stop 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 msgr2=0x7f5ae4152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 0x7f5ae4152a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5ad401d0c0 tx=0x7f5ad40a67b0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 shutdown_connections 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5ac80733e0 0x7f5ac80758f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ae40cea80 0x7f5ae4140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ae40b2fc0 0x7f5ae4152a20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 --2- 172.21.15.170:0/291568779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ae406ef40 0x7f5ae4166320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 >> 172.21.15.170:0/291568779 conn(0x7f5ae40d0170 msgr2=0x7f5ae4153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 shutdown_connections 2024-09-21T02:15:27.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.638+0000 7f5aeb95b640 1 -- 172.21.15.170:0/291568779 wait complete. 2024-09-21T02:15:27.662 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 -- 172.21.15.170:0/2849133525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 msgr2=0x7f81780b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 --2- 172.21.15.170:0/2849133525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b3350 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f81780c7150 tx=0x7f816801c800 comp rx=0 tx=0).stop 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 -- 172.21.15.170:0/2849133525 shutdown_connections 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 --2- 172.21.15.170:0/2849133525 >> v2:172.21.15.191:3300/0 conn(0x7f81780ccb00 0x7f81780ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 --2- 172.21.15.170:0/2849133525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b3350 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 -- 172.21.15.170:0/2849133525 >> 172.21.15.170:0/2849133525 conn(0x7f81780b1c40 msgr2=0x7f81780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 -- 172.21.15.170:0/2849133525 shutdown_connections 2024-09-21T02:15:27.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.814+0000 7f8180642640 1 -- 172.21.15.170:0/2849133525 wait complete. 2024-09-21T02:15:27.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 Processor -- start 2024-09-21T02:15:27.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 -- start start 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81780b5410 0x7f817815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81780ccb00 0x7f817815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8178169e50 con 0x7f81780b2f40 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f81780cf970 con 0x7f81780ccb00 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f8180642640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8178169fd0 con 0x7f81780b5410 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f817e3de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f817e3de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47130/0 (socket says 172.21.15.170:47130) 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f817e3de640 1 -- 172.21.15.170:0/144119987 learned_addr learned my addr 172.21.15.170:0/144119987 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.818+0000 7f817ebdf640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81780ccb00 0x7f817815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:27.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817dbdd640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81780b5410 0x7f817815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817e3de640 1 -- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81780b5410 msgr2=0x7f817815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817e3de640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81780b5410 0x7f817815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817e3de640 1 -- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81780ccb00 msgr2=0x7f817815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817e3de640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81780ccb00 0x7f817815ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817e3de640 1 -- 172.21.15.170:0/144119987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81781fa260 con 0x7f81780b2f40 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817dbdd640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81780b5410 0x7f817815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817e3de640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b0740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f817400c420 tx=0x7f817400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174011790 con 0x7f81780b2f40 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8174017070 con 0x7f81780b2f40 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81781fa540 con 0x7f81780b2f40 2024-09-21T02:15:27.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174011ce0 con 0x7f81780b2f40 2024-09-21T02:15:27.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81781fab00 con 0x7f81780b2f40 2024-09-21T02:15:27.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81780b3350 con 0x7f81780b2f40 2024-09-21T02:15:27.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f817401c460 con 0x7f81780b2f40 2024-09-21T02:15:27.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f816effd640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f815c0733b0 0x7f815c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:27.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f8174094510 con 0x7f81780b2f40 2024-09-21T02:15:27.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817dbdd640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f815c0733b0 0x7f815c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:27.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.822+0000 7f817dbdd640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f815c0733b0 0x7f815c0758c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81680af000 tx=0x7f816801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:27.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:27.826+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f81780cd450 con 0x7f81780b2f40 2024-09-21T02:15:28.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.034+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81780b4bc0 con 0x7f81780b2f40 2024-09-21T02:15:28.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.034+0000 7f816effd640 1 -- 172.21.15.170:0/144119987 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f817405f110 con 0x7f81780b2f40 2024-09-21T02:15:28.038 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:28.038 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:28.038 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:28.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f815c0733b0 msgr2=0x7f815c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f815c0733b0 0x7f815c0758c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81680af000 tx=0x7f816801b040 comp rx=0 tx=0).stop 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 msgr2=0x7f81780b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b0740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f817400c420 tx=0x7f817400c8e0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 shutdown_connections 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f815c0733b0 0x7f815c0758c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81780b5410 0x7f817815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81780ccb00 0x7f817815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 --2- 172.21.15.170:0/144119987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81780b2f40 0x7f81780b0740 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 >> 172.21.15.170:0/144119987 conn(0x7f81780b1c40 msgr2=0x7f81780cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 shutdown_connections 2024-09-21T02:15:28.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.038+0000 7f8180642640 1 -- 172.21.15.170:0/144119987 wait complete. 2024-09-21T02:15:28.059 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:28.236 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.627983713,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.968822064510455,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:28.237 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.968822064510455 }, "state": "open", "uptime": 0.627983713 } ] 2024-09-21T02:15:28.237 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:28.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 -- 172.21.15.170:0/1863304928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0cb460 msgr2=0x7f432c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 --2- 172.21.15.170:0/1863304928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0cb460 0x7f432c0cb870 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f431c009840 tx=0x7f431c01c820 comp rx=0 tx=0).stop 2024-09-21T02:15:28.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 -- 172.21.15.170:0/1863304928 shutdown_connections 2024-09-21T02:15:28.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 --2- 172.21.15.170:0/1863304928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0cb460 0x7f432c0cb870 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 -- 172.21.15.170:0/1863304928 >> 172.21.15.170:0/1863304928 conn(0x7f432c0b1c40 msgr2=0x7f432c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:28.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 -- 172.21.15.170:0/1863304928 shutdown_connections 2024-09-21T02:15:28.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 -- 172.21.15.170:0/1863304928 wait complete. 2024-09-21T02:15:28.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 Processor -- start 2024-09-21T02:15:28.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.394+0000 7f4334568640 1 -- start start 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f432c0cb460 0x7f432c153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 0x7f432c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f432c0cf7e0 0x7f432c155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f432c06f3e0 con 0x7f432c0ce1c0 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f432c06f260 con 0x7f432c0cb460 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f432c06f560 con 0x7f432c0cf7e0 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 0x7f432c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 0x7f432c153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47152/0 (socket says 172.21.15.170:47152) 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 -- 172.21.15.170:0/4040504240 learned_addr learned my addr 172.21.15.170:0/4040504240 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4332304640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f432c0cb460 0x7f432c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4332b05640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f432c0cf7e0 0x7f432c155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 -- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f432c0cf7e0 msgr2=0x7f432c155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f432c0cf7e0 0x7f432c155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 -- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f432c0cb460 msgr2=0x7f432c153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f432c0cb460 0x7f432c153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 -- 172.21.15.170:0/4040504240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c157e70 con 0x7f432c0ce1c0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4332304640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f432c0cb460 0x7f432c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4332b05640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f432c0cf7e0 0x7f432c155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4331b03640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 0x7f432c153ab0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f431c01cd00 tx=0x7f431c005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0bb070 con 0x7f432c0ce1c0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f431c004330 con 0x7f432c0ce1c0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c09d460 con 0x7f432c0ce1c0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0b2040 con 0x7f432c0ce1c0 2024-09-21T02:15:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c156150 con 0x7f432c0ce1c0 2024-09-21T02:15:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c155ec0 con 0x7f432c0ce1c0 2024-09-21T02:15:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 7) ==== 97387+0+0 (secure 0 0 0) 0x7f431c0be070 con 0x7f432c0ce1c0 2024-09-21T02:15:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4322ffd640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f43100733b0 0x7f43100758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.398+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f431c12e350 con 0x7f432c0ce1c0 2024-09-21T02:15:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.402+0000 7f4332304640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f43100733b0 0x7f43100758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:28.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.402+0000 7f4332304640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f43100733b0 0x7f43100758c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4328001630 tx=0x7f4328009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:28.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.402+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f431c0c5020 con 0x7f432c0ce1c0 2024-09-21T02:15:28.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.614+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f432c06fa50 con 0x7f432c0ce1c0 2024-09-21T02:15:28.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4322ffd640 1 -- 172.21.15.170:0/4040504240 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f431c016410 con 0x7f432c0ce1c0 2024-09-21T02:15:28.619 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:28.619 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:28.619 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f43100733b0 msgr2=0x7f43100758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f43100733b0 0x7f43100758c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4328001630 tx=0x7f4328009040 comp rx=0 tx=0).stop 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 msgr2=0x7f432c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 0x7f432c153ab0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f431c01cd00 tx=0x7f431c005d10 comp rx=0 tx=0).stop 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 shutdown_connections 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f43100733b0 0x7f43100758c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f432c0cf7e0 0x7f432c155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f432c0ce1c0 0x7f432c153ab0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 --2- 172.21.15.170:0/4040504240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f432c0cb460 0x7f432c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:28.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 >> 172.21.15.170:0/4040504240 conn(0x7f432c0b1c40 msgr2=0x7f432c0ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:28.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 shutdown_connections 2024-09-21T02:15:28.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:28.618+0000 7f4334568640 1 -- 172.21.15.170:0/4040504240 wait complete. 2024-09-21T02:15:28.641 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:28.821 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":1.2119685680000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.939199708305154,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:28.821 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.939199708305154 }, "state": "open", "uptime": 1.211968568 } ] 2024-09-21T02:15:28.821 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:15:33.822 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- 172.21.15.170:0/116303948 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 msgr2=0x7f0a54169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 --2- 172.21.15.170:0/116303948 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 0x7f0a54169870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009840 tx=0x7f0a4401c790 comp rx=0 tx=0).stop 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- 172.21.15.170:0/116303948 shutdown_connections 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 --2- 172.21.15.170:0/116303948 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 0x7f0a54169870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 --2- 172.21.15.170:0/116303948 >> v2:172.21.15.191:3300/0 conn(0x7f0a5406d510 0x7f0a540adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- 172.21.15.170:0/116303948 >> 172.21.15.170:0/116303948 conn(0x7f0a5409a9c0 msgr2=0x7f0a540d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- 172.21.15.170:0/116303948 shutdown_connections 2024-09-21T02:15:33.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- 172.21.15.170:0/116303948 wait complete. 2024-09-21T02:15:33.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 Processor -- start 2024-09-21T02:15:33.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- start start 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 0x7f0a541576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 0x7f0a541513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a541504e0 0x7f0a54142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5416c750 con 0x7f0a54162e80 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5416c5d0 con 0x7f0a541504e0 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a59f83640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5416c8d0 con 0x7f0a540ae500 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a52ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 0x7f0a541513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a52ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 0x7f0a541513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47158/0 (socket says 172.21.15.170:47158) 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a52ffd640 1 -- 172.21.15.170:0/741932125 learned_addr learned my addr 172.21.15.170:0/741932125 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a53fff640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a541504e0 0x7f0a54142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.990+0000 7f0a537fe640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 0x7f0a541576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:33.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a52ffd640 1 -- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 msgr2=0x7f0a541576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a52ffd640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 0x7f0a541576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a52ffd640 1 -- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a541504e0 msgr2=0x7f0a54142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a52ffd640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a541504e0 0x7f0a54142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a52ffd640 1 -- 172.21.15.170:0/741932125 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a54156610 con 0x7f0a54162e80 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a52ffd640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 0x7f0a541513d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800e970 tx=0x7f0a4800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800cd80 con 0x7f0a54162e80 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a48004020 con 0x7f0a54162e80 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48004470 con 0x7f0a54162e80 2024-09-21T02:15:33.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a541568f0 con 0x7f0a54162e80 2024-09-21T02:15:33.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a54156eb0 con 0x7f0a54162e80 2024-09-21T02:15:33.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a540adfc0 con 0x7f0a54162e80 2024-09-21T02:15:33.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f0a48003600 con 0x7f0a54162e80 2024-09-21T02:15:33.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a3bfff640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a40073440 0x7f0a40075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:33.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f0a48095100 con 0x7f0a54162e80 2024-09-21T02:15:33.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.994+0000 7f0a537fe640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a40073440 0x7f0a40075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:33.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.998+0000 7f0a537fe640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a40073440 0x7f0a40075950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0020b0 tx=0x7f0a4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:34.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:33.998+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0a4805fcb0 con 0x7f0a54162e80 2024-09-21T02:15:34.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.202+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a5406d930 con 0x7f0a54162e80 2024-09-21T02:15:34.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.202+0000 7f0a3bfff640 1 -- 172.21.15.170:0/741932125 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f0a4805f5b0 con 0x7f0a54162e80 2024-09-21T02:15:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:34.205 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:34.206 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:34.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a40073440 msgr2=0x7f0a40075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a40073440 0x7f0a40075950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0020b0 tx=0x7f0a4c009040 comp rx=0 tx=0).stop 2024-09-21T02:15:34.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 msgr2=0x7f0a541513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:34.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 0x7f0a541513d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800e970 tx=0x7f0a4800ee30 comp rx=0 tx=0).stop 2024-09-21T02:15:34.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 shutdown_connections 2024-09-21T02:15:34.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0a40073440 0x7f0a40075950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a541504e0 0x7f0a54142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a54162e80 0x7f0a541513d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 --2- 172.21.15.170:0/741932125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a540ae500 0x7f0a541576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:34.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 >> 172.21.15.170:0/741932125 conn(0x7f0a5409a9c0 msgr2=0x7f0a54070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:34.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 shutdown_connections 2024-09-21T02:15:34.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:34.206+0000 7f0a59f83640 1 -- 172.21.15.170:0/741932125 wait complete. 2024-09-21T02:15:34.227 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:34.407 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":6.7958339050000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.657969856766233,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:34.528 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.657969856766233 }, "state": "open", "uptime": 6.795833905 } ] 2024-09-21T02:15:34.529 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:15:39.530 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 -- 172.21.15.170:0/3334754212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b5410 msgr2=0x7f4aa406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 --2- 172.21.15.170:0/3334754212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b5410 0x7f4aa406fa30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4070e90 tx=0x7f4a9801c800 comp rx=0 tx=0).stop 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 -- 172.21.15.170:0/3334754212 shutdown_connections 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 --2- 172.21.15.170:0/3334754212 >> v2:172.21.15.191:3300/0 conn(0x7f4aa40b2f40 0x7f4aa40b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 --2- 172.21.15.170:0/3334754212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b5410 0x7f4aa406fa30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 -- 172.21.15.170:0/3334754212 >> 172.21.15.170:0/3334754212 conn(0x7f4aa40b1c40 msgr2=0x7f4aa40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 -- 172.21.15.170:0/3334754212 shutdown_connections 2024-09-21T02:15:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.694+0000 7f4aab59f640 1 -- 172.21.15.170:0/3334754212 wait complete. 2024-09-21T02:15:39.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 Processor -- start 2024-09-21T02:15:39.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- start start 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 0x7f4aa40c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aa40b5410 0x7f4aa413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa40ccb00 0x7f4aa413f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4169fd0 con 0x7f4aa40b2f40 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa4169e50 con 0x7f4aa40ccb00 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa416a150 con 0x7f4aa40b5410 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 0x7f4aa40c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 0x7f4aa40c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36360/0 (socket says 172.21.15.170:36360) 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 -- 172.21.15.170:0/1047640454 learned_addr learned my addr 172.21.15.170:0/1047640454 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 -- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aa40b5410 msgr2=0x7f4aa413ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aa40b5410 0x7f4aa413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 -- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa40ccb00 msgr2=0x7f4aa413f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa40ccb00 0x7f4aa413f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 -- 172.21.15.170:0/1047640454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa4150a20 con 0x7f4aa40b2f40 2024-09-21T02:15:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aa933b640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 0x7f4aa40c5c90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4a980982f0 tx=0x7f4a98005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a980b2040 con 0x7f4aa40b2f40 2024-09-21T02:15:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a980bb070 con 0x7f4aa40b2f40 2024-09-21T02:15:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa41fa2a0 con 0x7f4aa40b2f40 2024-09-21T02:15:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a980a5bb0 con 0x7f4aa40b2f40 2024-09-21T02:15:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa41fa860 con 0x7f4aa40b2f40 2024-09-21T02:15:39.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.698+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa40b3350 con 0x7f4aa40b2f40 2024-09-21T02:15:39.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.702+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f4a98003600 con 0x7f4aa40b2f40 2024-09-21T02:15:39.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.702+0000 7f4a95ffb640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a88073470 0x7f4a88075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:39.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.702+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f4a9812ece0 con 0x7f4aa40b2f40 2024-09-21T02:15:39.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.702+0000 7f4aa8b3a640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a88073470 0x7f4a88075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:39.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.702+0000 7f4aa8b3a640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a88073470 0x7f4a88075980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00092f0 tx=0x7f4aa0009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:39.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.706+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a980163f0 con 0x7f4aa40b2f40 2024-09-21T02:15:39.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.910+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aa40b22a0 con 0x7f4aa40b2f40 2024-09-21T02:15:39.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.910+0000 7f4a95ffb640 1 -- 172.21.15.170:0/1047640454 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f4a980f9800 con 0x7f4aa40b2f40 2024-09-21T02:15:39.913 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:39.913 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:39.914 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:39.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a88073470 msgr2=0x7f4a88075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:39.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a88073470 0x7f4a88075980 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00092f0 tx=0x7f4aa0009240 comp rx=0 tx=0).stop 2024-09-21T02:15:39.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 msgr2=0x7f4aa40c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:39.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 0x7f4aa40c5c90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4a980982f0 tx=0x7f4a98005760 comp rx=0 tx=0).stop 2024-09-21T02:15:39.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 shutdown_connections 2024-09-21T02:15:39.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4a88073470 0x7f4a88075980 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4aa40b5410 0x7f4aa413ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa40ccb00 0x7f4aa413f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 --2- 172.21.15.170:0/1047640454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa40b2f40 0x7f4aa40c5c90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:39.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 >> 172.21.15.170:0/1047640454 conn(0x7f4aa40b1c40 msgr2=0x7f4aa40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:39.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 shutdown_connections 2024-09-21T02:15:39.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:39.914+0000 7f4aab59f640 1 -- 172.21.15.170:0/1047640454 wait complete. 2024-09-21T02:15:39.935 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:40.112 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.503710268000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.37421982473294,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:40.113 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.37421982473294 }, "state": "open", "uptime": 12.503710268 } ] 2024-09-21T02:15:40.113 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:15:45.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2620763472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe79815faf0 msgr2=0x7fe79815ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2620763472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe79815faf0 0x7fe79815ff00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe794009840 tx=0x7fe79401c820 comp rx=0 tx=0).stop 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2620763472 shutdown_connections 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2620763472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe79815faf0 0x7fe79815ff00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2620763472 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7980b25f0 0x7fe7980cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2620763472 >> 172.21.15.170:0/2620763472 conn(0x7fe79809a9c0 msgr2=0x7fe7980d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2620763472 shutdown_connections 2024-09-21T02:15:45.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2620763472 wait complete. 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 Processor -- start 2024-09-21T02:15:45.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- start start 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 0x7fe79806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe79815faf0 0x7fe798147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe798160440 0x7fe7980b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe798169350 con 0x7fe7980b25f0 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981691d0 con 0x7fe79815faf0 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe7a03e3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe7981694d0 con 0x7fe798160440 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe79e17f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 0x7fe79806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe79e17f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 0x7fe79806d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36382/0 (socket says 172.21.15.170:36382) 2024-09-21T02:15:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.277+0000 7fe79e17f640 1 -- 172.21.15.170:0/2986544119 learned_addr learned my addr 172.21.15.170:0/2986544119 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e980640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe798160440 0x7fe7980b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79d97e640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe79815faf0 0x7fe798147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e17f640 1 -- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe798160440 msgr2=0x7fe7980b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e17f640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe798160440 0x7fe7980b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e17f640 1 -- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe79815faf0 msgr2=0x7fe798147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e17f640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe79815faf0 0x7fe798147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e17f640 1 -- 172.21.15.170:0/2986544119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe798159d10 con 0x7fe7980b25f0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79e17f640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 0x7fe79806d7f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe78800d730 tx=0x7fe78800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788017070 con 0x7fe7980b25f0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe788009a30 con 0x7fe7980b25f0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79806dd30 con 0x7fe7980b25f0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788010460 con 0x7fe7980b25f0 2024-09-21T02:15:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79815bae0 con 0x7fe7980b25f0 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79814b490 con 0x7fe7980b25f0 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7fe78801a070 con 0x7fe7980b25f0 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe78effd640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe77c073440 0x7fe77c075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fe788094960 con 0x7fe7980b25f0 2024-09-21T02:15:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.281+0000 7fe79d97e640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe77c073440 0x7fe77c075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.285+0000 7fe79d97e640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe77c073440 0x7fe77c075950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe7940af000 tx=0x7fe79401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.285+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe79806dd30 con 0x7fe7980b25f0 2024-09-21T02:15:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.493+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7980cf950 con 0x7fe7980b25f0 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.493+0000 7fe78effd640 1 -- 172.21.15.170:0/2986544119 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7fe78805f510 con 0x7fe7980b25f0 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:45.497 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe77c073440 msgr2=0x7fe77c075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe77c073440 0x7fe77c075950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe7940af000 tx=0x7fe79401b060 comp rx=0 tx=0).stop 2024-09-21T02:15:45.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 msgr2=0x7fe79806d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 0x7fe79806d7f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe78800d730 tx=0x7fe78800dbf0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 shutdown_connections 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe77c073440 0x7fe77c075950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe798160440 0x7fe7980b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe79815faf0 0x7fe798147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 --2- 172.21.15.170:0/2986544119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7980b25f0 0x7fe79806d7f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 >> 172.21.15.170:0/2986544119 conn(0x7fe79809a9c0 msgr2=0x7fe798156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 shutdown_connections 2024-09-21T02:15:45.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:45.497+0000 7fe7a03e3640 1 -- 172.21.15.170:0/2986544119 wait complete. 2024-09-21T02:15:45.518 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:45.700 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.087601637999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.10023095584533,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:45.700 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.10023095584533 }, "state": "open", "uptime": 18.087601638 } ] 2024-09-21T02:15:45.700 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:15:50.701 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 -- 172.21.15.170:0/2855973878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b5410 msgr2=0x7f46a806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 --2- 172.21.15.170:0/2855973878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b5410 0x7f46a806fa30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f46a80b3a10 tx=0x7f469c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 -- 172.21.15.170:0/2855973878 shutdown_connections 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 --2- 172.21.15.170:0/2855973878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46a80b2f40 0x7f46a80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 --2- 172.21.15.170:0/2855973878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b5410 0x7f46a806fa30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 -- 172.21.15.170:0/2855973878 >> 172.21.15.170:0/2855973878 conn(0x7f46a80b1c40 msgr2=0x7f46a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 -- 172.21.15.170:0/2855973878 shutdown_connections 2024-09-21T02:15:50.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.865+0000 7f46adf24640 1 -- 172.21.15.170:0/2855973878 wait complete. 2024-09-21T02:15:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 Processor -- start 2024-09-21T02:15:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- start start 2024-09-21T02:15:50.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 0x7f46a814a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:50.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46a80b5410 0x7f46a81fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:50.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46a80ccb00 0x7f46a81fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8169fd0 con 0x7f46a80b2f40 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8169e50 con 0x7f46a80b5410 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f46a816a150 con 0x7f46a80ccb00 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 0x7f46a814a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 0x7f46a814a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51280/0 (socket says 172.21.15.170:51280) 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 -- 172.21.15.170:0/2009533856 learned_addr learned my addr 172.21.15.170:0/2009533856 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 -- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46a80ccb00 msgr2=0x7f46a81fcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a7fff640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46a80ccb00 0x7f46a81fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a6ffd640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46a80b5410 0x7f46a81fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46a80ccb00 0x7f46a81fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 -- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46a80b5410 msgr2=0x7f46a81fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46a80b5410 0x7f46a81fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:50.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 -- 172.21.15.170:0/2009533856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a809b6e0 con 0x7f46a80b2f40 2024-09-21T02:15:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46a77fe640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 0x7f46a814a040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f469c01d180 tx=0x7f469c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0bc070 con 0x7f46a80b2f40 2024-09-21T02:15:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f469c0a6ca0 con 0x7f46a80b2f40 2024-09-21T02:15:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a814a7f0 con 0x7f46a80b2f40 2024-09-21T02:15:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0b2040 con 0x7f46a80b2f40 2024-09-21T02:15:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a809a9c0 con 0x7f46a80b2f40 2024-09-21T02:15:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.869+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a80b3350 con 0x7f46a80b2f40 2024-09-21T02:15:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.873+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f469c0bf070 con 0x7f46a80b2f40 2024-09-21T02:15:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.873+0000 7f468ffff640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4694073440 0x7f4694075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.873+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f469c12e140 con 0x7f46a80b2f40 2024-09-21T02:15:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.873+0000 7f46a6ffd640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4694073440 0x7f4694075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.873+0000 7f46a6ffd640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4694073440 0x7f4694075950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46a806aa00 tx=0x7f4698005bf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:50.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:50.877+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f469c0164d0 con 0x7f46a80b2f40 2024-09-21T02:15:51.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.085+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a80b1a20 con 0x7f46a80b2f40 2024-09-21T02:15:51.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.085+0000 7f468ffff640 1 -- 172.21.15.170:0/2009533856 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f469c0f8d00 con 0x7f46a80b2f40 2024-09-21T02:15:51.090 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:51.090 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:51.090 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:51.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4694073440 msgr2=0x7f4694075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4694073440 0x7f4694075950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f46a806aa00 tx=0x7f4698005bf0 comp rx=0 tx=0).stop 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 msgr2=0x7f46a814a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 0x7f46a814a040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f469c01d180 tx=0x7f469c0a56b0 comp rx=0 tx=0).stop 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 shutdown_connections 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4694073440 0x7f4694075950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46a80ccb00 0x7f46a81fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46a80b5410 0x7f46a81fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 --2- 172.21.15.170:0/2009533856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46a80b2f40 0x7f46a814a040 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 >> 172.21.15.170:0/2009533856 conn(0x7f46a80b1c40 msgr2=0x7f46a80c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 shutdown_connections 2024-09-21T02:15:51.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:51.089+0000 7f46adf24640 1 -- 172.21.15.170:0/2009533856 wait complete. 2024-09-21T02:15:51.113 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:51.292 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":23.683502227999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.829176298138531,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:51.293 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.82917629813853 }, "state": "open", "uptime": 23.683502228 } ] 2024-09-21T02:15:51.293 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:15:53.810 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:15:53.813 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:15:56.294 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:15:56.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.457+0000 7f22f0174640 1 -- 172.21.15.170:0/1736654688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22d4003fe0 msgr2=0x7f22d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:56.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.457+0000 7f22f0174640 1 --2- 172.21.15.170:0/1736654688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22d4003fe0 0x7f22d4004810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009b60 tx=0x7f22dc01d270 comp rx=0 tx=0).stop 2024-09-21T02:15:56.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.457+0000 7f22ee711640 1 -- 172.21.15.170:0/1736654688 reap_dead start 2024-09-21T02:15:56.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- 172.21.15.170:0/1736654688 shutdown_connections 2024-09-21T02:15:56.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- 172.21.15.170:0/1736654688 >> 172.21.15.170:0/1736654688 conn(0x7f22e80b1c40 msgr2=0x7f22e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:56.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- 172.21.15.170:0/1736654688 shutdown_connections 2024-09-21T02:15:56.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- 172.21.15.170:0/1736654688 wait complete. 2024-09-21T02:15:56.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 Processor -- start 2024-09-21T02:15:56.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- start start 2024-09-21T02:15:56.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22d4003fe0 0x7f22e809ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 0x7f22e815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e8155110 0x7f22e8143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8074cf0 con 0x7f22e80c63a0 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8070850 con 0x7f22e8155110 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22f0174640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f22e8074e70 con 0x7f22d4003fe0 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 0x7f22e815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 0x7f22e815e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48970/0 (socket says 172.21.15.170:48970) 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 -- 172.21.15.170:0/525888080 learned_addr learned my addr 172.21.15.170:0/525888080 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ee711640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e8155110 0x7f22e8143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22edf10640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22d4003fe0 0x7f22e809ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 -- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22d4003fe0 msgr2=0x7f22e809ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22d4003fe0 0x7f22e809ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 -- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e8155110 msgr2=0x7f22e8143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e8155110 0x7f22e8143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ed70f640 1 -- 172.21.15.170:0/525888080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e815c070 con 0x7f22e80c63a0 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.461+0000 7f22ee711640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e8155110 0x7f22e8143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22ed70f640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 0x7f22e815e970 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f22dc01d0a0 tx=0x7f22dc0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0b1040 con 0x7f22e80c63a0 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22dc0bb070 con 0x7f22e80c63a0 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004290 con 0x7f22e80c63a0 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e815e490 con 0x7f22e80c63a0 2024-09-21T02:15:56.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e815c310 con 0x7f22e80c63a0 2024-09-21T02:15:56.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e80cb410 con 0x7f22e80c63a0 2024-09-21T02:15:56.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f22dc0046d0 con 0x7f22e80c63a0 2024-09-21T02:15:56.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22da7fc640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22cc073440 0x7f22cc075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:15:56.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f22dc12f760 con 0x7f22e80c63a0 2024-09-21T02:15:56.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22edf10640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22cc073440 0x7f22cc075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:15:56.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.465+0000 7f22edf10640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22cc073440 0x7f22cc075950 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f22e4001060 tx=0x7f22e40011c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:15:56.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.469+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f22dc016fc0 con 0x7f22e80c63a0 2024-09-21T02:15:56.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.669+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22e80c7b50 con 0x7f22e80c63a0 2024-09-21T02:15:56.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.669+0000 7f22da7fc640 1 -- 172.21.15.170:0/525888080 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f22dc0fa470 con 0x7f22e80c63a0 2024-09-21T02:15:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:15:56.674 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:15:56.674 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22cc073440 msgr2=0x7f22cc075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22cc073440 0x7f22cc075950 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f22e4001060 tx=0x7f22e40011c0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 msgr2=0x7f22e815e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 0x7f22e815e970 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f22dc01d0a0 tx=0x7f22dc0055d0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 shutdown_connections 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f22cc073440 0x7f22cc075950 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22e8155110 0x7f22e8143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22e80c63a0 0x7f22e815e970 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 --2- 172.21.15.170:0/525888080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22d4003fe0 0x7f22e809ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:15:56.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 >> 172.21.15.170:0/525888080 conn(0x7f22e80b1c40 msgr2=0x7f22e80d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:15:56.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 shutdown_connections 2024-09-21T02:15:56.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:15:56.673+0000 7f22f0174640 1 -- 172.21.15.170:0/525888080 wait complete. 2024-09-21T02:15:56.694 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:15:56.871 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":29.263410598,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.56236338527756,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:15:56.872 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.56236338527756 }, "state": "open", "uptime": 29.263410598 } ] 2024-09-21T02:15:56.872 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:01.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 -- 172.21.15.170:0/1246476557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700cb460 msgr2=0x7f27700cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 --2- 172.21.15.170:0/1246476557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700cb460 0x7f27700cb870 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2764009840 tx=0x7f276401c820 comp rx=0 tx=0).stop 2024-09-21T02:16:02.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 -- 172.21.15.170:0/1246476557 shutdown_connections 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 --2- 172.21.15.170:0/1246476557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700ce1c0 0x7f27700ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 --2- 172.21.15.170:0/1246476557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700cb460 0x7f27700cb870 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 -- 172.21.15.170:0/1246476557 >> 172.21.15.170:0/1246476557 conn(0x7f27700b1c40 msgr2=0x7f27700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 -- 172.21.15.170:0/1246476557 shutdown_connections 2024-09-21T02:16:02.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.033+0000 7f27763e4640 1 -- 172.21.15.170:0/1246476557 wait complete. 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 Processor -- start 2024-09-21T02:16:02.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 -- start start 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27700cb460 0x7f27701f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 0x7f2770148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700cf7e0 0x7f2770149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27701690d0 con 0x7f27700ce1c0 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2770168f50 con 0x7f27700cb460 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2770169250 con 0x7f27700cf7e0 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 0x7f2770148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 0x7f2770148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48990/0 (socket says 172.21.15.170:48990) 2024-09-21T02:16:02.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 -- 172.21.15.170:0/367871347 learned_addr learned my addr 172.21.15.170:0/367871347 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f2774981640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700cf7e0 0x7f2770149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276ffff640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27700cb460 0x7f27701f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 -- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700cf7e0 msgr2=0x7f2770149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700cf7e0 0x7f2770149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 -- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27700cb460 msgr2=0x7f27701f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27700cb460 0x7f27701f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 -- 172.21.15.170:0/367871347 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277006a8a0 con 0x7f27700ce1c0 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276ffff640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27700cb460 0x7f27701f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f2774981640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700cf7e0 0x7f2770149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276f7fe640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 0x7f2770148e50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2764009790 tx=0x7f2764004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27640bb070 con 0x7f27700ce1c0 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2764005030 con 0x7f27700ce1c0 2024-09-21T02:16:02.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27700cd4c0 con 0x7f27700ce1c0 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27640b2040 con 0x7f27700ce1c0 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.037+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27700c71c0 con 0x7f27700ce1c0 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.041+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27700c64d0 con 0x7f27700ce1c0 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.041+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f2764005460 con 0x7f27700ce1c0 2024-09-21T02:16:02.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.041+0000 7f276cff9640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f275c073470 0x7f275c075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.041+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f276412e320 con 0x7f27700ce1c0 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.041+0000 7f276ffff640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f275c073470 0x7f275c075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:02.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.041+0000 7f276ffff640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f275c073470 0x7f275c075980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2760001630 tx=0x7f2760009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:02.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.045+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f27640f2290 con 0x7f27700ce1c0 2024-09-21T02:16:02.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.245+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27700cb870 con 0x7f27700ce1c0 2024-09-21T02:16:02.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.245+0000 7f276cff9640 1 -- 172.21.15.170:0/367871347 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f2764016410 con 0x7f27700ce1c0 2024-09-21T02:16:02.248 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:02.248 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:02.248 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f275c073470 msgr2=0x7f275c075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f275c073470 0x7f275c075980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2760001630 tx=0x7f2760009040 comp rx=0 tx=0).stop 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 msgr2=0x7f2770148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 0x7f2770148e50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2764009790 tx=0x7f2764004040 comp rx=0 tx=0).stop 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 shutdown_connections 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f275c073470 0x7f275c075980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27700cf7e0 0x7f2770149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27700ce1c0 0x7f2770148e50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 --2- 172.21.15.170:0/367871347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27700cb460 0x7f27701f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 >> 172.21.15.170:0/367871347 conn(0x7f27700b1c40 msgr2=0x7f27700b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:02.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 shutdown_connections 2024-09-21T02:16:02.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:02.249+0000 7f27763e4640 1 -- 172.21.15.170:0/367871347 wait complete. 2024-09-21T02:16:02.269 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:02.447 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.839326188000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.299155497194057,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:16:02.447 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.299155497194057 }, "state": "open", "uptime": 34.839326188 } ] 2024-09-21T02:16:02.448 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:07.449 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 -- 172.21.15.170:0/1159640827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 msgr2=0x7f16d80d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 --2- 172.21.15.170:0/1159640827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d80d0750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009c80 tx=0x7f16d001cd70 comp rx=0 tx=0).stop 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 -- 172.21.15.170:0/1159640827 shutdown_connections 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 --2- 172.21.15.170:0/1159640827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d80d0750 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 --2- 172.21.15.170:0/1159640827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16d8160f60 0x7f16d80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 -- 172.21.15.170:0/1159640827 >> 172.21.15.170:0/1159640827 conn(0x7f16d80711c0 msgr2=0x7f16d80715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 -- 172.21.15.170:0/1159640827 shutdown_connections 2024-09-21T02:16:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.613+0000 7f16de00c640 1 -- 172.21.15.170:0/1159640827 wait complete. 2024-09-21T02:16:07.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 Processor -- start 2024-09-21T02:16:07.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- start start 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d81507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16d8160b50 0x7f16d8161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16d8160f60 0x7f16d8161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8169d00 con 0x7f16d80cf1b0 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8169b80 con 0x7f16d8160b50 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f16d8169e80 con 0x7f16d8160f60 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d81507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d81507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57000/0 (socket says 172.21.15.170:57000) 2024-09-21T02:16:07.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 -- 172.21.15.170:0/964292787 learned_addr learned my addr 172.21.15.170:0/964292787 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d7fff640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16d8160f60 0x7f16d8161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d6ffd640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16d8160b50 0x7f16d8161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 -- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16d8160f60 msgr2=0x7f16d8161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16d8160f60 0x7f16d8161d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 -- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16d8160b50 msgr2=0x7f16d8161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16d8160b50 0x7f16d8161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 -- 172.21.15.170:0/964292787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81fa710 con 0x7f16d80cf1b0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16d77fe640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d81507d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c3c0 tx=0x7f16cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc017070 con 0x7f16d80cf1b0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16cc0053c0 con 0x7f16d80cf1b0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81fa9f0 con 0x7f16d80cf1b0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc003860 con 0x7f16d80cf1b0 2024-09-21T02:16:07.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81fafb0 con 0x7f16d80cf1b0 2024-09-21T02:16:07.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.617+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d80d0750 con 0x7f16d80cf1b0 2024-09-21T02:16:07.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.621+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f16cc01a070 con 0x7f16d80cf1b0 2024-09-21T02:16:07.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.621+0000 7f16bffff640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16c4073440 0x7f16c4075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:07.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.621+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f16cc093d10 con 0x7f16d80cf1b0 2024-09-21T02:16:07.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.621+0000 7f16d6ffd640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16c4073440 0x7f16c4075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:07.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.621+0000 7f16d6ffd640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16c4073440 0x7f16c4075950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16d815c8e0 tx=0x7f16c80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:07.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.625+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f16cc021020 con 0x7f16d80cf1b0 2024-09-21T02:16:07.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d8071750 con 0x7f16d80cf1b0 2024-09-21T02:16:07.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16bffff640 1 -- 172.21.15.170:0/964292787 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f16cc05e420 con 0x7f16d80cf1b0 2024-09-21T02:16:07.836 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:07.836 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:07.836 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:07.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16c4073440 msgr2=0x7f16c4075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16c4073440 0x7f16c4075950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16d815c8e0 tx=0x7f16c80091e0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 msgr2=0x7f16d81507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:07.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d81507d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c3c0 tx=0x7f16cc00c880 comp rx=0 tx=0).stop 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 shutdown_connections 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f16c4073440 0x7f16c4075950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16d8160f60 0x7f16d8161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16d8160b50 0x7f16d8161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.833+0000 7f16de00c640 1 --2- 172.21.15.170:0/964292787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16d80cf1b0 0x7f16d81507d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.837+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 >> 172.21.15.170:0/964292787 conn(0x7f16d80711c0 msgr2=0x7f16d8156850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.837+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 shutdown_connections 2024-09-21T02:16:07.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:07.837+0000 7f16de00c640 1 -- 172.21.15.170:0/964292787 wait complete. 2024-09-21T02:16:07.857 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:08.036 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":40.427246781000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.038760961662074,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:16:08.036 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.038760961662074 }, "state": "open", "uptime": 40.427246781 } ] 2024-09-21T02:16:08.037 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:13.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe70d5fc640 1 -- 172.21.15.170:0/1321056769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f0003a40 msgr2=0x7fe6f0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/1321056769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f0003a40 0x7fe6f0003eb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c2e0 tx=0x7fe6f801cf60 comp rx=0 tx=0).stop 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe7077fe640 1 -- 172.21.15.170:0/1321056769 reap_dead start 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe70d5fc640 1 -- 172.21.15.170:0/1321056769 shutdown_connections 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe70d5fc640 1 -- 172.21.15.170:0/1321056769 >> 172.21.15.170:0/1321056769 conn(0x7fe7080b1c40 msgr2=0x7fe7080b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe70d5fc640 1 -- 172.21.15.170:0/1321056769 shutdown_connections 2024-09-21T02:16:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.201+0000 7fe70d5fc640 1 -- 172.21.15.170:0/1321056769 wait complete. 2024-09-21T02:16:13.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 Processor -- start 2024-09-21T02:16:13.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- start start 2024-09-21T02:16:13.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f0003a40 0x7fe7081f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:13.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe708072e50 0x7fe70815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:13.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 0x7fe708074660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:13.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081691e0 con 0x7fe7080c7180 2024-09-21T02:16:13.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe708169060 con 0x7fe708072e50 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe708169360 con 0x7fe6f0003a40 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 0x7fe708074660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 0x7fe708074660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57024/0 (socket says 172.21.15.170:57024) 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 -- 172.21.15.170:0/2767971847 learned_addr learned my addr 172.21.15.170:0/2767971847 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe706ffd640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f0003a40 0x7fe7081f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7067fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe708072e50 0x7fe70815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 -- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f0003a40 msgr2=0x7fe7081f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f0003a40 0x7fe7081f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 -- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe708072e50 msgr2=0x7fe70815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe708072e50 0x7fe70815b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 -- 172.21.15.170:0/2767971847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe708076420 con 0x7fe7080c7180 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7067fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe708072e50 0x7fe70815b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe7077fe640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 0x7fe708074660 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe708070c40 tx=0x7fe70000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:13.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000f040 con 0x7fe7080c7180 2024-09-21T02:16:13.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70815bd90 con 0x7fe7080c7180 2024-09-21T02:16:13.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe700018070 con 0x7fe7080c7180 2024-09-21T02:16:13.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000c900 con 0x7fe7080c7180 2024-09-21T02:16:13.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7080738a0 con 0x7fe7080c7180 2024-09-21T02:16:13.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.205+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70806fa30 con 0x7fe7080c7180 2024-09-21T02:16:13.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.209+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7fe70000caa0 con 0x7fe7080c7180 2024-09-21T02:16:13.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.209+0000 7fe6ef7fe640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6f4073470 0x7fe6f4075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:13.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.209+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fe700094e40 con 0x7fe7080c7180 2024-09-21T02:16:13.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.209+0000 7fe706ffd640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6f4073470 0x7fe6f4075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:13.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.209+0000 7fe706ffd640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6f4073470 0x7fe6f4075980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80ab000 tx=0x7fe6f8019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.213+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe70005fab0 con 0x7fe7080c7180 2024-09-21T02:16:13.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.421+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe708073310 con 0x7fe7080c7180 2024-09-21T02:16:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.421+0000 7fe6ef7fe640 1 -- 172.21.15.170:0/2767971847 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7fe70005f3b0 con 0x7fe7080c7180 2024-09-21T02:16:13.425 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:13.425 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6f4073470 msgr2=0x7fe6f4075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6f4073470 0x7fe6f4075980 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80ab000 tx=0x7fe6f8019da0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 msgr2=0x7fe708074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 0x7fe708074660 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe708070c40 tx=0x7fe70000ec20 comp rx=0 tx=0).stop 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 shutdown_connections 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe6f4073470 0x7fe6f4075980 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7080c7180 0x7fe708074660 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe708072e50 0x7fe70815b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 --2- 172.21.15.170:0/2767971847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f0003a40 0x7fe7081f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 >> 172.21.15.170:0/2767971847 conn(0x7fe7080b1c40 msgr2=0x7fe708165c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 shutdown_connections 2024-09-21T02:16:13.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:13.425+0000 7fe70d5fc640 1 -- 172.21.15.170:0/2767971847 wait complete. 2024-09-21T02:16:13.447 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:13.627 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":46.019172470999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.781523674444657,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:16:13.627 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.781523674444657 }, "state": "open", "uptime": 46.019172471 } ] 2024-09-21T02:16:13.627 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:18.628 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 -- 172.21.15.170:0/3668511779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cf7e0 msgr2=0x7fefa0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 --2- 172.21.15.170:0/3668511779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cf7e0 0x7fefa0070340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fef90009840 tx=0x7fef9001c820 comp rx=0 tx=0).stop 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 -- 172.21.15.170:0/3668511779 shutdown_connections 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 --2- 172.21.15.170:0/3668511779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cf7e0 0x7fefa0070340 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 -- 172.21.15.170:0/3668511779 >> 172.21.15.170:0/3668511779 conn(0x7fefa00b1c40 msgr2=0x7fefa00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 -- 172.21.15.170:0/3668511779 shutdown_connections 2024-09-21T02:16:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.793+0000 7fefa5664640 1 -- 172.21.15.170:0/3668511779 wait complete. 2024-09-21T02:16:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 Processor -- start 2024-09-21T02:16:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 -- start start 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 0x7fefa0151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa00ce1c0 0x7fefa006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa00cf7e0 0x7fefa006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fefa016a360 con 0x7fefa00cb460 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fefa016a1e0 con 0x7fefa00cf7e0 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fefa5664640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fefa016a4e0 con 0x7fefa00ce1c0 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 0x7fefa0151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 0x7fefa0151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40262/0 (socket says 172.21.15.170:40262) 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 -- 172.21.15.170:0/3179894075 learned_addr learned my addr 172.21.15.170:0/3179894075 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9e7fc640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa00ce1c0 0x7fefa006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 -- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa00ce1c0 msgr2=0x7fefa006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa00ce1c0 0x7fefa006eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9f7fe640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa00cf7e0 0x7fefa006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 -- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa00cf7e0 msgr2=0x7fefa006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa00cf7e0 0x7fefa006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 -- 172.21.15.170:0/3179894075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01fd7f0 con 0x7fefa00cb460 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9f7fe640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa00cf7e0 0x7fefa006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef9effd640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 0x7fefa0151360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fef90009790 tx=0x7fef90004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.797+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900bb070 con 0x7fefa00cb460 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef90005030 con 0x7fefa00cb460 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa0152a80 con 0x7fefa00cb460 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900b2040 con 0x7fefa00cb460 2024-09-21T02:16:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa009d190 con 0x7fefa00cb460 2024-09-21T02:16:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa0070340 con 0x7fefa00cb460 2024-09-21T02:16:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7fef900be070 con 0x7fefa00cb460 2024-09-21T02:16:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef877fe640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef8c073440 0x7fef8c075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fef9012dff0 con 0x7fefa00cb460 2024-09-21T02:16:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef9e7fc640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef8c073440 0x7fef8c075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.801+0000 7fef9e7fc640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef8c073440 0x7fef8c075950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fefa00cd540 tx=0x7fef94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:18.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:18.805+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef90016410 con 0x7fefa00cb460 2024-09-21T02:16:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.009+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefa0158960 con 0x7fefa00cb460 2024-09-21T02:16:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fef877fe640 1 -- 172.21.15.170:0/3179894075 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7fef900f8b10 con 0x7fefa00cb460 2024-09-21T02:16:19.014 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:19.014 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef8c073440 msgr2=0x7fef8c075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef8c073440 0x7fef8c075950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fefa00cd540 tx=0x7fef94008040 comp rx=0 tx=0).stop 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 msgr2=0x7fefa0151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 0x7fefa0151360 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fef90009790 tx=0x7fef90004040 comp rx=0 tx=0).stop 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 shutdown_connections 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef8c073440 0x7fef8c075950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa00ce1c0 0x7fefa006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa00cf7e0 0x7fefa006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 --2- 172.21.15.170:0/3179894075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa00cb460 0x7fefa0151360 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 >> 172.21.15.170:0/3179894075 conn(0x7fefa00b1c40 msgr2=0x7fefa00ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 shutdown_connections 2024-09-21T02:16:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:19.013+0000 7fefa5664640 1 -- 172.21.15.170:0/3179894075 wait complete. 2024-09-21T02:16:19.036 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:19.217 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":51.611102344999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.527588349820849,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:16:19.218 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.52758834982085 }, "state": "open", "uptime": 51.611102345 } ] 2024-09-21T02:16:19.218 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:23.862 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:16:23.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:16:24.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:24.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.381+0000 7f8563b98640 1 -- 172.21.15.170:0/3976044182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 msgr2=0x7f8548004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:24.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.381+0000 7f8563b98640 1 --2- 172.21.15.170:0/3976044182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f8548004810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8558001690 tx=0x7f855801c7a0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.381+0000 7f8562135640 1 -- 172.21.15.170:0/3976044182 reap_dead start 2024-09-21T02:16:24.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.381+0000 7f8563b98640 1 -- 172.21.15.170:0/3976044182 shutdown_connections 2024-09-21T02:16:24.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.381+0000 7f8563b98640 1 -- 172.21.15.170:0/3976044182 >> 172.21.15.170:0/3976044182 conn(0x7f855c09a9c0 msgr2=0x7f855c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:24.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.381+0000 7f8563b98640 1 -- 172.21.15.170:0/3976044182 shutdown_connections 2024-09-21T02:16:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 -- 172.21.15.170:0/3976044182 wait complete. 2024-09-21T02:16:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 Processor -- start 2024-09-21T02:16:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 -- start start 2024-09-21T02:16:24.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f855c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:24.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f855c0c9b80 0x7f855c0c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:24.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f855c0c9f90 0x7f855c0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:24.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f855c0b6cc0 con 0x7f8548004060 2024-09-21T02:16:24.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f855c0b6b40 con 0x7f855c0c9f90 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8563b98640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f855c160aa0 con 0x7f855c0c9b80 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f855c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f855c15f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40280/0 (socket says 172.21.15.170:40280) 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 -- 172.21.15.170:0/4010892791 learned_addr learned my addr 172.21.15.170:0/4010892791 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561133640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f855c0c9b80 0x7f855c0c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 -- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f855c0c9b80 msgr2=0x7f855c0c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f855c0c9b80 0x7f855c0c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 -- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f855c0c9f90 msgr2=0x7f855c0c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f855c0c9f90 0x7f855c0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 -- 172.21.15.170:0/4010892791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c1f60f0 con 0x7f8548004060 2024-09-21T02:16:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f8561934640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f855c15f0b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8550006fd0 tx=0x7f85500107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:24.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.385+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550017030 con 0x7f8548004060 2024-09-21T02:16:24.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8550003c80 con 0x7f8548004060 2024-09-21T02:16:24.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c1f63d0 con 0x7f8548004060 2024-09-21T02:16:24.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550006370 con 0x7f8548004060 2024-09-21T02:16:24.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c1f6990 con 0x7f8548004060 2024-09-21T02:16:24.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f855c160fb0 con 0x7f8548004060 2024-09-21T02:16:24.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f855001a070 con 0x7f8548004060 2024-09-21T02:16:24.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f854e7fc640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8540073440 0x7f8540075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:24.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f8550097900 con 0x7f8548004060 2024-09-21T02:16:24.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f8561133640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8540073440 0x7f8540075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:24.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.389+0000 7f8561133640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8540073440 0x7f8540075950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8558001660 tx=0x7f8558019540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:24.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.393+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8550062300 con 0x7f8548004060 2024-09-21T02:16:24.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.605+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f855c0b6220 con 0x7f8548004060 2024-09-21T02:16:24.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f854e7fc640 1 -- 172.21.15.170:0/4010892791 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f8550061c00 con 0x7f8548004060 2024-09-21T02:16:24.610 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:24.610 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:24.610 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:24.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8540073440 msgr2=0x7f8540075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:24.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8540073440 0x7f8540075950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8558001660 tx=0x7f8558019540 comp rx=0 tx=0).stop 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 msgr2=0x7f855c15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f855c15f0b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8550006fd0 tx=0x7f85500107f0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 shutdown_connections 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8540073440 0x7f8540075950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f855c0c9b80 0x7f855c0c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f855c0c9f90 0x7f855c0c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 --2- 172.21.15.170:0/4010892791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548004060 0x7f855c15f0b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 >> 172.21.15.170:0/4010892791 conn(0x7f855c09a9c0 msgr2=0x7f855c0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 shutdown_connections 2024-09-21T02:16:24.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:24.609+0000 7f8563b98640 1 -- 172.21.15.170:0/4010892791 wait complete. 2024-09-21T02:16:24.633 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:24.813 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":57.206979527999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.276736982851919,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:16:24.814 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.27673698285192 }, "state": "open", "uptime": 57.206979528 } ] 2024-09-21T02:16:24.814 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:29.814 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:29.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 -- 172.21.15.170:0/3618250550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5158003a60 msgr2=0x7f5158003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:29.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 --2- 172.21.15.170:0/3618250550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5158003a60 0x7f5158003ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f516c0b3bc0 tx=0x7f515c01cf10 comp rx=0 tx=0).stop 2024-09-21T02:16:29.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 -- 172.21.15.170:0/3618250550 shutdown_connections 2024-09-21T02:16:29.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 --2- 172.21.15.170:0/3618250550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5158003a60 0x7f5158003ed0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:29.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 -- 172.21.15.170:0/3618250550 >> 172.21.15.170:0/3618250550 conn(0x7f516c08bf50 msgr2=0x7f516c09d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:29.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 -- 172.21.15.170:0/3618250550 shutdown_connections 2024-09-21T02:16:29.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.977+0000 7f517417c640 1 -- 172.21.15.170:0/3618250550 wait complete. 2024-09-21T02:16:29.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 Processor -- start 2024-09-21T02:16:29.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 -- start start 2024-09-21T02:16:29.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5158003a60 0x7f516c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 0x7f516c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f516c0ccd70 0x7f516c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0d1780 con 0x7f516c159a80 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0d1600 con 0x7f5158003a60 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f516c0d2b80 con 0x7f516c0ccd70 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 0x7f516c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 0x7f516c14dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:42696/0 (socket says 172.21.15.170:42696) 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 -- 172.21.15.170:0/699798195 learned_addr learned my addr 172.21.15.170:0/699798195 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5172719640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f516c0ccd70 0x7f516c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171f18640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5158003a60 0x7f516c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 -- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f516c0ccd70 msgr2=0x7f516c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f516c0ccd70 0x7f516c0cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 -- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5158003a60 msgr2=0x7f516c14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5158003a60 0x7f516c14d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 -- 172.21.15.170:0/699798195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c0cc890 con 0x7f516c159a80 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5172719640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f516c0ccd70 0x7f516c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f5171717640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 0x7f516c14dc70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f516800d560 tx=0x7f516800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680064e0 con 0x7f516c159a80 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5168003620 con 0x7f516c159a80 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c15ca10 con 0x7f516c159a80 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168003a50 con 0x7f516c159a80 2024-09-21T02:16:29.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.981+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c15f6a0 con 0x7f516c159a80 2024-09-21T02:16:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.985+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516c0d3050 con 0x7f516c159a80 2024-09-21T02:16:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.985+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f5168003bf0 con 0x7f516c159a80 2024-09-21T02:16:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.985+0000 7f51627fc640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5150073440 0x7f5150075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.985+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f5168095760 con 0x7f516c159a80 2024-09-21T02:16:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.985+0000 7f5171f18640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5150073440 0x7f5150075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:29.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.985+0000 7f5171f18640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5150073440 0x7f5150075950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f515c0ac000 tx=0x7f515c019d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:29.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:29.989+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5168060360 con 0x7f516c159a80 2024-09-21T02:16:30.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.193+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f516c161e00 con 0x7f516c159a80 2024-09-21T02:16:30.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f51627fc640 1 -- 172.21.15.170:0/699798195 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f516805fc60 con 0x7f516c159a80 2024-09-21T02:16:30.199 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:30.199 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:30.199 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5150073440 msgr2=0x7f5150075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5150073440 0x7f5150075950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f515c0ac000 tx=0x7f515c019d50 comp rx=0 tx=0).stop 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 msgr2=0x7f516c14dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 0x7f516c14dc70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f516800d560 tx=0x7f516800da20 comp rx=0 tx=0).stop 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 shutdown_connections 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5150073440 0x7f5150075950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f516c0ccd70 0x7f516c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f516c159a80 0x7f516c14dc70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 --2- 172.21.15.170:0/699798195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5158003a60 0x7f516c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 >> 172.21.15.170:0/699798195 conn(0x7f516c08bf50 msgr2=0x7f516c0b51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 shutdown_connections 2024-09-21T02:16:30.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:30.197+0000 7f517417c640 1 -- 172.21.15.170:0/699798195 wait complete. 2024-09-21T02:16:30.219 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:30.397 INFO:teuthology.orchestra.run.smithi191.stdout:[{"id":4925,"entity":{"name":{"type":"client","num":4925},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":2747198642}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":62.790844849999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.029636179839454,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.4925 172.21.15.191:0/2747198642","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:16:30.398 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 2747198642, "type": "any" }, "name": { "num": 4925, "type": "client" } }, "id": 4925, "inst": "client.4925 172.21.15.191:0/2747198642", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.029636179839454 }, "state": "open", "uptime": 62.79084485 } ] 2024-09-21T02:16:30.398 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T02:16:35.398 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 -- 172.21.15.170:0/3636063295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 msgr2=0x7f452c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 --2- 172.21.15.170:0/3636063295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f452c004c20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f453800bb10 tx=0x7f453801cda0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 -- 172.21.15.170:0/3636063295 shutdown_connections 2024-09-21T02:16:35.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 --2- 172.21.15.170:0/3636063295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f452c004c20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 -- 172.21.15.170:0/3636063295 >> 172.21.15.170:0/3636063295 conn(0x7f45440b1c40 msgr2=0x7f45440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:35.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 -- 172.21.15.170:0/3636063295 shutdown_connections 2024-09-21T02:16:35.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.561+0000 7f4548a1d640 1 -- 172.21.15.170:0/3636063295 wait complete. 2024-09-21T02:16:35.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 Processor -- start 2024-09-21T02:16:35.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- start start 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f45440cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45440ce530 0x7f454414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45441504c0 0x7f454409cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45440cf760 con 0x7f452c004810 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f45440c3a10 con 0x7f45440ce530 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f45440cf8e0 con 0x7f45441504c0 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f45440cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:35.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f45440cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44544/0 (socket says 172.21.15.170:44544) 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 -- 172.21.15.170:0/2064879926 learned_addr learned my addr 172.21.15.170:0/2064879926 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 -- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45441504c0 msgr2=0x7f454409cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45441504c0 0x7f454409cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4541d9b640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45440ce530 0x7f454414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 -- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45440ce530 msgr2=0x7f454414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45440ce530 0x7f454414ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 -- 172.21.15.170:0/2064879926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45440ce940 con 0x7f452c004810 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f454259c640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f45440cdff0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f45340017f0 tx=0x7f453400e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534012850 con 0x7f452c004810 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4534012ae0 con 0x7f452c004810 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4544151350 con 0x7f452c004810 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453401a400 con 0x7f452c004810 2024-09-21T02:16:35.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4544151070 con 0x7f452c004810 2024-09-21T02:16:35.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.565+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454415c770 con 0x7f452c004810 2024-09-21T02:16:35.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.569+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7f453400d030 con 0x7f452c004810 2024-09-21T02:16:35.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.569+0000 7f4532ffd640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45240734a0 0x7f45240759b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:35.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.569+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f4534097920 con 0x7f452c004810 2024-09-21T02:16:35.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.569+0000 7f4541d9b640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45240734a0 0x7f45240759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:35.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.569+0000 7f4541d9b640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45240734a0 0x7f45240759b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f45380b0000 tx=0x7f4538019b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:35.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.573+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f454415c770 con 0x7f452c004810 2024-09-21T02:16:35.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.781+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45440b3640 con 0x7f452c004810 2024-09-21T02:16:35.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4532ffd640 1 -- 172.21.15.170:0/2064879926 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f45440ce940 con 0x7f452c004810 2024-09-21T02:16:35.786 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:35.787 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:35.787 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45240734a0 msgr2=0x7f45240759b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45240734a0 0x7f45240759b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f45380b0000 tx=0x7f4538019b40 comp rx=0 tx=0).stop 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 msgr2=0x7f45440cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f45440cdff0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f45340017f0 tx=0x7f453400e770 comp rx=0 tx=0).stop 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 shutdown_connections 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45240734a0 0x7f45240759b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45441504c0 0x7f454409cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45440ce530 0x7f454414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 --2- 172.21.15.170:0/2064879926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f452c004810 0x7f45440cdff0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 >> 172.21.15.170:0/2064879926 conn(0x7f45440b1c40 msgr2=0x7f45440c8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 shutdown_connections 2024-09-21T02:16:35.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:35.785+0000 7f4548a1d640 1 -- 172.21.15.170:0/2064879926 wait complete. 2024-09-21T02:16:35.809 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-21T02:16:35.988 INFO:teuthology.orchestra.run.smithi191.stdout:[] 2024-09-21T02:16:35.989 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T02:16:35.989 DEBUG:tasks.ceph_test_case:wait_until_true: success in 60s and 0 retries 2024-09-21T02:16:35.989 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:16:35.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:16:36.051 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:16:36.117 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:16:36.117 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:16:36.117 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:16:36.117 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:36.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:16:36.123 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:16:36.124 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:16:36.124 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:36.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:16:36.129 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:16:36.129 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:16:36.129 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:36.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:16:36.170 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:16:36.171 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:16:36.171 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:36.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:16:36.176 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 -- 172.21.15.170:0/1727815410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cb460 msgr2=0x7facd00cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 --2- 172.21.15.170:0/1727815410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cb460 0x7facd00cb870 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7facb8009900 tx=0x7facb801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 -- 172.21.15.170:0/1727815410 shutdown_connections 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 --2- 172.21.15.170:0/1727815410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cb460 0x7facd00cb870 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 --2- 172.21.15.170:0/1727815410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00cf7e0 0x7facd0070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 -- 172.21.15.170:0/1727815410 >> 172.21.15.170:0/1727815410 conn(0x7facd00b1c40 msgr2=0x7facd00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 -- 172.21.15.170:0/1727815410 shutdown_connections 2024-09-21T02:16:36.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.377+0000 7facd6047640 1 -- 172.21.15.170:0/1727815410 wait complete. 2024-09-21T02:16:36.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 Processor -- start 2024-09-21T02:16:36.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- start start 2024-09-21T02:16:36.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00cb460 0x7facd009c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 0x7facd01f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00cf7e0 0x7facd015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7facd01690d0 con 0x7facd00cc8c0 2024-09-21T02:16:36.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7facd0168f50 con 0x7facd00cb460 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7facd0169250 con 0x7facd00cf7e0 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 0x7facd01f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 0x7facd01f9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44552/0 (socket says 172.21.15.170:44552) 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 -- 172.21.15.170:0/31146766 learned_addr learned my addr 172.21.15.170:0/31146766 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7faccffff640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00cf7e0 0x7facd015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7faccf7fe640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00cb460 0x7facd009c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 -- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00cf7e0 msgr2=0x7facd015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00cf7e0 0x7facd015c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 -- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00cb460 msgr2=0x7facd009c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00cb460 0x7facd009c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 -- 172.21.15.170:0/31146766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd009b620 con 0x7facd00cc8c0 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7faccf7fe640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00cb460 0x7facd009c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facc7fff640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 0x7facd01f9820 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7facb8009850 tx=0x7facb80a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:36.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80bc020 con 0x7facd00cc8c0 2024-09-21T02:16:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facb80044b0 con 0x7facd00cc8c0 2024-09-21T02:16:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd009b900 con 0x7facd00cc8c0 2024-09-21T02:16:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80b2040 con 0x7facd00cc8c0 2024-09-21T02:16:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd009bec0 con 0x7facd00cc8c0 2024-09-21T02:16:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.381+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd00cb870 con 0x7facd00cc8c0 2024-09-21T02:16:36.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.385+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7facb8003840 con 0x7facd00cc8c0 2024-09-21T02:16:36.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.385+0000 7facccff9640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7facb0073440 0x7facb0075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.385+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7facb812f470 con 0x7facd00cc8c0 2024-09-21T02:16:36.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.385+0000 7faccf7fe640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7facb0073440 0x7facb0075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.385+0000 7faccf7fe640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7facb0073440 0x7facb0075950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7facc00058d0 tx=0x7facc0005ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:36.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.389+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7facb80c6020 con 0x7facd00cc8c0 2024-09-21T02:16:36.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.597+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facd00ae890 con 0x7facd00cc8c0 2024-09-21T02:16:36.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facccff9640 1 -- 172.21.15.170:0/31146766 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7facb80164d0 con 0x7facd00cc8c0 2024-09-21T02:16:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:36.603 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:36.603 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7facb0073440 msgr2=0x7facb0075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7facb0073440 0x7facb0075950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7facc00058d0 tx=0x7facc0005ca0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 msgr2=0x7facd01f9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 0x7facd01f9820 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7facb8009850 tx=0x7facb80a5790 comp rx=0 tx=0).stop 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 shutdown_connections 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7facb0073440 0x7facb0075950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facd00cf7e0 0x7facd015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd00cc8c0 0x7facd01f9820 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 --2- 172.21.15.170:0/31146766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facd00cb460 0x7facd009c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 >> 172.21.15.170:0/31146766 conn(0x7facd00b1c40 msgr2=0x7facd00ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:36.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 shutdown_connections 2024-09-21T02:16:36.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.601+0000 7facd6047640 1 -- 172.21.15.170:0/31146766 wait complete. 2024-09-21T02:16:36.626 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 -- 172.21.15.170:0/2423368314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe18005310 msgr2=0x7efe1800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 --2- 172.21.15.170:0/2423368314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe18005310 0x7efe1800b9d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efe28009e30 tx=0x7efe2801d1b0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 -- 172.21.15.170:0/2423368314 shutdown_connections 2024-09-21T02:16:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 --2- 172.21.15.170:0/2423368314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe18005310 0x7efe1800b9d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 -- 172.21.15.170:0/2423368314 >> 172.21.15.170:0/2423368314 conn(0x7efe300b1c40 msgr2=0x7efe300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 -- 172.21.15.170:0/2423368314 shutdown_connections 2024-09-21T02:16:36.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 -- 172.21.15.170:0/2423368314 wait complete. 2024-09-21T02:16:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 Processor -- start 2024-09-21T02:16:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.769+0000 7efe36883640 1 -- start start 2024-09-21T02:16:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe18005310 0x7efe30074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe300c66b0 0x7efe30150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 0x7efe301f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe300cf760 con 0x7efe301f8a20 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe300c3a10 con 0x7efe300c66b0 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efe300cf8e0 con 0x7efe18005310 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 0x7efe301f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 0x7efe301f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44562/0 (socket says 172.21.15.170:44562) 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 -- 172.21.15.170:0/4059726978 learned_addr learned my addr 172.21.15.170:0/4059726978 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2ffff640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe18005310 0x7efe30074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2f7fe640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe300c66b0 0x7efe30150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 -- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe18005310 msgr2=0x7efe30074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe18005310 0x7efe30074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 -- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe300c66b0 msgr2=0x7efe30150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe300c66b0 0x7efe30150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 -- 172.21.15.170:0/4059726978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30151340 con 0x7efe301f8a20 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2f7fe640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe300c66b0 0x7efe30150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe34e20640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 0x7efe301f8e30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efe2801cfe0 tx=0x7efe280aca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280bd070 con 0x7efe301f8a20 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe28003980 con 0x7efe301f8a20 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301f9370 con 0x7efe301f8a20 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28005030 con 0x7efe301f8a20 2024-09-21T02:16:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe300736d0 con 0x7efe301f8a20 2024-09-21T02:16:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30073160 con 0x7efe301f8a20 2024-09-21T02:16:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) ==== 97438+0+0 (secure 0 0 0) 0x7efe280b5040 con 0x7efe301f8a20 2024-09-21T02:16:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.773+0000 7efe2cff9640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efe1c073440 0x7efe1c075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.777+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7efe2812f6b0 con 0x7efe301f8a20 2024-09-21T02:16:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.777+0000 7efe2ffff640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efe1c073440 0x7efe1c075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.777+0000 7efe2ffff640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efe1c073440 0x7efe1c075950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efe24001770 tx=0x7efe24006ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.777+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efe280c6080 con 0x7efe301f8a20 2024-09-21T02:16:36.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.981+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe300b34c0 con 0x7efe301f8a20 2024-09-21T02:16:36.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe2cff9640 1 -- 172.21.15.170:0/4059726978 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7efe301f9370 con 0x7efe301f8a20 2024-09-21T02:16:36.986 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:36.986 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:36.986 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efe1c073440 msgr2=0x7efe1c075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efe1c073440 0x7efe1c075950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efe24001770 tx=0x7efe24006ad0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 msgr2=0x7efe301f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 0x7efe301f8e30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efe2801cfe0 tx=0x7efe280aca80 comp rx=0 tx=0).stop 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 shutdown_connections 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7efe1c073440 0x7efe1c075950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe301f8a20 0x7efe301f8e30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe300c66b0 0x7efe30150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 --2- 172.21.15.170:0/4059726978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe18005310 0x7efe30074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 >> 172.21.15.170:0/4059726978 conn(0x7efe300b1c40 msgr2=0x7efe300b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 shutdown_connections 2024-09-21T02:16:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:36.985+0000 7efe36883640 1 -- 172.21.15.170:0/4059726978 wait complete. 2024-09-21T02:16:37.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 -- 172.21.15.170:0/1962382235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f252015f990 msgr2=0x7f2520070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1962382235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f252015f990 0x7f2520070030 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f25200cdc00 tx=0x7f251001c6b0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 -- 172.21.15.170:0/1962382235 shutdown_connections 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1962382235 >> v2:172.21.15.191:3300/0 conn(0x7f25200cadf0 0x7f25200cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1962382235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f252015f990 0x7f2520070030 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 -- 172.21.15.170:0/1962382235 >> 172.21.15.170:0/1962382235 conn(0x7f25200b1c40 msgr2=0x7f25200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 -- 172.21.15.170:0/1962382235 shutdown_connections 2024-09-21T02:16:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 -- 172.21.15.170:0/1962382235 wait complete. 2024-09-21T02:16:37.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 Processor -- start 2024-09-21T02:16:37.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.145+0000 7f2526bbe640 1 -- start start 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 0x7f25200aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25200cadf0 0x7f252015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f252015f990 0x7f252015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2520165fe0 con 0x7f2520070570 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25200cede0 con 0x7f25200cadf0 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2520166160 con 0x7f252015f990 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 0x7f25200aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 0x7f25200aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44584/0 (socket says 172.21.15.170:44584) 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 -- 172.21.15.170:0/1412501269 learned_addr learned my addr 172.21.15.170:0/1412501269 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251ffff640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25200cadf0 0x7f252015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 -- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f252015f990 msgr2=0x7f252015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f252015f990 0x7f252015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 -- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25200cadf0 msgr2=0x7f252015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25200cadf0 0x7f252015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 -- 172.21.15.170:0/1412501269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252015bc50 con 0x7f2520070570 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251ffff640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25200cadf0 0x7f252015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f252495a640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 0x7f25200aeef0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f251001cb90 tx=0x7f2510004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510004430 con 0x7f2520070570 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25100b9030 con 0x7f2520070570 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25100039b0 con 0x7f2520070570 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2520154850 con 0x7f2520070570 2024-09-21T02:16:37.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2520154e10 con 0x7f2520070570 2024-09-21T02:16:37.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25201554d0 con 0x7f2520070570 2024-09-21T02:16:37.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f2510003b50 con 0x7f2520070570 2024-09-21T02:16:37.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251d7fa640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f250c0733e0 0x7f250c0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2510130a50 con 0x7f2520070570 2024-09-21T02:16:37.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.149+0000 7f251ffff640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f250c0733e0 0x7f250c0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.153+0000 7f251ffff640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f250c0733e0 0x7f250c0758f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f252009d6e0 tx=0x7f2514008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:37.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.157+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25201554d0 con 0x7f2520070570 2024-09-21T02:16:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.313+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25200cb200 con 0x7f2520070570 2024-09-21T02:16:37.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.317+0000 7f251d7fa640 1 -- 172.21.15.170:0/1412501269 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+17009 (secure 0 0 0) 0x7f25200cb740 con 0x7f2520070570 2024-09-21T02:16:37.320 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:37.320 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:15:06.957556+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:52.409252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:53.674497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.28","mappings":[{"from":5,"to":0}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f250c0733e0 msgr2=0x7f250c0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f250c0733e0 0x7f250c0758f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f252009d6e0 tx=0x7f2514008040 comp rx=0 tx=0).stop 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 msgr2=0x7f25200aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 0x7f25200aeef0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f251001cb90 tx=0x7f2510004040 comp rx=0 tx=0).stop 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 shutdown_connections 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f250c0733e0 0x7f250c0758f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f252015f990 0x7f252015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25200cadf0 0x7f252015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 --2- 172.21.15.170:0/1412501269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2520070570 0x7f25200aeef0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 >> 172.21.15.170:0/1412501269 conn(0x7f25200b1c40 msgr2=0x7f25200ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 shutdown_connections 2024-09-21T02:16:37.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.321+0000 7f2526bbe640 1 -- 172.21.15.170:0/1412501269 wait complete. 2024-09-21T02:16:37.341 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:16:37.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 -- 172.21.15.170:0/1265646785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000cb460 msgr2=0x7f48000cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 --2- 172.21.15.170:0/1265646785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000cb460 0x7f48000cb870 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009900 tx=0x7f47f401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 -- 172.21.15.170:0/1265646785 shutdown_connections 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 --2- 172.21.15.170:0/1265646785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000ce1c0 0x7f48000ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 --2- 172.21.15.170:0/1265646785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000cb460 0x7f48000cb870 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 -- 172.21.15.170:0/1265646785 >> 172.21.15.170:0/1265646785 conn(0x7f48000b1c40 msgr2=0x7f48000b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 -- 172.21.15.170:0/1265646785 shutdown_connections 2024-09-21T02:16:37.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.489+0000 7f48055fe640 1 -- 172.21.15.170:0/1265646785 wait complete. 2024-09-21T02:16:37.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 Processor -- start 2024-09-21T02:16:37.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 -- start start 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48000cb460 0x7f4800156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 0x7f48000c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000cf7e0 0x7f48000c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f48001670c0 con 0x7f48000ce1c0 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4800166f40 con 0x7f48000cb460 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4800167240 con 0x7f48000cf7e0 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 0x7f48000c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 0x7f48000c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44594/0 (socket says 172.21.15.170:44594) 2024-09-21T02:16:37.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 -- 172.21.15.170:0/2167729331 learned_addr learned my addr 172.21.15.170:0/2167729331 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47ff7fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000cf7e0 0x7f48000c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 -- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000cf7e0 msgr2=0x7f48000c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47feffd640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48000cb460 0x7f4800156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000cf7e0 0x7f48000c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 -- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48000cb460 msgr2=0x7f4800156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48000cb460 0x7f4800156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 -- 172.21.15.170:0/2167729331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4800156c90 con 0x7f48000ce1c0 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47ff7fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000cf7e0 0x7f48000c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47fe7fc640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 0x7f48000c5650 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009850 tx=0x7f47f4004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40b2040 con 0x7f48000ce1c0 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f40036f0 con 0x7f48000ce1c0 2024-09-21T02:16:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480009d020 con 0x7f48000ce1c0 2024-09-21T02:16:37.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4003b00 con 0x7f48000ce1c0 2024-09-21T02:16:37.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.493+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480009d5e0 con 0x7f48000ce1c0 2024-09-21T02:16:37.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.497+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48000cb870 con 0x7f48000ce1c0 2024-09-21T02:16:37.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.497+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f47f4003ca0 con 0x7f48000ce1c0 2024-09-21T02:16:37.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.497+0000 7f47e77fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f47ec0733e0 0x7f47ec0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.497+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f47f412e9d0 con 0x7f48000ce1c0 2024-09-21T02:16:37.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.497+0000 7f47feffd640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f47ec0733e0 0x7f47ec0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.497+0000 7f47feffd640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f47ec0733e0 0x7f47ec0758f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47f0001570 tx=0x7f47f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:37.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.501+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f47f40bd060 con 0x7f48000ce1c0 2024-09-21T02:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.709+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48000b4670 con 0x7f48000ce1c0 2024-09-21T02:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.709+0000 7f47e77fe640 1 -- 172.21.15.170:0/2167729331 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f47f40164d0 con 0x7f48000ce1c0 2024-09-21T02:16:37.713 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:37.713 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f47ec0733e0 msgr2=0x7f47ec0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f47ec0733e0 0x7f47ec0758f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47f0001570 tx=0x7f47f0009040 comp rx=0 tx=0).stop 2024-09-21T02:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 msgr2=0x7f48000c5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 0x7f48000c5650 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009850 tx=0x7f47f4004220 comp rx=0 tx=0).stop 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 shutdown_connections 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f47ec0733e0 0x7f47ec0758f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48000cf7e0 0x7f48000c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48000ce1c0 0x7f48000c5650 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 --2- 172.21.15.170:0/2167729331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48000cb460 0x7f4800156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 >> 172.21.15.170:0/2167729331 conn(0x7f48000b1c40 msgr2=0x7f48000cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 shutdown_connections 2024-09-21T02:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.713+0000 7f48055fe640 1 -- 172.21.15.170:0/2167729331 wait complete. 2024-09-21T02:16:37.736 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 -- 172.21.15.170:0/672435551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 msgr2=0x7f3de815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 --2- 172.21.15.170:0/672435551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 0x7f3de815c850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009900 tx=0x7f3dd001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 -- 172.21.15.170:0/672435551 shutdown_connections 2024-09-21T02:16:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 --2- 172.21.15.170:0/672435551 >> v2:172.21.15.191:3300/0 conn(0x7f3de80d0aa0 0x7f3de80d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 --2- 172.21.15.170:0/672435551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 0x7f3de815c850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 -- 172.21.15.170:0/672435551 >> 172.21.15.170:0/672435551 conn(0x7f3de80cee30 msgr2=0x7f3de80cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 -- 172.21.15.170:0/672435551 shutdown_connections 2024-09-21T02:16:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.873+0000 7f3ded842640 1 -- 172.21.15.170:0/672435551 wait complete. 2024-09-21T02:16:37.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 Processor -- start 2024-09-21T02:16:37.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 -- start start 2024-09-21T02:16:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 0x7f3de806f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 0x7f3de8165a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3de80d0aa0 0x7f3de8159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3de816a560 con 0x7f3de80b3e30 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3de816a3e0 con 0x7f3de80d0aa0 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3de816a6e0 con 0x7f3de80b7ab0 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 0x7f3de806f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 0x7f3de806f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44604/0 (socket says 172.21.15.170:44604) 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 -- 172.21.15.170:0/1412054370 learned_addr learned my addr 172.21.15.170:0/1412054370 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de77fe640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3de80d0aa0 0x7f3de8159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de67fc640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 0x7f3de8165a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 -- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 msgr2=0x7f3de8165a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 0x7f3de8165a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 -- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3de80d0aa0 msgr2=0x7f3de8159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3de80d0aa0 0x7f3de8159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:37.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 -- 172.21.15.170:0/1412054370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de81fc720 con 0x7f3de80b3e30 2024-09-21T02:16:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3de6ffd640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 0x7f3de806f430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009850 tx=0x7f3dd0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00b2040 con 0x7f3de80b3e30 2024-09-21T02:16:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dd00bd070 con 0x7f3de80b3e30 2024-09-21T02:16:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de81fca00 con 0x7f3de80b3e30 2024-09-21T02:16:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0003930 con 0x7f3de80b3e30 2024-09-21T02:16:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.877+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de81fcfc0 con 0x7f3de80b3e30 2024-09-21T02:16:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.881+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de815c850 con 0x7f3de80b3e30 2024-09-21T02:16:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.881+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f3dd0003ad0 con 0x7f3de80b3e30 2024-09-21T02:16:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.881+0000 7f3ddffff640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3dc80733b0 0x7f3dc80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.881+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f3dd012f5f0 con 0x7f3de80b3e30 2024-09-21T02:16:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.881+0000 7f3de67fc640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3dc80733b0 0x7f3dc80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.881+0000 7f3de67fc640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3dc80733b0 0x7f3dc80758c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3de00012f0 tx=0x7f3de0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:37.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:37.885+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3dd00164d0 con 0x7f3de80b3e30 2024-09-21T02:16:38.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.081+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3de8072cc0 con 0x7f3de80b3e30 2024-09-21T02:16:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.081+0000 7f3ddffff640 1 -- 172.21.15.170:0/1412054370 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5235 (secure 0 0 0) 0x7f3dd00fa280 con 0x7f3de80b3e30 2024-09-21T02:16:38.084 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:38.084 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"btime":"2024-09-21T02:15:11:993337+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:14:54.692519+0000","modified":"2024-09-21T02:14:57.742100+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4984},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4984":{"gid":4984,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/2965172270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":2965172270},{"type":"v1","addr":"172.21.15.191:6837","nonce":2965172270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4984,"qdb_cluster":[4984]},"id":3}]} 2024-09-21T02:16:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 28 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3dc80733b0 msgr2=0x7f3dc80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3dc80733b0 0x7f3dc80758c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3de00012f0 tx=0x7f3de0009040 comp rx=0 tx=0).stop 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 msgr2=0x7f3de806f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 0x7f3de806f430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009850 tx=0x7f3dd0005760 comp rx=0 tx=0).stop 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 shutdown_connections 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3dc80733b0 0x7f3dc80758c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3de80b7ab0 0x7f3de8165a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3de80d0aa0 0x7f3de8159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 --2- 172.21.15.170:0/1412054370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de80b3e30 0x7f3de806f430 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 >> 172.21.15.170:0/1412054370 conn(0x7f3de80cee30 msgr2=0x7f3de80b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:38.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 shutdown_connections 2024-09-21T02:16:38.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.085+0000 7f3ded842640 1 -- 172.21.15.170:0/1412054370 wait complete. 2024-09-21T02:16:38.105 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:16:38.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 -- 172.21.15.170:0/3708218798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00ccb00 msgr2=0x7f7ce00ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/3708218798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00ccb00 0x7f7ce00ccf10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009c80 tx=0x7f7cd801cbc0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 -- 172.21.15.170:0/3708218798 shutdown_connections 2024-09-21T02:16:38.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/3708218798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00ccb00 0x7f7ce00ccf10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/3708218798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ce00b2f40 0x7f7ce00b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 -- 172.21.15.170:0/3708218798 >> 172.21.15.170:0/3708218798 conn(0x7f7ce00b1c40 msgr2=0x7f7ce00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:38.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 -- 172.21.15.170:0/3708218798 shutdown_connections 2024-09-21T02:16:38.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.257+0000 7f7ce68fb640 1 -- 172.21.15.170:0/3708218798 wait complete. 2024-09-21T02:16:38.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 Processor -- start 2024-09-21T02:16:38.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- start start 2024-09-21T02:16:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ce00b2f40 0x7f7ce015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 0x7f7ce0200850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce00ccb00 0x7f7ce0200d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce0169e50 con 0x7f7ce00b5410 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00cf970 con 0x7f7ce00ccb00 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce0169fd0 con 0x7f7ce00b2f40 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 0x7f7ce0200850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 0x7f7ce0200850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44614/0 (socket says 172.21.15.170:44614) 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 -- 172.21.15.170:0/1005445150 learned_addr learned my addr 172.21.15.170:0/1005445150 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdffff640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ce00b2f40 0x7f7ce015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce4e98640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce00ccb00 0x7f7ce0200d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 -- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ce00b2f40 msgr2=0x7f7ce015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ce00b2f40 0x7f7ce015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 -- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce00ccb00 msgr2=0x7f7ce0200d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce00ccb00 0x7f7ce0200d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 -- 172.21.15.170:0/1005445150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce00717c0 con 0x7f7ce00b5410 2024-09-21T02:16:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdf7fe640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 0x7f7ce0200850 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400c7a0 tx=0x7f7cd400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4007bb0 con 0x7f7ce00b5410 2024-09-21T02:16:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd4007e40 con 0x7f7ce00b5410 2024-09-21T02:16:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce0160020 con 0x7f7ce00b5410 2024-09-21T02:16:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4005280 con 0x7f7ce00b5410 2024-09-21T02:16:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce015f540 con 0x7f7ce00b5410 2024-09-21T02:16:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.261+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce015d8f0 con 0x7f7ce00b5410 2024-09-21T02:16:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.265+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f7cd4005420 con 0x7f7ce00b5410 2024-09-21T02:16:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.265+0000 7f7cdcff9640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7ccc0733b0 0x7f7ccc0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.265+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f7cd4094ba0 con 0x7f7ce00b5410 2024-09-21T02:16:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.265+0000 7f7cdffff640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7ccc0733b0 0x7f7ccc0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.265+0000 7f7cdffff640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7ccc0733b0 0x7f7ccc0758c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000b810 tx=0x7f7cd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:38.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.269+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7cd4014030 con 0x7f7ce00b5410 2024-09-21T02:16:38.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.433+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ce00b3350 con 0x7f7ce00b5410 2024-09-21T02:16:38.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.433+0000 7f7cdcff9640 1 -- 172.21.15.170:0/1005445150 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+17009 (secure 0 0 0) 0x7f7cd405f7a0 con 0x7f7ce00b5410 2024-09-21T02:16:38.437 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:38.437 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:15:06.957556+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:14:52.409252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T02:14:53.674497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.28","mappings":[{"from":5,"to":0}]},{"pgid":"7.31","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7ccc0733b0 msgr2=0x7f7ccc0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7ccc0733b0 0x7f7ccc0758c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000b810 tx=0x7f7cd0008040 comp rx=0 tx=0).stop 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 msgr2=0x7f7ce0200850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 0x7f7ce0200850 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400c7a0 tx=0x7f7cd400cc60 comp rx=0 tx=0).stop 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 shutdown_connections 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7ccc0733b0 0x7f7ccc0758c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce00ccb00 0x7f7ce0200d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ce00b5410 0x7f7ce0200850 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 --2- 172.21.15.170:0/1005445150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ce00b2f40 0x7f7ce015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 >> 172.21.15.170:0/1005445150 conn(0x7f7ce00b1c40 msgr2=0x7f7ce015c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:38.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 shutdown_connections 2024-09-21T02:16:38.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.437+0000 7f7ce68fb640 1 -- 172.21.15.170:0/1005445150 wait complete. 2024-09-21T02:16:38.459 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:16:38.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e847c640 1 -- 172.21.15.170:0/1516714417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc003fe0 msgr2=0x7f37cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e847c640 1 --2- 172.21.15.170:0/1516714417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37cc003fe0 0x7f37cc004810 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ba1a0 tx=0x7f37d001d130 comp rx=0 tx=0).stop 2024-09-21T02:16:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e6a19640 1 -- 172.21.15.170:0/1516714417 reap_dead start 2024-09-21T02:16:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e847c640 1 -- 172.21.15.170:0/1516714417 shutdown_connections 2024-09-21T02:16:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e847c640 1 -- 172.21.15.170:0/1516714417 >> 172.21.15.170:0/1516714417 conn(0x7f37e009d830 msgr2=0x7f37e008bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e847c640 1 -- 172.21.15.170:0/1516714417 shutdown_connections 2024-09-21T02:16:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.601+0000 7f37e847c640 1 -- 172.21.15.170:0/1516714417 wait complete. 2024-09-21T02:16:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 Processor -- start 2024-09-21T02:16:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 -- start start 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37cc003fe0 0x7f37e015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e01f9b00 0x7f37e00cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 0x7f37e00cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00cfde0 con 0x7f37e006de20 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00cfc60 con 0x7f37e01f9b00 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f37e016a1c0 con 0x7f37cc003fe0 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 0x7f37e00cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 0x7f37e00cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44634/0 (socket says 172.21.15.170:44634) 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 -- 172.21.15.170:0/1959080173 learned_addr learned my addr 172.21.15.170:0/1959080173 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e5a17640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e01f9b00 0x7f37e00cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6218640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37cc003fe0 0x7f37e015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 -- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37cc003fe0 msgr2=0x7f37e015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37cc003fe0 0x7f37e015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 -- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e01f9b00 msgr2=0x7f37e00cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e01f9b00 0x7f37e00cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 -- 172.21.15.170:0/1959080173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e015b0a0 con 0x7f37e006de20 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e5a17640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e01f9b00 0x7f37e00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e6a19640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 0x7f37e00cc370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f37e00b4ab0 tx=0x7f37d800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800b820 con 0x7f37e006de20 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37d800b9e0 con 0x7f37e006de20 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.605+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e015ba50 con 0x7f37e006de20 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800bc30 con 0x7f37e006de20 2024-09-21T02:16:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e015c010 con 0x7f37e006de20 2024-09-21T02:16:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e01f9fe0 con 0x7f37e006de20 2024-09-21T02:16:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f37d801a020 con 0x7f37e006de20 2024-09-21T02:16:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37d6ffd640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37c40733b0 0x7f37c40758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f37d8094160 con 0x7f37e006de20 2024-09-21T02:16:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37e6218640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37c40733b0 0x7f37c40758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.609+0000 7f37e6218640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37c40733b0 0x7f37c40758c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009a10 tx=0x7f37dc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.613+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f37d805edf0 con 0x7f37e006de20 2024-09-21T02:16:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.805+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f37e00b5df0 con 0x7f37e006de20 2024-09-21T02:16:38.811 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:16:38.809+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:16:38.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.809+0000 7f37d6ffd640 1 -- 172.21.15.170:0/1959080173 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f37d805e6f0 con 0x7f37e006de20 2024-09-21T02:16:38.813 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:16:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37c40733b0 msgr2=0x7f37c40758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37c40733b0 0x7f37c40758c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009a10 tx=0x7f37dc0091e0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 msgr2=0x7f37e00cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 0x7f37e00cc370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f37e00b4ab0 tx=0x7f37d800dcd0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 shutdown_connections 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37c40733b0 0x7f37c40758c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e006de20 0x7f37e00cc370 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37e01f9b00 0x7f37e00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 --2- 172.21.15.170:0/1959080173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f37cc003fe0 0x7f37e015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 >> 172.21.15.170:0/1959080173 conn(0x7f37e009d830 msgr2=0x7f37e00b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 shutdown_connections 2024-09-21T02:16:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.813+0000 7f37e847c640 1 -- 172.21.15.170:0/1959080173 wait complete. 2024-09-21T02:16:38.836 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:16:38.949 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:16:38.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0c779640 1 -- 172.21.15.170:0/209451449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9f0005280 msgr2=0x7fa9f000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0c779640 1 --2- 172.21.15.170:0/209451449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9f0005280 0x7fa9f000b9d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7faa040b60f0 tx=0x7fa9fc01cad0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0ad16640 1 -- 172.21.15.170:0/209451449 reap_dead start 2024-09-21T02:16:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0c779640 1 -- 172.21.15.170:0/209451449 shutdown_connections 2024-09-21T02:16:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0c779640 1 -- 172.21.15.170:0/209451449 >> 172.21.15.170:0/209451449 conn(0x7faa040d0170 msgr2=0x7faa040d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0c779640 1 -- 172.21.15.170:0/209451449 shutdown_connections 2024-09-21T02:16:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.989+0000 7faa0c779640 1 -- 172.21.15.170:0/209451449 wait complete. 2024-09-21T02:16:38.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 Processor -- start 2024-09-21T02:16:38.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 -- start start 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9f0005280 0x7faa041e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0414c970 0x7faa041e9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 0x7faa040b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa040b3550 con 0x7faa040b2130 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faa040b33d0 con 0x7faa0414c970 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7faa0406ff50 con 0x7fa9f0005280 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 0x7faa040b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 0x7faa040b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44638/0 (socket says 172.21.15.170:44638) 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 -- 172.21.15.170:0/106962622 learned_addr learned my addr 172.21.15.170:0/106962622 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa09d14640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0414c970 0x7faa041e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0a515640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9f0005280 0x7faa041e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:38.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 -- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9f0005280 msgr2=0x7faa041e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9f0005280 0x7faa041e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 -- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0414c970 msgr2=0x7faa041e9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0414c970 0x7faa041e9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 -- 172.21.15.170:0/106962622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa04157be0 con 0x7faa040b2130 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0ad16640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 0x7faa040b2540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0ab040 tx=0x7fa9fc003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc006cb0 con 0x7faa040b2130 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9fc0019f0 con 0x7faa040b2130 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa040b2a80 con 0x7faa040b2130 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc006330 con 0x7faa040b2130 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.993+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041f9ac0 con 0x7faa040b2130 2024-09-21T02:16:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.997+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa041f9550 con 0x7faa040b2130 2024-09-21T02:16:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.997+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fa9fc0064d0 con 0x7faa040b2130 2024-09-21T02:16:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.997+0000 7fa9faffd640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9e80733e0 0x7fa9e80758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.997+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5940+0+0 (secure 0 0 0) 0x7fa9fc12e6d0 con 0x7faa040b2130 2024-09-21T02:16:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.997+0000 7faa0a515640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9e80733e0 0x7fa9e80758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:38.997+0000 7faa0a515640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9e80733e0 0x7fa9e80758f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa00000fc0 tx=0x7faa00006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:39.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.001+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faa040b2a80 con 0x7faa040b2130 2024-09-21T02:16:39.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.201+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faa040704e0 con 0x7faa040b2130 2024-09-21T02:16:39.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.821+0000 7fa9faffd640 1 -- 172.21.15.170:0/106962622 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fa9fc016830 con 0x7faa040b2130 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9e80733e0 msgr2=0x7fa9e80758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9e80733e0 0x7fa9e80758f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa00000fc0 tx=0x7faa00006040 comp rx=0 tx=0).stop 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 msgr2=0x7faa040b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 0x7faa040b2540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0ab040 tx=0x7fa9fc003f40 comp rx=0 tx=0).stop 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 shutdown_connections 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9e80733e0 0x7fa9e80758f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa040b2130 0x7faa040b2540 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0414c970 0x7faa041e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 --2- 172.21.15.170:0/106962622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9f0005280 0x7faa041e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 >> 172.21.15.170:0/106962622 conn(0x7faa040d0170 msgr2=0x7faa0415d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:39.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 shutdown_connections 2024-09-21T02:16:39.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:39.825+0000 7faa0c779640 1 -- 172.21.15.170:0/106962622 wait complete. 2024-09-21T02:16:39.852 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 -- 172.21.15.170:0/1580079305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 msgr2=0x7f63e40b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/1580079305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40b5e90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009900 tx=0x7f63e001c820 comp rx=0 tx=0).stop 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 -- 172.21.15.170:0/1580079305 shutdown_connections 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/1580079305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40b5e90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/1580079305 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63e415b810 0x7f63e40736b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 -- 172.21.15.170:0/1580079305 >> 172.21.15.170:0/1580079305 conn(0x7f63e40b2720 msgr2=0x7f63e40b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 -- 172.21.15.170:0/1580079305 shutdown_connections 2024-09-21T02:16:40.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.009+0000 7f63ebd4b640 1 -- 172.21.15.170:0/1580079305 wait complete. 2024-09-21T02:16:40.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 Processor -- start 2024-09-21T02:16:40.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- start start 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63e40704c0 0x7f63e40c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e415b810 0x7f63e41f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41691a0 con 0x7f63e40700b0 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4169020 con 0x7f63e415b810 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f63e4169320 con 0x7f63e40704c0 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44662/0 (socket says 172.21.15.170:44662) 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 -- 172.21.15.170:0/2322317194 learned_addr learned my addr 172.21.15.170:0/2322317194 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 -- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63e40704c0 msgr2=0x7f63e40c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:40.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e92e6640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63e40704c0 0x7f63e40c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ea2e8640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e415b810 0x7f63e41f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63e40704c0 0x7f63e40c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 -- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e415b810 msgr2=0x7f63e41f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e415b810 0x7f63e41f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 -- 172.21.15.170:0/2322317194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40c7c50 con 0x7f63e40700b0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63e9ae7640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40c6b60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f63d400d750 tx=0x7f63d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4017070 con 0x7f63e40700b0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63d4009d20 con 0x7f63e40700b0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e40c7f30 con 0x7f63e40700b0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4005280 con 0x7f63e40700b0 2024-09-21T02:16:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e40c84f0 con 0x7f63e40700b0 2024-09-21T02:16:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.013+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e40b5e90 con 0x7f63e40700b0 2024-09-21T02:16:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.017+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f63d4005420 con 0x7f63e40700b0 2024-09-21T02:16:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.017+0000 7f63da7fc640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63c80733b0 0x7f63c80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.017+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5940+0+0 (secure 0 0 0) 0x7f63d4094890 con 0x7f63e40700b0 2024-09-21T02:16:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.017+0000 7f63e92e6640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63c80733b0 0x7f63c80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:40.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.017+0000 7f63e92e6640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63c80733b0 0x7f63c80758c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63e00af000 tx=0x7f63e001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.021+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f63d400de60 con 0x7f63e40700b0 2024-09-21T02:16:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:40.201+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63e40b2d80 con 0x7f63e40700b0 2024-09-21T02:16:41.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.041+0000 7f63da7fc640 1 -- 172.21.15.170:0/2322317194 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7f63d405f430 con 0x7f63e40700b0 2024-09-21T02:16:41.042 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:16:41.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63c80733b0 msgr2=0x7f63c80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:41.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63c80733b0 0x7f63c80758c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63e00af000 tx=0x7f63e001b060 comp rx=0 tx=0).stop 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 msgr2=0x7f63e40c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40c6b60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f63d400d750 tx=0x7f63d400dc10 comp rx=0 tx=0).stop 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 shutdown_connections 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f63c80733b0 0x7f63c80758c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.045+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63e40704c0 0x7f63e40c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.049+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63e415b810 0x7f63e41f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.049+0000 7f63ebd4b640 1 --2- 172.21.15.170:0/2322317194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e40700b0 0x7f63e40c6b60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.049+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 >> 172.21.15.170:0/2322317194 conn(0x7f63e40b2720 msgr2=0x7f63e40d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:41.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.049+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 shutdown_connections 2024-09-21T02:16:41.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.049+0000 7f63ebd4b640 1 -- 172.21.15.170:0/2322317194 wait complete. 2024-09-21T02:16:41.076 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:16:41.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 -- 172.21.15.170:0/3085239354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a18004810 msgr2=0x7f1a18004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:41.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 --2- 172.21.15.170:0/3085239354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a18004810 0x7f1a18004c20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1a240016c0 tx=0x7f1a2401c730 comp rx=0 tx=0).stop 2024-09-21T02:16:41.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 -- 172.21.15.170:0/3085239354 shutdown_connections 2024-09-21T02:16:41.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 --2- 172.21.15.170:0/3085239354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a18004810 0x7f1a18004c20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 -- 172.21.15.170:0/3085239354 >> 172.21.15.170:0/3085239354 conn(0x7f1a300b29c0 msgr2=0x7f1a300b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:41.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 -- 172.21.15.170:0/3085239354 shutdown_connections 2024-09-21T02:16:41.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.229+0000 7f1a355be640 1 -- 172.21.15.170:0/3085239354 wait complete. 2024-09-21T02:16:41.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 Processor -- start 2024-09-21T02:16:41.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- start start 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a18004810 0x7f1a300ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a30143d30 0x7f1a301f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 0x7f1a300b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015d1d0 con 0x7f1a301f9d90 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015d050 con 0x7f1a18004810 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3015d350 con 0x7f1a30143d30 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 0x7f1a300b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 0x7f1a300b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44674/0 (socket says 172.21.15.170:44674) 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 -- 172.21.15.170:0/2325990221 learned_addr learned my addr 172.21.15.170:0/2325990221 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2effd640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a18004810 0x7f1a300ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2e7fc640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a30143d30 0x7f1a301f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 -- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a30143d30 msgr2=0x7f1a301f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a30143d30 0x7f1a301f9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 -- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a18004810 msgr2=0x7f1a300ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a18004810 0x7f1a300ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 -- 172.21.15.170:0/2325990221 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301fa1a0 con 0x7f1a301f9d90 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a2f7fe640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 0x7f1a300b5870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1a300cfe60 tx=0x7f1a2800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28004b30 con 0x7f1a301f9d90 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a28004dc0 con 0x7f1a301f9d90 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a300c7af0 con 0x7f1a301f9d90 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800dd00 con 0x7f1a301f9d90 2024-09-21T02:16:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a300c80b0 con 0x7f1a301f9d90 2024-09-21T02:16:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.233+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a301f7110 con 0x7f1a301f9d90 2024-09-21T02:16:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.237+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f1a2801c020 con 0x7f1a301f9d90 2024-09-21T02:16:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.237+0000 7f1a177fe640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a1c0733b0 0x7f1a1c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.237+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5502+0+0 (secure 0 0 0) 0x7f1a280942a0 con 0x7f1a301f9d90 2024-09-21T02:16:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.237+0000 7f1a2effd640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a1c0733b0 0x7f1a1c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.237+0000 7f1a2effd640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a1c0733b0 0x7f1a1c0758c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a20001870 tx=0x7f1a20000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.241+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a301fa1a0 con 0x7f1a301f9d90 2024-09-21T02:16:41.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:41.401+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1a3006e0f0 con 0x7f1a301f9d90 2024-09-21T02:16:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.041+0000 7f1a177fe640 1 -- 172.21.15.170:0/2325990221 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7f1a2805f090 con 0x7f1a301f9d90 2024-09-21T02:16:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:16:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a1c0733b0 msgr2=0x7f1a1c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a1c0733b0 0x7f1a1c0758c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a20001870 tx=0x7f1a20000ee0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 msgr2=0x7f1a300b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 0x7f1a300b5870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1a300cfe60 tx=0x7f1a2800fd90 comp rx=0 tx=0).stop 2024-09-21T02:16:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 shutdown_connections 2024-09-21T02:16:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a1c0733b0 0x7f1a1c0758c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a30143d30 0x7f1a301f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a301f9d90 0x7f1a300b5870 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 --2- 172.21.15.170:0/2325990221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a18004810 0x7f1a300ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.045+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 >> 172.21.15.170:0/2325990221 conn(0x7f1a300b29c0 msgr2=0x7f1a30161230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:42.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.049+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 shutdown_connections 2024-09-21T02:16:42.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.049+0000 7f1a355be640 1 -- 172.21.15.170:0/2325990221 wait complete. 2024-09-21T02:16:42.076 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:16:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 -- 172.21.15.170:0/276989337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 msgr2=0x7fa048004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 --2- 172.21.15.170:0/276989337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa048004c20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa0600b3bc0 tx=0x7fa05401cf10 comp rx=0 tx=0).stop 2024-09-21T02:16:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 -- 172.21.15.170:0/276989337 shutdown_connections 2024-09-21T02:16:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 --2- 172.21.15.170:0/276989337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa048004c20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 -- 172.21.15.170:0/276989337 >> 172.21.15.170:0/276989337 conn(0x7fa06008bf50 msgr2=0x7fa06009d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 -- 172.21.15.170:0/276989337 shutdown_connections 2024-09-21T02:16:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.225+0000 7fa065b37640 1 -- 172.21.15.170:0/276989337 wait complete. 2024-09-21T02:16:42.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 Processor -- start 2024-09-21T02:16:42.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 -- start start 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa0600cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0601f9bc0 0x7fa06014d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0601f8f40 0x7fa0601f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0600b1db0 con 0x7fa048004810 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0600b1c30 con 0x7fa0601f8f40 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa0600b1f30 con 0x7fa0601f9bc0 2024-09-21T02:16:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa0600cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa0600cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44684/0 (socket says 172.21.15.170:44684) 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 -- 172.21.15.170:0/1492161200 learned_addr learned my addr 172.21.15.170:0/1492161200 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05effd640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0601f9bc0 0x7fa06014d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 -- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0601f9bc0 msgr2=0x7fa06014d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0601f9bc0 0x7fa06014d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 -- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0601f8f40 msgr2=0x7fa0601f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0601f8f40 0x7fa0601f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 -- 172.21.15.170:0/1492161200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0600cd930 con 0x7fa048004810 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05effd640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0601f9bc0 0x7fa06014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa05f7fe640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa0600cd030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa050004d70 tx=0x7fa050001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050011660 con 0x7fa048004810 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa050016070 con 0x7fa048004810 2024-09-21T02:16:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0600cdc10 con 0x7fa048004810 2024-09-21T02:16:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.229+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050011b70 con 0x7fa048004810 2024-09-21T02:16:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06015b1a0 con 0x7fa048004810 2024-09-21T02:16:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0600d26f0 con 0x7fa048004810 2024-09-21T02:16:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fa05001b460 con 0x7fa048004810 2024-09-21T02:16:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa047fff640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa04c0733b0 0x7fa04c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fa050094c40 con 0x7fa048004810 2024-09-21T02:16:42.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa05effd640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa04c0733b0 0x7fa04c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.233+0000 7fa05effd640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa04c0733b0 0x7fa04c0758c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0540ac000 tx=0x7fa054019d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:42.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.237+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa05005fc60 con 0x7fa048004810 2024-09-21T02:16:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.413+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fa060161500 con 0x7fa048004810 2024-09-21T02:16:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.417+0000 7fa047fff640 1 -- 172.21.15.170:0/1492161200 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7fa05005f560 con 0x7fa048004810 2024-09-21T02:16:42.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa04c0733b0 msgr2=0x7fa04c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa04c0733b0 0x7fa04c0758c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0540ac000 tx=0x7fa054019d50 comp rx=0 tx=0).stop 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 msgr2=0x7fa0600cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa0600cd030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa050004d70 tx=0x7fa050001640 comp rx=0 tx=0).stop 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 shutdown_connections 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa04c0733b0 0x7fa04c0758c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0601f9bc0 0x7fa06014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0601f8f40 0x7fa0601f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 --2- 172.21.15.170:0/1492161200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048004810 0x7fa0600cd030 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 >> 172.21.15.170:0/1492161200 conn(0x7fa06008bf50 msgr2=0x7fa0600b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 shutdown_connections 2024-09-21T02:16:42.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.421+0000 7fa065b37640 1 -- 172.21.15.170:0/1492161200 wait complete. 2024-09-21T02:16:42.443 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-21T02:16:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 -- 172.21.15.170:0/3893451590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 msgr2=0x7f91780b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 --2- 172.21.15.170:0/3893451590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f91780b5ef0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f917809de30 tx=0x7f916001c7b0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 -- 172.21.15.170:0/3893451590 shutdown_connections 2024-09-21T02:16:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 --2- 172.21.15.170:0/3893451590 >> v2:172.21.15.191:3300/0 conn(0x7f91780b6430 0x7f91780cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 --2- 172.21.15.170:0/3893451590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f91780b5ef0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 -- 172.21.15.170:0/3893451590 >> 172.21.15.170:0/3893451590 conn(0x7f917809d7d0 msgr2=0x7f91780d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 -- 172.21.15.170:0/3893451590 shutdown_connections 2024-09-21T02:16:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.581+0000 7f917d627640 1 -- 172.21.15.170:0/3893451590 wait complete. 2024-09-21T02:16:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 Processor -- start 2024-09-21T02:16:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- start start 2024-09-21T02:16:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91780b6430 0x7f91780cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91780b9ba0 0x7f9178158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f917815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f917816c310 con 0x7f91780d4e00 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f917816c190 con 0x7f91780b6430 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f917816c490 con 0x7f91780b9ba0 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f917815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f917815a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44692/0 (socket says 172.21.15.170:44692) 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 -- 172.21.15.170:0/193903011 learned_addr learned my addr 172.21.15.170:0/193903011 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f9176ffd640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91780b6430 0x7f91780cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f916e7fc640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91780b9ba0 0x7f9178158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 -- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91780b9ba0 msgr2=0x7f9178158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91780b9ba0 0x7f9178158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 -- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91780b6430 msgr2=0x7f91780cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91780b6430 0x7f91780cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 -- 172.21.15.170:0/193903011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91781fcca0 con 0x7f91780d4e00 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f9176ffd640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91780b6430 0x7f91780cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f91777fe640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f917815a0f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f917000b970 tx=0x7f917000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170019070 con 0x7f91780d4e00 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f917000cbf0 con 0x7f91780d4e00 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91781fcf80 con 0x7f91780d4e00 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170014440 con 0x7f91780d4e00 2024-09-21T02:16:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9178158ec0 con 0x7f91780d4e00 2024-09-21T02:16:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.585+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91781fca10 con 0x7f91780d4e00 2024-09-21T02:16:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.589+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f917001c070 con 0x7f91780d4e00 2024-09-21T02:16:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.589+0000 7f916ffff640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f91580733b0 0x7f91580758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.589+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f9170094020 con 0x7f91780d4e00 2024-09-21T02:16:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.589+0000 7f9176ffd640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f91580733b0 0x7f91580758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.589+0000 7f9176ffd640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f91580733b0 0x7f91580758c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9168001630 tx=0x7f9168009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:42.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.593+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f917005f040 con 0x7f91780d4e00 2024-09-21T02:16:42.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:42.829+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f91780cfc60 con 0x7f91780d4e00 2024-09-21T02:16:43.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.065+0000 7f916ffff640 1 -- 172.21.15.170:0/193903011 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v174) ==== 140+0+0 (secure 0 0 0) 0x7f917005e940 con 0x7f91780d4e00 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f91580733b0 msgr2=0x7f91580758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f91580733b0 0x7f91580758c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9168001630 tx=0x7f9168009040 comp rx=0 tx=0).stop 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 msgr2=0x7f917815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f917815a0f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f917000b970 tx=0x7f917000be30 comp rx=0 tx=0).stop 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 shutdown_connections 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f91580733b0 0x7f91580758c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91780b9ba0 0x7f9178158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91780d4e00 0x7f917815a0f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 --2- 172.21.15.170:0/193903011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91780b6430 0x7f91780cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 >> 172.21.15.170:0/193903011 conn(0x7f917809d7d0 msgr2=0x7f917815e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 shutdown_connections 2024-09-21T02:16:43.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.069+0000 7f917d627640 1 -- 172.21.15.170:0/193903011 wait complete. 2024-09-21T02:16:43.095 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-21T02:16:43.096 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-21T02:16:43.096 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:16:43.096 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:16:43.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1909908084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd2c003a60 msgr2=0x7fcd2c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1909908084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd2c003a60 0x7fcd2c003ed0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcd38006fd0 tx=0x7fcd3801cf40 comp rx=0 tx=0).stop 2024-09-21T02:16:43.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1909908084 shutdown_connections 2024-09-21T02:16:43.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1909908084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd2c003a60 0x7fcd2c003ed0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1909908084 >> 172.21.15.170:0/1909908084 conn(0x7fcd440b1c40 msgr2=0x7fcd440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:43.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1909908084 shutdown_connections 2024-09-21T02:16:43.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.245+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1909908084 wait complete. 2024-09-21T02:16:43.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 Processor -- start 2024-09-21T02:16:43.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 -- start start 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 0x7fcd4414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd441f8c30 0x7fcd4414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd441fc620 0x7fcd441fca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440c3980 con 0x7fcd441f8820 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440c3800 con 0x7fcd441fc620 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd4a59e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcd440cf760 con 0x7fcd441f8c30 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 0x7fcd4414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 0x7fcd4414dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44708/0 (socket says 172.21.15.170:44708) 2024-09-21T02:16:43.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 -- 172.21.15.170:0/1278276930 learned_addr learned my addr 172.21.15.170:0/1278276930 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd48b3b640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd441fc620 0x7fcd441fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd437fe640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd441f8c30 0x7fcd4414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 -- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd441f8c30 msgr2=0x7fcd4414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd441f8c30 0x7fcd4414e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 -- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd441fc620 msgr2=0x7fcd441fca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd441fc620 0x7fcd441fca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 -- 172.21.15.170:0/1278276930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4409cf00 con 0x7fcd441f8820 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd48b3b640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd441fc620 0x7fcd441fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd43fff640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 0x7fcd4414dc80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801ca00 tx=0x7fcd38001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.249+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380a7e00 con 0x7fcd441f8820 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd38001500 con 0x7fcd441f8820 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4409d1e0 con 0x7fcd441f8820 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38006280 con 0x7fcd441f8820 2024-09-21T02:16:43.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4409adf0 con 0x7fcd441f8820 2024-09-21T02:16:43.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd440ce0d0 con 0x7fcd441f8820 2024-09-21T02:16:43.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fcd38006440 con 0x7fcd441f8820 2024-09-21T02:16:43.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd40ff9640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fcd300733b0 0x7fcd300758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fcd3812dc80 con 0x7fcd441f8820 2024-09-21T02:16:43.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd437fe640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fcd300733b0 0x7fcd300758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.253+0000 7fcd437fe640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fcd300733b0 0x7fcd300758c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd44070d90 tx=0x7fcd340069b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:43.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.257+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcd380bf030 con 0x7fcd441f8820 2024-09-21T02:16:43.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.421+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd440c6b30 con 0x7fcd441f8820 2024-09-21T02:16:43.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.421+0000 7fcd40ff9640 1 -- 172.21.15.170:0/1278276930 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13082 (secure 0 0 0) 0x7fcd38016cf0 con 0x7fcd441f8820 2024-09-21T02:16:43.425 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:43.425 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":51,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:16:42.042547+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/2965172270":"2024-09-22T02:16:38.811529+0000","172.21.15.191:6836/2965172270":"2024-09-22T02:16:38.811529+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fcd300733b0 msgr2=0x7fcd300758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fcd300733b0 0x7fcd300758c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd44070d90 tx=0x7fcd340069b0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 msgr2=0x7fcd4414dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 0x7fcd4414dc80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801ca00 tx=0x7fcd38001d90 comp rx=0 tx=0).stop 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 shutdown_connections 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fcd300733b0 0x7fcd300758c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd441f8c30 0x7fcd4414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd441fc620 0x7fcd441fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 --2- 172.21.15.170:0/1278276930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd441f8820 0x7fcd4414dc80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 >> 172.21.15.170:0/1278276930 conn(0x7fcd440b1c40 msgr2=0x7fcd440c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 shutdown_connections 2024-09-21T02:16:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.425+0000 7fcd4a59e640 1 -- 172.21.15.170:0/1278276930 wait complete. 2024-09-21T02:16:43.446 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:16:43.446 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:16:43.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 -- 172.21.15.170:0/614622523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f329c005360 msgr2=0x7f329c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/614622523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f329c005360 0x7f329c00b9d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f32ac001dc0 tx=0x7f32ac01c870 comp rx=0 tx=0).stop 2024-09-21T02:16:43.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 -- 172.21.15.170:0/614622523 shutdown_connections 2024-09-21T02:16:43.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/614622523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f329c005360 0x7f329c00b9d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 -- 172.21.15.170:0/614622523 >> 172.21.15.170:0/614622523 conn(0x7f32b40d0170 msgr2=0x7f32b40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:43.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 -- 172.21.15.170:0/614622523 shutdown_connections 2024-09-21T02:16:43.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.589+0000 7f32b8f1a640 1 -- 172.21.15.170:0/614622523 wait complete. 2024-09-21T02:16:43.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 Processor -- start 2024-09-21T02:16:43.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- start start 2024-09-21T02:16:43.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329c005360 0x7f32b40c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 0x7f32b41538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32b4153df0 0x7f32b409bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32b40ce070 con 0x7f32b41534a0 2024-09-21T02:16:43.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f32b406fe20 con 0x7f329c005360 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f32b40ce1f0 con 0x7f32b4153df0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 0x7f32b41538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 0x7f32b41538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44724/0 (socket says 172.21.15.170:44724) 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 -- 172.21.15.170:0/4168834229 learned_addr learned my addr 172.21.15.170:0/4168834229 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b2d9d640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32b4153df0 0x7f32b409bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b259c640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329c005360 0x7f32b40c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 -- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32b4153df0 msgr2=0x7f32b409bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32b4153df0 0x7f32b409bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 -- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329c005360 msgr2=0x7f32b40c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329c005360 0x7f32b40c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 -- 172.21.15.170:0/4168834229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b41f6f80 con 0x7f32b41534a0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b1d9b640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 0x7f32b41538b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f32a400f3a0 tx=0x7f32a400f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a401a070 con 0x7f32b41534a0 2024-09-21T02:16:43.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4008520 con 0x7f32b41534a0 2024-09-21T02:16:43.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b41f7260 con 0x7f32b41534a0 2024-09-21T02:16:43.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4006980 con 0x7f32b41534a0 2024-09-21T02:16:43.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b40c6880 con 0x7f32b41534a0 2024-09-21T02:16:43.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.593+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b40ce6e0 con 0x7f32b41534a0 2024-09-21T02:16:43.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.597+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f32a401d070 con 0x7f32b41534a0 2024-09-21T02:16:43.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.597+0000 7f32a2ffd640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f32940733b0 0x7f32940758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.597+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f32a4096310 con 0x7f32b41534a0 2024-09-21T02:16:43.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.597+0000 7f32b259c640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f32940733b0 0x7f32940758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.597+0000 7f32b259c640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f32940733b0 0x7f32940758c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009970 tx=0x7f32a80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:43.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.601+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f32a4060d70 con 0x7f32b41534a0 2024-09-21T02:16:43.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.773+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32b41f6cd0 con 0x7f32b41534a0 2024-09-21T02:16:43.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.773+0000 7f32a2ffd640 1 -- 172.21.15.170:0/4168834229 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f32a40657a0 con 0x7f32b41534a0 2024-09-21T02:16:43.777 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f32940733b0 msgr2=0x7f32940758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f32940733b0 0x7f32940758c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009970 tx=0x7f32a80091e0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 msgr2=0x7f32b41538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 0x7f32b41538b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f32a400f3a0 tx=0x7f32a400f860 comp rx=0 tx=0).stop 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 shutdown_connections 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f32940733b0 0x7f32940758c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32b4153df0 0x7f32b409bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b41534a0 0x7f32b41538b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 --2- 172.21.15.170:0/4168834229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f329c005360 0x7f32b40c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 >> 172.21.15.170:0/4168834229 conn(0x7f32b40d0170 msgr2=0x7f32b40c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:43.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 shutdown_connections 2024-09-21T02:16:43.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.777+0000 7f32b8f1a640 1 -- 172.21.15.170:0/4168834229 wait complete. 2024-09-21T02:16:43.800 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-21T02:16:43.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/925531940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 msgr2=0x7f4a14070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/925531940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a14070340 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a140710f0 tx=0x7f4a0401c800 comp rx=0 tx=0).stop 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/925531940 shutdown_connections 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/925531940 >> v2:172.21.15.191:3300/0 conn(0x7f4a140ce1c0 0x7f4a140ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/925531940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a14070340 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/925531940 >> 172.21.15.170:0/925531940 conn(0x7f4a140b1c40 msgr2=0x7f4a140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/925531940 shutdown_connections 2024-09-21T02:16:43.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/925531940 wait complete. 2024-09-21T02:16:43.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 Processor -- start 2024-09-21T02:16:43.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.945+0000 7f4a1a1b0640 1 -- start start 2024-09-21T02:16:43.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a140cb460 0x7f4a1406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a140ce1c0 0x7f4a1406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a1413e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141690d0 con 0x7f4a140cf7e0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14168f50 con 0x7f4a140ce1c0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a14169250 con 0x7f4a140cb460 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a1413e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a1413e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44736/0 (socket says 172.21.15.170:44736) 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 -- 172.21.15.170:0/2511355300 learned_addr learned my addr 172.21.15.170:0/2511355300 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a137fe640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a140cb460 0x7f4a1406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a0bfff640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a140ce1c0 0x7f4a1406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 -- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a140cb460 msgr2=0x7f4a1406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a140cb460 0x7f4a1406eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 -- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a140ce1c0 msgr2=0x7f4a1406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a140ce1c0 0x7f4a1406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 -- 172.21.15.170:0/2511355300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1409deb0 con 0x7f4a140cf7e0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a13fff640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a1413e390 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e970 tx=0x7f4a0c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:43.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004d30 con 0x7f4a140cf7e0 2024-09-21T02:16:43.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a0c00cb20 con 0x7f4a140cf7e0 2024-09-21T02:16:43.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1409ce10 con 0x7f4a140cf7e0 2024-09-21T02:16:43.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c005020 con 0x7f4a140cf7e0 2024-09-21T02:16:43.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1409d3d0 con 0x7f4a140cf7e0 2024-09-21T02:16:43.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a140ce5d0 con 0x7f4a140cf7e0 2024-09-21T02:16:43.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.949+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f4a0c0051c0 con 0x7f4a140cf7e0 2024-09-21T02:16:43.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.953+0000 7f4a10ff9640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f40733b0 0x7f49f40758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:43.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.953+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f4a0c094c10 con 0x7f4a140cf7e0 2024-09-21T02:16:43.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.953+0000 7f4a137fe640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f40733b0 0x7f49f40758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:43.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.953+0000 7f4a137fe640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f40733b0 0x7f49f40758c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4a040af000 tx=0x7f4a0401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:43.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:43.957+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a0c016030 con 0x7f4a140cf7e0 2024-09-21T02:16:44.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:44.177+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f4a1406de10 con 0x7f4a140cf7e0 2024-09-21T02:16:45.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.081+0000 7f4a10ff9640 1 -- 172.21.15.170:0/2511355300 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v176) ==== 164+0+0 (secure 0 0 0) 0x7f4a0c05fc30 con 0x7f4a140cf7e0 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f40733b0 msgr2=0x7f49f40758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f40733b0 0x7f49f40758c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4a040af000 tx=0x7f4a0401b040 comp rx=0 tx=0).stop 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 msgr2=0x7f4a1413e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a1413e390 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00e970 tx=0x7f4a0c00ee30 comp rx=0 tx=0).stop 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 shutdown_connections 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f40733b0 0x7f49f40758c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a140cf7e0 0x7f4a1413e390 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a140ce1c0 0x7f4a1406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 --2- 172.21.15.170:0/2511355300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a140cb460 0x7f4a1406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 >> 172.21.15.170:0/2511355300 conn(0x7f4a140b1c40 msgr2=0x7f4a140b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:45.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 shutdown_connections 2024-09-21T02:16:45.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.085+0000 7f4a1a1b0640 1 -- 172.21.15.170:0/2511355300 wait complete. 2024-09-21T02:16:45.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 -- 172.21.15.170:0/4090358177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b2f40 msgr2=0x7f99040b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 --2- 172.21.15.170:0/4090358177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b2f40 0x7f99040b3350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009840 tx=0x7f98f401c820 comp rx=0 tx=0).stop 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 -- 172.21.15.170:0/4090358177 shutdown_connections 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 --2- 172.21.15.170:0/4090358177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b2f40 0x7f99040b3350 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 -- 172.21.15.170:0/4090358177 >> 172.21.15.170:0/4090358177 conn(0x7f99040b1c40 msgr2=0x7f99040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 -- 172.21.15.170:0/4090358177 shutdown_connections 2024-09-21T02:16:45.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.273+0000 7f9909d00640 1 -- 172.21.15.170:0/4090358177 wait complete. 2024-09-21T02:16:45.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 Processor -- start 2024-09-21T02:16:45.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- start start 2024-09-21T02:16:45.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99040b2f40 0x7f99041463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 0x7f99040add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99040ccb00 0x7f99040ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f990416afe0 con 0x7f99040b5410 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f990416ae60 con 0x7f99040ccb00 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f990416b160 con 0x7f99040b2f40 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 0x7f99040add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 0x7f99040add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44752/0 (socket says 172.21.15.170:44752) 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 -- 172.21.15.170:0/2438443203 learned_addr learned my addr 172.21.15.170:0/2438443203 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9903fff640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99040ccb00 0x7f99040ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f99037fe640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99040b2f40 0x7f99041463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 -- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99040b2f40 msgr2=0x7f99041463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99040b2f40 0x7f99041463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 -- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99040ccb00 msgr2=0x7f99040ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99040ccb00 0x7f99040ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 -- 172.21.15.170:0/2438443203 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99041580d0 con 0x7f99040b5410 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f99037fe640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99040b2f40 0x7f99041463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9903fff640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99040ccb00 0x7f99040ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:45.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9902ffd640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 0x7f99040add10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f98f401aed0 tx=0x7f98f40a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:45.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40bc070 con 0x7f99040b5410 2024-09-21T02:16:45.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98f40044b0 con 0x7f99040b5410 2024-09-21T02:16:45.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f40b2040 con 0x7f99040b5410 2024-09-21T02:16:45.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99041583b0 con 0x7f99040b5410 2024-09-21T02:16:45.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9904158970 con 0x7f99040b5410 2024-09-21T02:16:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.277+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9904157e40 con 0x7f99040b5410 2024-09-21T02:16:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.281+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f98f4003860 con 0x7f99040b5410 2024-09-21T02:16:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.281+0000 7f98fbfff640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f98e40733e0 0x7f98e40758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.281+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f98f412df00 con 0x7f99040b5410 2024-09-21T02:16:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.281+0000 7f99037fe640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f98e40733e0 0x7f98e40758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.281+0000 7f99037fe640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f98e40733e0 0x7f98e40758f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00a850 tx=0x7f98ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.285+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f98f4016410 con 0x7f99040b5410 2024-09-21T02:16:45.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.457+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f99040b3350 con 0x7f99040b5410 2024-09-21T02:16:45.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.457+0000 7f98fbfff640 1 -- 172.21.15.170:0/2438443203 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7f98f40f8df0 con 0x7f99040b5410 2024-09-21T02:16:45.460 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:16:45.461 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:16:45.462 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:16:45.463 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:16:45.463 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:16:45.463 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:16:45.463 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:16:45.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f98e40733e0 msgr2=0x7f98e40758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f98e40733e0 0x7f98e40758f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00a850 tx=0x7f98ec008040 comp rx=0 tx=0).stop 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 msgr2=0x7f99040add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 0x7f99040add10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f98f401aed0 tx=0x7f98f40a5740 comp rx=0 tx=0).stop 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 shutdown_connections 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f98e40733e0 0x7f98e40758f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99040ccb00 0x7f99040ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99040b5410 0x7f99040add10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 --2- 172.21.15.170:0/2438443203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99040b2f40 0x7f99041463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 >> 172.21.15.170:0/2438443203 conn(0x7f99040b1c40 msgr2=0x7f99040b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 shutdown_connections 2024-09-21T02:16:45.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.461+0000 7f9909d00640 1 -- 172.21.15.170:0/2438443203 wait complete. 2024-09-21T02:16:45.484 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 -- 172.21.15.170:0/55268363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33580b2fc0 msgr2=0x7f33580ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 --2- 172.21.15.170:0/55268363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33580b2fc0 0x7f33580ce540 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3350009930 tx=0x7f335001c900 comp rx=0 tx=0).stop 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 -- 172.21.15.170:0/55268363 shutdown_connections 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 --2- 172.21.15.170:0/55268363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33580cea80 0x7f33580cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 --2- 172.21.15.170:0/55268363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33580b2fc0 0x7f33580ce540 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 -- 172.21.15.170:0/55268363 >> 172.21.15.170:0/55268363 conn(0x7f33580d0170 msgr2=0x7f33580d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 -- 172.21.15.170:0/55268363 shutdown_connections 2024-09-21T02:16:45.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.633+0000 7f335d1a6640 1 -- 172.21.15.170:0/55268363 wait complete. 2024-09-21T02:16:45.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 Processor -- start 2024-09-21T02:16:45.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- start start 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 0x7f33581545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33580b2fc0 0x7f3358142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33580cea80 0x7f3358143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3358169fe0 con 0x7f335806ef40 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33580b3640 con 0x7f33580b2fc0 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f335816a160 con 0x7f33580cea80 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 0x7f33581545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 0x7f33581545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47950/0 (socket says 172.21.15.170:47950) 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 -- 172.21.15.170:0/4026960330 learned_addr learned my addr 172.21.15.170:0/4026960330 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335659c640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33580b2fc0 0x7f3358142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335759e640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33580cea80 0x7f3358143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 -- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33580cea80 msgr2=0x7f3358143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33580cea80 0x7f3358143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 -- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33580b2fc0 msgr2=0x7f3358142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33580b2fc0 0x7f3358142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 -- 172.21.15.170:0/4026960330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3358154b20 con 0x7f335806ef40 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f3356d9d640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 0x7f33581545e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f33500b0040 tx=0x7f33500a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33500b2040 con 0x7f335806ef40 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33500043b0 con 0x7f335806ef40 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33580b2740 con 0x7f335806ef40 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350003860 con 0x7f335806ef40 2024-09-21T02:16:45.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3358143890 con 0x7f335806ef40 2024-09-21T02:16:45.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.637+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335815eca0 con 0x7f335806ef40 2024-09-21T02:16:45.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.641+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f33500bf060 con 0x7f335806ef40 2024-09-21T02:16:45.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.641+0000 7f334f7fe640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33380733b0 0x7f33380758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:45.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.641+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f335012e290 con 0x7f335806ef40 2024-09-21T02:16:45.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.641+0000 7f335659c640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33380733b0 0x7f33380758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:45.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.641+0000 7f335659c640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33380733b0 0x7f33380758c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3340006fd0 tx=0x7f3340008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:45.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.645+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33500164f0 con 0x7f335806ef40 2024-09-21T02:16:45.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.801+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f33580705f0 con 0x7f335806ef40 2024-09-21T02:16:45.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.809+0000 7f334f7fe640 1 -- 172.21.15.170:0/4026960330 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f33500f92c0 con 0x7f335806ef40 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33380733b0 msgr2=0x7f33380758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33380733b0 0x7f33380758c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3340006fd0 tx=0x7f3340008040 comp rx=0 tx=0).stop 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 msgr2=0x7f33581545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 0x7f33581545e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f33500b0040 tx=0x7f33500a55c0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 shutdown_connections 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f33380733b0 0x7f33380758c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33580cea80 0x7f3358143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33580b2fc0 0x7f3358142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 --2- 172.21.15.170:0/4026960330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f335806ef40 0x7f33581545e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 >> 172.21.15.170:0/4026960330 conn(0x7f33580d0170 msgr2=0x7f335815a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 shutdown_connections 2024-09-21T02:16:45.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:45.813+0000 7f335d1a6640 1 -- 172.21.15.170:0/4026960330 wait complete. 2024-09-21T02:16:45.833 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:16:45.833 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:16:45.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:16:45.961 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:16:45.961 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:16:45.961 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:45.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:16:45.968 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:16:45.968 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:45.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:16:45.974 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:16:45.974 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:46.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:16:46.014 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:16:46.014 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:16:46.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:16:46.020 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:16:46.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:46.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 -- 172.21.15.170:0/3871324901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040cb500 msgr2=0x7feb040ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 --2- 172.21.15.170:0/3871324901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040cb500 0x7feb040ce8d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7feafc009bc0 tx=0x7feafc01cba0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 -- 172.21.15.170:0/3871324901 shutdown_connections 2024-09-21T02:16:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 --2- 172.21.15.170:0/3871324901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040cb500 0x7feb040ce8d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 -- 172.21.15.170:0/3871324901 >> 172.21.15.170:0/3871324901 conn(0x7feb040b1c40 msgr2=0x7feb040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 -- 172.21.15.170:0/3871324901 shutdown_connections 2024-09-21T02:16:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 -- 172.21.15.170:0/3871324901 wait complete. 2024-09-21T02:16:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 Processor -- start 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.217+0000 7feb09b96640 1 -- start start 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 0x7feb04155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb0415f990 0x7feb041fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb04154350 0x7feb04154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb04169390 con 0x7feb040ce4c0 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feb04169210 con 0x7feb04154350 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7feb04169510 con 0x7feb0415f990 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 0x7feb04155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 0x7feb04155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47980/0 (socket says 172.21.15.170:47980) 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 -- 172.21.15.170:0/256278183 learned_addr learned my addr 172.21.15.170:0/256278183 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb03fff640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb04154350 0x7feb04154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb02ffd640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb0415f990 0x7feb041fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 -- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb0415f990 msgr2=0x7feb041fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb0415f990 0x7feb041fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 -- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb04154350 msgr2=0x7feb04154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb04154350 0x7feb04154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 -- 172.21.15.170:0/256278183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb04154ca0 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb03fff640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb04154350 0x7feb04154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb037fe640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 0x7feb04155110 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c3e0 tx=0x7feaf400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4011790 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaf4017070 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0409cb30 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4011ca0 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb040b4c10 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb040ce8d0 con 0x7feb040ce4c0 2024-09-21T02:16:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7feaf401c460 con 0x7feb040ce4c0 2024-09-21T02:16:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.221+0000 7feaebfff640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feaf00733b0 0x7feaf00758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.225+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7feaf4094120 con 0x7feb040ce4c0 2024-09-21T02:16:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.225+0000 7feb02ffd640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feaf00733b0 0x7feaf00758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.225+0000 7feb02ffd640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feaf00733b0 0x7feaf00758c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feb0415ec50 tx=0x7feaf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.229+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feb04154ca0 con 0x7feb040ce4c0 2024-09-21T02:16:46.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.429+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb040ce2a0 con 0x7feb040ce4c0 2024-09-21T02:16:46.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feaebfff640 1 -- 172.21.15.170:0/256278183 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3920 (secure 0 0 0) 0x7feaf405f120 con 0x7feb040ce4c0 2024-09-21T02:16:46.434 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:46.434 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":30,"btime":"2024-09-21T02:16:39:815116+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3751367574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3751367574},{"type":"v1","addr":"172.21.15.170:6835","nonce":3751367574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1466124428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1466124428},{"type":"v1","addr":"172.21.15.191:6835","nonce":1466124428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/1675962654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1675962654},{"type":"v1","addr":"172.21.15.170:6834","nonce":1675962654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/456108989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":456108989},{"type":"v1","addr":"172.21.15.191:6837","nonce":456108989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-21T02:16:46.434 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 30 2024-09-21T02:16:46.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feaf00733b0 msgr2=0x7feaf00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feaf00733b0 0x7feaf00758c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feb0415ec50 tx=0x7feaf8008040 comp rx=0 tx=0).stop 2024-09-21T02:16:46.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 msgr2=0x7feb04155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 0x7feb04155110 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c3e0 tx=0x7feaf400c8a0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 shutdown_connections 2024-09-21T02:16:46.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feaf00733b0 0x7feaf00758c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb0415f990 0x7feb041fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb04154350 0x7feb04154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 --2- 172.21.15.170:0/256278183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb040ce4c0 0x7feb04155110 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 >> 172.21.15.170:0/256278183 conn(0x7feb040b1c40 msgr2=0x7feb0406f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:46.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 shutdown_connections 2024-09-21T02:16:46.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.433+0000 7feb09b96640 1 -- 172.21.15.170:0/256278183 wait complete. 2024-09-21T02:16:46.455 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:16:46.455 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:16:46.455 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:16:46.455 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:16:46.455 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:16:46.455 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:16:46.456 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:16:46.456 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:16:46.456 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:16:46.456 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:16:46.456 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:16:46.456 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:16:46.456 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:16:46.456 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:16:46.457 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:16:46.457 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:16:46.468 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:16:46.468 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:16:46.474 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:16:46.474 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:16:46.475 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:16:46.475 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:16:46.476 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:16:46.476 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:16:46.517 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:16:46.517 DEBUG:teuthology.parallel:result is None 2024-09-21T02:16:46.518 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:16:46.518 DEBUG:teuthology.parallel:result is None 2024-09-21T02:16:46.518 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:16:46.519 DEBUG:teuthology.parallel:result is None 2024-09-21T02:16:46.519 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:16:46.520 DEBUG:teuthology.parallel:result is None 2024-09-21T02:16:46.520 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:16:46.606 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:16:46.606 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:16:46.618 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:16:46.620 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 -- 172.21.15.170:0/3209752243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b4540 msgr2=0x7f72c815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 --2- 172.21.15.170:0/3209752243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b4540 0x7f72c815c850 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009840 tx=0x7f72b801c820 comp rx=0 tx=0).stop 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 -- 172.21.15.170:0/3209752243 shutdown_connections 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 --2- 172.21.15.170:0/3209752243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72c80d0aa0 0x7f72c80d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 --2- 172.21.15.170:0/3209752243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b4540 0x7f72c815c850 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 -- 172.21.15.170:0/3209752243 >> 172.21.15.170:0/3209752243 conn(0x7f72c80cee30 msgr2=0x7f72c80cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 -- 172.21.15.170:0/3209752243 shutdown_connections 2024-09-21T02:16:46.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 -- 172.21.15.170:0/3209752243 wait complete. 2024-09-21T02:16:46.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 Processor -- start 2024-09-21T02:16:46.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.657+0000 7f72cf559640 1 -- start start 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 0x7f72c80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c80d0aa0 0x7f72c8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72c815a8c0 0x7f72c815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72c816a590 con 0x7f72c80b7ab0 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72c816a410 con 0x7f72c80d0aa0 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f72c816a710 con 0x7f72c815a8c0 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 0x7f72c80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 0x7f72c80c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48030/0 (socket says 172.21.15.170:48030) 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 -- 172.21.15.170:0/3111721707 learned_addr learned my addr 172.21.15.170:0/3111721707 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cdaf6640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72c815a8c0 0x7f72c815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72ccaf4640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c80d0aa0 0x7f72c8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 -- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72c815a8c0 msgr2=0x7f72c815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72c815a8c0 0x7f72c815acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 -- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c80d0aa0 msgr2=0x7f72c8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c80d0aa0 0x7f72c8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 -- 172.21.15.170:0/3111721707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c815b210 con 0x7f72c80b7ab0 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cd2f5640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 0x7f72c80c8120 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c430 tx=0x7f72c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4011790 con 0x7f72c80b7ab0 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72c4017070 con 0x7f72c80b7ab0 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c80b72c0 con 0x7f72c80b7ab0 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4011ce0 con 0x7f72c80b7ab0 2024-09-21T02:16:46.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c80b58f0 con 0x7f72c80b7ab0 2024-09-21T02:16:46.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c815c850 con 0x7f72c80b7ab0 2024-09-21T02:16:46.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.661+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f72c401c460 con 0x7f72c80b7ab0 2024-09-21T02:16:46.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.665+0000 7f72bdffb640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f72ac0733b0 0x7f72ac0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:46.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.665+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f72c4094130 con 0x7f72c80b7ab0 2024-09-21T02:16:46.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.665+0000 7f72ccaf4640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f72ac0733b0 0x7f72ac0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:46.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.665+0000 7f72ccaf4640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f72ac0733b0 0x7f72ac0758c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f72b80af000 tx=0x7f72b801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:46.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.669+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f72c815b210 con 0x7f72c80b7ab0 2024-09-21T02:16:46.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:46.837+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f72c80731f0 con 0x7f72c80b7ab0 2024-09-21T02:16:47.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.105+0000 7f72bdffb640 1 -- 172.21.15.170:0/3111721707 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) ==== 97+0+0 (secure 0 0 0) 0x7f72c405f180 con 0x7f72c80b7ab0 2024-09-21T02:16:47.108 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f72ac0733b0 msgr2=0x7f72ac0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f72ac0733b0 0x7f72ac0758c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f72b80af000 tx=0x7f72b801b060 comp rx=0 tx=0).stop 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 msgr2=0x7f72c80c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 0x7f72c80c8120 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c430 tx=0x7f72c400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 shutdown_connections 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f72ac0733b0 0x7f72ac0758c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72c815a8c0 0x7f72c815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72c80d0aa0 0x7f72c8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 --2- 172.21.15.170:0/3111721707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72c80b7ab0 0x7f72c80c8120 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 >> 172.21.15.170:0/3111721707 conn(0x7f72c80cee30 msgr2=0x7f72c80726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:47.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 shutdown_connections 2024-09-21T02:16:47.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.109+0000 7f72cf559640 1 -- 172.21.15.170:0/3111721707 wait complete. 2024-09-21T02:16:47.136 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:16:47.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 -- 172.21.15.170:0/3040657927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49f8004b20 msgr2=0x7f49f8004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 --2- 172.21.15.170:0/3040657927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49f8004b20 0x7f49f8004f70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c074030 tx=0x7f49fc01cf60 comp rx=0 tx=0).stop 2024-09-21T02:16:47.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 -- 172.21.15.170:0/3040657927 shutdown_connections 2024-09-21T02:16:47.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 --2- 172.21.15.170:0/3040657927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49f8004b20 0x7f49f8004f70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 -- 172.21.15.170:0/3040657927 >> 172.21.15.170:0/3040657927 conn(0x7f4a0c0b1c40 msgr2=0x7f4a0c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:47.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 -- 172.21.15.170:0/3040657927 shutdown_connections 2024-09-21T02:16:47.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.285+0000 7f4a139c4640 1 -- 172.21.15.170:0/3040657927 wait complete. 2024-09-21T02:16:47.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 Processor -- start 2024-09-21T02:16:47.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- start start 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49f8004b20 0x7f4a0c09b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0c09d370 0x7f4a0c1f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 0x7f4a0c15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0703c0 con 0x7f4a0c09d780 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c070240 con 0x7f4a0c09d370 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c070540 con 0x7f49f8004b20 2024-09-21T02:16:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 0x7f4a0c15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 0x7f4a0c15bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48050/0 (socket says 172.21.15.170:48050) 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 -- 172.21.15.170:0/2800443838 learned_addr learned my addr 172.21.15.170:0/2800443838 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11760640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49f8004b20 0x7f4a0c09b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a10f5f640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0c09d370 0x7f4a0c1f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 -- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49f8004b20 msgr2=0x7f4a0c09b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49f8004b20 0x7f4a0c09b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 -- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0c09d370 msgr2=0x7f4a0c1f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0c09d370 0x7f4a0c1f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 -- 172.21.15.170:0/2800443838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1f9990 con 0x7f4a0c09d780 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a10f5f640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0c09d370 0x7f4a0c1f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:16:47.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a11f61640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 0x7f4a0c15bed0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c074c60 tx=0x7f4a0400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04003980 con 0x7f4a0c09d780 2024-09-21T02:16:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a04003c10 con 0x7f4a0c09d780 2024-09-21T02:16:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c09ab60 con 0x7f4a0c09d780 2024-09-21T02:16:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400b920 con 0x7f4a0c09d780 2024-09-21T02:16:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c09b120 con 0x7f4a0c09d780 2024-09-21T02:16:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.289+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c1f9700 con 0x7f4a0c09d780 2024-09-21T02:16:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.293+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f4a0401b020 con 0x7f4a0c09d780 2024-09-21T02:16:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.293+0000 7f4a01ffb640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f00733b0 0x7f49f00758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.293+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f4a04093f20 con 0x7f4a0c09d780 2024-09-21T02:16:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.293+0000 7f4a11760640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f00733b0 0x7f49f00758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.293+0000 7f4a11760640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f00733b0 0x7f49f00758c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ab000 tx=0x7f49fc019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:47.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.297+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a0405ee60 con 0x7f4a0c09d780 2024-09-21T02:16:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.533+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4a0c0cadf0 con 0x7f4a0c09d780 2024-09-21T02:16:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a01ffb640 1 -- 172.21.15.170:0/2800443838 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f4a0405e760 con 0x7f4a0c09d780 2024-09-21T02:16:47.538 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:47.538 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:16:47.539 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:16:47.540 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:16:47.541 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.543 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:16:47.544 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.545 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:16:47.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.547 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.548 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:16:47.549 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:16:47.550 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:16:47.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f00733b0 msgr2=0x7f49f00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f00733b0 0x7f49f00758c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ab000 tx=0x7f49fc019da0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 msgr2=0x7f4a0c15bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 0x7f4a0c15bed0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c074c60 tx=0x7f4a0400dd10 comp rx=0 tx=0).stop 2024-09-21T02:16:47.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 shutdown_connections 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f49f00733b0 0x7f49f00758c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0c09d780 0x7f4a0c15bed0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a0c09d370 0x7f4a0c1f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 --2- 172.21.15.170:0/2800443838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49f8004b20 0x7f4a0c09b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 >> 172.21.15.170:0/2800443838 conn(0x7f4a0c0b1c40 msgr2=0x7f4a0c0aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 shutdown_connections 2024-09-21T02:16:47.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.537+0000 7f4a139c4640 1 -- 172.21.15.170:0/2800443838 wait complete. 2024-09-21T02:16:47.560 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 -- 172.21.15.170:0/3636294995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 msgr2=0x7fd3bc06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/3636294995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc06fa30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc070e90 tx=0x7fd3ac01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 -- 172.21.15.170:0/3636294995 shutdown_connections 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/3636294995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3bc0ccb00 0x7fd3bc0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/3636294995 >> v2:172.21.15.191:3300/0 conn(0x7fd3bc0b2f40 0x7fd3bc0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/3636294995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc06fa30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 -- 172.21.15.170:0/3636294995 >> 172.21.15.170:0/3636294995 conn(0x7fd3bc0b1c40 msgr2=0x7fd3bc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 -- 172.21.15.170:0/3636294995 shutdown_connections 2024-09-21T02:16:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.705+0000 7fd3c2b23640 1 -- 172.21.15.170:0/3636294995 wait complete. 2024-09-21T02:16:47.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 Processor -- start 2024-09-21T02:16:47.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 -- start start 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3bc0b2f40 0x7fd3bc14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3bc0ccb00 0x7fd3bc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc0700f0 con 0x7fd3bc0b5410 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc06ff70 con 0x7fd3bc0b2f40 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd3bc0ce800 con 0x7fd3bc0ccb00 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48068/0 (socket says 172.21.15.170:48068) 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 -- 172.21.15.170:0/2204060775 learned_addr learned my addr 172.21.15.170:0/2204060775 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c08bf640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3bc0b2f40 0x7fd3bc14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c10c0640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3bc0ccb00 0x7fd3bc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 -- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3bc0ccb00 msgr2=0x7fd3bc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3bc0ccb00 0x7fd3bc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 -- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3bc0b2f40 msgr2=0x7fd3bc14e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3bc0b2f40 0x7fd3bc14e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 -- 172.21.15.170:0/2204060775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc15db00 con 0x7fd3bc0b5410 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3bbfff640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc0afac0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c9a0 tx=0x7fd3b000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0007bb0 con 0x7fd3bc0b5410 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b0007e40 con 0x7fd3bc0b5410 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc0b0540 con 0x7fd3bc0b5410 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0005260 con 0x7fd3bc0b5410 2024-09-21T02:16:47.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.709+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc15e250 con 0x7fd3bc0b5410 2024-09-21T02:16:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.713+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3bc0ced60 con 0x7fd3bc0b5410 2024-09-21T02:16:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.713+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fd3b0005400 con 0x7fd3bc0b5410 2024-09-21T02:16:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.713+0000 7fd3b97fa640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd3a80733b0 0x7fd3a80758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.713+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fd3b00947b0 con 0x7fd3bc0b5410 2024-09-21T02:16:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.713+0000 7fd3c08bf640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd3a80733b0 0x7fd3a80758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:47.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.713+0000 7fd3c08bf640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd3a80733b0 0x7fd3a80758c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0af000 tx=0x7fd3ac01b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:47.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.717+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd3b0014030 con 0x7fd3bc0b5410 2024-09-21T02:16:47.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.885+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd3bc15dcd0 con 0x7fd3bc0b5410 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.885+0000 7fd3b97fa640 1 -- 172.21.15.170:0/2204060775 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7fd3b005f780 con 0x7fd3bc0b5410 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:16:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:47.890 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:16:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd3a80733b0 msgr2=0x7fd3a80758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd3a80733b0 0x7fd3a80758c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0af000 tx=0x7fd3ac01b100 comp rx=0 tx=0).stop 2024-09-21T02:16:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 msgr2=0x7fd3bc0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc0afac0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c9a0 tx=0x7fd3b000ce60 comp rx=0 tx=0).stop 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 shutdown_connections 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd3a80733b0 0x7fd3a80758c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3bc0ccb00 0x7fd3bc0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3bc0b5410 0x7fd3bc0afac0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 --2- 172.21.15.170:0/2204060775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3bc0b2f40 0x7fd3bc14e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 >> 172.21.15.170:0/2204060775 conn(0x7fd3bc0b1c40 msgr2=0x7fd3bc09dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 shutdown_connections 2024-09-21T02:16:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:47.889+0000 7fd3c2b23640 1 -- 172.21.15.170:0/2204060775 wait complete. 2024-09-21T02:16:47.909 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:16:47.909 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:16:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- 172.21.15.170:0/1173614979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 msgr2=0x7fa9140cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 --2- 172.21.15.170:0/1173614979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa9140cabc0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa904005fd0 tx=0x7fa90401c820 comp rx=0 tx=0).stop 2024-09-21T02:16:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- 172.21.15.170:0/1173614979 shutdown_connections 2024-09-21T02:16:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 --2- 172.21.15.170:0/1173614979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa9140cabc0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- 172.21.15.170:0/1173614979 >> 172.21.15.170:0/1173614979 conn(0x7fa9140b1c40 msgr2=0x7fa9140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:48.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- 172.21.15.170:0/1173614979 shutdown_connections 2024-09-21T02:16:48.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- 172.21.15.170:0/1173614979 wait complete. 2024-09-21T02:16:48.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 Processor -- start 2024-09-21T02:16:48.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- start start 2024-09-21T02:16:48.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91406ffc0 0x7fa9140c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:48.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9140735d0 0x7fa9141f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:48.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa914152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa914169cf0 con 0x7fa9140ca7b0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa914169b70 con 0x7fa91406ffc0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa91a714640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa914169e70 con 0x7fa9140735d0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa918cb1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa914152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa918cb1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa914152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48078/0 (socket says 172.21.15.170:48078) 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.061+0000 7fa918cb1640 1 -- 172.21.15.170:0/1780267197 learned_addr learned my addr 172.21.15.170:0/1780267197 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa913fff640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91406ffc0 0x7fa9140c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa918cb1640 1 -- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9140735d0 msgr2=0x7fa9141f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa918cb1640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9140735d0 0x7fa9141f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa918cb1640 1 -- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91406ffc0 msgr2=0x7fa9140c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa918cb1640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91406ffc0 0x7fa9140c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa918cb1640 1 -- 172.21.15.170:0/1780267197 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9141f9db0 con 0x7fa9140ca7b0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa918cb1640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa914152c40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00ba10 tx=0x7fa90c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:48.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c019070 con 0x7fa9140ca7b0 2024-09-21T02:16:48.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa90c00cc30 con 0x7fa9140ca7b0 2024-09-21T02:16:48.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00ce60 con 0x7fa9140ca7b0 2024-09-21T02:16:48.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9140ae730 con 0x7fa9140ca7b0 2024-09-21T02:16:48.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91415c330 con 0x7fa9140ca7b0 2024-09-21T02:16:48.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9140cabc0 con 0x7fa9140ca7b0 2024-09-21T02:16:48.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fa90c01c070 con 0x7fa9140ca7b0 2024-09-21T02:16:48.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa910ff9640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9000733e0 0x7fa9000758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:48.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fa90c094100 con 0x7fa9140ca7b0 2024-09-21T02:16:48.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.065+0000 7fa913fff640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9000733e0 0x7fa9000758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:48.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.069+0000 7fa913fff640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9000733e0 0x7fa9000758f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9040ae000 tx=0x7fa90401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:48.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.069+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9141f9db0 con 0x7fa9140ca7b0 2024-09-21T02:16:48.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:48.237+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa914154ca0 con 0x7fa9140ca7b0 2024-09-21T02:16:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.117+0000 7fa910ff9640 1 -- 172.21.15.170:0/1780267197 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7fa90c05f250 con 0x7fa9140ca7b0 2024-09-21T02:16:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:16:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9000733e0 msgr2=0x7fa9000758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9000733e0 0x7fa9000758f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa9040ae000 tx=0x7fa90401b060 comp rx=0 tx=0).stop 2024-09-21T02:16:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 msgr2=0x7fa914152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa914152c40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00ba10 tx=0x7fa90c00bed0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 shutdown_connections 2024-09-21T02:16:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa9000733e0 0x7fa9000758f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9140735d0 0x7fa9141f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9140ca7b0 0x7fa914152c40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 --2- 172.21.15.170:0/1780267197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa91406ffc0 0x7fa9140c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 >> 172.21.15.170:0/1780267197 conn(0x7fa9140b1c40 msgr2=0x7fa9140b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 shutdown_connections 2024-09-21T02:16:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.121+0000 7fa91a714640 1 -- 172.21.15.170:0/1780267197 wait complete. 2024-09-21T02:16:49.154 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:16:49.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 -- 172.21.15.170:0/1240748896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0cf7d0 msgr2=0x7f25ac073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:49.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1240748896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0cf7d0 0x7f25ac073b60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f259c009840 tx=0x7f259c01c820 comp rx=0 tx=0).stop 2024-09-21T02:16:49.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 -- 172.21.15.170:0/1240748896 shutdown_connections 2024-09-21T02:16:49.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1240748896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0cf7d0 0x7f25ac073b60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 -- 172.21.15.170:0/1240748896 >> 172.21.15.170:0/1240748896 conn(0x7f25ac0d0170 msgr2=0x7f25ac0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:49.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 -- 172.21.15.170:0/1240748896 shutdown_connections 2024-09-21T02:16:49.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.317+0000 7f25b1b99640 1 -- 172.21.15.170:0/1240748896 wait complete. 2024-09-21T02:16:49.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 Processor -- start 2024-09-21T02:16:49.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 -- start start 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ac0740a0 0x7f25ac152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 0x7f25ac0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ac0cf7d0 0x7f25ac15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac169ed0 con 0x7f25ac0744b0 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac0b5c90 con 0x7f25ac0cf7d0 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f25ac16a050 con 0x7f25ac0740a0 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 0x7f25ac0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 0x7f25ac0aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48092/0 (socket says 172.21.15.170:48092) 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 -- 172.21.15.170:0/1603606063 learned_addr learned my addr 172.21.15.170:0/1603606063 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25ab7fe640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ac0740a0 0x7f25ac152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25abfff640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ac0cf7d0 0x7f25ac15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 -- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ac0740a0 msgr2=0x7f25ac152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:49.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ac0740a0 0x7f25ac152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 -- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ac0cf7d0 msgr2=0x7f25ac15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ac0cf7d0 0x7f25ac15e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 -- 172.21.15.170:0/1603606063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac0c3ca0 con 0x7f25ac0744b0 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25aaffd640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 0x7f25ac0aee50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c420 tx=0x7f25a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011790 con 0x7f25ac0744b0 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a0017070 con 0x7f25ac0744b0 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1fc0c0 con 0x7f25ac0744b0 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.321+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011ce0 con 0x7f25ac0744b0 2024-09-21T02:16:49.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac1fc680 con 0x7f25ac0744b0 2024-09-21T02:16:49.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac073b60 con 0x7f25ac0744b0 2024-09-21T02:16:49.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f25a001c460 con 0x7f25ac0744b0 2024-09-21T02:16:49.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f2593fff640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f25980733b0 0x7f25980758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:49.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5227+0+0 (secure 0 0 0) 0x7f25a0094cc0 con 0x7f25ac0744b0 2024-09-21T02:16:49.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f25ab7fe640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f25980733b0 0x7f25980758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.325+0000 7f25ab7fe640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f25980733b0 0x7f25980758c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f259c0af000 tx=0x7f259c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.329+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f25a005fc10 con 0x7f25ac0744b0 2024-09-21T02:16:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:49.501+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f25ac0c5f00 con 0x7f25ac0744b0 2024-09-21T02:16:50.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.121+0000 7f2593fff640 1 -- 172.21.15.170:0/1603606063 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7f25a005f510 con 0x7f25ac0744b0 2024-09-21T02:16:50.124 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:16:50.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f25980733b0 msgr2=0x7f25980758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:50.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f25980733b0 0x7f25980758c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f259c0af000 tx=0x7f259c01b060 comp rx=0 tx=0).stop 2024-09-21T02:16:50.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 msgr2=0x7f25ac0aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:50.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 0x7f25ac0aee50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c420 tx=0x7f25a000c8e0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 shutdown_connections 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f25980733b0 0x7f25980758c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ac0cf7d0 0x7f25ac15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac0744b0 0x7f25ac0aee50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 --2- 172.21.15.170:0/1603606063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ac0740a0 0x7f25ac152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 >> 172.21.15.170:0/1603606063 conn(0x7f25ac0d0170 msgr2=0x7f25ac06f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 shutdown_connections 2024-09-21T02:16:50.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.125+0000 7f25b1b99640 1 -- 172.21.15.170:0/1603606063 wait complete. 2024-09-21T02:16:50.168 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:16:50.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 -- 172.21.15.170:0/480472167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 msgr2=0x7f1298071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:50.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 --2- 172.21.15.170:0/480472167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f1298071110 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1290009cb0 tx=0x7f129001cd10 comp rx=0 tx=0).stop 2024-09-21T02:16:50.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 -- 172.21.15.170:0/480472167 shutdown_connections 2024-09-21T02:16:50.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 --2- 172.21.15.170:0/480472167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f1298071110 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 --2- 172.21.15.170:0/480472167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 0x7f129815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 -- 172.21.15.170:0/480472167 >> 172.21.15.170:0/480472167 conn(0x7f12980d2a30 msgr2=0x7f12980cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:50.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 -- 172.21.15.170:0/480472167 shutdown_connections 2024-09-21T02:16:50.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.321+0000 7f129f829640 1 -- 172.21.15.170:0/480472167 wait complete. 2024-09-21T02:16:50.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 Processor -- start 2024-09-21T02:16:50.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 -- start start 2024-09-21T02:16:50.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f12981f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:50.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12981606e0 0x7f12980c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:50.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 0x7f12981f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:50.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f129816a040 con 0x7f129815d920 2024-09-21T02:16:50.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1298169ec0 con 0x7f12981606e0 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129f829640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f129816a1c0 con 0x7f1298160af0 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f12981f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f12981f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48120/0 (socket says 172.21.15.170:48120) 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 -- 172.21.15.170:0/2663142300 learned_addr learned my addr 172.21.15.170:0/2663142300 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129cdc4640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12981606e0 0x7f12980c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129ddc6640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 0x7f12981f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 -- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 msgr2=0x7f12981f9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 0x7f12981f9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 -- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12981606e0 msgr2=0x7f12980c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12981606e0 0x7f12980c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 -- 172.21.15.170:0/2663142300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12980ca6a0 con 0x7f129815d920 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129ddc6640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 0x7f12981f9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f129d5c5640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f12981f6e90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f129400b410 tx=0x7f129400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294015030 con 0x7f129815d920 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1294004020 con 0x7f129815d920 2024-09-21T02:16:50.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.325+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940044b0 con 0x7f129815d920 2024-09-21T02:16:50.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12980cab10 con 0x7f129815d920 2024-09-21T02:16:50.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12980cb0d0 con 0x7f129815d920 2024-09-21T02:16:50.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1298071110 con 0x7f129815d920 2024-09-21T02:16:50.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f1294018070 con 0x7f129815d920 2024-09-21T02:16:50.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f128dffb640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f127c0733e0 0x7f127c0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:50.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5609+0+0 (secure 0 0 0) 0x7f1294094490 con 0x7f129815d920 2024-09-21T02:16:50.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f129cdc4640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f127c0733e0 0x7f127c0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:50.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.329+0000 7f129cdc4640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f127c0733e0 0x7f127c0758f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f12880058d0 tx=0x7f1288005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:50.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.333+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f129405f260 con 0x7f129815d920 2024-09-21T02:16:50.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:50.545+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f12980d0830 con 0x7f129815d920 2024-09-21T02:16:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.149+0000 7f128dffb640 1 -- 172.21.15.170:0/2663142300 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) ==== 172+0+0 (secure 0 0 0) 0x7f129405eb60 con 0x7f129815d920 2024-09-21T02:16:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 8 and data pool 9 2024-09-21T02:16:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.157+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f127c0733e0 msgr2=0x7f127c0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.157+0000 7f129f829640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f127c0733e0 0x7f127c0758f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f12880058d0 tx=0x7f1288005860 comp rx=0 tx=0).stop 2024-09-21T02:16:51.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.157+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 msgr2=0x7f12981f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.157+0000 7f129f829640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f12981f6e90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f129400b410 tx=0x7f129400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.161+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 shutdown_connections 2024-09-21T02:16:51.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.161+0000 7f129f829640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f127c0733e0 0x7f127c0758f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.165+0000 7f129f829640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1298160af0 0x7f12981f9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.165+0000 7f129f829640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12981606e0 0x7f12980c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.165+0000 7f129f829640 1 --2- 172.21.15.170:0/2663142300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f129815d920 0x7f12981f6e90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.165+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 >> 172.21.15.170:0/2663142300 conn(0x7f12980d2a30 msgr2=0x7f129815b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:51.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.169+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 shutdown_connections 2024-09-21T02:16:51.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.169+0000 7f129f829640 1 -- 172.21.15.170:0/2663142300 wait complete. 2024-09-21T02:16:51.212 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:16:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 -- 172.21.15.170:0/3836400654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 msgr2=0x7f08c00d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 --2- 172.21.15.170:0/3836400654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c00d4e00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009840 tx=0x7f08b001c7f0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 -- 172.21.15.170:0/3836400654 shutdown_connections 2024-09-21T02:16:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 --2- 172.21.15.170:0/3836400654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c00d4e00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 -- 172.21.15.170:0/3836400654 >> 172.21.15.170:0/3836400654 conn(0x7f08c009d7d0 msgr2=0x7f08c00d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 -- 172.21.15.170:0/3836400654 shutdown_connections 2024-09-21T02:16:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.357+0000 7f08c53a5640 1 -- 172.21.15.170:0/3836400654 wait complete. 2024-09-21T02:16:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 Processor -- start 2024-09-21T02:16:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 -- start start 2024-09-21T02:16:51.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08c006dd80 0x7f08c0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c01fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08c00b5610 0x7f08c01fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08c016d8b0 con 0x7f08c00b9ba0 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08c016d730 con 0x7f08c006dd80 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f08c016da30 con 0x7f08c00b5610 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c01fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c01fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48130/0 (socket says 172.21.15.170:48130) 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 -- 172.21.15.170:0/1243560416 learned_addr learned my addr 172.21.15.170:0/1243560416 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08bf7fe640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08c00b5610 0x7f08c01fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 -- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08c00b5610 msgr2=0x7f08c01fa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08c00b5610 0x7f08c01fa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 -- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08c006dd80 msgr2=0x7f08c0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08c006dd80 0x7f08c0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 -- 172.21.15.170:0/1243560416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00b4570 con 0x7f08c00b9ba0 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08bf7fe640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08c00b5610 0x7f08c01fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b67fc640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c01fa2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f08a800b3f0 tx=0x7f08a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8015020 con 0x7f08c00b9ba0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a8011940 con 0x7f08c00b9ba0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c00b4850 con 0x7f08c00b9ba0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8011d90 con 0x7f08c00b9ba0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.361+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c00b4e10 con 0x7f08c00b9ba0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.365+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c00ca800 con 0x7f08c00b9ba0 2024-09-21T02:16:51.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.365+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f08a8018020 con 0x7f08c00b9ba0 2024-09-21T02:16:51.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.365+0000 7f08b7fff640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08a00733e0 0x7f08a00758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.365+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f08a8094360 con 0x7f08c00b9ba0 2024-09-21T02:16:51.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.365+0000 7f08beffd640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08a00733e0 0x7f08a00758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.365+0000 7f08beffd640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08a00733e0 0x7f08a00758f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08b00af000 tx=0x7f08b0019590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:51.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.369+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f08a805ef70 con 0x7f08c00b9ba0 2024-09-21T02:16:51.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.545+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08c0076290 con 0x7f08c00b9ba0 2024-09-21T02:16:51.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.549+0000 7f08b7fff640 1 -- 172.21.15.170:0/1243560416 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16786 (secure 0 0 0) 0x7f08a805e870 con 0x7f08c00b9ba0 2024-09-21T02:16:51.552 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:51.552 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":58,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:16:50.547607+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:16:48.250782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T02:16:49.517893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:51.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08a00733e0 msgr2=0x7f08a00758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08a00733e0 0x7f08a00758f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08b00af000 tx=0x7f08b0019590 comp rx=0 tx=0).stop 2024-09-21T02:16:51.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 msgr2=0x7f08c01fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c01fa2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f08a800b3f0 tx=0x7f08a800b8b0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 shutdown_connections 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08a00733e0 0x7f08a00758f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08c00b5610 0x7f08c01fa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c00b9ba0 0x7f08c01fa2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 --2- 172.21.15.170:0/1243560416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08c006dd80 0x7f08c0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 >> 172.21.15.170:0/1243560416 conn(0x7f08c009d7d0 msgr2=0x7f08c006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 shutdown_connections 2024-09-21T02:16:51.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.553+0000 7f08c53a5640 1 -- 172.21.15.170:0/1243560416 wait complete. 2024-09-21T02:16:51.576 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:16:51.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- 172.21.15.170:0/1434897651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 msgr2=0x7feba8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 --2- 172.21.15.170:0/1434897651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7feba8004c20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7febb4006fd0 tx=0x7febb401f6a0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc6372640 1 -- 172.21.15.170:0/1434897651 reap_dead start 2024-09-21T02:16:51.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- 172.21.15.170:0/1434897651 shutdown_connections 2024-09-21T02:16:51.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- 172.21.15.170:0/1434897651 >> 172.21.15.170:0/1434897651 conn(0x7febc00b1c40 msgr2=0x7febc00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:51.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- 172.21.15.170:0/1434897651 shutdown_connections 2024-09-21T02:16:51.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- 172.21.15.170:0/1434897651 wait complete. 2024-09-21T02:16:51.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 Processor -- start 2024-09-21T02:16:51.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- start start 2024-09-21T02:16:51.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7febc014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febc014e730 0x7febc01f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febc01f7770 0x7febc015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7febc0074f00 con 0x7feba8004810 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7febc009ddc0 con 0x7febc014e730 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.725+0000 7febc7dd5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7febc00cb500 con 0x7febc01f7770 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7febc014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7febc014b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48146/0 (socket says 172.21.15.170:48146) 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 -- 172.21.15.170:0/2887717236 learned_addr learned my addr 172.21.15.170:0/2887717236 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5370640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febc014e730 0x7febc01f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc6372640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febc01f7770 0x7febc015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 -- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febc01f7770 msgr2=0x7febc015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febc01f7770 0x7febc015d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 -- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febc014e730 msgr2=0x7febc01f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febc014e730 0x7febc01f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 -- 172.21.15.170:0/2887717236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc014dc70 con 0x7feba8004810 2024-09-21T02:16:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5370640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febc014e730 0x7febc01f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc6372640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febc01f7770 0x7febc015d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc5b71640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7febc014b0e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7febb00018d0 tx=0x7febb0001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0015070 con 0x7feba8004810 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb000eb50 con 0x7feba8004810 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc01f7f60 con 0x7feba8004810 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00020d0 con 0x7feba8004810 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc014c650 con 0x7feba8004810 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc014c8b0 con 0x7feba8004810 2024-09-21T02:16:51.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7febb0018070 con 0x7feba8004810 2024-09-21T02:16:51.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.729+0000 7febbe7fc640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febac0733e0 0x7febac0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:51.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.733+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7febb0095200 con 0x7feba8004810 2024-09-21T02:16:51.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.733+0000 7febc5370640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febac0733e0 0x7febac0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.733+0000 7febc5370640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febac0733e0 0x7febac0758f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7febb40b0000 tx=0x7febb401c4e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:51.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.733+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7febb005ff70 con 0x7feba8004810 2024-09-21T02:16:51.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.897+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febc00cfbc0 con 0x7feba8004810 2024-09-21T02:16:51.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.897+0000 7febbe7fc640 1 -- 172.21.15.170:0/2887717236 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16786 (secure 0 0 0) 0x7febb005f870 con 0x7feba8004810 2024-09-21T02:16:51.901 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:51.901 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":58,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:16:50.547607+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:16:48.250782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T02:16:49.517893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:51.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febac0733e0 msgr2=0x7febac0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febac0733e0 0x7febac0758f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7febb40b0000 tx=0x7febb401c4e0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 msgr2=0x7febc014b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:51.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7febc014b0e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7febb00018d0 tx=0x7febb0001d90 comp rx=0 tx=0).stop 2024-09-21T02:16:51.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 shutdown_connections 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7febac0733e0 0x7febac0758f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7febc01f7770 0x7febc015d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febc014e730 0x7febc01f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 --2- 172.21.15.170:0/2887717236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba8004810 0x7febc014b0e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 >> 172.21.15.170:0/2887717236 conn(0x7febc00b1c40 msgr2=0x7febc00ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 shutdown_connections 2024-09-21T02:16:51.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:51.901+0000 7febc7dd5640 1 -- 172.21.15.170:0/2887717236 wait complete. 2024-09-21T02:16:51.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:16:52.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 -- 172.21.15.170:0/2880936165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d17c0 msgr2=0x7f50140b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:52.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 --2- 172.21.15.170:0/2880936165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d17c0 0x7f50140b3a10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5004009bc0 tx=0x7f500401cba0 comp rx=0 tx=0).stop 2024-09-21T02:16:52.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 -- 172.21.15.170:0/2880936165 shutdown_connections 2024-09-21T02:16:52.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 --2- 172.21.15.170:0/2880936165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d1bd0 0x7f50140d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:52.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 --2- 172.21.15.170:0/2880936165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d17c0 0x7f50140b3a10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:52.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 -- 172.21.15.170:0/2880936165 >> 172.21.15.170:0/2880936165 conn(0x7f501408bf50 msgr2=0x7f501409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:52.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 -- 172.21.15.170:0/2880936165 shutdown_connections 2024-09-21T02:16:52.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.065+0000 7f501c6a0640 1 -- 172.21.15.170:0/2880936165 wait complete. 2024-09-21T02:16:52.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 Processor -- start 2024-09-21T02:16:52.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 -- start start 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50140d17c0 0x7f50140caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 0x7f50140cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d4a50 0x7f5014157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f501416bfe0 con 0x7f50140d1bd0 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f501416be60 con 0x7f50140d17c0 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501c6a0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f501416c160 con 0x7f50140d4a50 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 0x7f50140cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 0x7f50140cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48170/0 (socket says 172.21.15.170:48170) 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 -- 172.21.15.170:0/4062064461 learned_addr learned my addr 172.21.15.170:0/4062064461 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501ac3d640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d4a50 0x7f5014157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:52.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501a43c640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50140d17c0 0x7f50140caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 -- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d4a50 msgr2=0x7f5014157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d4a50 0x7f5014157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 -- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50140d17c0 msgr2=0x7f50140caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50140d17c0 0x7f50140caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 -- 172.21.15.170:0/4062064461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50140cbce0 con 0x7f50140d1bd0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f501ac3d640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d4a50 0x7f5014157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.069+0000 7f5019c3b640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 0x7f50140cb490 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f500800d6a0 tx=0x7f500800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008017070 con 0x7f50140d1bd0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5008009b10 con 0x7f50140d1bd0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50140cbfc0 con 0x7f50140d1bd0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008010460 con 0x7f50140d1bd0 2024-09-21T02:16:52.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50140cd9e0 con 0x7f50140d1bd0 2024-09-21T02:16:52.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50140b3a10 con 0x7f50140d1bd0 2024-09-21T02:16:52.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f500801a070 con 0x7f50140d1bd0 2024-09-21T02:16:52.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f5012ffd640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f50000733b0 0x7f50000758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:52.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5008094b20 con 0x7f50140d1bd0 2024-09-21T02:16:52.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f501a43c640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f50000733b0 0x7f50000758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:52.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.073+0000 7f501a43c640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f50000733b0 0x7f50000758c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f50040af000 tx=0x7f500401b3e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:52.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.077+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f500805f920 con 0x7f50140d1bd0 2024-09-21T02:16:52.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:52.289+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f50140cba70 con 0x7f50140d1bd0 2024-09-21T02:16:53.111 INFO:tasks.ceph.mds.d.smithi191.stderr:2024-09-21T02:16:53.111+0000 7f0a1bd0f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:16:53.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.189+0000 7f5012ffd640 1 -- 172.21.15.170:0/4062064461 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) ==== 118+0+0 (secure 0 0 0) 0x7f500805f220 con 0x7f50140d1bd0 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f50000733b0 msgr2=0x7f50000758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f50000733b0 0x7f50000758c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f50040af000 tx=0x7f500401b3e0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 msgr2=0x7f50140cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 0x7f50140cb490 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f500800d6a0 tx=0x7f500800db60 comp rx=0 tx=0).stop 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 shutdown_connections 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f50000733b0 0x7f50000758c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50140d4a50 0x7f5014157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50140d1bd0 0x7f50140cb490 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 --2- 172.21.15.170:0/4062064461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50140d17c0 0x7f50140caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 >> 172.21.15.170:0/4062064461 conn(0x7f501408bf50 msgr2=0x7f50140b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 shutdown_connections 2024-09-21T02:16:53.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.193+0000 7f501c6a0640 1 -- 172.21.15.170:0/4062064461 wait complete. 2024-09-21T02:16:53.221 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:16:53.222 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 -- 172.21.15.170:0/15470077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0cb380 msgr2=0x7f371c0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 --2- 172.21.15.170:0/15470077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0cb380 0x7f371c0ce070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a950 tx=0x7f370c01dae0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 -- 172.21.15.170:0/15470077 shutdown_connections 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 --2- 172.21.15.170:0/15470077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0cb380 0x7f371c0ce070 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 -- 172.21.15.170:0/15470077 >> 172.21.15.170:0/15470077 conn(0x7f371c0d0170 msgr2=0x7f371c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 -- 172.21.15.170:0/15470077 shutdown_connections 2024-09-21T02:16:53.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 -- 172.21.15.170:0/15470077 wait complete. 2024-09-21T02:16:53.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 Processor -- start 2024-09-21T02:16:53.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.381+0000 7f372266f640 1 -- start start 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 0x7f371c165cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0cb380 0x7f371c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0cb790 0x7f371c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f371c0744c0 con 0x7f371c0700b0 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f371c074340 con 0x7f371c0cb380 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f371c074640 con 0x7f371c0cb790 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 0x7f371c165cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 0x7f371c165cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48196/0 (socket says 172.21.15.170:48196) 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 -- 172.21.15.170:0/2987017483 learned_addr learned my addr 172.21.15.170:0/2987017483 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371b7fe640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0cb380 0x7f371c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:53.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 -- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0cb790 msgr2=0x7f371c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f3720c0c640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0cb790 0x7f371c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0cb790 0x7f371c157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 -- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0cb380 msgr2=0x7f371c1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0cb380 0x7f371c1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 -- 172.21.15.170:0/2987017483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c09dcd0 con 0x7f371c0700b0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371b7fe640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0cb380 0x7f371c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f371bfff640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 0x7f371c165cc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a8a0 tx=0x7f370c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c0b3040 con 0x7f371c0700b0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f370c0036f0 con 0x7f371c0700b0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c003ac0 con 0x7f371c0700b0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c151ce0 con 0x7f371c0700b0 2024-09-21T02:16:53.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1522a0 con 0x7f371c0700b0 2024-09-21T02:16:53.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.385+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c0c9d50 con 0x7f371c0700b0 2024-09-21T02:16:53.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.389+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f370c0c0060 con 0x7f371c0700b0 2024-09-21T02:16:53.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.389+0000 7f3718ff9640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37080733b0 0x7f37080758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:53.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.389+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f370c12ed10 con 0x7f371c0700b0 2024-09-21T02:16:53.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.389+0000 7f371b7fe640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37080733b0 0x7f37080758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:53.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.389+0000 7f371b7fe640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37080733b0 0x7f37080758c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f371c0cae70 tx=0x7f3710008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.393+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f370c134020 con 0x7f371c0700b0 2024-09-21T02:16:53.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:53.585+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f371c0ce070 con 0x7f371c0700b0 2024-09-21T02:16:54.027 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:16:54.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:16:54.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.201+0000 7f3718ff9640 1 -- 172.21.15.170:0/2987017483 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f370c017760 con 0x7f371c0700b0 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37080733b0 msgr2=0x7f37080758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37080733b0 0x7f37080758c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f371c0cae70 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 msgr2=0x7f371c165cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 0x7f371c165cc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a8a0 tx=0x7f370c0056e0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 shutdown_connections 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f37080733b0 0x7f37080758c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0cb790 0x7f371c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0cb380 0x7f371c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 --2- 172.21.15.170:0/2987017483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f371c0700b0 0x7f371c165cc0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 >> 172.21.15.170:0/2987017483 conn(0x7f371c0d0170 msgr2=0x7f371c075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:54.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 shutdown_connections 2024-09-21T02:16:54.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.205+0000 7f372266f640 1 -- 172.21.15.170:0/2987017483 wait complete. 2024-09-21T02:16:54.234 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:16:54.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 -- 172.21.15.170:0/1088434287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0cb500 msgr2=0x7f0fcc0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/1088434287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0cb500 0x7f0fcc0ce8d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400acd0 tx=0x7f0fc401def0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 -- 172.21.15.170:0/1088434287 shutdown_connections 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/1088434287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0cb500 0x7f0fcc0ce8d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 -- 172.21.15.170:0/1088434287 >> 172.21.15.170:0/1088434287 conn(0x7f0fcc0b1c40 msgr2=0x7f0fcc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 -- 172.21.15.170:0/1088434287 shutdown_connections 2024-09-21T02:16:54.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.389+0000 7f0fd2c73640 1 -- 172.21.15.170:0/1088434287 wait complete. 2024-09-21T02:16:54.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 Processor -- start 2024-09-21T02:16:54.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 -- start start 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fcc0ce4c0 0x7f0fcc0b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fcc15f990 0x7f0fcc0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 0x7f0fcc0adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc169390 con 0x7f0fcc0ad830 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc169210 con 0x7f0fcc15f990 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc169510 con 0x7f0fcc0ce4c0 2024-09-21T02:16:54.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 0x7f0fcc0adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 0x7f0fcc0adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48208/0 (socket says 172.21.15.170:48208) 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 -- 172.21.15.170:0/679267057 learned_addr learned my addr 172.21.15.170:0/679267057 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fcbfff640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fcc15f990 0x7f0fcc0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd0a0f640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fcc0ce4c0 0x7f0fcc0b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 -- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fcc0ce4c0 msgr2=0x7f0fcc0b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fcc0ce4c0 0x7f0fcc0b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 -- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fcc15f990 msgr2=0x7f0fcc0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fcc15f990 0x7f0fcc0b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 -- 172.21.15.170:0/679267057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc0b4f50 con 0x7f0fcc0ad830 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fcbfff640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fcc15f990 0x7f0fcc0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd0a0f640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fcc0ce4c0 0x7f0fcc0b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd1210640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 0x7f0fcc0adc40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40b1040 tx=0x7f0fc4005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40b3040 con 0x7f0fcc0ad830 2024-09-21T02:16:54.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fc4004330 con 0x7f0fcc0ad830 2024-09-21T02:16:54.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc155a00 con 0x7f0fcc0ad830 2024-09-21T02:16:54.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4004760 con 0x7f0fcc0ad830 2024-09-21T02:16:54.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.393+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc0b4c70 con 0x7f0fcc0ad830 2024-09-21T02:16:54.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.397+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fcc0ce8d0 con 0x7f0fcc0ad830 2024-09-21T02:16:54.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.397+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f0fc40be060 con 0x7f0fcc0ad830 2024-09-21T02:16:54.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.397+0000 7f0fc97fa640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0fac0733e0 0x7f0fac0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.397+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f0fc412e230 con 0x7f0fcc0ad830 2024-09-21T02:16:54.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.397+0000 7f0fd0a0f640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0fac0733e0 0x7f0fac0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.397+0000 7f0fd0a0f640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0fac0733e0 0x7f0fac0758f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400a850 tx=0x7f0fb4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:54.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.401+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0fc4017ae0 con 0x7f0fcc0ad830 2024-09-21T02:16:54.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.593+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fcc0ce2a0 con 0x7f0fcc0ad830 2024-09-21T02:16:54.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fc97fa640 1 -- 172.21.15.170:0/679267057 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f0fc40f8ec0 con 0x7f0fcc0ad830 2024-09-21T02:16:54.598 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:54.598 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":37,"btime":"2024-09-21T02:16:54:193427+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:16:50.547545+0000","modified":"2024-09-21T02:16:53.589716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/3667702062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3667702062},{"type":"v1","addr":"172.21.15.191:6837","nonce":3667702062}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5889,"qdb_cluster":[5889]},"id":4}]} 2024-09-21T02:16:54.598 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 37 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0fac0733e0 msgr2=0x7f0fac0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0fac0733e0 0x7f0fac0758f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400a850 tx=0x7f0fb4008040 comp rx=0 tx=0).stop 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 msgr2=0x7f0fcc0adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 0x7f0fcc0adc40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40b1040 tx=0x7f0fc4005a60 comp rx=0 tx=0).stop 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 shutdown_connections 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0fac0733e0 0x7f0fac0758f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fcc0ad830 0x7f0fcc0adc40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fcc15f990 0x7f0fcc0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 --2- 172.21.15.170:0/679267057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fcc0ce4c0 0x7f0fcc0b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 >> 172.21.15.170:0/679267057 conn(0x7f0fcc0b1c40 msgr2=0x7f0fcc15ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:54.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 shutdown_connections 2024-09-21T02:16:54.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.597+0000 7f0fd2c73640 1 -- 172.21.15.170:0/679267057 wait complete. 2024-09-21T02:16:54.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:16:54.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- 172.21.15.170:0/1563627148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 msgr2=0x7fa6000b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 --2- 172.21.15.170:0/1563627148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa6000b25b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00098f0 tx=0x7fa5f001c7f0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- 172.21.15.170:0/1563627148 shutdown_connections 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 --2- 172.21.15.170:0/1563627148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6000b60e0 0x7fa6000b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 --2- 172.21.15.170:0/1563627148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa6000b25b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- 172.21.15.170:0/1563627148 >> 172.21.15.170:0/1563627148 conn(0x7fa60009a9c0 msgr2=0x7fa6000d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- 172.21.15.170:0/1563627148 shutdown_connections 2024-09-21T02:16:54.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- 172.21.15.170:0/1563627148 wait complete. 2024-09-21T02:16:54.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 Processor -- start 2024-09-21T02:16:54.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- start start 2024-09-21T02:16:54.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa600070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6000b60e0 0x7fa600142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6000ccce0 0x7fa600142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6000b1910 con 0x7fa6000b21a0 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6000b1790 con 0x7fa6000b60e0 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa605754640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa60016a4f0 con 0x7fa6000ccce0 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa5feffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa600070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa5feffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa600070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48220/0 (socket says 172.21.15.170:48220) 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.777+0000 7fa5feffd640 1 -- 172.21.15.170:0/1687977207 learned_addr learned my addr 172.21.15.170:0/1687977207 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5feffd640 1 -- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6000ccce0 msgr2=0x7fa600142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5feffd640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6000ccce0 0x7fa600142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5feffd640 1 -- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6000b60e0 msgr2=0x7fa600142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5fe7fc640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6000b60e0 0x7fa600142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5feffd640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6000b60e0 0x7fa600142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5feffd640 1 -- 172.21.15.170:0/1687977207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6000ae450 con 0x7fa6000b21a0 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5fe7fc640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6000b60e0 0x7fa600142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5feffd640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa600070de0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009840 tx=0x7fa5f00a8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00a8610 con 0x7fa6000b21a0 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5f00a8a60 con 0x7fa6000b21a0 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa600142d90 con 0x7fa6000b21a0 2024-09-21T02:16:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00a8e50 con 0x7fa6000b21a0 2024-09-21T02:16:54.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6000ae6f0 con 0x7fa6000b21a0 2024-09-21T02:16:54.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6000b25b0 con 0x7fa6000b21a0 2024-09-21T02:16:54.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fa5f0004040 con 0x7fa6000b21a0 2024-09-21T02:16:54.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5e77fe640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa5ec0736f0 0x7fa5ec075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:54.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa5f012e9a0 con 0x7fa6000b21a0 2024-09-21T02:16:54.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.781+0000 7fa5fe7fc640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa5ec0736f0 0x7fa5ec075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:54.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.785+0000 7fa5fe7fc640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa5ec0736f0 0x7fa5ec075c00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa600153160 tx=0x7fa5f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.789+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa5f00bf060 con 0x7fa6000b21a0 2024-09-21T02:16:54.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.961+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6000cd3d0 con 0x7fa6000b21a0 2024-09-21T02:16:54.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa5e77fe640 1 -- 172.21.15.170:0/1687977207 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16773 (secure 0 0 0) 0x7fa5f00ac030 con 0x7fa6000b21a0 2024-09-21T02:16:54.966 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:54.967 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":59,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:16:52.152448+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:16:48.250782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T02:16:49.517893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:16:54.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa5ec0736f0 msgr2=0x7fa5ec075c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa5ec0736f0 0x7fa5ec075c00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa600153160 tx=0x7fa5f4009040 comp rx=0 tx=0).stop 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 msgr2=0x7fa600070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa600070de0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009840 tx=0x7fa5f00a8420 comp rx=0 tx=0).stop 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 shutdown_connections 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa5ec0736f0 0x7fa5ec075c00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6000ccce0 0x7fa600142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6000b60e0 0x7fa600142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 --2- 172.21.15.170:0/1687977207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6000b21a0 0x7fa600070de0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:54.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 >> 172.21.15.170:0/1687977207 conn(0x7fa60009a9c0 msgr2=0x7fa6000b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:54.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 shutdown_connections 2024-09-21T02:16:54.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:54.965+0000 7fa605754640 1 -- 172.21.15.170:0/1687977207 wait complete. 2024-09-21T02:16:54.990 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:16:54.990 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:16:55.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 -- 172.21.15.170:0/1944716670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a00b25f0 msgr2=0x7f51a00cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 --2- 172.21.15.170:0/1944716670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a00b25f0 0x7f51a00cf950 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5190005fd0 tx=0x7f519001c840 comp rx=0 tx=0).stop 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 -- 172.21.15.170:0/1944716670 shutdown_connections 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 --2- 172.21.15.170:0/1944716670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a015faf0 0x7f51a015ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 --2- 172.21.15.170:0/1944716670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a00b25f0 0x7f51a00cf950 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 -- 172.21.15.170:0/1944716670 >> 172.21.15.170:0/1944716670 conn(0x7f51a009a9c0 msgr2=0x7f51a00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 -- 172.21.15.170:0/1944716670 shutdown_connections 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.137+0000 7f51a6b25640 1 -- 172.21.15.170:0/1944716670 wait complete. 2024-09-21T02:16:55.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 Processor -- start 2024-09-21T02:16:55.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- start start 2024-09-21T02:16:55.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a00b25f0 0x7f51a014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a015faf0 0x7f51a0070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 0x7f51a015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0169400 con 0x7f51a0160440 2024-09-21T02:16:55.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0169280 con 0x7f51a015faf0 2024-09-21T02:16:55.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f51a0169580 con 0x7f51a00b25f0 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 0x7f51a015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 0x7f51a015dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48246/0 (socket says 172.21.15.170:48246) 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 -- 172.21.15.170:0/3654466482 learned_addr learned my addr 172.21.15.170:0/3654466482 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a48c1640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a00b25f0 0x7f51a014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 -- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a00b25f0 msgr2=0x7f51a014d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a00b25f0 0x7f51a014d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 -- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a015faf0 msgr2=0x7f51a0070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f519ffff640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a015faf0 0x7f51a0070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a015faf0 0x7f51a0070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 -- 172.21.15.170:0/3654466482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a01fc4b0 con 0x7f51a0160440 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a50c2640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 0x7f51a015dbf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f519800cb10 tx=0x7f519800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198016020 con 0x7f51a0160440 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f519800ce50 con 0x7f51a0160440 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a01fc740 con 0x7f51a0160440 2024-09-21T02:16:55.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519800ade0 con 0x7f51a0160440 2024-09-21T02:16:55.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a01fcd00 con 0x7f51a0160440 2024-09-21T02:16:55.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.141+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a01fd3c0 con 0x7f51a0160440 2024-09-21T02:16:55.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.145+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f5198019020 con 0x7f51a0160440 2024-09-21T02:16:55.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.145+0000 7f519d7fa640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f518c0733b0 0x7f518c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.145+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5198094540 con 0x7f51a0160440 2024-09-21T02:16:55.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.145+0000 7f51a48c1640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f518c0733b0 0x7f518c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.145+0000 7f51a48c1640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f518c0733b0 0x7f518c0758c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f51900ae000 tx=0x7f519001b080 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:55.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.149+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f51a01fd3c0 con 0x7f51a0160440 2024-09-21T02:16:55.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.321+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51a014d8d0 con 0x7f518c0733b0 2024-09-21T02:16:55.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.333+0000 7f519d7fa640 1 -- 172.21.15.170:0/3654466482 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f51a014d8d0 con 0x7f518c0733b0 2024-09-21T02:16:55.338 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:55.339 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:16:55.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f518c0733b0 msgr2=0x7f518c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f518c0733b0 0x7f518c0758c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f51900ae000 tx=0x7f519001b080 comp rx=0 tx=0).stop 2024-09-21T02:16:55.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 msgr2=0x7f51a015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 0x7f51a015dbf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f519800cb10 tx=0x7f519800cc00 comp rx=0 tx=0).stop 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 shutdown_connections 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f518c0733b0 0x7f518c0758c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0160440 0x7f51a015dbf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a015faf0 0x7f51a0070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 --2- 172.21.15.170:0/3654466482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a00b25f0 0x7f51a014d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 >> 172.21.15.170:0/3654466482 conn(0x7f51a009a9c0 msgr2=0x7f51a00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 shutdown_connections 2024-09-21T02:16:55.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.337+0000 7f51a6b25640 1 -- 172.21.15.170:0/3654466482 wait complete. 2024-09-21T02:16:55.359 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-21T02:16:54.946702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264520,"kb_used_data":6684,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754710200,"statfs":{"total":773094113280,"available":772823244800,"internally_reserved":0,"allocated":6844416,"data_stored":3499696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.824918"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164110+0000","last_change":"2024-09-21T02:16:51.189665+0000","last_active":"2024-09-21T02:16:52.164110+0000","last_peered":"2024-09-21T02:16:52.164110+0000","last_clean":"2024-09-21T02:16:52.164110+0000","last_became_active":"2024-09-21T02:16:51.188346+0000","last_became_peered":"2024-09-21T02:16:51.188346+0000","last_unstale":"2024-09-21T02:16:52.164110+0000","last_undegraded":"2024-09-21T02:16:52.164110+0000","last_fullsized":"2024-09-21T02:16:52.164110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:11.621698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732554+0000","last_change":"2024-09-21T02:16:51.196632+0000","last_active":"2024-09-21T02:16:52.732554+0000","last_peered":"2024-09-21T02:16:52.732554+0000","last_clean":"2024-09-21T02:16:52.732554+0000","last_became_active":"2024-09-21T02:16:51.196304+0000","last_became_peered":"2024-09-21T02:16:51.196304+0000","last_unstale":"2024-09-21T02:16:52.732554+0000","last_undegraded":"2024-09-21T02:16:52.732554+0000","last_fullsized":"2024-09-21T02:16:52.732554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.943160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159667+0000","last_change":"2024-09-21T02:16:51.165597+0000","last_active":"2024-09-21T02:16:52.159667+0000","last_peered":"2024-09-21T02:16:52.159667+0000","last_clean":"2024-09-21T02:16:52.159667+0000","last_became_active":"2024-09-21T02:16:51.165284+0000","last_became_peered":"2024-09-21T02:16:51.165284+0000","last_unstale":"2024-09-21T02:16:52.159667+0000","last_undegraded":"2024-09-21T02:16:52.159667+0000","last_fullsized":"2024-09-21T02:16:52.159667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:18.046819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731462+0000","last_change":"2024-09-21T02:16:51.195743+0000","last_active":"2024-09-21T02:16:52.731462+0000","last_peered":"2024-09-21T02:16:52.731462+0000","last_clean":"2024-09-21T02:16:52.731462+0000","last_became_active":"2024-09-21T02:16:51.195402+0000","last_became_peered":"2024-09-21T02:16:51.195402+0000","last_unstale":"2024-09-21T02:16:52.731462+0000","last_undegraded":"2024-09-21T02:16:52.731462+0000","last_fullsized":"2024-09-21T02:16:52.731462+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:17:56.566142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162240+0000","last_change":"2024-09-21T02:16:51.190063+0000","last_active":"2024-09-21T02:16:52.162240+0000","last_peered":"2024-09-21T02:16:52.162240+0000","last_clean":"2024-09-21T02:16:52.162240+0000","last_became_active":"2024-09-21T02:16:51.187717+0000","last_became_peered":"2024-09-21T02:16:51.187717+0000","last_unstale":"2024-09-21T02:16:52.162240+0000","last_undegraded":"2024-09-21T02:16:52.162240+0000","last_fullsized":"2024-09-21T02:16:52.162240+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:50.623434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161739+0000","last_change":"2024-09-21T02:16:51.200131+0000","last_active":"2024-09-21T02:16:52.161739+0000","last_peered":"2024-09-21T02:16:52.161739+0000","last_clean":"2024-09-21T02:16:52.161739+0000","last_became_active":"2024-09-21T02:16:51.199648+0000","last_became_peered":"2024-09-21T02:16:51.199648+0000","last_unstale":"2024-09-21T02:16:52.161739+0000","last_undegraded":"2024-09-21T02:16:52.161739+0000","last_fullsized":"2024-09-21T02:16:52.161739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:10.966326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732134+0000","last_change":"2024-09-21T02:16:51.186325+0000","last_active":"2024-09-21T02:16:52.732134+0000","last_peered":"2024-09-21T02:16:52.732134+0000","last_clean":"2024-09-21T02:16:52.732134+0000","last_became_active":"2024-09-21T02:16:51.185672+0000","last_became_peered":"2024-09-21T02:16:51.185672+0000","last_unstale":"2024-09-21T02:16:52.732134+0000","last_undegraded":"2024-09-21T02:16:52.732134+0000","last_fullsized":"2024-09-21T02:16:52.732134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:08.267398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161305+0000","last_change":"2024-09-21T02:16:51.163709+0000","last_active":"2024-09-21T02:16:52.161305+0000","last_peered":"2024-09-21T02:16:52.161305+0000","last_clean":"2024-09-21T02:16:52.161305+0000","last_became_active":"2024-09-21T02:16:51.163247+0000","last_became_peered":"2024-09-21T02:16:51.163247+0000","last_unstale":"2024-09-21T02:16:52.161305+0000","last_undegraded":"2024-09-21T02:16:52.161305+0000","last_fullsized":"2024-09-21T02:16:52.161305+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:05.830044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674251+0000","last_change":"2024-09-21T02:16:51.191738+0000","last_active":"2024-09-21T02:16:52.674251+0000","last_peered":"2024-09-21T02:16:52.674251+0000","last_clean":"2024-09-21T02:16:52.674251+0000","last_became_active":"2024-09-21T02:16:51.191310+0000","last_became_peered":"2024-09-21T02:16:51.191310+0000","last_unstale":"2024-09-21T02:16:52.674251+0000","last_undegraded":"2024-09-21T02:16:52.674251+0000","last_fullsized":"2024-09-21T02:16:52.674251+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:03.031340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163407+0000","last_change":"2024-09-21T02:16:51.186870+0000","last_active":"2024-09-21T02:16:52.163407+0000","last_peered":"2024-09-21T02:16:52.163407+0000","last_clean":"2024-09-21T02:16:52.163407+0000","last_became_active":"2024-09-21T02:16:51.186413+0000","last_became_peered":"2024-09-21T02:16:51.186413+0000","last_unstale":"2024-09-21T02:16:52.163407+0000","last_undegraded":"2024-09-21T02:16:52.163407+0000","last_fullsized":"2024-09-21T02:16:52.163407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:47.278485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730719+0000","last_change":"2024-09-21T02:16:51.201899+0000","last_active":"2024-09-21T02:16:52.730719+0000","last_peered":"2024-09-21T02:16:52.730719+0000","last_clean":"2024-09-21T02:16:52.730719+0000","last_became_active":"2024-09-21T02:16:51.201406+0000","last_became_peered":"2024-09-21T02:16:51.201406+0000","last_unstale":"2024-09-21T02:16:52.730719+0000","last_undegraded":"2024-09-21T02:16:52.730719+0000","last_fullsized":"2024-09-21T02:16:52.730719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:43.840495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673814+0000","last_change":"2024-09-21T02:16:51.173933+0000","last_active":"2024-09-21T02:16:52.673814+0000","last_peered":"2024-09-21T02:16:52.673814+0000","last_clean":"2024-09-21T02:16:52.673814+0000","last_became_active":"2024-09-21T02:16:51.173638+0000","last_became_peered":"2024-09-21T02:16:51.173638+0000","last_unstale":"2024-09-21T02:16:52.673814+0000","last_undegraded":"2024-09-21T02:16:52.673814+0000","last_fullsized":"2024-09-21T02:16:52.673814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:00.100917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730926+0000","last_change":"2024-09-21T02:16:51.177724+0000","last_active":"2024-09-21T02:16:52.730926+0000","last_peered":"2024-09-21T02:16:52.730926+0000","last_clean":"2024-09-21T02:16:52.730926+0000","last_became_active":"2024-09-21T02:16:51.177413+0000","last_became_peered":"2024-09-21T02:16:51.177413+0000","last_unstale":"2024-09-21T02:16:52.730926+0000","last_undegraded":"2024-09-21T02:16:52.730926+0000","last_fullsized":"2024-09-21T02:16:52.730926+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.221491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158873+0000","last_change":"2024-09-21T02:16:51.170286+0000","last_active":"2024-09-21T02:16:52.158873+0000","last_peered":"2024-09-21T02:16:52.158873+0000","last_clean":"2024-09-21T02:16:52.158873+0000","last_became_active":"2024-09-21T02:16:51.167554+0000","last_became_peered":"2024-09-21T02:16:51.167554+0000","last_unstale":"2024-09-21T02:16:52.158873+0000","last_undegraded":"2024-09-21T02:16:52.158873+0000","last_fullsized":"2024-09-21T02:16:52.158873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:39.560466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730467+0000","last_change":"2024-09-21T02:16:51.193017+0000","last_active":"2024-09-21T02:16:52.730467+0000","last_peered":"2024-09-21T02:16:52.730467+0000","last_clean":"2024-09-21T02:16:52.730467+0000","last_became_active":"2024-09-21T02:16:51.192378+0000","last_became_peered":"2024-09-21T02:16:51.192378+0000","last_unstale":"2024-09-21T02:16:52.730467+0000","last_undegraded":"2024-09-21T02:16:52.730467+0000","last_fullsized":"2024-09-21T02:16:52.730467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:06.695079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160276+0000","last_change":"2024-09-21T02:16:51.187525+0000","last_active":"2024-09-21T02:16:52.160276+0000","last_peered":"2024-09-21T02:16:52.160276+0000","last_clean":"2024-09-21T02:16:52.160276+0000","last_became_active":"2024-09-21T02:16:51.186631+0000","last_became_peered":"2024-09-21T02:16:51.186631+0000","last_unstale":"2024-09-21T02:16:52.160276+0000","last_undegraded":"2024-09-21T02:16:52.160276+0000","last_fullsized":"2024-09-21T02:16:52.160276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:06.599990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164368+0000","last_change":"2024-09-21T02:16:51.188548+0000","last_active":"2024-09-21T02:16:52.164368+0000","last_peered":"2024-09-21T02:16:52.164368+0000","last_clean":"2024-09-21T02:16:52.164368+0000","last_became_active":"2024-09-21T02:16:51.188198+0000","last_became_peered":"2024-09-21T02:16:51.188198+0000","last_unstale":"2024-09-21T02:16:52.164368+0000","last_undegraded":"2024-09-21T02:16:52.164368+0000","last_fullsized":"2024-09-21T02:16:52.164368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:48.755766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159271+0000","last_change":"2024-09-21T02:16:51.188743+0000","last_active":"2024-09-21T02:16:52.159271+0000","last_peered":"2024-09-21T02:16:52.159271+0000","last_clean":"2024-09-21T02:16:52.159271+0000","last_became_active":"2024-09-21T02:16:51.188159+0000","last_became_peered":"2024-09-21T02:16:51.188159+0000","last_unstale":"2024-09-21T02:16:52.159271+0000","last_undegraded":"2024-09-21T02:16:52.159271+0000","last_fullsized":"2024-09-21T02:16:52.159271+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:00.091492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732154+0000","last_change":"2024-09-21T02:16:51.207799+0000","last_active":"2024-09-21T02:16:52.732154+0000","last_peered":"2024-09-21T02:16:52.732154+0000","last_clean":"2024-09-21T02:16:52.732154+0000","last_became_active":"2024-09-21T02:16:51.207427+0000","last_became_peered":"2024-09-21T02:16:51.207427+0000","last_unstale":"2024-09-21T02:16:52.732154+0000","last_undegraded":"2024-09-21T02:16:52.732154+0000","last_fullsized":"2024-09-21T02:16:52.732154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:21.646065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730456+0000","last_change":"2024-09-21T02:16:51.201460+0000","last_active":"2024-09-21T02:16:52.730456+0000","last_peered":"2024-09-21T02:16:52.730456+0000","last_clean":"2024-09-21T02:16:52.730456+0000","last_became_active":"2024-09-21T02:16:51.201201+0000","last_became_peered":"2024-09-21T02:16:51.201201+0000","last_unstale":"2024-09-21T02:16:52.730456+0000","last_undegraded":"2024-09-21T02:16:52.730456+0000","last_fullsized":"2024-09-21T02:16:52.730456+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:09.256231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731089+0000","last_change":"2024-09-21T02:16:51.192385+0000","last_active":"2024-09-21T02:16:52.731089+0000","last_peered":"2024-09-21T02:16:52.731089+0000","last_clean":"2024-09-21T02:16:52.731089+0000","last_became_active":"2024-09-21T02:16:51.191985+0000","last_became_peered":"2024-09-21T02:16:51.191985+0000","last_unstale":"2024-09-21T02:16:52.731089+0000","last_undegraded":"2024-09-21T02:16:52.731089+0000","last_fullsized":"2024-09-21T02:16:52.731089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.185759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730957+0000","last_change":"2024-09-21T02:16:51.185648+0000","last_active":"2024-09-21T02:16:52.730957+0000","last_peered":"2024-09-21T02:16:52.730957+0000","last_clean":"2024-09-21T02:16:52.730957+0000","last_became_active":"2024-09-21T02:16:51.185205+0000","last_became_peered":"2024-09-21T02:16:51.185205+0000","last_unstale":"2024-09-21T02:16:52.730957+0000","last_undegraded":"2024-09-21T02:16:52.730957+0000","last_fullsized":"2024-09-21T02:16:52.730957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:46.624090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732254+0000","last_change":"2024-09-21T02:16:51.181593+0000","last_active":"2024-09-21T02:16:52.732254+0000","last_peered":"2024-09-21T02:16:52.732254+0000","last_clean":"2024-09-21T02:16:52.732254+0000","last_became_active":"2024-09-21T02:16:51.180706+0000","last_became_peered":"2024-09-21T02:16:51.180706+0000","last_unstale":"2024-09-21T02:16:52.732254+0000","last_undegraded":"2024-09-21T02:16:52.732254+0000","last_fullsized":"2024-09-21T02:16:52.732254+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:11.406001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674362+0000","last_change":"2024-09-21T02:16:51.191576+0000","last_active":"2024-09-21T02:16:52.674362+0000","last_peered":"2024-09-21T02:16:52.674362+0000","last_clean":"2024-09-21T02:16:52.674362+0000","last_became_active":"2024-09-21T02:16:51.191231+0000","last_became_peered":"2024-09-21T02:16:51.191231+0000","last_unstale":"2024-09-21T02:16:52.674362+0000","last_undegraded":"2024-09-21T02:16:52.674362+0000","last_fullsized":"2024-09-21T02:16:52.674362+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:56.578375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732499+0000","last_change":"2024-09-21T02:16:51.194953+0000","last_active":"2024-09-21T02:16:52.732499+0000","last_peered":"2024-09-21T02:16:52.732499+0000","last_clean":"2024-09-21T02:16:52.732499+0000","last_became_active":"2024-09-21T02:16:51.194446+0000","last_became_peered":"2024-09-21T02:16:51.194446+0000","last_unstale":"2024-09-21T02:16:52.732499+0000","last_undegraded":"2024-09-21T02:16:52.732499+0000","last_fullsized":"2024-09-21T02:16:52.732499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:56.126563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732172+0000","last_change":"2024-09-21T02:16:51.180259+0000","last_active":"2024-09-21T02:16:52.732172+0000","last_peered":"2024-09-21T02:16:52.732172+0000","last_clean":"2024-09-21T02:16:52.732172+0000","last_became_active":"2024-09-21T02:16:51.179958+0000","last_became_peered":"2024-09-21T02:16:51.179958+0000","last_unstale":"2024-09-21T02:16:52.732172+0000","last_undegraded":"2024-09-21T02:16:52.732172+0000","last_fullsized":"2024-09-21T02:16:52.732172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:41.099926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732339+0000","last_change":"2024-09-21T02:16:51.196428+0000","last_active":"2024-09-21T02:16:52.732339+0000","last_peered":"2024-09-21T02:16:52.732339+0000","last_clean":"2024-09-21T02:16:52.732339+0000","last_became_active":"2024-09-21T02:16:51.196142+0000","last_became_peered":"2024-09-21T02:16:51.196142+0000","last_unstale":"2024-09-21T02:16:52.732339+0000","last_undegraded":"2024-09-21T02:16:52.732339+0000","last_fullsized":"2024-09-21T02:16:52.732339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:23.102351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162444+0000","last_change":"2024-09-21T02:16:51.173717+0000","last_active":"2024-09-21T02:16:52.162444+0000","last_peered":"2024-09-21T02:16:52.162444+0000","last_clean":"2024-09-21T02:16:52.162444+0000","last_became_active":"2024-09-21T02:16:51.173297+0000","last_became_peered":"2024-09-21T02:16:51.173297+0000","last_unstale":"2024-09-21T02:16:52.162444+0000","last_undegraded":"2024-09-21T02:16:52.162444+0000","last_fullsized":"2024-09-21T02:16:52.162444+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:29.989701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162449+0000","last_change":"2024-09-21T02:16:51.166203+0000","last_active":"2024-09-21T02:16:52.162449+0000","last_peered":"2024-09-21T02:16:52.162449+0000","last_clean":"2024-09-21T02:16:52.162449+0000","last_became_active":"2024-09-21T02:16:51.165293+0000","last_became_peered":"2024-09-21T02:16:51.165293+0000","last_unstale":"2024-09-21T02:16:52.162449+0000","last_undegraded":"2024-09-21T02:16:52.162449+0000","last_fullsized":"2024-09-21T02:16:52.162449+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:12.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730819+0000","last_change":"2024-09-21T02:16:51.173606+0000","last_active":"2024-09-21T02:16:52.730819+0000","last_peered":"2024-09-21T02:16:52.730819+0000","last_clean":"2024-09-21T02:16:52.730819+0000","last_became_active":"2024-09-21T02:16:51.173178+0000","last_became_peered":"2024-09-21T02:16:51.173178+0000","last_unstale":"2024-09-21T02:16:52.730819+0000","last_undegraded":"2024-09-21T02:16:52.730819+0000","last_fullsized":"2024-09-21T02:16:52.730819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.809804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731257+0000","last_change":"2024-09-21T02:16:51.194905+0000","last_active":"2024-09-21T02:16:52.731257+0000","last_peered":"2024-09-21T02:16:52.731257+0000","last_clean":"2024-09-21T02:16:52.731257+0000","last_became_active":"2024-09-21T02:16:51.185685+0000","last_became_peered":"2024-09-21T02:16:51.185685+0000","last_unstale":"2024-09-21T02:16:52.731257+0000","last_undegraded":"2024-09-21T02:16:52.731257+0000","last_fullsized":"2024-09-21T02:16:52.731257+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:11.411735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730147+0000","last_change":"2024-09-21T02:16:51.187912+0000","last_active":"2024-09-21T02:16:52.730147+0000","last_peered":"2024-09-21T02:16:52.730147+0000","last_clean":"2024-09-21T02:16:52.730147+0000","last_became_active":"2024-09-21T02:16:51.187104+0000","last_became_peered":"2024-09-21T02:16:51.187104+0000","last_unstale":"2024-09-21T02:16:52.730147+0000","last_undegraded":"2024-09-21T02:16:52.730147+0000","last_fullsized":"2024-09-21T02:16:52.730147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:58.703357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164087+0000","last_change":"2024-09-21T02:16:51.185956+0000","last_active":"2024-09-21T02:16:52.164087+0000","last_peered":"2024-09-21T02:16:52.164087+0000","last_clean":"2024-09-21T02:16:52.164087+0000","last_became_active":"2024-09-21T02:16:51.185566+0000","last_became_peered":"2024-09-21T02:16:51.185566+0000","last_unstale":"2024-09-21T02:16:52.164087+0000","last_undegraded":"2024-09-21T02:16:52.164087+0000","last_fullsized":"2024-09-21T02:16:52.164087+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:55.352497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731912+0000","last_change":"2024-09-21T02:16:50.174848+0000","last_active":"2024-09-21T02:16:52.731912+0000","last_peered":"2024-09-21T02:16:52.731912+0000","last_clean":"2024-09-21T02:16:52.731912+0000","last_became_active":"2024-09-21T02:16:50.173914+0000","last_became_peered":"2024-09-21T02:16:50.173914+0000","last_unstale":"2024-09-21T02:16:52.731912+0000","last_undegraded":"2024-09-21T02:16:52.731912+0000","last_fullsized":"2024-09-21T02:16:52.731912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:35.166011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164466+0000","last_change":"2024-09-21T02:16:51.203099+0000","last_active":"2024-09-21T02:16:52.164466+0000","last_peered":"2024-09-21T02:16:52.164466+0000","last_clean":"2024-09-21T02:16:52.164466+0000","last_became_active":"2024-09-21T02:16:51.202719+0000","last_became_peered":"2024-09-21T02:16:51.202719+0000","last_unstale":"2024-09-21T02:16:52.164466+0000","last_undegraded":"2024-09-21T02:16:52.164466+0000","last_fullsized":"2024-09-21T02:16:52.164466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:50.693876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732588+0000","last_change":"2024-09-21T02:16:50.152214+0000","last_active":"2024-09-21T02:16:52.732588+0000","last_peered":"2024-09-21T02:16:52.732588+0000","last_clean":"2024-09-21T02:16:52.732588+0000","last_became_active":"2024-09-21T02:16:50.151357+0000","last_became_peered":"2024-09-21T02:16:50.151357+0000","last_unstale":"2024-09-21T02:16:52.732588+0000","last_undegraded":"2024-09-21T02:16:52.732588+0000","last_fullsized":"2024-09-21T02:16:52.732588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:08.263342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161674+0000","last_change":"2024-09-21T02:16:51.204940+0000","last_active":"2024-09-21T02:16:52.161674+0000","last_peered":"2024-09-21T02:16:52.161674+0000","last_clean":"2024-09-21T02:16:52.161674+0000","last_became_active":"2024-09-21T02:16:51.204488+0000","last_became_peered":"2024-09-21T02:16:51.204488+0000","last_unstale":"2024-09-21T02:16:52.161674+0000","last_undegraded":"2024-09-21T02:16:52.161674+0000","last_fullsized":"2024-09-21T02:16:52.161674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:51.137409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163434+0000","last_change":"2024-09-21T02:16:50.171963+0000","last_active":"2024-09-21T02:16:52.163434+0000","last_peered":"2024-09-21T02:16:52.163434+0000","last_clean":"2024-09-21T02:16:52.163434+0000","last_became_active":"2024-09-21T02:16:50.171043+0000","last_became_peered":"2024-09-21T02:16:50.171043+0000","last_unstale":"2024-09-21T02:16:52.163434+0000","last_undegraded":"2024-09-21T02:16:52.163434+0000","last_fullsized":"2024-09-21T02:16:52.163434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:00.490366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163140+0000","last_change":"2024-09-21T02:16:51.187593+0000","last_active":"2024-09-21T02:16:52.163140+0000","last_peered":"2024-09-21T02:16:52.163140+0000","last_clean":"2024-09-21T02:16:52.163140+0000","last_became_active":"2024-09-21T02:16:51.187233+0000","last_became_peered":"2024-09-21T02:16:51.187233+0000","last_unstale":"2024-09-21T02:16:52.163140+0000","last_undegraded":"2024-09-21T02:16:52.163140+0000","last_fullsized":"2024-09-21T02:16:52.163140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:11.892830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731337+0000","last_change":"2024-09-21T02:16:50.150419+0000","last_active":"2024-09-21T02:16:52.731337+0000","last_peered":"2024-09-21T02:16:52.731337+0000","last_clean":"2024-09-21T02:16:52.731337+0000","last_became_active":"2024-09-21T02:16:50.150084+0000","last_became_peered":"2024-09-21T02:16:50.150084+0000","last_unstale":"2024-09-21T02:16:52.731337+0000","last_undegraded":"2024-09-21T02:16:52.731337+0000","last_fullsized":"2024-09-21T02:16:52.731337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:36.885015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161200+0000","last_change":"2024-09-21T02:16:51.204546+0000","last_active":"2024-09-21T02:16:52.161200+0000","last_peered":"2024-09-21T02:16:52.161200+0000","last_clean":"2024-09-21T02:16:52.161200+0000","last_became_active":"2024-09-21T02:16:51.204176+0000","last_became_peered":"2024-09-21T02:16:51.204176+0000","last_unstale":"2024-09-21T02:16:52.161200+0000","last_undegraded":"2024-09-21T02:16:52.161200+0000","last_fullsized":"2024-09-21T02:16:52.161200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:37.896294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158905+0000","last_change":"2024-09-21T02:16:50.175912+0000","last_active":"2024-09-21T02:16:52.158905+0000","last_peered":"2024-09-21T02:16:52.158905+0000","last_clean":"2024-09-21T02:16:52.158905+0000","last_became_active":"2024-09-21T02:16:50.175612+0000","last_became_peered":"2024-09-21T02:16:50.175612+0000","last_unstale":"2024-09-21T02:16:52.158905+0000","last_undegraded":"2024-09-21T02:16:52.158905+0000","last_fullsized":"2024-09-21T02:16:52.158905+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:19.224005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730697+0000","last_change":"2024-09-21T02:16:51.195629+0000","last_active":"2024-09-21T02:16:52.730697+0000","last_peered":"2024-09-21T02:16:52.730697+0000","last_clean":"2024-09-21T02:16:52.730697+0000","last_became_active":"2024-09-21T02:16:51.195047+0000","last_became_peered":"2024-09-21T02:16:51.195047+0000","last_unstale":"2024-09-21T02:16:52.730697+0000","last_undegraded":"2024-09-21T02:16:52.730697+0000","last_fullsized":"2024-09-21T02:16:52.730697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:03.957096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161943+0000","last_change":"2024-09-21T02:16:50.171947+0000","last_active":"2024-09-21T02:16:52.161943+0000","last_peered":"2024-09-21T02:16:52.161943+0000","last_clean":"2024-09-21T02:16:52.161943+0000","last_became_active":"2024-09-21T02:16:50.171098+0000","last_became_peered":"2024-09-21T02:16:50.171098+0000","last_unstale":"2024-09-21T02:16:52.161943+0000","last_undegraded":"2024-09-21T02:16:52.161943+0000","last_fullsized":"2024-09-21T02:16:52.161943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:31.894529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158803+0000","last_change":"2024-09-21T02:16:51.174713+0000","last_active":"2024-09-21T02:16:52.158803+0000","last_peered":"2024-09-21T02:16:52.158803+0000","last_clean":"2024-09-21T02:16:52.158803+0000","last_became_active":"2024-09-21T02:16:51.174379+0000","last_became_peered":"2024-09-21T02:16:51.174379+0000","last_unstale":"2024-09-21T02:16:52.158803+0000","last_undegraded":"2024-09-21T02:16:52.158803+0000","last_fullsized":"2024-09-21T02:16:52.158803+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:31.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673332+0000","last_change":"2024-09-21T02:16:50.132772+0000","last_active":"2024-09-21T02:16:52.673332+0000","last_peered":"2024-09-21T02:16:52.673332+0000","last_clean":"2024-09-21T02:16:52.673332+0000","last_became_active":"2024-09-21T02:16:50.132313+0000","last_became_peered":"2024-09-21T02:16:50.132313+0000","last_unstale":"2024-09-21T02:16:52.673332+0000","last_undegraded":"2024-09-21T02:16:52.673332+0000","last_fullsized":"2024-09-21T02:16:52.673332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:39.207003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160678+0000","last_change":"2024-09-21T02:16:51.188083+0000","last_active":"2024-09-21T02:16:52.160678+0000","last_peered":"2024-09-21T02:16:52.160678+0000","last_clean":"2024-09-21T02:16:52.160678+0000","last_became_active":"2024-09-21T02:16:51.187786+0000","last_became_peered":"2024-09-21T02:16:51.187786+0000","last_unstale":"2024-09-21T02:16:52.160678+0000","last_undegraded":"2024-09-21T02:16:52.160678+0000","last_fullsized":"2024-09-21T02:16:52.160678+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:01.714526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.729907+0000","last_change":"2024-09-21T02:16:50.164996+0000","last_active":"2024-09-21T02:16:52.729907+0000","last_peered":"2024-09-21T02:16:52.729907+0000","last_clean":"2024-09-21T02:16:52.729907+0000","last_became_active":"2024-09-21T02:16:50.164668+0000","last_became_peered":"2024-09-21T02:16:50.164668+0000","last_unstale":"2024-09-21T02:16:52.729907+0000","last_undegraded":"2024-09-21T02:16:52.729907+0000","last_fullsized":"2024-09-21T02:16:52.729907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:19.224005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732257+0000","last_change":"2024-09-21T02:16:51.199388+0000","last_active":"2024-09-21T02:16:52.732257+0000","last_peered":"2024-09-21T02:16:52.732257+0000","last_clean":"2024-09-21T02:16:52.732257+0000","last_became_active":"2024-09-21T02:16:51.198999+0000","last_became_peered":"2024-09-21T02:16:51.198999+0000","last_unstale":"2024-09-21T02:16:52.732257+0000","last_undegraded":"2024-09-21T02:16:52.732257+0000","last_fullsized":"2024-09-21T02:16:52.732257+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:20.111417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673791+0000","last_change":"2024-09-21T02:16:50.156976+0000","last_active":"2024-09-21T02:16:52.673791+0000","last_peered":"2024-09-21T02:16:52.673791+0000","last_clean":"2024-09-21T02:16:52.673791+0000","last_became_active":"2024-09-21T02:16:50.156519+0000","last_became_peered":"2024-09-21T02:16:50.156519+0000","last_unstale":"2024-09-21T02:16:52.673791+0000","last_undegraded":"2024-09-21T02:16:52.673791+0000","last_fullsized":"2024-09-21T02:16:52.673791+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:04.725484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159753+0000","last_change":"2024-09-21T02:16:51.172272+0000","last_active":"2024-09-21T02:16:52.159753+0000","last_peered":"2024-09-21T02:16:52.159753+0000","last_clean":"2024-09-21T02:16:52.159753+0000","last_became_active":"2024-09-21T02:16:51.167774+0000","last_became_peered":"2024-09-21T02:16:51.167774+0000","last_unstale":"2024-09-21T02:16:52.159753+0000","last_undegraded":"2024-09-21T02:16:52.159753+0000","last_fullsized":"2024-09-21T02:16:52.159753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:11.406004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160342+0000","last_change":"2024-09-21T02:16:50.165192+0000","last_active":"2024-09-21T02:16:52.160342+0000","last_peered":"2024-09-21T02:16:52.160342+0000","last_clean":"2024-09-21T02:16:52.160342+0000","last_became_active":"2024-09-21T02:16:50.164365+0000","last_became_peered":"2024-09-21T02:16:50.164365+0000","last_unstale":"2024-09-21T02:16:52.160342+0000","last_undegraded":"2024-09-21T02:16:52.160342+0000","last_fullsized":"2024-09-21T02:16:52.160342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:30.765265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162024+0000","last_change":"2024-09-21T02:16:51.204939+0000","last_active":"2024-09-21T02:16:52.162024+0000","last_peered":"2024-09-21T02:16:52.162024+0000","last_clean":"2024-09-21T02:16:52.162024+0000","last_became_active":"2024-09-21T02:16:51.204436+0000","last_became_peered":"2024-09-21T02:16:51.204436+0000","last_unstale":"2024-09-21T02:16:52.162024+0000","last_undegraded":"2024-09-21T02:16:52.162024+0000","last_fullsized":"2024-09-21T02:16:52.162024+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.152781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732500+0000","last_change":"2024-09-21T02:16:50.153393+0000","last_active":"2024-09-21T02:16:52.732500+0000","last_peered":"2024-09-21T02:16:52.732500+0000","last_clean":"2024-09-21T02:16:52.732500+0000","last_became_active":"2024-09-21T02:16:50.153055+0000","last_became_peered":"2024-09-21T02:16:50.153055+0000","last_unstale":"2024-09-21T02:16:52.732500+0000","last_undegraded":"2024-09-21T02:16:52.732500+0000","last_fullsized":"2024-09-21T02:16:52.732500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:04.818765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732312+0000","last_change":"2024-09-21T02:16:51.178490+0000","last_active":"2024-09-21T02:16:52.732312+0000","last_peered":"2024-09-21T02:16:52.732312+0000","last_clean":"2024-09-21T02:16:52.732312+0000","last_became_active":"2024-09-21T02:16:51.178036+0000","last_became_peered":"2024-09-21T02:16:51.178036+0000","last_unstale":"2024-09-21T02:16:52.732312+0000","last_undegraded":"2024-09-21T02:16:52.732312+0000","last_fullsized":"2024-09-21T02:16:52.732312+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:29.492707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731371+0000","last_change":"2024-09-21T02:16:50.143980+0000","last_active":"2024-09-21T02:16:52.731371+0000","last_peered":"2024-09-21T02:16:52.731371+0000","last_clean":"2024-09-21T02:16:52.731371+0000","last_became_active":"2024-09-21T02:16:50.143616+0000","last_became_peered":"2024-09-21T02:16:50.143616+0000","last_unstale":"2024-09-21T02:16:52.731371+0000","last_undegraded":"2024-09-21T02:16:52.731371+0000","last_fullsized":"2024-09-21T02:16:52.731371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:41.273913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673557+0000","last_change":"2024-09-21T02:16:51.178265+0000","last_active":"2024-09-21T02:16:52.673557+0000","last_peered":"2024-09-21T02:16:52.673557+0000","last_clean":"2024-09-21T02:16:52.673557+0000","last_became_active":"2024-09-21T02:16:51.177817+0000","last_became_peered":"2024-09-21T02:16:51.177817+0000","last_unstale":"2024-09-21T02:16:52.673557+0000","last_undegraded":"2024-09-21T02:16:52.673557+0000","last_fullsized":"2024-09-21T02:16:52.673557+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:29.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161632+0000","last_change":"2024-09-21T02:16:50.176339+0000","last_active":"2024-09-21T02:16:52.161632+0000","last_peered":"2024-09-21T02:16:52.161632+0000","last_clean":"2024-09-21T02:16:52.161632+0000","last_became_active":"2024-09-21T02:16:50.175487+0000","last_became_peered":"2024-09-21T02:16:50.175487+0000","last_unstale":"2024-09-21T02:16:52.161632+0000","last_undegraded":"2024-09-21T02:16:52.161632+0000","last_fullsized":"2024-09-21T02:16:52.161632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:28.997124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158850+0000","last_change":"2024-09-21T02:16:51.164191+0000","last_active":"2024-09-21T02:16:52.158850+0000","last_peered":"2024-09-21T02:16:52.158850+0000","last_clean":"2024-09-21T02:16:52.158850+0000","last_became_active":"2024-09-21T02:16:51.163850+0000","last_became_peered":"2024-09-21T02:16:51.163850+0000","last_unstale":"2024-09-21T02:16:52.158850+0000","last_undegraded":"2024-09-21T02:16:52.158850+0000","last_fullsized":"2024-09-21T02:16:52.158850+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:35.060289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731039+0000","last_change":"2024-09-21T02:16:50.145198+0000","last_active":"2024-09-21T02:16:52.731039+0000","last_peered":"2024-09-21T02:16:52.731039+0000","last_clean":"2024-09-21T02:16:52.731039+0000","last_became_active":"2024-09-21T02:16:50.144362+0000","last_became_peered":"2024-09-21T02:16:50.144362+0000","last_unstale":"2024-09-21T02:16:52.731039+0000","last_undegraded":"2024-09-21T02:16:52.731039+0000","last_fullsized":"2024-09-21T02:16:52.731039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:22.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673772+0000","last_change":"2024-09-21T02:16:51.174767+0000","last_active":"2024-09-21T02:16:52.673772+0000","last_peered":"2024-09-21T02:16:52.673772+0000","last_clean":"2024-09-21T02:16:52.673772+0000","last_became_active":"2024-09-21T02:16:51.174472+0000","last_became_peered":"2024-09-21T02:16:51.174472+0000","last_unstale":"2024-09-21T02:16:52.673772+0000","last_undegraded":"2024-09-21T02:16:52.673772+0000","last_fullsized":"2024-09-21T02:16:52.673772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:39.378405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732367+0000","last_change":"2024-09-21T02:16:50.148152+0000","last_active":"2024-09-21T02:16:52.732367+0000","last_peered":"2024-09-21T02:16:52.732367+0000","last_clean":"2024-09-21T02:16:52.732367+0000","last_became_active":"2024-09-21T02:16:50.147793+0000","last_became_peered":"2024-09-21T02:16:50.147793+0000","last_unstale":"2024-09-21T02:16:52.732367+0000","last_undegraded":"2024-09-21T02:16:52.732367+0000","last_fullsized":"2024-09-21T02:16:52.732367+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:15.613216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732804+0000","last_change":"2024-09-21T02:16:51.184539+0000","last_active":"2024-09-21T02:16:52.732804+0000","last_peered":"2024-09-21T02:16:52.732804+0000","last_clean":"2024-09-21T02:16:52.732804+0000","last_became_active":"2024-09-21T02:16:51.181220+0000","last_became_peered":"2024-09-21T02:16:51.181220+0000","last_unstale":"2024-09-21T02:16:52.732804+0000","last_undegraded":"2024-09-21T02:16:52.732804+0000","last_fullsized":"2024-09-21T02:16:52.732804+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:52.839564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730252+0000","last_change":"2024-09-21T02:16:50.144290+0000","last_active":"2024-09-21T02:16:52.730252+0000","last_peered":"2024-09-21T02:16:52.730252+0000","last_clean":"2024-09-21T02:16:52.730252+0000","last_became_active":"2024-09-21T02:16:50.143920+0000","last_became_peered":"2024-09-21T02:16:50.143920+0000","last_unstale":"2024-09-21T02:16:52.730252+0000","last_undegraded":"2024-09-21T02:16:52.730252+0000","last_fullsized":"2024-09-21T02:16:52.730252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:54.939937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159885+0000","last_change":"2024-09-21T02:16:51.199498+0000","last_active":"2024-09-21T02:16:52.159885+0000","last_peered":"2024-09-21T02:16:52.159885+0000","last_clean":"2024-09-21T02:16:52.159885+0000","last_became_active":"2024-09-21T02:16:51.199130+0000","last_became_peered":"2024-09-21T02:16:51.199130+0000","last_unstale":"2024-09-21T02:16:52.159885+0000","last_undegraded":"2024-09-21T02:16:52.159885+0000","last_fullsized":"2024-09-21T02:16:52.159885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:54.877624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732422+0000","last_change":"2024-09-21T02:16:50.180686+0000","last_active":"2024-09-21T02:16:52.732422+0000","last_peered":"2024-09-21T02:16:52.732422+0000","last_clean":"2024-09-21T02:16:52.732422+0000","last_became_active":"2024-09-21T02:16:50.179399+0000","last_became_peered":"2024-09-21T02:16:50.179399+0000","last_unstale":"2024-09-21T02:16:52.732422+0000","last_undegraded":"2024-09-21T02:16:52.732422+0000","last_fullsized":"2024-09-21T02:16:52.732422+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:59.798714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160119+0000","last_change":"2024-09-21T02:16:51.176600+0000","last_active":"2024-09-21T02:16:52.160119+0000","last_peered":"2024-09-21T02:16:52.160119+0000","last_clean":"2024-09-21T02:16:52.160119+0000","last_became_active":"2024-09-21T02:16:51.176223+0000","last_became_peered":"2024-09-21T02:16:51.176223+0000","last_unstale":"2024-09-21T02:16:52.160119+0000","last_undegraded":"2024-09-21T02:16:52.160119+0000","last_fullsized":"2024-09-21T02:16:52.160119+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:32.399050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160541+0000","last_change":"2024-09-21T02:16:50.165003+0000","last_active":"2024-09-21T02:16:52.160541+0000","last_peered":"2024-09-21T02:16:52.160541+0000","last_clean":"2024-09-21T02:16:52.160541+0000","last_became_active":"2024-09-21T02:16:50.164130+0000","last_became_peered":"2024-09-21T02:16:50.164130+0000","last_unstale":"2024-09-21T02:16:52.160541+0000","last_undegraded":"2024-09-21T02:16:52.160541+0000","last_fullsized":"2024-09-21T02:16:52.160541+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732100+0000","last_change":"2024-09-21T02:16:51.195247+0000","last_active":"2024-09-21T02:16:52.732100+0000","last_peered":"2024-09-21T02:16:52.732100+0000","last_clean":"2024-09-21T02:16:52.732100+0000","last_became_active":"2024-09-21T02:16:51.191137+0000","last_became_peered":"2024-09-21T02:16:51.191137+0000","last_unstale":"2024-09-21T02:16:52.732100+0000","last_undegraded":"2024-09-21T02:16:52.732100+0000","last_fullsized":"2024-09-21T02:16:52.732100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:54.611197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159628+0000","last_change":"2024-09-21T02:16:50.166657+0000","last_active":"2024-09-21T02:16:52.159628+0000","last_peered":"2024-09-21T02:16:52.159628+0000","last_clean":"2024-09-21T02:16:52.159628+0000","last_became_active":"2024-09-21T02:16:50.166335+0000","last_became_peered":"2024-09-21T02:16:50.166335+0000","last_unstale":"2024-09-21T02:16:52.159628+0000","last_undegraded":"2024-09-21T02:16:52.159628+0000","last_fullsized":"2024-09-21T02:16:52.159628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:41.273913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732888+0000","last_change":"2024-09-21T02:16:51.189237+0000","last_active":"2024-09-21T02:16:52.732888+0000","last_peered":"2024-09-21T02:16:52.732888+0000","last_clean":"2024-09-21T02:16:52.732888+0000","last_became_active":"2024-09-21T02:16:51.188831+0000","last_became_peered":"2024-09-21T02:16:51.188831+0000","last_unstale":"2024-09-21T02:16:52.732888+0000","last_undegraded":"2024-09-21T02:16:52.732888+0000","last_fullsized":"2024-09-21T02:16:52.732888+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:12.834157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162121+0000","last_change":"2024-09-21T02:16:50.174041+0000","last_active":"2024-09-21T02:16:52.162121+0000","last_peered":"2024-09-21T02:16:52.162121+0000","last_clean":"2024-09-21T02:16:52.162121+0000","last_became_active":"2024-09-21T02:16:50.173299+0000","last_became_peered":"2024-09-21T02:16:50.173299+0000","last_unstale":"2024-09-21T02:16:52.162121+0000","last_undegraded":"2024-09-21T02:16:52.162121+0000","last_fullsized":"2024-09-21T02:16:52.162121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:34.049010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673470+0000","last_change":"2024-09-21T02:16:51.188933+0000","last_active":"2024-09-21T02:16:52.673470+0000","last_peered":"2024-09-21T02:16:52.673470+0000","last_clean":"2024-09-21T02:16:52.673470+0000","last_became_active":"2024-09-21T02:16:51.188270+0000","last_became_peered":"2024-09-21T02:16:51.188270+0000","last_unstale":"2024-09-21T02:16:52.673470+0000","last_undegraded":"2024-09-21T02:16:52.673470+0000","last_fullsized":"2024-09-21T02:16:52.673470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:19.891292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731290+0000","last_change":"2024-09-21T02:16:50.166462+0000","last_active":"2024-09-21T02:16:52.731290+0000","last_peered":"2024-09-21T02:16:52.731290+0000","last_clean":"2024-09-21T02:16:52.731290+0000","last_became_active":"2024-09-21T02:16:50.162999+0000","last_became_peered":"2024-09-21T02:16:50.162999+0000","last_unstale":"2024-09-21T02:16:52.731290+0000","last_undegraded":"2024-09-21T02:16:52.731290+0000","last_fullsized":"2024-09-21T02:16:52.731290+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:07.447223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161423+0000","last_change":"2024-09-21T02:16:51.184256+0000","last_active":"2024-09-21T02:16:52.161423+0000","last_peered":"2024-09-21T02:16:52.161423+0000","last_clean":"2024-09-21T02:16:52.161423+0000","last_became_active":"2024-09-21T02:16:51.175450+0000","last_became_peered":"2024-09-21T02:16:51.175450+0000","last_unstale":"2024-09-21T02:16:52.161423+0000","last_undegraded":"2024-09-21T02:16:52.161423+0000","last_fullsized":"2024-09-21T02:16:52.161423+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:16.624495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161300+0000","last_change":"2024-09-21T02:16:50.167281+0000","last_active":"2024-09-21T02:16:52.161300+0000","last_peered":"2024-09-21T02:16:52.161300+0000","last_clean":"2024-09-21T02:16:52.161300+0000","last_became_active":"2024-09-21T02:16:50.166028+0000","last_became_peered":"2024-09-21T02:16:50.166028+0000","last_unstale":"2024-09-21T02:16:52.161300+0000","last_undegraded":"2024-09-21T02:16:52.161300+0000","last_fullsized":"2024-09-21T02:16:52.161300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:38.549187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161538+0000","last_change":"2024-09-21T02:16:51.173435+0000","last_active":"2024-09-21T02:16:52.161538+0000","last_peered":"2024-09-21T02:16:52.161538+0000","last_clean":"2024-09-21T02:16:52.161538+0000","last_became_active":"2024-09-21T02:16:51.172945+0000","last_became_peered":"2024-09-21T02:16:51.172945+0000","last_unstale":"2024-09-21T02:16:52.161538+0000","last_undegraded":"2024-09-21T02:16:52.161538+0000","last_fullsized":"2024-09-21T02:16:52.161538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:26.595491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674089+0000","last_change":"2024-09-21T02:16:50.173233+0000","last_active":"2024-09-21T02:16:52.674089+0000","last_peered":"2024-09-21T02:16:52.674089+0000","last_clean":"2024-09-21T02:16:52.674089+0000","last_became_active":"2024-09-21T02:16:50.172749+0000","last_became_peered":"2024-09-21T02:16:50.172749+0000","last_unstale":"2024-09-21T02:16:52.674089+0000","last_undegraded":"2024-09-21T02:16:52.674089+0000","last_fullsized":"2024-09-21T02:16:52.674089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:55.679512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159747+0000","last_change":"2024-09-21T02:16:51.188449+0000","last_active":"2024-09-21T02:16:52.159747+0000","last_peered":"2024-09-21T02:16:52.159747+0000","last_clean":"2024-09-21T02:16:52.159747+0000","last_became_active":"2024-09-21T02:16:51.188135+0000","last_became_peered":"2024-09-21T02:16:51.188135+0000","last_unstale":"2024-09-21T02:16:52.159747+0000","last_undegraded":"2024-09-21T02:16:52.159747+0000","last_fullsized":"2024-09-21T02:16:52.159747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:26.578649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732681+0000","last_change":"2024-09-21T02:16:50.171404+0000","last_active":"2024-09-21T02:16:52.732681+0000","last_peered":"2024-09-21T02:16:52.732681+0000","last_clean":"2024-09-21T02:16:52.732681+0000","last_became_active":"2024-09-21T02:16:50.170131+0000","last_became_peered":"2024-09-21T02:16:50.170131+0000","last_unstale":"2024-09-21T02:16:52.732681+0000","last_undegraded":"2024-09-21T02:16:52.732681+0000","last_fullsized":"2024-09-21T02:16:52.732681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:22.389587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160386+0000","last_change":"2024-09-21T02:16:51.175941+0000","last_active":"2024-09-21T02:16:52.160386+0000","last_peered":"2024-09-21T02:16:52.160386+0000","last_clean":"2024-09-21T02:16:52.160386+0000","last_became_active":"2024-09-21T02:16:51.174587+0000","last_became_peered":"2024-09-21T02:16:51.174587+0000","last_unstale":"2024-09-21T02:16:52.160386+0000","last_undegraded":"2024-09-21T02:16:52.160386+0000","last_fullsized":"2024-09-21T02:16:52.160386+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:52.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163821+0000","last_change":"2024-09-21T02:16:50.174347+0000","last_active":"2024-09-21T02:16:52.163821+0000","last_peered":"2024-09-21T02:16:52.163821+0000","last_clean":"2024-09-21T02:16:52.163821+0000","last_became_active":"2024-09-21T02:16:50.173542+0000","last_became_peered":"2024-09-21T02:16:50.173542+0000","last_unstale":"2024-09-21T02:16:52.163821+0000","last_undegraded":"2024-09-21T02:16:52.163821+0000","last_fullsized":"2024-09-21T02:16:52.163821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:39.790101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732549+0000","last_change":"2024-09-21T02:16:51.184576+0000","last_active":"2024-09-21T02:16:52.732549+0000","last_peered":"2024-09-21T02:16:52.732549+0000","last_clean":"2024-09-21T02:16:52.732549+0000","last_became_active":"2024-09-21T02:16:51.184223+0000","last_became_peered":"2024-09-21T02:16:51.184223+0000","last_unstale":"2024-09-21T02:16:52.732549+0000","last_undegraded":"2024-09-21T02:16:52.732549+0000","last_fullsized":"2024-09-21T02:16:52.732549+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:53.992884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674113+0000","last_change":"2024-09-21T02:16:50.152016+0000","last_active":"2024-09-21T02:16:52.674113+0000","last_peered":"2024-09-21T02:16:52.674113+0000","last_clean":"2024-09-21T02:16:52.674113+0000","last_became_active":"2024-09-21T02:16:50.151639+0000","last_became_peered":"2024-09-21T02:16:50.151639+0000","last_unstale":"2024-09-21T02:16:52.674113+0000","last_undegraded":"2024-09-21T02:16:52.674113+0000","last_fullsized":"2024-09-21T02:16:52.674113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:32.534455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162137+0000","last_change":"2024-09-21T02:16:51.173919+0000","last_active":"2024-09-21T02:16:52.162137+0000","last_peered":"2024-09-21T02:16:52.162137+0000","last_clean":"2024-09-21T02:16:52.162137+0000","last_became_active":"2024-09-21T02:16:51.173253+0000","last_became_peered":"2024-09-21T02:16:51.173253+0000","last_unstale":"2024-09-21T02:16:52.162137+0000","last_undegraded":"2024-09-21T02:16:52.162137+0000","last_fullsized":"2024-09-21T02:16:52.162137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:10.276293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159201+0000","last_change":"2024-09-21T02:16:50.149394+0000","last_active":"2024-09-21T02:16:52.159201+0000","last_peered":"2024-09-21T02:16:52.159201+0000","last_clean":"2024-09-21T02:16:52.159201+0000","last_became_active":"2024-09-21T02:16:50.148962+0000","last_became_peered":"2024-09-21T02:16:50.148962+0000","last_unstale":"2024-09-21T02:16:52.159201+0000","last_undegraded":"2024-09-21T02:16:52.159201+0000","last_fullsized":"2024-09-21T02:16:52.159201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:02.873090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161880+0000","last_change":"2024-09-21T02:16:51.200663+0000","last_active":"2024-09-21T02:16:52.161880+0000","last_peered":"2024-09-21T02:16:52.161880+0000","last_clean":"2024-09-21T02:16:52.161880+0000","last_became_active":"2024-09-21T02:16:51.198965+0000","last_became_peered":"2024-09-21T02:16:51.198965+0000","last_unstale":"2024-09-21T02:16:52.161880+0000","last_undegraded":"2024-09-21T02:16:52.161880+0000","last_fullsized":"2024-09-21T02:16:52.161880+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:34.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731550+0000","last_change":"2024-09-21T02:16:50.175039+0000","last_active":"2024-09-21T02:16:52.731550+0000","last_peered":"2024-09-21T02:16:52.731550+0000","last_clean":"2024-09-21T02:16:52.731550+0000","last_became_active":"2024-09-21T02:16:50.174228+0000","last_became_peered":"2024-09-21T02:16:50.174228+0000","last_unstale":"2024-09-21T02:16:52.731550+0000","last_undegraded":"2024-09-21T02:16:52.731550+0000","last_fullsized":"2024-09-21T02:16:52.731550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.688724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731551+0000","last_change":"2024-09-21T02:16:51.208663+0000","last_active":"2024-09-21T02:16:52.731551+0000","last_peered":"2024-09-21T02:16:52.731551+0000","last_clean":"2024-09-21T02:16:52.731551+0000","last_became_active":"2024-09-21T02:16:51.208258+0000","last_became_peered":"2024-09-21T02:16:51.208258+0000","last_unstale":"2024-09-21T02:16:52.731551+0000","last_undegraded":"2024-09-21T02:16:52.731551+0000","last_fullsized":"2024-09-21T02:16:52.731551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:23.348286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159016+0000","last_change":"2024-09-21T02:16:50.155612+0000","last_active":"2024-09-21T02:16:52.159016+0000","last_peered":"2024-09-21T02:16:52.159016+0000","last_clean":"2024-09-21T02:16:52.159016+0000","last_became_active":"2024-09-21T02:16:50.155095+0000","last_became_peered":"2024-09-21T02:16:50.155095+0000","last_unstale":"2024-09-21T02:16:52.159016+0000","last_undegraded":"2024-09-21T02:16:52.159016+0000","last_fullsized":"2024-09-21T02:16:52.159016+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:03.680139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161577+0000","last_change":"2024-09-21T02:16:51.167090+0000","last_active":"2024-09-21T02:16:52.161577+0000","last_peered":"2024-09-21T02:16:52.161577+0000","last_clean":"2024-09-21T02:16:52.161577+0000","last_became_active":"2024-09-21T02:16:51.166677+0000","last_became_peered":"2024-09-21T02:16:51.166677+0000","last_unstale":"2024-09-21T02:16:52.161577+0000","last_undegraded":"2024-09-21T02:16:52.161577+0000","last_fullsized":"2024-09-21T02:16:52.161577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.940950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160858+0000","last_change":"2024-09-21T02:16:50.179411+0000","last_active":"2024-09-21T02:16:52.160858+0000","last_peered":"2024-09-21T02:16:52.160858+0000","last_clean":"2024-09-21T02:16:52.160858+0000","last_became_active":"2024-09-21T02:16:50.178589+0000","last_became_peered":"2024-09-21T02:16:50.178589+0000","last_unstale":"2024-09-21T02:16:52.160858+0000","last_undegraded":"2024-09-21T02:16:52.160858+0000","last_fullsized":"2024-09-21T02:16:52.160858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:15.960076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731407+0000","last_change":"2024-09-21T02:16:51.195546+0000","last_active":"2024-09-21T02:16:52.731407+0000","last_peered":"2024-09-21T02:16:52.731407+0000","last_clean":"2024-09-21T02:16:52.731407+0000","last_became_active":"2024-09-21T02:16:51.195082+0000","last_became_peered":"2024-09-21T02:16:51.195082+0000","last_unstale":"2024-09-21T02:16:52.731407+0000","last_undegraded":"2024-09-21T02:16:52.731407+0000","last_fullsized":"2024-09-21T02:16:52.731407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:03.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161522+0000","last_change":"2024-09-21T02:16:50.155940+0000","last_active":"2024-09-21T02:16:52.161522+0000","last_peered":"2024-09-21T02:16:52.161522+0000","last_clean":"2024-09-21T02:16:52.161522+0000","last_became_active":"2024-09-21T02:16:50.155127+0000","last_became_peered":"2024-09-21T02:16:50.155127+0000","last_unstale":"2024-09-21T02:16:52.161522+0000","last_undegraded":"2024-09-21T02:16:52.161522+0000","last_fullsized":"2024-09-21T02:16:52.161522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:36.216279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162584+0000","last_change":"2024-09-21T02:16:51.181492+0000","last_active":"2024-09-21T02:16:52.162584+0000","last_peered":"2024-09-21T02:16:52.162584+0000","last_clean":"2024-09-21T02:16:52.162584+0000","last_became_active":"2024-09-21T02:16:51.181154+0000","last_became_peered":"2024-09-21T02:16:51.181154+0000","last_unstale":"2024-09-21T02:16:52.162584+0000","last_undegraded":"2024-09-21T02:16:52.162584+0000","last_fullsized":"2024-09-21T02:16:52.162584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:27.432178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730884+0000","last_change":"2024-09-21T02:16:50.150588+0000","last_active":"2024-09-21T02:16:52.730884+0000","last_peered":"2024-09-21T02:16:52.730884+0000","last_clean":"2024-09-21T02:16:52.730884+0000","last_became_active":"2024-09-21T02:16:50.150147+0000","last_became_peered":"2024-09-21T02:16:50.150147+0000","last_unstale":"2024-09-21T02:16:52.730884+0000","last_undegraded":"2024-09-21T02:16:52.730884+0000","last_fullsized":"2024-09-21T02:16:52.730884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:02.873090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":176,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.672634+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:16:52.672634+0000","last_peered":"2024-09-21T02:16:52.672634+0000","last_clean":"2024-09-21T02:16:52.672634+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:16:52.672634+0000","last_undegraded":"2024-09-21T02:16:52.672634+0000","last_fullsized":"2024-09-21T02:16:52.672634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":290147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":704512,"data_stored":290077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":880457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36036,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335804,"statfs":{"total":96636764160,"available":96599863296,"internally_reserved":0,"allocated":704512,"data_stored":290145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":720896,"data_stored":291827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1298432,"data_stored":881005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":712704,"data_stored":291667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":696320,"data_stored":284371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:16:55.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:16:55.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 -- 172.21.15.170:0/3957871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce40b6930 msgr2=0x7fbce406ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/3957871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce40b6930 0x7fbce406ce00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009870 tx=0x7fbcd401c820 comp rx=0 tx=0).stop 2024-09-21T02:16:55.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 -- 172.21.15.170:0/3957871535 shutdown_connections 2024-09-21T02:16:55.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/3957871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce40b6930 0x7fbce406ce00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/3957871535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbce40b22b0 0x7fbce40b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 -- 172.21.15.170:0/3957871535 >> 172.21.15.170:0/3957871535 conn(0x7fbce409a9c0 msgr2=0x7fbce40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:55.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 -- 172.21.15.170:0/3957871535 shutdown_connections 2024-09-21T02:16:55.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.545+0000 7fbce9a7c640 1 -- 172.21.15.170:0/3957871535 wait complete. 2024-09-21T02:16:55.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 Processor -- start 2024-09-21T02:16:55.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- start start 2024-09-21T02:16:55.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 0x7fbce40c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbce40b22b0 0x7fbce40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbce40b6930 0x7fbce40ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbce415c4d0 con 0x7fbce406d340 2024-09-21T02:16:55.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbce415c350 con 0x7fbce40b22b0 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbce415c650 con 0x7fbce40b6930 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 0x7fbce40c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 0x7fbce40c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47024/0 (socket says 172.21.15.170:47024) 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 -- 172.21.15.170:0/832803705 learned_addr learned my addr 172.21.15.170:0/832803705 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce2ffd640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbce40b22b0 0x7fbce40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce3fff640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbce40b6930 0x7fbce40ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 -- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbce40b6930 msgr2=0x7fbce40ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbce40b6930 0x7fbce40ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 -- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbce40b22b0 msgr2=0x7fbce40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbce40b22b0 0x7fbce40ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 -- 172.21.15.170:0/832803705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce40ca910 con 0x7fbce406d340 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce37fe640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 0x7fbce40c8290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c9a0 tx=0x7fbccc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc007bb0 con 0x7fbce406d340 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbccc007e40 con 0x7fbce406d340 2024-09-21T02:16:55.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce40cabf0 con 0x7fbce406d340 2024-09-21T02:16:55.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc005270 con 0x7fbce406d340 2024-09-21T02:16:55.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce40cb1b0 con 0x7fbce406d340 2024-09-21T02:16:55.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.549+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce415cb00 con 0x7fbce406d340 2024-09-21T02:16:55.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.553+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fbccc005410 con 0x7fbce406d340 2024-09-21T02:16:55.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.553+0000 7fbcdbfff640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbcc40733e0 0x7fbcc40758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.553+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fbccc094a60 con 0x7fbce406d340 2024-09-21T02:16:55.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.553+0000 7fbce2ffd640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbcc40733e0 0x7fbcc40758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.553+0000 7fbce2ffd640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbcc40733e0 0x7fbcc40758f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40af000 tx=0x7fbcd401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:55.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.557+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbccc014070 con 0x7fbce406d340 2024-09-21T02:16:55.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.721+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbce40b4240 con 0x7fbcc40733e0 2024-09-21T02:16:55.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.733+0000 7fbcdbfff640 1 -- 172.21.15.170:0/832803705 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fbce40b4240 con 0x7fbcc40733e0 2024-09-21T02:16:55.738 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:55.740 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbcc40733e0 msgr2=0x7fbcc40758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbcc40733e0 0x7fbcc40758f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40af000 tx=0x7fbcd401b060 comp rx=0 tx=0).stop 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 msgr2=0x7fbce40c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 0x7fbce40c8290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c9a0 tx=0x7fbccc00ce60 comp rx=0 tx=0).stop 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 shutdown_connections 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbcc40733e0 0x7fbcc40758f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbce40b6930 0x7fbce40ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbce40b22b0 0x7fbce40ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 --2- 172.21.15.170:0/832803705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbce406d340 0x7fbce40c8290 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 >> 172.21.15.170:0/832803705 conn(0x7fbce409a9c0 msgr2=0x7fbce40b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 shutdown_connections 2024-09-21T02:16:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.737+0000 7fbce9a7c640 1 -- 172.21.15.170:0/832803705 wait complete. 2024-09-21T02:16:55.759 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-21T02:16:54.946702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264520,"kb_used_data":6684,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754710200,"statfs":{"total":773094113280,"available":772823244800,"internally_reserved":0,"allocated":6844416,"data_stored":3499696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.824918"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164110+0000","last_change":"2024-09-21T02:16:51.189665+0000","last_active":"2024-09-21T02:16:52.164110+0000","last_peered":"2024-09-21T02:16:52.164110+0000","last_clean":"2024-09-21T02:16:52.164110+0000","last_became_active":"2024-09-21T02:16:51.188346+0000","last_became_peered":"2024-09-21T02:16:51.188346+0000","last_unstale":"2024-09-21T02:16:52.164110+0000","last_undegraded":"2024-09-21T02:16:52.164110+0000","last_fullsized":"2024-09-21T02:16:52.164110+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:11.621698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732554+0000","last_change":"2024-09-21T02:16:51.196632+0000","last_active":"2024-09-21T02:16:52.732554+0000","last_peered":"2024-09-21T02:16:52.732554+0000","last_clean":"2024-09-21T02:16:52.732554+0000","last_became_active":"2024-09-21T02:16:51.196304+0000","last_became_peered":"2024-09-21T02:16:51.196304+0000","last_unstale":"2024-09-21T02:16:52.732554+0000","last_undegraded":"2024-09-21T02:16:52.732554+0000","last_fullsized":"2024-09-21T02:16:52.732554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:06.943160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159667+0000","last_change":"2024-09-21T02:16:51.165597+0000","last_active":"2024-09-21T02:16:52.159667+0000","last_peered":"2024-09-21T02:16:52.159667+0000","last_clean":"2024-09-21T02:16:52.159667+0000","last_became_active":"2024-09-21T02:16:51.165284+0000","last_became_peered":"2024-09-21T02:16:51.165284+0000","last_unstale":"2024-09-21T02:16:52.159667+0000","last_undegraded":"2024-09-21T02:16:52.159667+0000","last_fullsized":"2024-09-21T02:16:52.159667+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:18.046819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731462+0000","last_change":"2024-09-21T02:16:51.195743+0000","last_active":"2024-09-21T02:16:52.731462+0000","last_peered":"2024-09-21T02:16:52.731462+0000","last_clean":"2024-09-21T02:16:52.731462+0000","last_became_active":"2024-09-21T02:16:51.195402+0000","last_became_peered":"2024-09-21T02:16:51.195402+0000","last_unstale":"2024-09-21T02:16:52.731462+0000","last_undegraded":"2024-09-21T02:16:52.731462+0000","last_fullsized":"2024-09-21T02:16:52.731462+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:17:56.566142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162240+0000","last_change":"2024-09-21T02:16:51.190063+0000","last_active":"2024-09-21T02:16:52.162240+0000","last_peered":"2024-09-21T02:16:52.162240+0000","last_clean":"2024-09-21T02:16:52.162240+0000","last_became_active":"2024-09-21T02:16:51.187717+0000","last_became_peered":"2024-09-21T02:16:51.187717+0000","last_unstale":"2024-09-21T02:16:52.162240+0000","last_undegraded":"2024-09-21T02:16:52.162240+0000","last_fullsized":"2024-09-21T02:16:52.162240+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:50.623434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161739+0000","last_change":"2024-09-21T02:16:51.200131+0000","last_active":"2024-09-21T02:16:52.161739+0000","last_peered":"2024-09-21T02:16:52.161739+0000","last_clean":"2024-09-21T02:16:52.161739+0000","last_became_active":"2024-09-21T02:16:51.199648+0000","last_became_peered":"2024-09-21T02:16:51.199648+0000","last_unstale":"2024-09-21T02:16:52.161739+0000","last_undegraded":"2024-09-21T02:16:52.161739+0000","last_fullsized":"2024-09-21T02:16:52.161739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:10.966326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732134+0000","last_change":"2024-09-21T02:16:51.186325+0000","last_active":"2024-09-21T02:16:52.732134+0000","last_peered":"2024-09-21T02:16:52.732134+0000","last_clean":"2024-09-21T02:16:52.732134+0000","last_became_active":"2024-09-21T02:16:51.185672+0000","last_became_peered":"2024-09-21T02:16:51.185672+0000","last_unstale":"2024-09-21T02:16:52.732134+0000","last_undegraded":"2024-09-21T02:16:52.732134+0000","last_fullsized":"2024-09-21T02:16:52.732134+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:08.267398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161305+0000","last_change":"2024-09-21T02:16:51.163709+0000","last_active":"2024-09-21T02:16:52.161305+0000","last_peered":"2024-09-21T02:16:52.161305+0000","last_clean":"2024-09-21T02:16:52.161305+0000","last_became_active":"2024-09-21T02:16:51.163247+0000","last_became_peered":"2024-09-21T02:16:51.163247+0000","last_unstale":"2024-09-21T02:16:52.161305+0000","last_undegraded":"2024-09-21T02:16:52.161305+0000","last_fullsized":"2024-09-21T02:16:52.161305+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:05.830044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674251+0000","last_change":"2024-09-21T02:16:51.191738+0000","last_active":"2024-09-21T02:16:52.674251+0000","last_peered":"2024-09-21T02:16:52.674251+0000","last_clean":"2024-09-21T02:16:52.674251+0000","last_became_active":"2024-09-21T02:16:51.191310+0000","last_became_peered":"2024-09-21T02:16:51.191310+0000","last_unstale":"2024-09-21T02:16:52.674251+0000","last_undegraded":"2024-09-21T02:16:52.674251+0000","last_fullsized":"2024-09-21T02:16:52.674251+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:03.031340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163407+0000","last_change":"2024-09-21T02:16:51.186870+0000","last_active":"2024-09-21T02:16:52.163407+0000","last_peered":"2024-09-21T02:16:52.163407+0000","last_clean":"2024-09-21T02:16:52.163407+0000","last_became_active":"2024-09-21T02:16:51.186413+0000","last_became_peered":"2024-09-21T02:16:51.186413+0000","last_unstale":"2024-09-21T02:16:52.163407+0000","last_undegraded":"2024-09-21T02:16:52.163407+0000","last_fullsized":"2024-09-21T02:16:52.163407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:47.278485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730719+0000","last_change":"2024-09-21T02:16:51.201899+0000","last_active":"2024-09-21T02:16:52.730719+0000","last_peered":"2024-09-21T02:16:52.730719+0000","last_clean":"2024-09-21T02:16:52.730719+0000","last_became_active":"2024-09-21T02:16:51.201406+0000","last_became_peered":"2024-09-21T02:16:51.201406+0000","last_unstale":"2024-09-21T02:16:52.730719+0000","last_undegraded":"2024-09-21T02:16:52.730719+0000","last_fullsized":"2024-09-21T02:16:52.730719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:43.840495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673814+0000","last_change":"2024-09-21T02:16:51.173933+0000","last_active":"2024-09-21T02:16:52.673814+0000","last_peered":"2024-09-21T02:16:52.673814+0000","last_clean":"2024-09-21T02:16:52.673814+0000","last_became_active":"2024-09-21T02:16:51.173638+0000","last_became_peered":"2024-09-21T02:16:51.173638+0000","last_unstale":"2024-09-21T02:16:52.673814+0000","last_undegraded":"2024-09-21T02:16:52.673814+0000","last_fullsized":"2024-09-21T02:16:52.673814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:00.100917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730926+0000","last_change":"2024-09-21T02:16:51.177724+0000","last_active":"2024-09-21T02:16:52.730926+0000","last_peered":"2024-09-21T02:16:52.730926+0000","last_clean":"2024-09-21T02:16:52.730926+0000","last_became_active":"2024-09-21T02:16:51.177413+0000","last_became_peered":"2024-09-21T02:16:51.177413+0000","last_unstale":"2024-09-21T02:16:52.730926+0000","last_undegraded":"2024-09-21T02:16:52.730926+0000","last_fullsized":"2024-09-21T02:16:52.730926+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.221491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158873+0000","last_change":"2024-09-21T02:16:51.170286+0000","last_active":"2024-09-21T02:16:52.158873+0000","last_peered":"2024-09-21T02:16:52.158873+0000","last_clean":"2024-09-21T02:16:52.158873+0000","last_became_active":"2024-09-21T02:16:51.167554+0000","last_became_peered":"2024-09-21T02:16:51.167554+0000","last_unstale":"2024-09-21T02:16:52.158873+0000","last_undegraded":"2024-09-21T02:16:52.158873+0000","last_fullsized":"2024-09-21T02:16:52.158873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:39.560466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730467+0000","last_change":"2024-09-21T02:16:51.193017+0000","last_active":"2024-09-21T02:16:52.730467+0000","last_peered":"2024-09-21T02:16:52.730467+0000","last_clean":"2024-09-21T02:16:52.730467+0000","last_became_active":"2024-09-21T02:16:51.192378+0000","last_became_peered":"2024-09-21T02:16:51.192378+0000","last_unstale":"2024-09-21T02:16:52.730467+0000","last_undegraded":"2024-09-21T02:16:52.730467+0000","last_fullsized":"2024-09-21T02:16:52.730467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:06.695079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160276+0000","last_change":"2024-09-21T02:16:51.187525+0000","last_active":"2024-09-21T02:16:52.160276+0000","last_peered":"2024-09-21T02:16:52.160276+0000","last_clean":"2024-09-21T02:16:52.160276+0000","last_became_active":"2024-09-21T02:16:51.186631+0000","last_became_peered":"2024-09-21T02:16:51.186631+0000","last_unstale":"2024-09-21T02:16:52.160276+0000","last_undegraded":"2024-09-21T02:16:52.160276+0000","last_fullsized":"2024-09-21T02:16:52.160276+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:06.599990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164368+0000","last_change":"2024-09-21T02:16:51.188548+0000","last_active":"2024-09-21T02:16:52.164368+0000","last_peered":"2024-09-21T02:16:52.164368+0000","last_clean":"2024-09-21T02:16:52.164368+0000","last_became_active":"2024-09-21T02:16:51.188198+0000","last_became_peered":"2024-09-21T02:16:51.188198+0000","last_unstale":"2024-09-21T02:16:52.164368+0000","last_undegraded":"2024-09-21T02:16:52.164368+0000","last_fullsized":"2024-09-21T02:16:52.164368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:48.755766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159271+0000","last_change":"2024-09-21T02:16:51.188743+0000","last_active":"2024-09-21T02:16:52.159271+0000","last_peered":"2024-09-21T02:16:52.159271+0000","last_clean":"2024-09-21T02:16:52.159271+0000","last_became_active":"2024-09-21T02:16:51.188159+0000","last_became_peered":"2024-09-21T02:16:51.188159+0000","last_unstale":"2024-09-21T02:16:52.159271+0000","last_undegraded":"2024-09-21T02:16:52.159271+0000","last_fullsized":"2024-09-21T02:16:52.159271+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:00.091492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732154+0000","last_change":"2024-09-21T02:16:51.207799+0000","last_active":"2024-09-21T02:16:52.732154+0000","last_peered":"2024-09-21T02:16:52.732154+0000","last_clean":"2024-09-21T02:16:52.732154+0000","last_became_active":"2024-09-21T02:16:51.207427+0000","last_became_peered":"2024-09-21T02:16:51.207427+0000","last_unstale":"2024-09-21T02:16:52.732154+0000","last_undegraded":"2024-09-21T02:16:52.732154+0000","last_fullsized":"2024-09-21T02:16:52.732154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:21.646065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730456+0000","last_change":"2024-09-21T02:16:51.201460+0000","last_active":"2024-09-21T02:16:52.730456+0000","last_peered":"2024-09-21T02:16:52.730456+0000","last_clean":"2024-09-21T02:16:52.730456+0000","last_became_active":"2024-09-21T02:16:51.201201+0000","last_became_peered":"2024-09-21T02:16:51.201201+0000","last_unstale":"2024-09-21T02:16:52.730456+0000","last_undegraded":"2024-09-21T02:16:52.730456+0000","last_fullsized":"2024-09-21T02:16:52.730456+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:09.256231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731089+0000","last_change":"2024-09-21T02:16:51.192385+0000","last_active":"2024-09-21T02:16:52.731089+0000","last_peered":"2024-09-21T02:16:52.731089+0000","last_clean":"2024-09-21T02:16:52.731089+0000","last_became_active":"2024-09-21T02:16:51.191985+0000","last_became_peered":"2024-09-21T02:16:51.191985+0000","last_unstale":"2024-09-21T02:16:52.731089+0000","last_undegraded":"2024-09-21T02:16:52.731089+0000","last_fullsized":"2024-09-21T02:16:52.731089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:13.185759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730957+0000","last_change":"2024-09-21T02:16:51.185648+0000","last_active":"2024-09-21T02:16:52.730957+0000","last_peered":"2024-09-21T02:16:52.730957+0000","last_clean":"2024-09-21T02:16:52.730957+0000","last_became_active":"2024-09-21T02:16:51.185205+0000","last_became_peered":"2024-09-21T02:16:51.185205+0000","last_unstale":"2024-09-21T02:16:52.730957+0000","last_undegraded":"2024-09-21T02:16:52.730957+0000","last_fullsized":"2024-09-21T02:16:52.730957+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:46.624090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732254+0000","last_change":"2024-09-21T02:16:51.181593+0000","last_active":"2024-09-21T02:16:52.732254+0000","last_peered":"2024-09-21T02:16:52.732254+0000","last_clean":"2024-09-21T02:16:52.732254+0000","last_became_active":"2024-09-21T02:16:51.180706+0000","last_became_peered":"2024-09-21T02:16:51.180706+0000","last_unstale":"2024-09-21T02:16:52.732254+0000","last_undegraded":"2024-09-21T02:16:52.732254+0000","last_fullsized":"2024-09-21T02:16:52.732254+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:11.406001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674362+0000","last_change":"2024-09-21T02:16:51.191576+0000","last_active":"2024-09-21T02:16:52.674362+0000","last_peered":"2024-09-21T02:16:52.674362+0000","last_clean":"2024-09-21T02:16:52.674362+0000","last_became_active":"2024-09-21T02:16:51.191231+0000","last_became_peered":"2024-09-21T02:16:51.191231+0000","last_unstale":"2024-09-21T02:16:52.674362+0000","last_undegraded":"2024-09-21T02:16:52.674362+0000","last_fullsized":"2024-09-21T02:16:52.674362+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:56.578375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732499+0000","last_change":"2024-09-21T02:16:51.194953+0000","last_active":"2024-09-21T02:16:52.732499+0000","last_peered":"2024-09-21T02:16:52.732499+0000","last_clean":"2024-09-21T02:16:52.732499+0000","last_became_active":"2024-09-21T02:16:51.194446+0000","last_became_peered":"2024-09-21T02:16:51.194446+0000","last_unstale":"2024-09-21T02:16:52.732499+0000","last_undegraded":"2024-09-21T02:16:52.732499+0000","last_fullsized":"2024-09-21T02:16:52.732499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:56.126563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732172+0000","last_change":"2024-09-21T02:16:51.180259+0000","last_active":"2024-09-21T02:16:52.732172+0000","last_peered":"2024-09-21T02:16:52.732172+0000","last_clean":"2024-09-21T02:16:52.732172+0000","last_became_active":"2024-09-21T02:16:51.179958+0000","last_became_peered":"2024-09-21T02:16:51.179958+0000","last_unstale":"2024-09-21T02:16:52.732172+0000","last_undegraded":"2024-09-21T02:16:52.732172+0000","last_fullsized":"2024-09-21T02:16:52.732172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:41.099926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732339+0000","last_change":"2024-09-21T02:16:51.196428+0000","last_active":"2024-09-21T02:16:52.732339+0000","last_peered":"2024-09-21T02:16:52.732339+0000","last_clean":"2024-09-21T02:16:52.732339+0000","last_became_active":"2024-09-21T02:16:51.196142+0000","last_became_peered":"2024-09-21T02:16:51.196142+0000","last_unstale":"2024-09-21T02:16:52.732339+0000","last_undegraded":"2024-09-21T02:16:52.732339+0000","last_fullsized":"2024-09-21T02:16:52.732339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:23.102351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162444+0000","last_change":"2024-09-21T02:16:51.173717+0000","last_active":"2024-09-21T02:16:52.162444+0000","last_peered":"2024-09-21T02:16:52.162444+0000","last_clean":"2024-09-21T02:16:52.162444+0000","last_became_active":"2024-09-21T02:16:51.173297+0000","last_became_peered":"2024-09-21T02:16:51.173297+0000","last_unstale":"2024-09-21T02:16:52.162444+0000","last_undegraded":"2024-09-21T02:16:52.162444+0000","last_fullsized":"2024-09-21T02:16:52.162444+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:29.989701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162449+0000","last_change":"2024-09-21T02:16:51.166203+0000","last_active":"2024-09-21T02:16:52.162449+0000","last_peered":"2024-09-21T02:16:52.162449+0000","last_clean":"2024-09-21T02:16:52.162449+0000","last_became_active":"2024-09-21T02:16:51.165293+0000","last_became_peered":"2024-09-21T02:16:51.165293+0000","last_unstale":"2024-09-21T02:16:52.162449+0000","last_undegraded":"2024-09-21T02:16:52.162449+0000","last_fullsized":"2024-09-21T02:16:52.162449+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:12.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730819+0000","last_change":"2024-09-21T02:16:51.173606+0000","last_active":"2024-09-21T02:16:52.730819+0000","last_peered":"2024-09-21T02:16:52.730819+0000","last_clean":"2024-09-21T02:16:52.730819+0000","last_became_active":"2024-09-21T02:16:51.173178+0000","last_became_peered":"2024-09-21T02:16:51.173178+0000","last_unstale":"2024-09-21T02:16:52.730819+0000","last_undegraded":"2024-09-21T02:16:52.730819+0000","last_fullsized":"2024-09-21T02:16:52.730819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.809804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731257+0000","last_change":"2024-09-21T02:16:51.194905+0000","last_active":"2024-09-21T02:16:52.731257+0000","last_peered":"2024-09-21T02:16:52.731257+0000","last_clean":"2024-09-21T02:16:52.731257+0000","last_became_active":"2024-09-21T02:16:51.185685+0000","last_became_peered":"2024-09-21T02:16:51.185685+0000","last_unstale":"2024-09-21T02:16:52.731257+0000","last_undegraded":"2024-09-21T02:16:52.731257+0000","last_fullsized":"2024-09-21T02:16:52.731257+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:01:11.411735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730147+0000","last_change":"2024-09-21T02:16:51.187912+0000","last_active":"2024-09-21T02:16:52.730147+0000","last_peered":"2024-09-21T02:16:52.730147+0000","last_clean":"2024-09-21T02:16:52.730147+0000","last_became_active":"2024-09-21T02:16:51.187104+0000","last_became_peered":"2024-09-21T02:16:51.187104+0000","last_unstale":"2024-09-21T02:16:52.730147+0000","last_undegraded":"2024-09-21T02:16:52.730147+0000","last_fullsized":"2024-09-21T02:16:52.730147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:58.703357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164087+0000","last_change":"2024-09-21T02:16:51.185956+0000","last_active":"2024-09-21T02:16:52.164087+0000","last_peered":"2024-09-21T02:16:52.164087+0000","last_clean":"2024-09-21T02:16:52.164087+0000","last_became_active":"2024-09-21T02:16:51.185566+0000","last_became_peered":"2024-09-21T02:16:51.185566+0000","last_unstale":"2024-09-21T02:16:52.164087+0000","last_undegraded":"2024-09-21T02:16:52.164087+0000","last_fullsized":"2024-09-21T02:16:52.164087+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:55.352497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731912+0000","last_change":"2024-09-21T02:16:50.174848+0000","last_active":"2024-09-21T02:16:52.731912+0000","last_peered":"2024-09-21T02:16:52.731912+0000","last_clean":"2024-09-21T02:16:52.731912+0000","last_became_active":"2024-09-21T02:16:50.173914+0000","last_became_peered":"2024-09-21T02:16:50.173914+0000","last_unstale":"2024-09-21T02:16:52.731912+0000","last_undegraded":"2024-09-21T02:16:52.731912+0000","last_fullsized":"2024-09-21T02:16:52.731912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:35.166011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.164466+0000","last_change":"2024-09-21T02:16:51.203099+0000","last_active":"2024-09-21T02:16:52.164466+0000","last_peered":"2024-09-21T02:16:52.164466+0000","last_clean":"2024-09-21T02:16:52.164466+0000","last_became_active":"2024-09-21T02:16:51.202719+0000","last_became_peered":"2024-09-21T02:16:51.202719+0000","last_unstale":"2024-09-21T02:16:52.164466+0000","last_undegraded":"2024-09-21T02:16:52.164466+0000","last_fullsized":"2024-09-21T02:16:52.164466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:50.693876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732588+0000","last_change":"2024-09-21T02:16:50.152214+0000","last_active":"2024-09-21T02:16:52.732588+0000","last_peered":"2024-09-21T02:16:52.732588+0000","last_clean":"2024-09-21T02:16:52.732588+0000","last_became_active":"2024-09-21T02:16:50.151357+0000","last_became_peered":"2024-09-21T02:16:50.151357+0000","last_unstale":"2024-09-21T02:16:52.732588+0000","last_undegraded":"2024-09-21T02:16:52.732588+0000","last_fullsized":"2024-09-21T02:16:52.732588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:08.263342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161674+0000","last_change":"2024-09-21T02:16:51.204940+0000","last_active":"2024-09-21T02:16:52.161674+0000","last_peered":"2024-09-21T02:16:52.161674+0000","last_clean":"2024-09-21T02:16:52.161674+0000","last_became_active":"2024-09-21T02:16:51.204488+0000","last_became_peered":"2024-09-21T02:16:51.204488+0000","last_unstale":"2024-09-21T02:16:52.161674+0000","last_undegraded":"2024-09-21T02:16:52.161674+0000","last_fullsized":"2024-09-21T02:16:52.161674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:51.137409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163434+0000","last_change":"2024-09-21T02:16:50.171963+0000","last_active":"2024-09-21T02:16:52.163434+0000","last_peered":"2024-09-21T02:16:52.163434+0000","last_clean":"2024-09-21T02:16:52.163434+0000","last_became_active":"2024-09-21T02:16:50.171043+0000","last_became_peered":"2024-09-21T02:16:50.171043+0000","last_unstale":"2024-09-21T02:16:52.163434+0000","last_undegraded":"2024-09-21T02:16:52.163434+0000","last_fullsized":"2024-09-21T02:16:52.163434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:00.490366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163140+0000","last_change":"2024-09-21T02:16:51.187593+0000","last_active":"2024-09-21T02:16:52.163140+0000","last_peered":"2024-09-21T02:16:52.163140+0000","last_clean":"2024-09-21T02:16:52.163140+0000","last_became_active":"2024-09-21T02:16:51.187233+0000","last_became_peered":"2024-09-21T02:16:51.187233+0000","last_unstale":"2024-09-21T02:16:52.163140+0000","last_undegraded":"2024-09-21T02:16:52.163140+0000","last_fullsized":"2024-09-21T02:16:52.163140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:11.892830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731337+0000","last_change":"2024-09-21T02:16:50.150419+0000","last_active":"2024-09-21T02:16:52.731337+0000","last_peered":"2024-09-21T02:16:52.731337+0000","last_clean":"2024-09-21T02:16:52.731337+0000","last_became_active":"2024-09-21T02:16:50.150084+0000","last_became_peered":"2024-09-21T02:16:50.150084+0000","last_unstale":"2024-09-21T02:16:52.731337+0000","last_undegraded":"2024-09-21T02:16:52.731337+0000","last_fullsized":"2024-09-21T02:16:52.731337+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:36.885015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161200+0000","last_change":"2024-09-21T02:16:51.204546+0000","last_active":"2024-09-21T02:16:52.161200+0000","last_peered":"2024-09-21T02:16:52.161200+0000","last_clean":"2024-09-21T02:16:52.161200+0000","last_became_active":"2024-09-21T02:16:51.204176+0000","last_became_peered":"2024-09-21T02:16:51.204176+0000","last_unstale":"2024-09-21T02:16:52.161200+0000","last_undegraded":"2024-09-21T02:16:52.161200+0000","last_fullsized":"2024-09-21T02:16:52.161200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:37.896294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158905+0000","last_change":"2024-09-21T02:16:50.175912+0000","last_active":"2024-09-21T02:16:52.158905+0000","last_peered":"2024-09-21T02:16:52.158905+0000","last_clean":"2024-09-21T02:16:52.158905+0000","last_became_active":"2024-09-21T02:16:50.175612+0000","last_became_peered":"2024-09-21T02:16:50.175612+0000","last_unstale":"2024-09-21T02:16:52.158905+0000","last_undegraded":"2024-09-21T02:16:52.158905+0000","last_fullsized":"2024-09-21T02:16:52.158905+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:19.224005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730697+0000","last_change":"2024-09-21T02:16:51.195629+0000","last_active":"2024-09-21T02:16:52.730697+0000","last_peered":"2024-09-21T02:16:52.730697+0000","last_clean":"2024-09-21T02:16:52.730697+0000","last_became_active":"2024-09-21T02:16:51.195047+0000","last_became_peered":"2024-09-21T02:16:51.195047+0000","last_unstale":"2024-09-21T02:16:52.730697+0000","last_undegraded":"2024-09-21T02:16:52.730697+0000","last_fullsized":"2024-09-21T02:16:52.730697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:03.957096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161943+0000","last_change":"2024-09-21T02:16:50.171947+0000","last_active":"2024-09-21T02:16:52.161943+0000","last_peered":"2024-09-21T02:16:52.161943+0000","last_clean":"2024-09-21T02:16:52.161943+0000","last_became_active":"2024-09-21T02:16:50.171098+0000","last_became_peered":"2024-09-21T02:16:50.171098+0000","last_unstale":"2024-09-21T02:16:52.161943+0000","last_undegraded":"2024-09-21T02:16:52.161943+0000","last_fullsized":"2024-09-21T02:16:52.161943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:31.894529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158803+0000","last_change":"2024-09-21T02:16:51.174713+0000","last_active":"2024-09-21T02:16:52.158803+0000","last_peered":"2024-09-21T02:16:52.158803+0000","last_clean":"2024-09-21T02:16:52.158803+0000","last_became_active":"2024-09-21T02:16:51.174379+0000","last_became_peered":"2024-09-21T02:16:51.174379+0000","last_unstale":"2024-09-21T02:16:52.158803+0000","last_undegraded":"2024-09-21T02:16:52.158803+0000","last_fullsized":"2024-09-21T02:16:52.158803+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:31.776543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673332+0000","last_change":"2024-09-21T02:16:50.132772+0000","last_active":"2024-09-21T02:16:52.673332+0000","last_peered":"2024-09-21T02:16:52.673332+0000","last_clean":"2024-09-21T02:16:52.673332+0000","last_became_active":"2024-09-21T02:16:50.132313+0000","last_became_peered":"2024-09-21T02:16:50.132313+0000","last_unstale":"2024-09-21T02:16:52.673332+0000","last_undegraded":"2024-09-21T02:16:52.673332+0000","last_fullsized":"2024-09-21T02:16:52.673332+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:39.207003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160678+0000","last_change":"2024-09-21T02:16:51.188083+0000","last_active":"2024-09-21T02:16:52.160678+0000","last_peered":"2024-09-21T02:16:52.160678+0000","last_clean":"2024-09-21T02:16:52.160678+0000","last_became_active":"2024-09-21T02:16:51.187786+0000","last_became_peered":"2024-09-21T02:16:51.187786+0000","last_unstale":"2024-09-21T02:16:52.160678+0000","last_undegraded":"2024-09-21T02:16:52.160678+0000","last_fullsized":"2024-09-21T02:16:52.160678+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:01.714526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.729907+0000","last_change":"2024-09-21T02:16:50.164996+0000","last_active":"2024-09-21T02:16:52.729907+0000","last_peered":"2024-09-21T02:16:52.729907+0000","last_clean":"2024-09-21T02:16:52.729907+0000","last_became_active":"2024-09-21T02:16:50.164668+0000","last_became_peered":"2024-09-21T02:16:50.164668+0000","last_unstale":"2024-09-21T02:16:52.729907+0000","last_undegraded":"2024-09-21T02:16:52.729907+0000","last_fullsized":"2024-09-21T02:16:52.729907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:19.224005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732257+0000","last_change":"2024-09-21T02:16:51.199388+0000","last_active":"2024-09-21T02:16:52.732257+0000","last_peered":"2024-09-21T02:16:52.732257+0000","last_clean":"2024-09-21T02:16:52.732257+0000","last_became_active":"2024-09-21T02:16:51.198999+0000","last_became_peered":"2024-09-21T02:16:51.198999+0000","last_unstale":"2024-09-21T02:16:52.732257+0000","last_undegraded":"2024-09-21T02:16:52.732257+0000","last_fullsized":"2024-09-21T02:16:52.732257+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:20.111417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673791+0000","last_change":"2024-09-21T02:16:50.156976+0000","last_active":"2024-09-21T02:16:52.673791+0000","last_peered":"2024-09-21T02:16:52.673791+0000","last_clean":"2024-09-21T02:16:52.673791+0000","last_became_active":"2024-09-21T02:16:50.156519+0000","last_became_peered":"2024-09-21T02:16:50.156519+0000","last_unstale":"2024-09-21T02:16:52.673791+0000","last_undegraded":"2024-09-21T02:16:52.673791+0000","last_fullsized":"2024-09-21T02:16:52.673791+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:04.725484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159753+0000","last_change":"2024-09-21T02:16:51.172272+0000","last_active":"2024-09-21T02:16:52.159753+0000","last_peered":"2024-09-21T02:16:52.159753+0000","last_clean":"2024-09-21T02:16:52.159753+0000","last_became_active":"2024-09-21T02:16:51.167774+0000","last_became_peered":"2024-09-21T02:16:51.167774+0000","last_unstale":"2024-09-21T02:16:52.159753+0000","last_undegraded":"2024-09-21T02:16:52.159753+0000","last_fullsized":"2024-09-21T02:16:52.159753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:11.406004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160342+0000","last_change":"2024-09-21T02:16:50.165192+0000","last_active":"2024-09-21T02:16:52.160342+0000","last_peered":"2024-09-21T02:16:52.160342+0000","last_clean":"2024-09-21T02:16:52.160342+0000","last_became_active":"2024-09-21T02:16:50.164365+0000","last_became_peered":"2024-09-21T02:16:50.164365+0000","last_unstale":"2024-09-21T02:16:52.160342+0000","last_undegraded":"2024-09-21T02:16:52.160342+0000","last_fullsized":"2024-09-21T02:16:52.160342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:30.765265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162024+0000","last_change":"2024-09-21T02:16:51.204939+0000","last_active":"2024-09-21T02:16:52.162024+0000","last_peered":"2024-09-21T02:16:52.162024+0000","last_clean":"2024-09-21T02:16:52.162024+0000","last_became_active":"2024-09-21T02:16:51.204436+0000","last_became_peered":"2024-09-21T02:16:51.204436+0000","last_unstale":"2024-09-21T02:16:52.162024+0000","last_undegraded":"2024-09-21T02:16:52.162024+0000","last_fullsized":"2024-09-21T02:16:52.162024+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:58.152781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732500+0000","last_change":"2024-09-21T02:16:50.153393+0000","last_active":"2024-09-21T02:16:52.732500+0000","last_peered":"2024-09-21T02:16:52.732500+0000","last_clean":"2024-09-21T02:16:52.732500+0000","last_became_active":"2024-09-21T02:16:50.153055+0000","last_became_peered":"2024-09-21T02:16:50.153055+0000","last_unstale":"2024-09-21T02:16:52.732500+0000","last_undegraded":"2024-09-21T02:16:52.732500+0000","last_fullsized":"2024-09-21T02:16:52.732500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:04.818765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732312+0000","last_change":"2024-09-21T02:16:51.178490+0000","last_active":"2024-09-21T02:16:52.732312+0000","last_peered":"2024-09-21T02:16:52.732312+0000","last_clean":"2024-09-21T02:16:52.732312+0000","last_became_active":"2024-09-21T02:16:51.178036+0000","last_became_peered":"2024-09-21T02:16:51.178036+0000","last_unstale":"2024-09-21T02:16:52.732312+0000","last_undegraded":"2024-09-21T02:16:52.732312+0000","last_fullsized":"2024-09-21T02:16:52.732312+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:29.492707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":21,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731371+0000","last_change":"2024-09-21T02:16:50.143980+0000","last_active":"2024-09-21T02:16:52.731371+0000","last_peered":"2024-09-21T02:16:52.731371+0000","last_clean":"2024-09-21T02:16:52.731371+0000","last_became_active":"2024-09-21T02:16:50.143616+0000","last_became_peered":"2024-09-21T02:16:50.143616+0000","last_unstale":"2024-09-21T02:16:52.731371+0000","last_undegraded":"2024-09-21T02:16:52.731371+0000","last_fullsized":"2024-09-21T02:16:52.731371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:41.273913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673557+0000","last_change":"2024-09-21T02:16:51.178265+0000","last_active":"2024-09-21T02:16:52.673557+0000","last_peered":"2024-09-21T02:16:52.673557+0000","last_clean":"2024-09-21T02:16:52.673557+0000","last_became_active":"2024-09-21T02:16:51.177817+0000","last_became_peered":"2024-09-21T02:16:51.177817+0000","last_unstale":"2024-09-21T02:16:52.673557+0000","last_undegraded":"2024-09-21T02:16:52.673557+0000","last_fullsized":"2024-09-21T02:16:52.673557+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:29.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161632+0000","last_change":"2024-09-21T02:16:50.176339+0000","last_active":"2024-09-21T02:16:52.161632+0000","last_peered":"2024-09-21T02:16:52.161632+0000","last_clean":"2024-09-21T02:16:52.161632+0000","last_became_active":"2024-09-21T02:16:50.175487+0000","last_became_peered":"2024-09-21T02:16:50.175487+0000","last_unstale":"2024-09-21T02:16:52.161632+0000","last_undegraded":"2024-09-21T02:16:52.161632+0000","last_fullsized":"2024-09-21T02:16:52.161632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:28.997124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.158850+0000","last_change":"2024-09-21T02:16:51.164191+0000","last_active":"2024-09-21T02:16:52.158850+0000","last_peered":"2024-09-21T02:16:52.158850+0000","last_clean":"2024-09-21T02:16:52.158850+0000","last_became_active":"2024-09-21T02:16:51.163850+0000","last_became_peered":"2024-09-21T02:16:51.163850+0000","last_unstale":"2024-09-21T02:16:52.158850+0000","last_undegraded":"2024-09-21T02:16:52.158850+0000","last_fullsized":"2024-09-21T02:16:52.158850+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:35.060289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731039+0000","last_change":"2024-09-21T02:16:50.145198+0000","last_active":"2024-09-21T02:16:52.731039+0000","last_peered":"2024-09-21T02:16:52.731039+0000","last_clean":"2024-09-21T02:16:52.731039+0000","last_became_active":"2024-09-21T02:16:50.144362+0000","last_became_peered":"2024-09-21T02:16:50.144362+0000","last_unstale":"2024-09-21T02:16:52.731039+0000","last_undegraded":"2024-09-21T02:16:52.731039+0000","last_fullsized":"2024-09-21T02:16:52.731039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:22.186394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673772+0000","last_change":"2024-09-21T02:16:51.174767+0000","last_active":"2024-09-21T02:16:52.673772+0000","last_peered":"2024-09-21T02:16:52.673772+0000","last_clean":"2024-09-21T02:16:52.673772+0000","last_became_active":"2024-09-21T02:16:51.174472+0000","last_became_peered":"2024-09-21T02:16:51.174472+0000","last_unstale":"2024-09-21T02:16:52.673772+0000","last_undegraded":"2024-09-21T02:16:52.673772+0000","last_fullsized":"2024-09-21T02:16:52.673772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:39.378405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732367+0000","last_change":"2024-09-21T02:16:50.148152+0000","last_active":"2024-09-21T02:16:52.732367+0000","last_peered":"2024-09-21T02:16:52.732367+0000","last_clean":"2024-09-21T02:16:52.732367+0000","last_became_active":"2024-09-21T02:16:50.147793+0000","last_became_peered":"2024-09-21T02:16:50.147793+0000","last_unstale":"2024-09-21T02:16:52.732367+0000","last_undegraded":"2024-09-21T02:16:52.732367+0000","last_fullsized":"2024-09-21T02:16:52.732367+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:15.613216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732804+0000","last_change":"2024-09-21T02:16:51.184539+0000","last_active":"2024-09-21T02:16:52.732804+0000","last_peered":"2024-09-21T02:16:52.732804+0000","last_clean":"2024-09-21T02:16:52.732804+0000","last_became_active":"2024-09-21T02:16:51.181220+0000","last_became_peered":"2024-09-21T02:16:51.181220+0000","last_unstale":"2024-09-21T02:16:52.732804+0000","last_undegraded":"2024-09-21T02:16:52.732804+0000","last_fullsized":"2024-09-21T02:16:52.732804+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:52.839564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730252+0000","last_change":"2024-09-21T02:16:50.144290+0000","last_active":"2024-09-21T02:16:52.730252+0000","last_peered":"2024-09-21T02:16:52.730252+0000","last_clean":"2024-09-21T02:16:52.730252+0000","last_became_active":"2024-09-21T02:16:50.143920+0000","last_became_peered":"2024-09-21T02:16:50.143920+0000","last_unstale":"2024-09-21T02:16:52.730252+0000","last_undegraded":"2024-09-21T02:16:52.730252+0000","last_fullsized":"2024-09-21T02:16:52.730252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:54.939937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159885+0000","last_change":"2024-09-21T02:16:51.199498+0000","last_active":"2024-09-21T02:16:52.159885+0000","last_peered":"2024-09-21T02:16:52.159885+0000","last_clean":"2024-09-21T02:16:52.159885+0000","last_became_active":"2024-09-21T02:16:51.199130+0000","last_became_peered":"2024-09-21T02:16:51.199130+0000","last_unstale":"2024-09-21T02:16:52.159885+0000","last_undegraded":"2024-09-21T02:16:52.159885+0000","last_fullsized":"2024-09-21T02:16:52.159885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:54.877624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732422+0000","last_change":"2024-09-21T02:16:50.180686+0000","last_active":"2024-09-21T02:16:52.732422+0000","last_peered":"2024-09-21T02:16:52.732422+0000","last_clean":"2024-09-21T02:16:52.732422+0000","last_became_active":"2024-09-21T02:16:50.179399+0000","last_became_peered":"2024-09-21T02:16:50.179399+0000","last_unstale":"2024-09-21T02:16:52.732422+0000","last_undegraded":"2024-09-21T02:16:52.732422+0000","last_fullsized":"2024-09-21T02:16:52.732422+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:59.798714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160119+0000","last_change":"2024-09-21T02:16:51.176600+0000","last_active":"2024-09-21T02:16:52.160119+0000","last_peered":"2024-09-21T02:16:52.160119+0000","last_clean":"2024-09-21T02:16:52.160119+0000","last_became_active":"2024-09-21T02:16:51.176223+0000","last_became_peered":"2024-09-21T02:16:51.176223+0000","last_unstale":"2024-09-21T02:16:52.160119+0000","last_undegraded":"2024-09-21T02:16:52.160119+0000","last_fullsized":"2024-09-21T02:16:52.160119+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:32.399050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160541+0000","last_change":"2024-09-21T02:16:50.165003+0000","last_active":"2024-09-21T02:16:52.160541+0000","last_peered":"2024-09-21T02:16:52.160541+0000","last_clean":"2024-09-21T02:16:52.160541+0000","last_became_active":"2024-09-21T02:16:50.164130+0000","last_became_peered":"2024-09-21T02:16:50.164130+0000","last_unstale":"2024-09-21T02:16:52.160541+0000","last_undegraded":"2024-09-21T02:16:52.160541+0000","last_fullsized":"2024-09-21T02:16:52.160541+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:59.080213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732100+0000","last_change":"2024-09-21T02:16:51.195247+0000","last_active":"2024-09-21T02:16:52.732100+0000","last_peered":"2024-09-21T02:16:52.732100+0000","last_clean":"2024-09-21T02:16:52.732100+0000","last_became_active":"2024-09-21T02:16:51.191137+0000","last_became_peered":"2024-09-21T02:16:51.191137+0000","last_unstale":"2024-09-21T02:16:52.732100+0000","last_undegraded":"2024-09-21T02:16:52.732100+0000","last_fullsized":"2024-09-21T02:16:52.732100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:54.611197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159628+0000","last_change":"2024-09-21T02:16:50.166657+0000","last_active":"2024-09-21T02:16:52.159628+0000","last_peered":"2024-09-21T02:16:52.159628+0000","last_clean":"2024-09-21T02:16:52.159628+0000","last_became_active":"2024-09-21T02:16:50.166335+0000","last_became_peered":"2024-09-21T02:16:50.166335+0000","last_unstale":"2024-09-21T02:16:52.159628+0000","last_undegraded":"2024-09-21T02:16:52.159628+0000","last_fullsized":"2024-09-21T02:16:52.159628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:41.273913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732888+0000","last_change":"2024-09-21T02:16:51.189237+0000","last_active":"2024-09-21T02:16:52.732888+0000","last_peered":"2024-09-21T02:16:52.732888+0000","last_clean":"2024-09-21T02:16:52.732888+0000","last_became_active":"2024-09-21T02:16:51.188831+0000","last_became_peered":"2024-09-21T02:16:51.188831+0000","last_unstale":"2024-09-21T02:16:52.732888+0000","last_undegraded":"2024-09-21T02:16:52.732888+0000","last_fullsized":"2024-09-21T02:16:52.732888+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:12.834157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162121+0000","last_change":"2024-09-21T02:16:50.174041+0000","last_active":"2024-09-21T02:16:52.162121+0000","last_peered":"2024-09-21T02:16:52.162121+0000","last_clean":"2024-09-21T02:16:52.162121+0000","last_became_active":"2024-09-21T02:16:50.173299+0000","last_became_peered":"2024-09-21T02:16:50.173299+0000","last_unstale":"2024-09-21T02:16:52.162121+0000","last_undegraded":"2024-09-21T02:16:52.162121+0000","last_fullsized":"2024-09-21T02:16:52.162121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:34.049010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.673470+0000","last_change":"2024-09-21T02:16:51.188933+0000","last_active":"2024-09-21T02:16:52.673470+0000","last_peered":"2024-09-21T02:16:52.673470+0000","last_clean":"2024-09-21T02:16:52.673470+0000","last_became_active":"2024-09-21T02:16:51.188270+0000","last_became_peered":"2024-09-21T02:16:51.188270+0000","last_unstale":"2024-09-21T02:16:52.673470+0000","last_undegraded":"2024-09-21T02:16:52.673470+0000","last_fullsized":"2024-09-21T02:16:52.673470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:19.891292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731290+0000","last_change":"2024-09-21T02:16:50.166462+0000","last_active":"2024-09-21T02:16:52.731290+0000","last_peered":"2024-09-21T02:16:52.731290+0000","last_clean":"2024-09-21T02:16:52.731290+0000","last_became_active":"2024-09-21T02:16:50.162999+0000","last_became_peered":"2024-09-21T02:16:50.162999+0000","last_unstale":"2024-09-21T02:16:52.731290+0000","last_undegraded":"2024-09-21T02:16:52.731290+0000","last_fullsized":"2024-09-21T02:16:52.731290+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:07.447223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161423+0000","last_change":"2024-09-21T02:16:51.184256+0000","last_active":"2024-09-21T02:16:52.161423+0000","last_peered":"2024-09-21T02:16:52.161423+0000","last_clean":"2024-09-21T02:16:52.161423+0000","last_became_active":"2024-09-21T02:16:51.175450+0000","last_became_peered":"2024-09-21T02:16:51.175450+0000","last_unstale":"2024-09-21T02:16:52.161423+0000","last_undegraded":"2024-09-21T02:16:52.161423+0000","last_fullsized":"2024-09-21T02:16:52.161423+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:16.624495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161300+0000","last_change":"2024-09-21T02:16:50.167281+0000","last_active":"2024-09-21T02:16:52.161300+0000","last_peered":"2024-09-21T02:16:52.161300+0000","last_clean":"2024-09-21T02:16:52.161300+0000","last_became_active":"2024-09-21T02:16:50.166028+0000","last_became_peered":"2024-09-21T02:16:50.166028+0000","last_unstale":"2024-09-21T02:16:52.161300+0000","last_undegraded":"2024-09-21T02:16:52.161300+0000","last_fullsized":"2024-09-21T02:16:52.161300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:27:38.549187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161538+0000","last_change":"2024-09-21T02:16:51.173435+0000","last_active":"2024-09-21T02:16:52.161538+0000","last_peered":"2024-09-21T02:16:52.161538+0000","last_clean":"2024-09-21T02:16:52.161538+0000","last_became_active":"2024-09-21T02:16:51.172945+0000","last_became_peered":"2024-09-21T02:16:51.172945+0000","last_unstale":"2024-09-21T02:16:52.161538+0000","last_undegraded":"2024-09-21T02:16:52.161538+0000","last_fullsized":"2024-09-21T02:16:52.161538+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:26.595491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674089+0000","last_change":"2024-09-21T02:16:50.173233+0000","last_active":"2024-09-21T02:16:52.674089+0000","last_peered":"2024-09-21T02:16:52.674089+0000","last_clean":"2024-09-21T02:16:52.674089+0000","last_became_active":"2024-09-21T02:16:50.172749+0000","last_became_peered":"2024-09-21T02:16:50.172749+0000","last_unstale":"2024-09-21T02:16:52.674089+0000","last_undegraded":"2024-09-21T02:16:52.674089+0000","last_fullsized":"2024-09-21T02:16:52.674089+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:55.679512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159747+0000","last_change":"2024-09-21T02:16:51.188449+0000","last_active":"2024-09-21T02:16:52.159747+0000","last_peered":"2024-09-21T02:16:52.159747+0000","last_clean":"2024-09-21T02:16:52.159747+0000","last_became_active":"2024-09-21T02:16:51.188135+0000","last_became_peered":"2024-09-21T02:16:51.188135+0000","last_unstale":"2024-09-21T02:16:52.159747+0000","last_undegraded":"2024-09-21T02:16:52.159747+0000","last_fullsized":"2024-09-21T02:16:52.159747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:26.578649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732681+0000","last_change":"2024-09-21T02:16:50.171404+0000","last_active":"2024-09-21T02:16:52.732681+0000","last_peered":"2024-09-21T02:16:52.732681+0000","last_clean":"2024-09-21T02:16:52.732681+0000","last_became_active":"2024-09-21T02:16:50.170131+0000","last_became_peered":"2024-09-21T02:16:50.170131+0000","last_unstale":"2024-09-21T02:16:52.732681+0000","last_undegraded":"2024-09-21T02:16:52.732681+0000","last_fullsized":"2024-09-21T02:16:52.732681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:22.389587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160386+0000","last_change":"2024-09-21T02:16:51.175941+0000","last_active":"2024-09-21T02:16:52.160386+0000","last_peered":"2024-09-21T02:16:52.160386+0000","last_clean":"2024-09-21T02:16:52.160386+0000","last_became_active":"2024-09-21T02:16:51.174587+0000","last_became_peered":"2024-09-21T02:16:51.174587+0000","last_unstale":"2024-09-21T02:16:52.160386+0000","last_undegraded":"2024-09-21T02:16:52.160386+0000","last_fullsized":"2024-09-21T02:16:52.160386+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:52.678040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.163821+0000","last_change":"2024-09-21T02:16:50.174347+0000","last_active":"2024-09-21T02:16:52.163821+0000","last_peered":"2024-09-21T02:16:52.163821+0000","last_clean":"2024-09-21T02:16:52.163821+0000","last_became_active":"2024-09-21T02:16:50.173542+0000","last_became_peered":"2024-09-21T02:16:50.173542+0000","last_unstale":"2024-09-21T02:16:52.163821+0000","last_undegraded":"2024-09-21T02:16:52.163821+0000","last_fullsized":"2024-09-21T02:16:52.163821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:39.790101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.732549+0000","last_change":"2024-09-21T02:16:51.184576+0000","last_active":"2024-09-21T02:16:52.732549+0000","last_peered":"2024-09-21T02:16:52.732549+0000","last_clean":"2024-09-21T02:16:52.732549+0000","last_became_active":"2024-09-21T02:16:51.184223+0000","last_became_peered":"2024-09-21T02:16:51.184223+0000","last_unstale":"2024-09-21T02:16:52.732549+0000","last_undegraded":"2024-09-21T02:16:52.732549+0000","last_fullsized":"2024-09-21T02:16:52.732549+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:53.992884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.674113+0000","last_change":"2024-09-21T02:16:50.152016+0000","last_active":"2024-09-21T02:16:52.674113+0000","last_peered":"2024-09-21T02:16:52.674113+0000","last_clean":"2024-09-21T02:16:52.674113+0000","last_became_active":"2024-09-21T02:16:50.151639+0000","last_became_peered":"2024-09-21T02:16:50.151639+0000","last_unstale":"2024-09-21T02:16:52.674113+0000","last_undegraded":"2024-09-21T02:16:52.674113+0000","last_fullsized":"2024-09-21T02:16:52.674113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:32.534455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162137+0000","last_change":"2024-09-21T02:16:51.173919+0000","last_active":"2024-09-21T02:16:52.162137+0000","last_peered":"2024-09-21T02:16:52.162137+0000","last_clean":"2024-09-21T02:16:52.162137+0000","last_became_active":"2024-09-21T02:16:51.173253+0000","last_became_peered":"2024-09-21T02:16:51.173253+0000","last_unstale":"2024-09-21T02:16:52.162137+0000","last_undegraded":"2024-09-21T02:16:52.162137+0000","last_fullsized":"2024-09-21T02:16:52.162137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:10.276293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159201+0000","last_change":"2024-09-21T02:16:50.149394+0000","last_active":"2024-09-21T02:16:52.159201+0000","last_peered":"2024-09-21T02:16:52.159201+0000","last_clean":"2024-09-21T02:16:52.159201+0000","last_became_active":"2024-09-21T02:16:50.148962+0000","last_became_peered":"2024-09-21T02:16:50.148962+0000","last_unstale":"2024-09-21T02:16:52.159201+0000","last_undegraded":"2024-09-21T02:16:52.159201+0000","last_fullsized":"2024-09-21T02:16:52.159201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:02.873090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161880+0000","last_change":"2024-09-21T02:16:51.200663+0000","last_active":"2024-09-21T02:16:52.161880+0000","last_peered":"2024-09-21T02:16:52.161880+0000","last_clean":"2024-09-21T02:16:52.161880+0000","last_became_active":"2024-09-21T02:16:51.198965+0000","last_became_peered":"2024-09-21T02:16:51.198965+0000","last_unstale":"2024-09-21T02:16:52.161880+0000","last_undegraded":"2024-09-21T02:16:52.161880+0000","last_fullsized":"2024-09-21T02:16:52.161880+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:34.824750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731550+0000","last_change":"2024-09-21T02:16:50.175039+0000","last_active":"2024-09-21T02:16:52.731550+0000","last_peered":"2024-09-21T02:16:52.731550+0000","last_clean":"2024-09-21T02:16:52.731550+0000","last_became_active":"2024-09-21T02:16:50.174228+0000","last_became_peered":"2024-09-21T02:16:50.174228+0000","last_unstale":"2024-09-21T02:16:52.731550+0000","last_undegraded":"2024-09-21T02:16:52.731550+0000","last_fullsized":"2024-09-21T02:16:52.731550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:59.688724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731551+0000","last_change":"2024-09-21T02:16:51.208663+0000","last_active":"2024-09-21T02:16:52.731551+0000","last_peered":"2024-09-21T02:16:52.731551+0000","last_clean":"2024-09-21T02:16:52.731551+0000","last_became_active":"2024-09-21T02:16:51.208258+0000","last_became_peered":"2024-09-21T02:16:51.208258+0000","last_unstale":"2024-09-21T02:16:52.731551+0000","last_undegraded":"2024-09-21T02:16:52.731551+0000","last_fullsized":"2024-09-21T02:16:52.731551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:23.348286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.159016+0000","last_change":"2024-09-21T02:16:50.155612+0000","last_active":"2024-09-21T02:16:52.159016+0000","last_peered":"2024-09-21T02:16:52.159016+0000","last_clean":"2024-09-21T02:16:52.159016+0000","last_became_active":"2024-09-21T02:16:50.155095+0000","last_became_peered":"2024-09-21T02:16:50.155095+0000","last_unstale":"2024-09-21T02:16:52.159016+0000","last_undegraded":"2024-09-21T02:16:52.159016+0000","last_fullsized":"2024-09-21T02:16:52.159016+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:03.680139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161577+0000","last_change":"2024-09-21T02:16:51.167090+0000","last_active":"2024-09-21T02:16:52.161577+0000","last_peered":"2024-09-21T02:16:52.161577+0000","last_clean":"2024-09-21T02:16:52.161577+0000","last_became_active":"2024-09-21T02:16:51.166677+0000","last_became_peered":"2024-09-21T02:16:51.166677+0000","last_unstale":"2024-09-21T02:16:52.161577+0000","last_undegraded":"2024-09-21T02:16:52.161577+0000","last_fullsized":"2024-09-21T02:16:52.161577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.940950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.160858+0000","last_change":"2024-09-21T02:16:50.179411+0000","last_active":"2024-09-21T02:16:52.160858+0000","last_peered":"2024-09-21T02:16:52.160858+0000","last_clean":"2024-09-21T02:16:52.160858+0000","last_became_active":"2024-09-21T02:16:50.178589+0000","last_became_peered":"2024-09-21T02:16:50.178589+0000","last_unstale":"2024-09-21T02:16:52.160858+0000","last_undegraded":"2024-09-21T02:16:52.160858+0000","last_fullsized":"2024-09-21T02:16:52.160858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:15.960076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.731407+0000","last_change":"2024-09-21T02:16:51.195546+0000","last_active":"2024-09-21T02:16:52.731407+0000","last_peered":"2024-09-21T02:16:52.731407+0000","last_clean":"2024-09-21T02:16:52.731407+0000","last_became_active":"2024-09-21T02:16:51.195082+0000","last_became_peered":"2024-09-21T02:16:51.195082+0000","last_unstale":"2024-09-21T02:16:52.731407+0000","last_undegraded":"2024-09-21T02:16:52.731407+0000","last_fullsized":"2024-09-21T02:16:52.731407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:03.626391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.161522+0000","last_change":"2024-09-21T02:16:50.155940+0000","last_active":"2024-09-21T02:16:52.161522+0000","last_peered":"2024-09-21T02:16:52.161522+0000","last_clean":"2024-09-21T02:16:52.161522+0000","last_became_active":"2024-09-21T02:16:50.155127+0000","last_became_peered":"2024-09-21T02:16:50.155127+0000","last_unstale":"2024-09-21T02:16:52.161522+0000","last_undegraded":"2024-09-21T02:16:52.161522+0000","last_fullsized":"2024-09-21T02:16:52.161522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:36.216279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.162584+0000","last_change":"2024-09-21T02:16:51.181492+0000","last_active":"2024-09-21T02:16:52.162584+0000","last_peered":"2024-09-21T02:16:52.162584+0000","last_clean":"2024-09-21T02:16:52.162584+0000","last_became_active":"2024-09-21T02:16:51.181154+0000","last_became_peered":"2024-09-21T02:16:51.181154+0000","last_unstale":"2024-09-21T02:16:52.162584+0000","last_undegraded":"2024-09-21T02:16:52.162584+0000","last_fullsized":"2024-09-21T02:16:52.162584+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:50.120496+0000","last_clean_scrub_stamp":"2024-09-21T02:16:50.120496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:27.432178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.730884+0000","last_change":"2024-09-21T02:16:50.150588+0000","last_active":"2024-09-21T02:16:52.730884+0000","last_peered":"2024-09-21T02:16:52.730884+0000","last_clean":"2024-09-21T02:16:52.730884+0000","last_became_active":"2024-09-21T02:16:50.150147+0000","last_became_peered":"2024-09-21T02:16:50.150147+0000","last_unstale":"2024-09-21T02:16:52.730884+0000","last_undegraded":"2024-09-21T02:16:52.730884+0000","last_fullsized":"2024-09-21T02:16:52.730884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:16:49.109217+0000","last_clean_scrub_stamp":"2024-09-21T02:16:49.109217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:02.873090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":176,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-21T02:16:52.672634+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:16:52.672634+0000","last_peered":"2024-09-21T02:16:52.672634+0000","last_clean":"2024-09-21T02:16:52.672634+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:16:52.672634+0000","last_undegraded":"2024-09-21T02:16:52.672634+0000","last_fullsized":"2024-09-21T02:16:52.672634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":290147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":704512,"data_stored":290077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1298432,"data_stored":880457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36036,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335804,"statfs":{"total":96636764160,"available":96599863296,"internally_reserved":0,"allocated":704512,"data_stored":290145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":720896,"data_stored":291827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1298432,"data_stored":881005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":712704,"data_stored":291667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":696320,"data_stored":284371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:16:55.762 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:16:55.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 -- 172.21.15.170:0/3373234342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0700b0 msgr2=0x7fb8ec0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/3373234342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0700b0 0x7fb8ec0ce5b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009c80 tx=0x7fb8e401cd10 comp rx=0 tx=0).stop 2024-09-21T02:16:55.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 -- 172.21.15.170:0/3373234342 shutdown_connections 2024-09-21T02:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/3373234342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0700b0 0x7fb8ec0ce5b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/3373234342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ec0cb380 0x7fb8ec0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 -- 172.21.15.170:0/3373234342 >> 172.21.15.170:0/3373234342 conn(0x7fb8ec0d0170 msgr2=0x7fb8ec0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 -- 172.21.15.170:0/3373234342 shutdown_connections 2024-09-21T02:16:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 -- 172.21.15.170:0/3373234342 wait complete. 2024-09-21T02:16:55.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 Processor -- start 2024-09-21T02:16:55.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.965+0000 7fb8f1e03640 1 -- start start 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ec0700b0 0x7fb8ec15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0cb380 0x7fb8ec14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 0x7fb8ec161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec168f10 con 0x7fb8ec0cb790 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec168d90 con 0x7fb8ec0cb380 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ec169090 con 0x7fb8ec0700b0 2024-09-21T02:16:55.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 0x7fb8ec161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 0x7fb8ec161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47036/0 (socket says 172.21.15.170:47036) 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 -- 172.21.15.170:0/1105100158 learned_addr learned my addr 172.21.15.170:0/1105100158 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8eaffd640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0cb380 0x7fb8ec14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8eb7fe640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ec0700b0 0x7fb8ec15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 -- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ec0700b0 msgr2=0x7fb8ec15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ec0700b0 0x7fb8ec15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 -- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0cb380 msgr2=0x7fb8ec14a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0cb380 0x7fb8ec14a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 -- 172.21.15.170:0/1105100158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec15c620 con 0x7fb8ec0cb790 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8ebfff640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 0x7fb8ec161020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009bd0 tx=0x7fb8e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40bb070 con 0x7fb8ec0cb790 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8e40a5960 con 0x7fb8ec0cb790 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec1593a0 con 0x7fb8ec0cb790 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e40b2040 con 0x7fb8ec0cb790 2024-09-21T02:16:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec14aaf0 con 0x7fb8ec0cb790 2024-09-21T02:16:55.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ec06f9a0 con 0x7fb8ec0cb790 2024-09-21T02:16:55.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.969+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fb8e40be070 con 0x7fb8ec0cb790 2024-09-21T02:16:55.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.973+0000 7fb8d3fff640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb8d80733e0 0x7fb8d80758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:55.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.973+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fb8e412e0b0 con 0x7fb8ec0cb790 2024-09-21T02:16:55.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.973+0000 7fb8eb7fe640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb8d80733e0 0x7fb8d80758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:55.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.973+0000 7fb8eb7fe640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb8d80733e0 0x7fb8d80758f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009ad0 tx=0x7fb8dc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:55.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:55.977+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb8e4016990 con 0x7fb8ec0cb790 2024-09-21T02:16:56.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.193+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb8ec0ca100 con 0x7fb8ec0cb790 2024-09-21T02:16:56.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8d3fff640 1 -- 172.21.15.170:0/1105100158 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fb8e40f8d40 con 0x7fb8ec0cb790 2024-09-21T02:16:56.202 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:16:56.202 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:16:56.202 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:16:56.202 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:16:56.203 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:16:56.203 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb8d80733e0 msgr2=0x7fb8d80758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb8d80733e0 0x7fb8d80758f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009ad0 tx=0x7fb8dc0091e0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 msgr2=0x7fb8ec161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 0x7fb8ec161020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009bd0 tx=0x7fb8e4004270 comp rx=0 tx=0).stop 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 shutdown_connections 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb8d80733e0 0x7fb8d80758f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8ec0cb790 0x7fb8ec161020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ec0cb380 0x7fb8ec14a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 --2- 172.21.15.170:0/1105100158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8ec0700b0 0x7fb8ec15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 >> 172.21.15.170:0/1105100158 conn(0x7fb8ec0d0170 msgr2=0x7fb8ec165b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:56.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 shutdown_connections 2024-09-21T02:16:56.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.201+0000 7fb8f1e03640 1 -- 172.21.15.170:0/1105100158 wait complete. 2024-09-21T02:16:56.223 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:16:56.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 -- 172.21.15.170:0/3440675106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf40b3e30 msgr2=0x7fabf40b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 --2- 172.21.15.170:0/3440675106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf40b3e30 0x7fabf40b4240 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fabf4160270 tx=0x7fabdc01c690 comp rx=0 tx=0).stop 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 -- 172.21.15.170:0/3440675106 shutdown_connections 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 --2- 172.21.15.170:0/3440675106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf40b3e30 0x7fabf40b4240 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 --2- 172.21.15.170:0/3440675106 >> v2:172.21.15.191:3300/0 conn(0x7fabf40b7ab0 0x7fabf415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 -- 172.21.15.170:0/3440675106 >> 172.21.15.170:0/3440675106 conn(0x7fabf40cee30 msgr2=0x7fabf40cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 -- 172.21.15.170:0/3440675106 shutdown_connections 2024-09-21T02:16:56.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.373+0000 7fabfa59c640 1 -- 172.21.15.170:0/3440675106 wait complete. 2024-09-21T02:16:56.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 Processor -- start 2024-09-21T02:16:56.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 -- start start 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 0x7fabf4158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabf40b3e30 0x7fabf4141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf40b7ab0 0x7fabf41fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fabf416a370 con 0x7fabf4072900 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fabf416a1f0 con 0x7fabf40b3e30 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fabf416a4f0 con 0x7fabf40b7ab0 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 0x7fabf4158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 0x7fabf4158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47050/0 (socket says 172.21.15.170:47050) 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 -- 172.21.15.170:0/139444574 learned_addr learned my addr 172.21.15.170:0/139444574 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:56.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf37fe640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabf40b3e30 0x7fabf4141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf8b39640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf40b7ab0 0x7fabf41fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 -- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf40b7ab0 msgr2=0x7fabf41fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf40b7ab0 0x7fabf41fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 -- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabf40b3e30 msgr2=0x7fabf4141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabf40b3e30 0x7fabf4141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 -- 172.21.15.170:0/139444574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41f9c80 con 0x7fabf4072900 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf8b39640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf40b7ab0 0x7fabf41fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf3fff640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 0x7fabf4158c00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fabe400ea20 tx=0x7fabe400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe400cde0 con 0x7fabf4072900 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabe4004c20 con 0x7fabf4072900 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4005020 con 0x7fabf4072900 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41f9f60 con 0x7fabf4072900 2024-09-21T02:16:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.377+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41fca00 con 0x7fabf4072900 2024-09-21T02:16:56.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.381+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf40b4240 con 0x7fabf4072900 2024-09-21T02:16:56.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.381+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fabe40051c0 con 0x7fabf4072900 2024-09-21T02:16:56.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.381+0000 7fabf0ff9640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd40733e0 0x7fabd40758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:56.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.381+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fabe40950a0 con 0x7fabf4072900 2024-09-21T02:16:56.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.381+0000 7fabf37fe640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd40733e0 0x7fabd40758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:56.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.381+0000 7fabf37fe640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd40733e0 0x7fabd40758f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0af000 tx=0x7fabdc01aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:56.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.385+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fabe4016030 con 0x7fabf4072900 2024-09-21T02:16:56.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.577+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabf4159a80 con 0x7fabf4072900 2024-09-21T02:16:56.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.577+0000 7fabf0ff9640 1 -- 172.21.15.170:0/139444574 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7fabe405fd30 con 0x7fabf4072900 2024-09-21T02:16:56.581 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:16:56.581 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":37,"btime":"2024-09-21T02:16:54:193427+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:16:50.547545+0000","modified":"2024-09-21T02:16:53.589716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/3667702062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3667702062},{"type":"v1","addr":"172.21.15.191:6837","nonce":3667702062}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5889,"qdb_cluster":[5889]},"id":4}]} 2024-09-21T02:16:56.581 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 37 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd40733e0 msgr2=0x7fabd40758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd40733e0 0x7fabd40758f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0af000 tx=0x7fabdc01aed0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 msgr2=0x7fabf4158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 0x7fabf4158c00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fabe400ea20 tx=0x7fabe400eee0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 shutdown_connections 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fabd40733e0 0x7fabd40758f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabf40b7ab0 0x7fabf41fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabf40b3e30 0x7fabf4141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 --2- 172.21.15.170:0/139444574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabf4072900 0x7fabf4158c00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 >> 172.21.15.170:0/139444574 conn(0x7fabf40cee30 msgr2=0x7fabf4076100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 shutdown_connections 2024-09-21T02:16:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:56.581+0000 7fabfa59c640 1 -- 172.21.15.170:0/139444574 wait complete. 2024-09-21T02:16:57.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:16:50.547545+0000', 'modified': '2024-09-21T02:16:53.589716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5889}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5889': {'gid': 5889, 'name': 'd', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6837/3667702062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 3667702062}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 3667702062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5889, 'qdb_cluster': [5889]} 2024-09-21T02:16:57.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:16:57.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- 172.21.15.170:0/2999819032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0cb500 msgr2=0x7fbf1c0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 --2- 172.21.15.170:0/2999819032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0cb500 0x7fbf1c0ce8d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009c80 tx=0x7fbf1401cc30 comp rx=0 tx=0).stop 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- 172.21.15.170:0/2999819032 shutdown_connections 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 --2- 172.21.15.170:0/2999819032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0cb500 0x7fbf1c0ce8d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- 172.21.15.170:0/2999819032 >> 172.21.15.170:0/2999819032 conn(0x7fbf1c0b1c40 msgr2=0x7fbf1c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- 172.21.15.170:0/2999819032 shutdown_connections 2024-09-21T02:16:57.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- 172.21.15.170:0/2999819032 wait complete. 2024-09-21T02:16:57.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 Processor -- start 2024-09-21T02:16:57.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- start start 2024-09-21T02:16:57.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 0x7fbf1c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c15f990 0x7fbf1c1f9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c1552d0 0x7fbf1c1f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c169ba0 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c169a20 con 0x7fbf1c15f990 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf22e30640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c169d20 con 0x7fbf1c1552d0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf20bcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 0x7fbf1c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf20bcc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 0x7fbf1c154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47054/0 (socket says 172.21.15.170:47054) 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.173+0000 7fbf20bcc640 1 -- 172.21.15.170:0/4057657789 learned_addr learned my addr 172.21.15.170:0/4057657789 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf20bcc640 1 -- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c1552d0 msgr2=0x7fbf1c1f9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf213cd640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c1552d0 0x7fbf1c1f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf13fff640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c15f990 0x7fbf1c1f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c1552d0 0x7fbf1c1f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf20bcc640 1 -- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c15f990 msgr2=0x7fbf1c1f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c15f990 0x7fbf1c1f9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf20bcc640 1 -- 172.21.15.170:0/4057657789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c09d260 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 0x7fbf1c154d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e940 tx=0x7fbf1800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf117fa640 1 -- 172.21.15.170:0/4057657789 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180052d0 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf117fa640 1 -- 172.21.15.170:0/4057657789 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf1800ca60 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c09d540 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf117fa640 1 -- 172.21.15.170:0/4057657789 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18003620 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c0c99a0 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf117fa640 1 -- 172.21.15.170:0/4057657789 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fbf180037c0 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf117fa640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbf080733b0 0x7fbf080758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf117fa640 1 -- 172.21.15.170:0/4057657789 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fbf18096c00 con 0x7fbf1c0ce4c0 2024-09-21T02:16:57.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf13fff640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbf080733b0 0x7fbf080758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf2a0fb000 1 Processor -- start 2024-09-21T02:16:57.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.177+0000 7fbf2a0fb000 1 -- start start 2024-09-21T02:16:57.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf13fff640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbf080733b0 0x7fbf080758c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c15f440 tx=0x7fbf0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557582185410 con 0x557582148ca0 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5575820c2b40 con 0x55758215f950 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5575820b1f70 con 0x557581d6f460 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47056/0 (socket says 172.21.15.170:47056) 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 -- 172.21.15.170:0/3763723432 learned_addr learned my addr 172.21.15.170:0/3763723432 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf13fff640 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 -- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 msgr2=0x5575822b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 -- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 msgr2=0x5575822b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 -- 172.21.15.170:0/3763723432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575820795b0 con 0x557582148ca0 2024-09-21T02:16:57.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:57.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf213cd640 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b1120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009bd0 tx=0x7fbf140199d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928b70e1b73be57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf027fc640 1 -- 172.21.15.170:0/3763723432 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140b2040 con 0x557582148ca0 2024-09-21T02:16:57.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf027fc640 1 -- 172.21.15.170:0/3763723432 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf140a5d30 con 0x557582148ca0 2024-09-21T02:16:57.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf027fc640 1 -- 172.21.15.170:0/3763723432 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14004c50 con 0x557582148ca0 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/3763723432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 msgr2=0x5575822b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b1120 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009bd0 tx=0x7fbf140199d0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/3763723432 shutdown_connections 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b1120 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/3763723432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/3763723432 >> 172.21.15.170:0/3763723432 conn(0x557582235d90 msgr2=0x557582265690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/3763723432 shutdown_connections 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/3763723432 wait complete. 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.181+0000 7fbf2a0fb000 1 -- start start 2024-09-21T02:16:57.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 Processor -- start 2024-09-21T02:16:57.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557582148ca0 0x5575822bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 0x5575822bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55758204c5f0 con 0x55758215f950 2024-09-21T02:16:57.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5575820b5d80 con 0x557582148ca0 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x557582058520 con 0x557581d6f460 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 10 client.0 _ll_register_callbacks cb 0x55758205c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 0x5575822bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 0x5575822bc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47064/0 (socket says 172.21.15.170:47064) 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 -- 172.21.15.170:0/477901585 learned_addr learned my addr 172.21.15.170:0/477901585 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf13fff640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557582148ca0 0x5575822bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 -- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 msgr2=0x5575822bbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822bbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 -- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557582148ca0 msgr2=0x5575822bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557582148ca0 0x5575822bc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 -- 172.21.15.170:0/477901585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557582050190 con 0x55758215f950 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822bbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:16:57.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf213cd640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 0x5575822bc780 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbf1401ca60 tx=0x7fbf14005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a975cbce442d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 1 -- 172.21.15.170:0/477901585 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140b2070 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 1 -- 172.21.15.170:0/477901585 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fbf14003b80 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55758204a110 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 1 -- 172.21.15.170:0/477901585 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbf140c7050 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 10 client.6024 fetch_fsmap learned FSMap version 37 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x557582136f10 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 1 -- 172.21.15.170:0/477901585 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55758204a110 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 1 -- 172.21.15.170:0/477901585 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fbf140ca020 con 0x55758215f950 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 10 client.6024 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 10 client.6024 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 20 client.6024 populate_metadata read hostname 'smithi170' 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d547d0 0x5575822c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 4 client.6024 mds_command: new command op to 5889 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:16:57.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 --> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575820c4a90 con 0x557581d547d0 2024-09-21T02:16:57.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d547d0 0x5575822c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbefb7fe640 10 client.6024 ms_handle_connect on v2:172.21.15.191:6836/3667702062 2024-09-21T02:16:57.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.185+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d547d0 0x5575822c60e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11f2c6326bc407d server_cookie=92d0e70dd6a15630 in_seq=0 out_seq=0 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbefb7fe640 1 -- 172.21.15.170:0/477901585 <== mds.0 v2:172.21.15.191:6836/3667702062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575820c4a90 con 0x557581d547d0 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbefb7fe640 10 client.6024 handle_command_reply: tid=0 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 client.6024 shutdown 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 msgr2=0x5575822bc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 0x5575822bc780 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbf1401ca60 tx=0x7fbf14005ae0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 shutdown_connections 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d547d0 0x5575822c60e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758215f950 0x5575822bc780 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557582148ca0 0x5575822bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbefb7fe640 0 client.6024 ms_handle_reset on v2:172.21.15.191:6836/3667702062 2024-09-21T02:16:57.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/477901585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822bbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 >> 172.21.15.170:0/477901585 conn(0x557582235d90 msgr2=0x5575822b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 shutdown_connections 2024-09-21T02:16:57.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/477901585 wait complete. 2024-09-21T02:16:57.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.189+0000 7fbf2a0fb000 20 client.6024 trim_cache size 0 max 16384 2024-09-21T02:16:57.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 Processor -- start 2024-09-21T02:16:57.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- start start 2024-09-21T02:16:57.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822c1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c200f80 con 0x557582148ca0 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c0cf850 con 0x55758215f950 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1c158150 con 0x557581d6f460 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47080/0 (socket says 172.21.15.170:47080) 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 -- 172.21.15.170:0/4292269084 learned_addr learned my addr 172.21.15.170:0/4292269084 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 -- 172.21.15.170:0/4292269084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 msgr2=0x5575822c1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 -- 172.21.15.170:0/4292269084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 msgr2=0x5575822b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 -- 172.21.15.170:0/4292269084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758210ddb0 con 0x557582148ca0 2024-09-21T02:16:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf13fff640 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b58e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c870 tx=0x7fbf0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e2ced571696a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf01ffb640 1 -- 172.21.15.170:0/4292269084 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c013b00 con 0x557582148ca0 2024-09-21T02:16:57.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf01ffb640 1 -- 172.21.15.170:0/4292269084 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf0c024070 con 0x557582148ca0 2024-09-21T02:16:57.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf01ffb640 1 -- 172.21.15.170:0/4292269084 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c01f430 con 0x557582148ca0 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/4292269084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 msgr2=0x5575822b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b58e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c870 tx=0x7fbf0c013000 comp rx=0 tx=0).stop 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/4292269084 shutdown_connections 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822b58e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/4292269084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822c1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/4292269084 >> 172.21.15.170:0/4292269084 conn(0x557582235d90 msgr2=0x557582267f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/4292269084 shutdown_connections 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.209+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/4292269084 wait complete. 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- start start 2024-09-21T02:16:57.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 Processor -- start 2024-09-21T02:16:57.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c0cf200 con 0x557582148ca0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557582171750 con 0x55758215f950 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5575820b5090 con 0x557581d6f460 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 10 client.0 _ll_register_callbacks cb 0x7fbf14009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822c12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47096/0 (socket says 172.21.15.170:47096) 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 -- 172.21.15.170:0/255029928 learned_addr learned my addr 172.21.15.170:0/255029928 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 -- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 msgr2=0x5575822c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 -- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 msgr2=0x5575822e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 -- 172.21.15.170:0/255029928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557582078ab0 con 0x557582148ca0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf13fff640 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822c12e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c008010 tx=0x7fbf0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e41f04cdd3a35a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 1 -- 172.21.15.170:0/255029928 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c02c070 con 0x557582148ca0 2024-09-21T02:16:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 1 -- 172.21.15.170:0/255029928 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fbf0c013d10 con 0x557582148ca0 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0c024070 con 0x557582148ca0 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 1 -- 172.21.15.170:0/255029928 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c039050 con 0x557582148ca0 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 10 client.6030 fetch_fsmap learned FSMap version 37 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x557582057070 con 0x557582148ca0 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 1 -- 172.21.15.170:0/255029928 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf0c024070 con 0x557582148ca0 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 1 -- 172.21.15.170:0/255029928 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fbf0c03c020 con 0x557582148ca0 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 10 client.6030 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 10 client.6030 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 20 client.6030 populate_metadata read hostname 'smithi170' 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d42830 0x55758232a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 4 client.6030 mds_command: new command op to 5889 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 --> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] -- command(tid 0: {"prefix": "status"}) -- 0x5575822cc400 con 0x557581d42830 2024-09-21T02:16:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d42830 0x55758232a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbefb7fe640 10 client.6030 ms_handle_connect on v2:172.21.15.191:6836/3667702062 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.213+0000 7fbf20bcc640 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d42830 0x55758232a060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a688bf701f45c1 server_cookie=bac57a0a0a8b31a9 in_seq=0 out_seq=0 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbefb7fe640 1 -- 172.21.15.170:0/255029928 <== mds.0 v2:172.21.15.191:6836/3667702062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5575822cc400 con 0x557581d42830 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbefb7fe640 10 client.6030 handle_command_reply: tid=0 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 client.6030 shutdown 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 msgr2=0x5575822c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822c12e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c008010 tx=0x7fbf0c004270 comp rx=0 tx=0).stop 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 shutdown_connections 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:6836/3667702062,v1:172.21.15.191:6837/3667702062] conn(0x557581d42830 0x55758232a060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55758215f950 0x5575822e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557582148ca0 0x5575822c12e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 --2- 172.21.15.170:0/255029928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557581d6f460 0x5575822c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbefb7fe640 0 client.6030 ms_handle_reset on v2:172.21.15.191:6836/3667702062 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 >> 172.21.15.170:0/255029928 conn(0x557582235d90 msgr2=0x557582268950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 shutdown_connections 2024-09-21T02:16:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 1 -- 172.21.15.170:0/255029928 wait complete. 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf2a0fb000 20 client.6030 trim_cache size 0 max 16384 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 5889, 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 6.0647942879999999, 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 37, 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 59, 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 59, 2024-09-21T02:16:57.221 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.595820158 2024-09-21T02:16:57.222 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:16:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbf080733b0 msgr2=0x7fbf080758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbf080733b0 0x7fbf080758c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c15f440 tx=0x7fbf0c008040 comp rx=0 tx=0).stop 2024-09-21T02:16:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 msgr2=0x7fbf1c154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 0x7fbf1c154d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800e940 tx=0x7fbf1800ee00 comp rx=0 tx=0).stop 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 shutdown_connections 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fbf080733b0 0x7fbf080758c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c1552d0 0x7fbf1c1f9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c15f990 0x7fbf1c1f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 --2- 172.21.15.170:0/4057657789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1c0ce4c0 0x7fbf1c154d90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 >> 172.21.15.170:0/4057657789 conn(0x7fbf1c0b1c40 msgr2=0x7fbf1c0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 shutdown_connections 2024-09-21T02:16:57.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.217+0000 7fbf22e30640 1 -- 172.21.15.170:0/4057657789 wait complete. 2024-09-21T02:16:57.238 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f3932c45640 1 -- 172.21.15.170:0/2283572149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3914003a40 msgr2=0x7f3914003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f3932c45640 1 --2- 172.21.15.170:0/2283572149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3914003a40 0x7f3914003eb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f391c0016f0 tx=0x7f391c01c780 comp rx=0 tx=0).stop 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f39311e2640 1 -- 172.21.15.170:0/2283572149 reap_dead start 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f3932c45640 1 -- 172.21.15.170:0/2283572149 shutdown_connections 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f3932c45640 1 -- 172.21.15.170:0/2283572149 >> 172.21.15.170:0/2283572149 conn(0x7f392c09d830 msgr2=0x7f392c08bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f3932c45640 1 -- 172.21.15.170:0/2283572149 shutdown_connections 2024-09-21T02:16:57.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.377+0000 7f3932c45640 1 -- 172.21.15.170:0/2283572149 wait complete. 2024-09-21T02:16:57.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 Processor -- start 2024-09-21T02:16:57.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 -- start start 2024-09-21T02:16:57.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f392c1fccb0 0x7f392c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 0x7f392c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f392c0b2730 0x7f392c1fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f392c16ab50 con 0x7f392c0b2320 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f392c16a9d0 con 0x7f392c0b2730 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f392c16acd0 con 0x7f392c1fccb0 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 0x7f392c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 0x7f392c0ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47126/0 (socket says 172.21.15.170:47126) 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 -- 172.21.15.170:0/1708020765 learned_addr learned my addr 172.21.15.170:0/1708020765 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f39309e1640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f392c1fccb0 0x7f392c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f39311e2640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f392c0b2730 0x7f392c1fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 -- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f392c1fccb0 msgr2=0x7f392c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f392c1fccb0 0x7f392c0cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 -- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f392c0b2730 msgr2=0x7f392c1fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f392c0b2730 0x7f392c1fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 -- 172.21.15.170:0/1708020765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c0b2c30 con 0x7f392c0b2320 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f392bfff640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 0x7f392c0ca990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f392000d900 tx=0x7f392000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39200064e0 con 0x7f392c0b2320 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f392000b920 con 0x7f392c0b2320 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392000bd70 con 0x7f392c0b2320 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392c160860 con 0x7f392c0b2320 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.381+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392c160e20 con 0x7f392c0b2320 2024-09-21T02:16:57.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.385+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392c0ced40 con 0x7f392c0b2320 2024-09-21T02:16:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.385+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f39200036e0 con 0x7f392c0b2320 2024-09-21T02:16:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.385+0000 7f39297fa640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f39180733b0 0x7f39180758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:16:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.385+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f3920094c00 con 0x7f392c0b2320 2024-09-21T02:16:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.385+0000 7f39309e1640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f39180733b0 0x7f39180758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:16:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.385+0000 7f39309e1640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f39180733b0 0x7f39180758c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f391c0016c0 tx=0x7f391c019570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:16:57.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.389+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f392009b080 con 0x7f392c0b2320 2024-09-21T02:16:57.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.597+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f392c167f70 con 0x7f392c0b2320 2024-09-21T02:16:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.597+0000 7f39297fa640 1 -- 172.21.15.170:0/1708020765 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) ==== 53+0+73 (secure 0 0 0) 0x7f392005f9b0 con 0x7f392c0b2320 2024-09-21T02:16:57.599 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f39180733b0 msgr2=0x7f39180758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f39180733b0 0x7f39180758c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f391c0016c0 tx=0x7f391c019570 comp rx=0 tx=0).stop 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 msgr2=0x7f392c0ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 0x7f392c0ca990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f392000d900 tx=0x7f392000ddc0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 shutdown_connections 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f39180733b0 0x7f39180758c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f392c0b2730 0x7f392c1fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f392c0b2320 0x7f392c0ca990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 --2- 172.21.15.170:0/1708020765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f392c1fccb0 0x7f392c0cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:16:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 >> 172.21.15.170:0/1708020765 conn(0x7f392c09d830 msgr2=0x7f392c0b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:16:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 shutdown_connections 2024-09-21T02:16:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57.601+0000 7f3932c45640 1 -- 172.21.15.170:0/1708020765 wait complete. 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:16:57.620 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:16:57.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:57.621 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:16:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42298sec preferred_lft 42298sec 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:16:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:16:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:16:57.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:57.630 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:16:57.678 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:16:57.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:57.679 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:16:57.729 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:16:57.730 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:16:57.731 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:16:57.731 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:16:57.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:16:57.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:16:57.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:16:57.731 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:16:57.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:16:57.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:16:57.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:57.850 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:16:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:16:57.898 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:16:57.898 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:16:57.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:57.898 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:16:57.953 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:16:57.954 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:16:57.954 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:58.002 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:16:58.002 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:16:58.002 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:58.049 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:16:58.107 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:16:58.155 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:16:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:58.157 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:58.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:58.158 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:16:58.206 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:16:58.206 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:16:58.251 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:16:58.264 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:16:58.310 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:16:58.310 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:16:58.310 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:16:58.310 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:16:58.311 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:58.312 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:58.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:58.313 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:16:58.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:16:58.361+0000 7ff929988580 -1 init, newargv = 0x7ff9180046b0 newargc=15 2024-09-21T02:16:58.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[40218]: starting ceph client 2024-09-21T02:16:58.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[40218]: starting fuse 2024-09-21T02:16:59.462 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:16:59.479 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:16:59.527 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:16:59.527 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:16:59.528 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:59.529 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:16:59.530 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:59.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:59.530 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:16:59.574 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:16:59.574 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:16:59.574 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:16:59.574 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:16:59.574 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:16:59.574 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:16:59.574 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:16:59.574 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:16:59.575 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:16:59.576 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:16:59.576 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:16:59.576 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:16:59.576 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:16:59.576 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:16:59.698 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.40218.asok 2024-09-21T02:16:59.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:16:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:16:59.710 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40218.asok 2024-09-21T02:16:59.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:59.710 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40218.asok status 2024-09-21T02:16:59.892 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:16:59.892 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "40218", 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6042, 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:16:59.893 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 6042 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3559839216 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3559839216 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:16:59.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.6042 192.168.0.1:0/3559839216", 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/3559839216", 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 37, 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 59, 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:16:59.895 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:16:59.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:59.908 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:16:59.908 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:59.914 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:16:59.914 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:59.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:16:59.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:16:59.969 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:17:00.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 -- 172.21.15.170:0/827823185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a80b7ab0 msgr2=0x7fc8a815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:00.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 --2- 172.21.15.170:0/827823185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a80b7ab0 0x7fc8a815c850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80d0450 tx=0x7fc89c01c890 comp rx=0 tx=0).stop 2024-09-21T02:17:00.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 -- 172.21.15.170:0/827823185 shutdown_connections 2024-09-21T02:17:00.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 --2- 172.21.15.170:0/827823185 >> v2:172.21.15.191:3300/0 conn(0x7fc8a8072900 0x7fc8a8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 --2- 172.21.15.170:0/827823185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80b3e30 0x7fc8a80b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 --2- 172.21.15.170:0/827823185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a80b7ab0 0x7fc8a815c850 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 -- 172.21.15.170:0/827823185 >> 172.21.15.170:0/827823185 conn(0x7fc8a80cee30 msgr2=0x7fc8a80cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:00.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 -- 172.21.15.170:0/827823185 shutdown_connections 2024-09-21T02:17:00.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.168+0000 7fc8af174640 1 -- 172.21.15.170:0/827823185 wait complete. 2024-09-21T02:17:00.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 Processor -- start 2024-09-21T02:17:00.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 -- start start 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a8072900 0x7fc8a81f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80b3e30 0x7fc8a80cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 0x7fc8a81f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a80d1800 con 0x7fc8a81f8250 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a80d1680 con 0x7fc8a8072900 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8af174640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a80d1980 con 0x7fc8a80b3e30 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 0x7fc8a81f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 0x7fc8a81f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47154/0 (socket says 172.21.15.170:47154) 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 -- 172.21.15.170:0/4036091920 learned_addr learned my addr 172.21.15.170:0/4036091920 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8acf10640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a8072900 0x7fc8a81f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8a7fff640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80b3e30 0x7fc8a80cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 -- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80b3e30 msgr2=0x7fc8a80cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80b3e30 0x7fc8a80cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 -- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a8072900 msgr2=0x7fc8a81f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a8072900 0x7fc8a81f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 -- 172.21.15.170:0/4036091920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a8159ee0 con 0x7fc8a81f8250 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8acf10640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a8072900 0x7fc8a81f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.172+0000 7fc8ad711640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 0x7fc8a81f8660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e990 tx=0x7fc8a000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0004d30 con 0x7fc8a81f8250 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8a000caa0 con 0x7fc8a81f8250 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a8158e40 con 0x7fc8a81f8250 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0005020 con 0x7fc8a81f8250 2024-09-21T02:17:00.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a8159400 con 0x7fc8a81f8250 2024-09-21T02:17:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a8072d10 con 0x7fc8a81f8250 2024-09-21T02:17:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fc8a00051c0 con 0x7fc8a81f8250 2024-09-21T02:17:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8a57fa640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8940733b0 0x7fc8940758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc8a0094f10 con 0x7fc8a81f8250 2024-09-21T02:17:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8acf10640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8940733b0 0x7fc8940758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.176+0000 7fc8acf10640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8940733b0 0x7fc8940758c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0af000 tx=0x7fc89c01b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:00.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.180+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc8a0016030 con 0x7fc8a81f8250 2024-09-21T02:17:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.388+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8a8158bd0 con 0x7fc8a81f8250 2024-09-21T02:17:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.388+0000 7fc8a57fa640 1 -- 172.21.15.170:0/4036091920 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) ==== 53+0+73 (secure 0 0 0) 0x7fc8a005fba0 con 0x7fc8a81f8250 2024-09-21T02:17:00.391 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8940733b0 msgr2=0x7fc8940758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8940733b0 0x7fc8940758c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0af000 tx=0x7fc89c01b0d0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 msgr2=0x7fc8a81f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 0x7fc8a81f8660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e990 tx=0x7fc8a000ee50 comp rx=0 tx=0).stop 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 shutdown_connections 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc8940733b0 0x7fc8940758c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80b3e30 0x7fc8a80cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a81f8250 0x7fc8a81f8660 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 --2- 172.21.15.170:0/4036091920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a8072900 0x7fc8a81f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 >> 172.21.15.170:0/4036091920 conn(0x7fc8a80cee30 msgr2=0x7fc8a80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 shutdown_connections 2024-09-21T02:17:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:00.392+0000 7fc8af174640 1 -- 172.21.15.170:0/4036091920 wait complete. 2024-09-21T02:17:00.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:17:00.411 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:17:00.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:00.411 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:00.418 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42289sec preferred_lft 42289sec 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:00.419 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:17:00.420 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:00.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:00.421 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:17:00.475 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:17:00.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:00.476 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:17:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:17:00.529 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:00.529 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:00.529 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:00.529 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:17:00.529 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:17:00.529 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:17:00.529 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:17:00.529 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:17:00.529 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:17:00.529 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:17:00.617 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:17:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:17:00.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:17:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:00.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:00.653 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:17:00.658 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:17:00.658 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:17:00.658 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:17:00.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:00.658 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:17:00.710 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:17:00.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:17:00.711 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:00.756 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:17:00.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:17:00.756 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:00.804 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:17:00.861 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:17:00.909 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:00.910 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:00.911 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:00.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:00.912 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:17:00.960 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:17:00.960 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:17:01.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:17:01.018 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:17:01.064 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:17:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:01.066 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:01.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:01.067 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:17:01.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:17:01.111+0000 7f1573e2a580 -1 init, newargv = 0x7f15600ae020 newargc=15 2024-09-21T02:17:01.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34584]: starting ceph client 2024-09-21T02:17:01.122 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34584]: starting fuse 2024-09-21T02:17:02.221 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:17:02.237 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:02.285 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:17:02.285 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:17:02.285 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:17:02.285 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:17:02.285 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:17:02.285 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:02.286 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:02.287 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:02.288 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:02.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:02.288 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:17:02.336 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:17:02.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:17:02.336 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:17:02.336 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:17:02.337 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:17:02.338 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:17:02.461 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.34584.asok 2024-09-21T02:17:02.465 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:17:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:02.472 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34584.asok 2024-09-21T02:17:02.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:02.473 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34584.asok status 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "34584", 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:17:02.657 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5308, 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5308 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 142571128 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:02.658 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 142571128 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5308 192.168.0.1:0/142571128", 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/142571128", 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 37, 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 59, 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:17:02.659 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:17:02.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:02.673 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:17:02.673 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:02.678 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:17:02.678 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:02.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:02.679 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:02.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:17:02.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 -- 172.21.15.170:0/3660790753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4004810 msgr2=0x7f00d4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:02.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 --2- 172.21.15.170:0/3660790753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4004810 0x7f00d4004c20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c2e0 tx=0x7f00e001cf60 comp rx=0 tx=0).stop 2024-09-21T02:17:02.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 -- 172.21.15.170:0/3660790753 shutdown_connections 2024-09-21T02:17:02.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 --2- 172.21.15.170:0/3660790753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00d4004810 0x7f00d4004c20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:02.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 -- 172.21.15.170:0/3660790753 >> 172.21.15.170:0/3660790753 conn(0x7f00ec0b1c40 msgr2=0x7f00ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:02.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 -- 172.21.15.170:0/3660790753 shutdown_connections 2024-09-21T02:17:02.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 -- 172.21.15.170:0/3660790753 wait complete. 2024-09-21T02:17:02.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 Processor -- start 2024-09-21T02:17:02.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.900+0000 7f00f4670640 1 -- start start 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00d4004810 0x7f00ec15f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec1f7d30 0x7f00ec0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 0x7f00ec0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec0ce690 con 0x7f00ec0aecf0 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec0ce510 con 0x7f00ec1f7d30 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec0ce810 con 0x7f00d4004810 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 0x7f00ec0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 0x7f00ec0c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47178/0 (socket says 172.21.15.170:47178) 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 -- 172.21.15.170:0/2637788905 learned_addr learned my addr 172.21.15.170:0/2637788905 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f240c640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00d4004810 0x7f00ec15f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f1c0b640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec1f7d30 0x7f00ec0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 -- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00d4004810 msgr2=0x7f00ec15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:02.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00d4004810 0x7f00ec15f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 -- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec1f7d30 msgr2=0x7f00ec0ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec1f7d30 0x7f00ec0ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 -- 172.21.15.170:0/2637788905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec15fd80 con 0x7f00ec0aecf0 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f2c0d640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 0x7f00ec0c8ce0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0afb20 tx=0x7f00dc00d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc005280 con 0x7f00ec0aecf0 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00dc005440 con 0x7f00ec0aecf0 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec0c7c60 con 0x7f00ec0aecf0 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003840 con 0x7f00ec0aecf0 2024-09-21T02:17:02.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec0c8220 con 0x7f00ec0aecf0 2024-09-21T02:17:02.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.904+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec0ced70 con 0x7f00ec0aecf0 2024-09-21T02:17:02.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.908+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f00dc0039e0 con 0x7f00ec0aecf0 2024-09-21T02:17:02.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.908+0000 7f00eaffd640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f00d80733e0 0x7f00d80758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:02.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.908+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f00dc094300 con 0x7f00ec0aecf0 2024-09-21T02:17:02.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.908+0000 7f00f240c640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f00d80733e0 0x7f00d80758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:02.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.908+0000 7f00f240c640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f00d80733e0 0x7f00d80758f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f00e4004490 tx=0x7f00e4003e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:02.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:02.912+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f00dc05ef90 con 0x7f00ec0aecf0 2024-09-21T02:17:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:03.120+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f00ec1611e0 con 0x7f00ec0aecf0 2024-09-21T02:17:04.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.004+0000 7f00eaffd640 1 -- 172.21.15.170:0/2637788905 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) ==== 136+0+0 (secure 0 0 0) 0x7f00dc05e890 con 0x7f00ec0aecf0 2024-09-21T02:17:04.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f00d80733e0 msgr2=0x7f00d80758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f00d80733e0 0x7f00d80758f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f00e4004490 tx=0x7f00e4003e30 comp rx=0 tx=0).stop 2024-09-21T02:17:04.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 msgr2=0x7f00ec0c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 0x7f00ec0c8ce0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0afb20 tx=0x7f00dc00d820 comp rx=0 tx=0).stop 2024-09-21T02:17:04.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 shutdown_connections 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f00d80733e0 0x7f00d80758f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec0aecf0 0x7f00ec0c8ce0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec1f7d30 0x7f00ec0ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 --2- 172.21.15.170:0/2637788905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00d4004810 0x7f00ec15f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 >> 172.21.15.170:0/2637788905 conn(0x7f00ec0b1c40 msgr2=0x7f00ec06f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 shutdown_connections 2024-09-21T02:17:04.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.008+0000 7f00f4670640 1 -- 172.21.15.170:0/2637788905 wait complete. 2024-09-21T02:17:04.032 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:17:04.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 -- 172.21.15.170:0/3117254012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 msgr2=0x7fc538004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 --2- 172.21.15.170:0/3117254012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc538004c20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc544006fd0 tx=0x7fc54401f6a0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 -- 172.21.15.170:0/3117254012 shutdown_connections 2024-09-21T02:17:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 --2- 172.21.15.170:0/3117254012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc538004c20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 -- 172.21.15.170:0/3117254012 >> 172.21.15.170:0/3117254012 conn(0x7fc5500b1c40 msgr2=0x7fc5500b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 -- 172.21.15.170:0/3117254012 shutdown_connections 2024-09-21T02:17:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.188+0000 7fc557056640 1 -- 172.21.15.170:0/3117254012 wait complete. 2024-09-21T02:17:04.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 Processor -- start 2024-09-21T02:17:04.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 -- start start 2024-09-21T02:17:04.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc55009b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55009d0f0 0x7fc55009d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5500ccc90 0x7fc5500cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500cbd10 con 0x7fc538004810 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500cbb90 con 0x7fc5500ccc90 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc550070340 con 0x7fc55009d0f0 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc55009b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc55009b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47200/0 (socket says 172.21.15.170:47200) 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 -- 172.21.15.170:0/3269641499 learned_addr learned my addr 172.21.15.170:0/3269641499 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 -- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55009d0f0 msgr2=0x7fc55009d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55009d0f0 0x7fc55009d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 -- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5500ccc90 msgr2=0x7fc5500cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc5555f3640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5500ccc90 0x7fc5500cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5500ccc90 0x7fc5500cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 -- 172.21.15.170:0/3269641499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5500cd5e0 con 0x7fc538004810 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc554df2640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc55009b720 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc540011a50 tx=0x7fc540011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54001f070 con 0x7fc538004810 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc54000fe80 con 0x7fc538004810 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5501f7090 con 0x7fc538004810 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540012690 con 0x7fc538004810 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.192+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5501f7650 con 0x7fc538004810 2024-09-21T02:17:04.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.196+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5500708a0 con 0x7fc538004810 2024-09-21T02:17:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.196+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fc540022070 con 0x7fc538004810 2024-09-21T02:17:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.196+0000 7fc54d7fa640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc53c0733b0 0x7fc53c0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.196+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc5400950a0 con 0x7fc538004810 2024-09-21T02:17:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.196+0000 7fc54ffff640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc53c0733b0 0x7fc53c0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.196+0000 7fc54ffff640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc53c0733b0 0x7fc53c0758c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5440b0000 tx=0x7fc538005840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.200+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc5500cd5e0 con 0x7fc538004810 2024-09-21T02:17:04.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.392+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc550155480 con 0x7fc53c0733b0 2024-09-21T02:17:04.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc54d7fa640 1 -- 172.21.15.170:0/3269641499 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc550155480 con 0x7fc53c0733b0 2024-09-21T02:17:04.399 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:17:04.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:04.399 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:17:04.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:04.399 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:17:04.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc53c0733b0 msgr2=0x7fc53c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc53c0733b0 0x7fc53c0758c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5440b0000 tx=0x7fc538005840 comp rx=0 tx=0).stop 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 msgr2=0x7fc55009b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc55009b720 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc540011a50 tx=0x7fc540011f10 comp rx=0 tx=0).stop 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 shutdown_connections 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc53c0733b0 0x7fc53c0758c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55009d0f0 0x7fc55009d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5500ccc90 0x7fc5500cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 --2- 172.21.15.170:0/3269641499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc538004810 0x7fc55009b720 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.396+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 >> 172.21.15.170:0/3269641499 conn(0x7fc5500b1c40 msgr2=0x7fc5500d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.400+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 shutdown_connections 2024-09-21T02:17:04.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.400+0000 7fc557056640 1 -- 172.21.15.170:0/3269641499 wait complete. 2024-09-21T02:17:04.419 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 -- 172.21.15.170:0/3532063202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52700cb790 msgr2=0x7f52700ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 --2- 172.21.15.170:0/3532063202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52700cb790 0x7f52700ce5b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52700723d0 tx=0x7f526001c810 comp rx=0 tx=0).stop 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 -- 172.21.15.170:0/3532063202 shutdown_connections 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 --2- 172.21.15.170:0/3532063202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527006f9a0 0x7f527006fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 --2- 172.21.15.170:0/3532063202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52700cb790 0x7f52700ce5b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 --2- 172.21.15.170:0/3532063202 >> v2:172.21.15.191:3300/0 conn(0x7f52700cb380 0x7f52700ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 -- 172.21.15.170:0/3532063202 >> 172.21.15.170:0/3532063202 conn(0x7f52700d0170 msgr2=0x7f52700d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 -- 172.21.15.170:0/3532063202 shutdown_connections 2024-09-21T02:17:04.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 -- 172.21.15.170:0/3532063202 wait complete. 2024-09-21T02:17:04.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 Processor -- start 2024-09-21T02:17:04.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.560+0000 7f527776d640 1 -- start start 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 0x7f52700c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52700cb380 0x7f52700c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52700cb790 0x7f5270155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f527015fd10 con 0x7f527006f9a0 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f527015fb90 con 0x7f52700cb380 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f527015fe90 con 0x7f52700cb790 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 0x7f52700c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 0x7f52700c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47216/0 (socket says 172.21.15.170:47216) 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 -- 172.21.15.170:0/915581583 learned_addr learned my addr 172.21.15.170:0/915581583 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 -- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52700cb790 msgr2=0x7f5270155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52700cb790 0x7f5270155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 -- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52700cb380 msgr2=0x7f52700c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52700cb380 0x7f52700c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 -- 172.21.15.170:0/915581583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5270155c90 con 0x7f527006f9a0 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5275509640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 0x7f52700c9d70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f526c00c9f0 tx=0x7f526c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c007bb0 con 0x7f527006f9a0 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f526c007e40 con 0x7f527006f9a0 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52700c7870 con 0x7f527006f9a0 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c00f690 con 0x7f527006f9a0 2024-09-21T02:17:04.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52700c7e30 con 0x7f527006f9a0 2024-09-21T02:17:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527006fdb0 con 0x7f527006f9a0 2024-09-21T02:17:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f526c00f830 con 0x7f527006f9a0 2024-09-21T02:17:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5265ffb640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f52540733b0 0x7f52540758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.564+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f526c094ed0 con 0x7f527006f9a0 2024-09-21T02:17:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.568+0000 7f5274d08640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f52540733b0 0x7f52540758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.568+0000 7f5274d08640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f52540733b0 0x7f52540758c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52600ae000 tx=0x7f52600195b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:04.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.572+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f526c014030 con 0x7f527006f9a0 2024-09-21T02:17:04.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.744+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5270076450 con 0x7f527006f9a0 2024-09-21T02:17:04.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.744+0000 7f5265ffb640 1 -- 172.21.15.170:0/915581583 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f526c05fb60 con 0x7f527006f9a0 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f52540733b0 msgr2=0x7f52540758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f52540733b0 0x7f52540758c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52600ae000 tx=0x7f52600195b0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 msgr2=0x7f52700c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 0x7f52700c9d70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f526c00c9f0 tx=0x7f526c00ceb0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 shutdown_connections 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f52540733b0 0x7f52540758c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52700cb790 0x7f5270155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52700cb380 0x7f52700c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 --2- 172.21.15.170:0/915581583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f527006f9a0 0x7f52700c9d70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 >> 172.21.15.170:0/915581583 conn(0x7f52700d0170 msgr2=0x7f52700ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 shutdown_connections 2024-09-21T02:17:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:04.748+0000 7f527776d640 1 -- 172.21.15.170:0/915581583 wait complete. 2024-09-21T02:17:04.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:04.769 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol41760) 2024-09-21T02:17:04.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:04.798 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:17:04.850 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:17:04.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:04.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol41760) 2024-09-21T02:17:04.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol85551 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 -- 172.21.15.170:0/724933138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19c8004d50 msgr2=0x7f19c80051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/724933138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19c8004d50 0x7f19c80051a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009e00 tx=0x7f19d801d170 comp rx=0 tx=0).stop 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19dffff640 1 -- 172.21.15.170:0/724933138 reap_dead start 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 -- 172.21.15.170:0/724933138 shutdown_connections 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 -- 172.21.15.170:0/724933138 >> 172.21.15.170:0/724933138 conn(0x7f19e008bf50 msgr2=0x7f19e009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 -- 172.21.15.170:0/724933138 shutdown_connections 2024-09-21T02:17:05.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 -- 172.21.15.170:0/724933138 wait complete. 2024-09-21T02:17:05.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 Processor -- start 2024-09-21T02:17:05.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.120+0000 7f19e5d1e640 1 -- start start 2024-09-21T02:17:05.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19c8004d50 0x7f19e00cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:05.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 0x7f19e0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e0151000 0x7f19e01f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0162630 con 0x7f19e00cc4e0 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f19e01624b0 con 0x7f19e0151000 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f19e01627b0 con 0x7f19c8004d50 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 0x7f19e0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 0x7f19e0150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47238/0 (socket says 172.21.15.170:47238) 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 -- 172.21.15.170:0/49714306 learned_addr learned my addr 172.21.15.170:0/49714306 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19dffff640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e0151000 0x7f19e01f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19df7fe640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19c8004d50 0x7f19e00cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 -- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19c8004d50 msgr2=0x7f19e00cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19c8004d50 0x7f19e00cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 -- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e0151000 msgr2=0x7f19e01f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e0151000 0x7f19e01f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 -- 172.21.15.170:0/49714306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e015f190 con 0x7f19e00cc4e0 2024-09-21T02:17:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19deffd640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 0x7f19e0150ac0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d920 tx=0x7f19d400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d40062e0 con 0x7f19e00cc4e0 2024-09-21T02:17:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d4015020 con 0x7f19e00cc4e0 2024-09-21T02:17:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e015f470 con 0x7f19e00cc4e0 2024-09-21T02:17:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d400ba30 con 0x7f19e00cc4e0 2024-09-21T02:17:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e015fa30 con 0x7f19e00cc4e0 2024-09-21T02:17:05.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00b6b30 con 0x7f19e00cc4e0 2024-09-21T02:17:05.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.124+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f19d400bbd0 con 0x7f19e00cc4e0 2024-09-21T02:17:05.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.128+0000 7f19c7fff640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19cc0733e0 0x7f19cc0758f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:05.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.128+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f19d4095070 con 0x7f19e00cc4e0 2024-09-21T02:17:05.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.128+0000 7f19df7fe640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19cc0733e0 0x7f19cc0758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:05.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.128+0000 7f19df7fe640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19cc0733e0 0x7f19cc0758f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005fd0 tx=0x7f19d0005f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:05.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.132+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19d401f020 con 0x7f19e00cc4e0 2024-09-21T02:17:05.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.316+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol85551", "target": ["mon-mgr", ""]}) -- 0x7f19e00cb240 con 0x7f19cc0733e0 2024-09-21T02:17:05.328 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:05.327+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:05.328 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:05.327+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:05.329 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:05.327+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:05.329 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:05.327+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:05.329 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:05.327+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:05.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.376+0000 7f19c7fff640 1 -- 172.21.15.170:0/49714306 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19e00cb240 con 0x7f19cc0733e0 2024-09-21T02:17:05.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19cc0733e0 msgr2=0x7f19cc0758f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:05.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19cc0733e0 0x7f19cc0758f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19d0005fd0 tx=0x7f19d0005f40 comp rx=0 tx=0).stop 2024-09-21T02:17:05.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 msgr2=0x7f19e0150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 0x7f19e0150ac0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d920 tx=0x7f19d400dd20 comp rx=0 tx=0).stop 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 shutdown_connections 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f19cc0733e0 0x7f19cc0758f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e0151000 0x7f19e01f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00cc4e0 0x7f19e0150ac0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 --2- 172.21.15.170:0/49714306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19c8004d50 0x7f19e00cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 >> 172.21.15.170:0/49714306 conn(0x7f19e008bf50 msgr2=0x7f19e00b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 shutdown_connections 2024-09-21T02:17:05.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:05.380+0000 7f19e5d1e640 1 -- 172.21.15.170:0/49714306 wait complete. 2024-09-21T02:17:05.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:05.401 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol85551/.meta ./volumes/_nogroup/subvol41760) 2024-09-21T02:17:06.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41760 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 -- 172.21.15.170:0/3499893704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 msgr2=0x7f6b9c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 --2- 172.21.15.170:0/3499893704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c0cb280 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009900 tx=0x7f6b9001c880 comp rx=0 tx=0).stop 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 -- 172.21.15.170:0/3499893704 shutdown_connections 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 --2- 172.21.15.170:0/3499893704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b9c074680 0x7f6b9c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 --2- 172.21.15.170:0/3499893704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c0cb280 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 -- 172.21.15.170:0/3499893704 >> 172.21.15.170:0/3499893704 conn(0x7f6b9c0d0170 msgr2=0x7f6b9c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 -- 172.21.15.170:0/3499893704 shutdown_connections 2024-09-21T02:17:06.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.812+0000 7f6ba23be640 1 -- 172.21.15.170:0/3499893704 wait complete. 2024-09-21T02:17:06.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 Processor -- start 2024-09-21T02:17:06.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 -- start start 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c09a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b9c074680 0x7f6b9c1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b9c0cf7d0 0x7f6b9c0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c169040 con 0x7f6b9c074270 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c168ec0 con 0x7f6b9c0cf7d0 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c1691c0 con 0x7f6b9c074680 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c09a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c09a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51202/0 (socket says 172.21.15.170:51202) 2024-09-21T02:17:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 -- 172.21.15.170:0/1127876501 learned_addr learned my addr 172.21.15.170:0/1127876501 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9b7fe640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b9c074680 0x7f6b9c1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba095b640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b9c0cf7d0 0x7f6b9c0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 -- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b9c074680 msgr2=0x7f6b9c1f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b9c074680 0x7f6b9c1f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 -- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b9c0cf7d0 msgr2=0x7f6b9c0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b9c0cf7d0 0x7f6b9c0c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 -- 172.21.15.170:0/1127876501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c159900 con 0x7f6b9c074270 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b9bfff640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c09a9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c3c0 tx=0x7f6b8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0052d0 con 0x7f6b9c074270 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c1fa860 con 0x7f6b9c074270 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b8c007020 con 0x7f6b9c074270 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c007470 con 0x7f6b9c074270 2024-09-21T02:17:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.816+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1f6b10 con 0x7f6b9c074270 2024-09-21T02:17:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.820+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c073b60 con 0x7f6b9c074270 2024-09-21T02:17:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.820+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f6b8c0067e0 con 0x7f6b9c074270 2024-09-21T02:17:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.820+0000 7f6b98ff9640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b880733b0 0x7f6b880758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.820+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5767+0+0 (secure 0 0 0) 0x7f6b8c093b00 con 0x7f6b9c074270 2024-09-21T02:17:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.820+0000 7f6b9b7fe640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b880733b0 0x7f6b880758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.820+0000 7f6b9b7fe640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b880733b0 0x7f6b880758c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b900af000 tx=0x7f6b9001b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:06.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:06.824+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6b8c05e760 con 0x7f6b9c074270 2024-09-21T02:17:07.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.020+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41760", "target": ["mon-mgr", ""]}) -- 0x7f6b9c0af2f0 con 0x7f6b880733b0 2024-09-21T02:17:07.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6b98ff9640 1 -- 172.21.15.170:0/1127876501 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f6b9c0af2f0 con 0x7f6b880733b0 2024-09-21T02:17:07.099 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol41760 2024-09-21T02:17:07.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b880733b0 msgr2=0x7f6b880758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b880733b0 0x7f6b880758c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b900af000 tx=0x7f6b9001b120 comp rx=0 tx=0).stop 2024-09-21T02:17:07.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 msgr2=0x7f6b9c09a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c09a9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c3c0 tx=0x7f6b8c00c880 comp rx=0 tx=0).stop 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 shutdown_connections 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6b880733b0 0x7f6b880758c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b9c074680 0x7f6b9c1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b9c0cf7d0 0x7f6b9c0c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 --2- 172.21.15.170:0/1127876501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9c074270 0x7f6b9c09a9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.096+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 >> 172.21.15.170:0/1127876501 conn(0x7f6b9c0d0170 msgr2=0x7f6b9c09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.100+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 shutdown_connections 2024-09-21T02:17:07.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.100+0000 7f6ba23be640 1 -- 172.21.15.170:0/1127876501 wait complete. 2024-09-21T02:17:07.120 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41760 2024-09-21T02:17:07.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 -- 172.21.15.170:0/4033169052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 msgr2=0x7f34c8073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 --2- 172.21.15.170:0/4033169052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 0x7f34c8073b60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009840 tx=0x7f34bc01c7c0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 -- 172.21.15.170:0/4033169052 shutdown_connections 2024-09-21T02:17:07.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 --2- 172.21.15.170:0/4033169052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 0x7f34c80744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 --2- 172.21.15.170:0/4033169052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 0x7f34c8073b60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 -- 172.21.15.170:0/4033169052 >> 172.21.15.170:0/4033169052 conn(0x7f34c80d0170 msgr2=0x7f34c80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:07.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 -- 172.21.15.170:0/4033169052 shutdown_connections 2024-09-21T02:17:07.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.280+0000 7f34cee8d640 1 -- 172.21.15.170:0/4033169052 wait complete. 2024-09-21T02:17:07.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 Processor -- start 2024-09-21T02:17:07.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 -- start start 2024-09-21T02:17:07.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 0x7f34c80c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:07.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 0x7f34c8159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:07.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 0x7f34c8158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:07.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8169040 con 0x7f34c80ce070 2024-09-21T02:17:07.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34c8168ec0 con 0x7f34c80cf7d0 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f34c81691c0 con 0x7f34c80740a0 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 0x7f34c8159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 0x7f34c8159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51218/0 (socket says 172.21.15.170:51218) 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 -- 172.21.15.170:0/3681344175 learned_addr learned my addr 172.21.15.170:0/3681344175 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34ccc29640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 0x7f34c80c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cd42a640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 0x7f34c8158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 -- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 msgr2=0x7f34c80c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 0x7f34c80c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 -- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 msgr2=0x7f34c8158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 0x7f34c8158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 -- 172.21.15.170:0/3681344175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c81f9810 con 0x7f34c80ce070 2024-09-21T02:17:07.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34ccc29640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 0x7f34c80c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:07.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c7fff640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 0x7f34c8159700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f34b400e9f0 tx=0x7f34b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:07.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b400cd80 con 0x7f34c80ce070 2024-09-21T02:17:07.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34b4004020 con 0x7f34c80ce070 2024-09-21T02:17:07.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004470 con 0x7f34c80ce070 2024-09-21T02:17:07.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81f8a50 con 0x7f34c80ce070 2024-09-21T02:17:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.284+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c81f9010 con 0x7f34c80ce070 2024-09-21T02:17:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.288+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c80744b0 con 0x7f34c80ce070 2024-09-21T02:17:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.288+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7f34b4003600 con 0x7f34c80ce070 2024-09-21T02:17:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.288+0000 7f34c57fa640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f34ac0733b0 0x7f34ac0758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.288+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f34b4095180 con 0x7f34c80ce070 2024-09-21T02:17:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.288+0000 7f34ccc29640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f34ac0733b0 0x7f34ac0758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:07.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.288+0000 7f34ccc29640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f34ac0733b0 0x7f34ac0758c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0af000 tx=0x7f34bc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:07.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.292+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f34b405fd50 con 0x7f34c80ce070 2024-09-21T02:17:07.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.492+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41760", "target": ["mon-mgr", ""]}) -- 0x7f34c80c6710 con 0x7f34ac0733b0 2024-09-21T02:17:07.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.508+0000 7f34c57fa640 1 -- 172.21.15.170:0/3681344175 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f34c80c6710 con 0x7f34ac0733b0 2024-09-21T02:17:07.512 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol41760 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f34ac0733b0 msgr2=0x7f34ac0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f34ac0733b0 0x7f34ac0758c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0af000 tx=0x7f34bc01b060 comp rx=0 tx=0).stop 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 msgr2=0x7f34c8159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 0x7f34c8159700 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f34b400e9f0 tx=0x7f34b400edf0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 shutdown_connections 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f34ac0733b0 0x7f34ac0758c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34c80cf7d0 0x7f34c8158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34c80ce070 0x7f34c8159700 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 --2- 172.21.15.170:0/3681344175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34c80740a0 0x7f34c80c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 >> 172.21.15.170:0/3681344175 conn(0x7f34c80d0170 msgr2=0x7f34c80aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 shutdown_connections 2024-09-21T02:17:07.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:07.512+0000 7f34cee8d640 1 -- 172.21.15.170:0/3681344175 wait complete. 2024-09-21T02:17:07.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:07.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/0cb6a0a479058da78073fa40c4c2f39d.meta) 2024-09-21T02:17:11.637 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:17:11.637 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:17:11.638 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:17:11.638 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol41760 2024-09-21T02:17:11.638 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:17:11.638 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:11.640 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol41760 alice 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- 172.21.15.170:0/1582120035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 msgr2=0x7f87d4005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 --2- 172.21.15.170:0/1582120035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87d4005250 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f87e80cdbe0 tx=0x7f87d801cad0 comp rx=0 tx=0).stop 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87ecfe1640 1 -- 172.21.15.170:0/1582120035 reap_dead start 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- 172.21.15.170:0/1582120035 shutdown_connections 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- 172.21.15.170:0/1582120035 >> 172.21.15.170:0/1582120035 conn(0x7f87e80b1c40 msgr2=0x7f87e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- 172.21.15.170:0/1582120035 shutdown_connections 2024-09-21T02:17:11.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- 172.21.15.170:0/1582120035 wait complete. 2024-09-21T02:17:11.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 Processor -- start 2024-09-21T02:17:11.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- start start 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87e81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e81e4580 0x7f87e81e4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e815d8c0 0x7f87e815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80740f0 con 0x7f87d4004da0 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80caf60 con 0x7f87e81e4580 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.808+0000 7f87eea44640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f87e8074270 con 0x7f87e815d8c0 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87e81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:11.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87e81594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51236/0 (socket says 172.21.15.170:51236) 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 -- 172.21.15.170:0/506505506 learned_addr learned my addr 172.21.15.170:0/506505506 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e77fe640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e81e4580 0x7f87e81e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 -- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e815d8c0 msgr2=0x7f87e815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e815d8c0 0x7f87e815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 -- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e81e4580 msgr2=0x7f87e81e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e81e4580 0x7f87e81e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 -- 172.21.15.170:0/506505506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e809aa60 con 0x7f87d4004da0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e7fff640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87e81594f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f87e00109c0 tx=0x7f87e0010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e000eda0 con 0x7f87d4004da0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87e0004020 con 0x7f87d4004da0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e815e210 con 0x7f87d4004da0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004470 con 0x7f87d4004da0 2024-09-21T02:17:11.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e809afb0 con 0x7f87d4004da0 2024-09-21T02:17:11.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8074800 con 0x7f87d4004da0 2024-09-21T02:17:11.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f87e0003600 con 0x7f87d4004da0 2024-09-21T02:17:11.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e4ff9640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073440 0x7f87c8075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:11.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f87e0095630 con 0x7f87d4004da0 2024-09-21T02:17:11.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.812+0000 7f87e77fe640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073440 0x7f87c8075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:11.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.816+0000 7f87e77fe640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073440 0x7f87c8075950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87e815f020 tx=0x7f87d0001560 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:11.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:11.816+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f87e815e210 con 0x7f87d4004da0 2024-09-21T02:17:12.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.016+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol41760", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f87e809e040 con 0x7f87c8073440 2024-09-21T02:17:12.044 INFO:tasks.ceph.mgr.x.smithi191.stderr:Creating meta for ID alice with tenant None 2024-09-21T02:17:12.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87e4ff9640 1 -- 172.21.15.170:0/506505506 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f87e809e040 con 0x7f87c8073440 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073440 msgr2=0x7f87c8075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073440 0x7f87c8075950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87e815f020 tx=0x7f87d0001560 comp rx=0 tx=0).stop 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 msgr2=0x7f87e81594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87e81594f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f87e00109c0 tx=0x7f87e0010e80 comp rx=0 tx=0).stop 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 shutdown_connections 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f87c8073440 0x7f87c8075950 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87e815d8c0 0x7f87e815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e81e4580 0x7f87e81e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 --2- 172.21.15.170:0/506505506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87d4004da0 0x7f87e81594f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 >> 172.21.15.170:0/506505506 conn(0x7f87e80b1c40 msgr2=0x7f87e80adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:12.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 shutdown_connections 2024-09-21T02:17:12.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.112+0000 7f87eea44640 1 -- 172.21.15.170:0/506505506 wait complete. 2024-09-21T02:17:12.141 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f03864ab640 1 -- 172.21.15.170:0/1482219558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 msgr2=0x7f036c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f03864ab640 1 --2- 172.21.15.170:0/1482219558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f036c003eb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f03700016f0 tx=0x7f037001c7b0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f0384a48640 1 -- 172.21.15.170:0/1482219558 reap_dead start 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f03864ab640 1 -- 172.21.15.170:0/1482219558 shutdown_connections 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f03864ab640 1 -- 172.21.15.170:0/1482219558 >> 172.21.15.170:0/1482219558 conn(0x7f03800b1c40 msgr2=0x7f03800b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f03864ab640 1 -- 172.21.15.170:0/1482219558 shutdown_connections 2024-09-21T02:17:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.296+0000 7f03864ab640 1 -- 172.21.15.170:0/1482219558 wait complete. 2024-09-21T02:17:12.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 Processor -- start 2024-09-21T02:17:12.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 -- start start 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f03801f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03800c97a0 0x7f0380157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03801f6e10 0x7f03801f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03801679f0 con 0x7f036c003a40 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0380167870 con 0x7f03800c97a0 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f03864ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0380167b70 con 0x7f03801f6e10 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f03801f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f03801f9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51246/0 (socket says 172.21.15.170:51246) 2024-09-21T02:17:12.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 -- 172.21.15.170:0/756536824 learned_addr learned my addr 172.21.15.170:0/756536824 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f0384a48640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03801f6e10 0x7f03801f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037f7fe640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03800c97a0 0x7f0380157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 -- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03801f6e10 msgr2=0x7f03801f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03801f6e10 0x7f03801f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 -- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03800c97a0 msgr2=0x7f0380157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03800c97a0 0x7f0380157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 -- 172.21.15.170:0/756536824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03800b40c0 con 0x7f036c003a40 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.300+0000 7f037ffff640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f03801f9d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0370000c10 tx=0x7f0370003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370001310 con 0x7f036c003a40 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0370006500 con 0x7f036c003a40 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03800b43a0 con 0x7f036c003a40 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370006930 con 0x7f036c003a40 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0380157870 con 0x7f036c003a40 2024-09-21T02:17:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03801f6710 con 0x7f036c003a40 2024-09-21T02:17:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f03700a6510 con 0x7f036c003a40 2024-09-21T02:17:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037cff9640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0360073470 0x7f0360075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f037012e850 con 0x7f036c003a40 2024-09-21T02:17:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037f7fe640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0360073470 0x7f0360075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.304+0000 7f037f7fe640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0360073470 0x7f0360075980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f03680018b0 tx=0x7f0368000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.308+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f03801f6710 con 0x7f036c003a40 2024-09-21T02:17:12.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.544+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7f03800cb870 con 0x7f036c003a40 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.544+0000 7f037cff9640 1 -- 172.21.15.170:0/756536824 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v22) ==== 107+0+280 (secure 0 0 0) 0x7f03700bf060 con 0x7f036c003a40 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.alice", 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCoLO5mVA40BBAA9j6DCK0ZfbzWtDD9jy8NDg==", 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol41760", 2024-09-21T02:17:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:17:12.550 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-21T02:17:12.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:12.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:12.550 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0360073470 msgr2=0x7f0360075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0360073470 0x7f0360075980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f03680018b0 tx=0x7f0368000ee0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 msgr2=0x7f03801f9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f03801f9d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0370000c10 tx=0x7f0370003b90 comp rx=0 tx=0).stop 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 shutdown_connections 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0360073470 0x7f0360075980 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03801f6e10 0x7f03801f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03800c97a0 0x7f0380157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 --2- 172.21.15.170:0/756536824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f036c003a40 0x7f03801f9d00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 >> 172.21.15.170:0/756536824 conn(0x7f03800b1c40 msgr2=0x7f038015a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:12.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 shutdown_connections 2024-09-21T02:17:12.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.548+0000 7f03864ab640 1 -- 172.21.15.170:0/756536824 wait complete. 2024-09-21T02:17:12.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41760 2024-09-21T02:17:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa2c962640 1 -- 172.21.15.170:0/49233141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa10004810 msgr2=0x7faa10004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa2c962640 1 --2- 172.21.15.170:0/49233141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa10004810 0x7faa10004c20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faa1c001690 tx=0x7faa1c01c7a0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa26d9d640 1 -- 172.21.15.170:0/49233141 reap_dead start 2024-09-21T02:17:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa2c962640 1 -- 172.21.15.170:0/49233141 shutdown_connections 2024-09-21T02:17:12.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa2c962640 1 -- 172.21.15.170:0/49233141 >> 172.21.15.170:0/49233141 conn(0x7faa280d0170 msgr2=0x7faa280d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:12.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa2c962640 1 -- 172.21.15.170:0/49233141 shutdown_connections 2024-09-21T02:17:12.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.728+0000 7faa2c962640 1 -- 172.21.15.170:0/49233141 wait complete. 2024-09-21T02:17:12.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 Processor -- start 2024-09-21T02:17:12.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 -- start start 2024-09-21T02:17:12.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa10004810 0x7faa281f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 0x7faa2814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa2806f7d0 0x7faa2806fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa280cbb10 con 0x7faa281513d0 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faa280cb990 con 0x7faa2806f7d0 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2c962640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7faa280cbc90 con 0x7faa10004810 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 0x7faa2814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 0x7faa2814ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51266/0 (socket says 172.21.15.170:51266) 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 -- 172.21.15.170:0/1839894970 learned_addr learned my addr 172.21.15.170:0/1839894970 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa2659c640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa10004810 0x7faa281f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa26d9d640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa2806f7d0 0x7faa2806fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 -- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa10004810 msgr2=0x7faa281f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa10004810 0x7faa281f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 -- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa2806f7d0 msgr2=0x7faa2806fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa2806f7d0 0x7faa2806fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 -- 172.21.15.170:0/1839894970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa28070120 con 0x7faa281513d0 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa26d9d640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa2806f7d0 0x7faa2806fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa25d9b640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 0x7faa2814ded0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faa1c006040 tx=0x7faa1c0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c003810 con 0x7faa281513d0 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa1c003aa0 con 0x7faa281513d0 2024-09-21T02:17:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.732+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0a55a0 con 0x7faa281513d0 2024-09-21T02:17:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa280c7500 con 0x7faa281513d0 2024-09-21T02:17:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa281508f0 con 0x7faa281513d0 2024-09-21T02:17:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa280cc220 con 0x7faa281513d0 2024-09-21T02:17:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7faa1c0a5740 con 0x7faa281513d0 2024-09-21T02:17:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa16ffd640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faa08073440 0x7faa08075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7faa1c12e5c0 con 0x7faa281513d0 2024-09-21T02:17:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa2659c640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faa08073440 0x7faa08075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.736+0000 7faa2659c640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faa08073440 0x7faa08075950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faa18001d40 tx=0x7faa180011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:12.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.740+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faa28070120 con 0x7faa281513d0 2024-09-21T02:17:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.936+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41760", "target": ["mon-mgr", ""]}) -- 0x7faa280c7290 con 0x7faa08073440 2024-09-21T02:17:12.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.972+0000 7faa16ffd640 1 -- 172.21.15.170:0/1839894970 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa280c7290 con 0x7faa08073440 2024-09-21T02:17:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faa08073440 msgr2=0x7faa08075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faa08073440 0x7faa08075950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faa18001d40 tx=0x7faa180011a0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 msgr2=0x7faa2814ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 0x7faa2814ded0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faa1c006040 tx=0x7faa1c0012b0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 shutdown_connections 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7faa08073440 0x7faa08075950 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa2806f7d0 0x7faa2806fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa281513d0 0x7faa2814ded0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 --2- 172.21.15.170:0/1839894970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa10004810 0x7faa281f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 >> 172.21.15.170:0/1839894970 conn(0x7faa280d0170 msgr2=0x7faa280b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 shutdown_connections 2024-09-21T02:17:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:12.976+0000 7faa2c962640 1 -- 172.21.15.170:0/1839894970 wait complete. 2024-09-21T02:17:12.983 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.978+0000 7f1a445f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:12.983 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.978+0000 7f1a445f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:12.983 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.978+0000 7f1a445f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:12.983 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.978+0000 7f1a445f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:12.983 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.978+0000 7f1a445f8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:12.998 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol85551 2024-09-21T02:17:13.001 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.998+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:13.002 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.998+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:13.002 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.998+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:13.002 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.998+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:13.002 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:12.998+0000 7f1a455fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 -- 172.21.15.170:0/2095634313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 msgr2=0x7f8be00cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 --2- 172.21.15.170:0/2095634313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be00cf020 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8be008c980 tx=0x7f8bd401c810 comp rx=0 tx=0).stop 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 -- 172.21.15.170:0/2095634313 shutdown_connections 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 --2- 172.21.15.170:0/2095634313 >> v2:172.21.15.191:3300/0 conn(0x7f8be00cf970 0x7f8be00d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 --2- 172.21.15.170:0/2095634313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be00cf020 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 -- 172.21.15.170:0/2095634313 >> 172.21.15.170:0/2095634313 conn(0x7f8be009d8b0 msgr2=0x7f8be00d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:13.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 -- 172.21.15.170:0/2095634313 shutdown_connections 2024-09-21T02:17:13.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.140+0000 7f8be676a640 1 -- 172.21.15.170:0/2095634313 wait complete. 2024-09-21T02:17:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 Processor -- start 2024-09-21T02:17:13.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 -- start start 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be00cf560 0x7f8be00a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be00cf970 0x7f8be0157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be01614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8be016be90 con 0x7f8be00d3020 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8be016bd10 con 0x7f8be00cf560 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be676a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8be016c010 con 0x7f8be00cf970 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be4d07640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be01614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be4d07640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be01614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51284/0 (socket says 172.21.15.170:51284) 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.144+0000 7f8be4d07640 1 -- 172.21.15.170:0/4173102496 learned_addr learned my addr 172.21.15.170:0/4173102496 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdffff640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be00cf560 0x7f8be00a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdf7fe640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be00cf970 0x7f8be0157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be4d07640 1 -- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be00cf970 msgr2=0x7f8be0157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be4d07640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be00cf970 0x7f8be0157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be4d07640 1 -- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be00cf560 msgr2=0x7f8be00a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be4d07640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be00cf560 0x7f8be00a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be4d07640 1 -- 172.21.15.170:0/4173102496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01fc1c0 con 0x7f8be00d3020 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdffff640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be00cf560 0x7f8be00a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be4d07640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be01614a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800ca90 tx=0x7f8bd800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8007df0 con 0x7f8be00d3020 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bd8004b30 con 0x7f8be00d3020 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8005020 con 0x7f8be00d3020 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01fc4a0 con 0x7f8be00d3020 2024-09-21T02:17:13.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01fca60 con 0x7f8be00d3020 2024-09-21T02:17:13.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be00cf020 con 0x7f8be00d3020 2024-09-21T02:17:13.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f8bd80051c0 con 0x7f8be00d3020 2024-09-21T02:17:13.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdcff9640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bcc073470 0x7f8bcc075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f8bd8094e20 con 0x7f8be00d3020 2024-09-21T02:17:13.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdffff640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bcc073470 0x7f8bcc075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.148+0000 7f8bdffff640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bcc073470 0x7f8bcc075980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40ae000 tx=0x7f8bd401b050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:13.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.152+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8bd8014030 con 0x7f8be00d3020 2024-09-21T02:17:13.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.356+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol85551", "target": ["mon-mgr", ""]}) -- 0x7f8be00b3fa0 con 0x7f8bcc073470 2024-09-21T02:17:13.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.396+0000 7f8bdcff9640 1 -- 172.21.15.170:0/4173102496 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8be00b3fa0 con 0x7f8bcc073470 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bcc073470 msgr2=0x7f8bcc075980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bcc073470 0x7f8bcc075980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40ae000 tx=0x7f8bd401b050 comp rx=0 tx=0).stop 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 msgr2=0x7f8be01614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be01614a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800ca90 tx=0x7f8bd800cf50 comp rx=0 tx=0).stop 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 shutdown_connections 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8bcc073470 0x7f8bcc075980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be00cf970 0x7f8be0157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be00d3020 0x7f8be01614a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 --2- 172.21.15.170:0/4173102496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be00cf560 0x7f8be00a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 >> 172.21.15.170:0/4173102496 conn(0x7f8be009d8b0 msgr2=0x7f8be0163080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 shutdown_connections 2024-09-21T02:17:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.400+0000 7f8be676a640 1 -- 172.21.15.170:0/4173102496 wait complete. 2024-09-21T02:17:13.423 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:17:13.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:17:13.479 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:17:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:13.485 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:17:13.485 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:17:13.594 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:17:13.660 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:17:13.661 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:13.661 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:13.661 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:13.667 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:13.667 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:13.714 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:17:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 -- 172.21.15.170:0/3210722045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700ce1c0 msgr2=0x7fd7700ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 --2- 172.21.15.170:0/3210722045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700ce1c0 0x7fd7700ce5d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd768009bc0 tx=0x7fd76801cba0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 -- 172.21.15.170:0/3210722045 shutdown_connections 2024-09-21T02:17:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 --2- 172.21.15.170:0/3210722045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700ce1c0 0x7fd7700ce5d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 --2- 172.21.15.170:0/3210722045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700cf7e0 0x7fd770070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 -- 172.21.15.170:0/3210722045 >> 172.21.15.170:0/3210722045 conn(0x7fd7700b1c40 msgr2=0x7fd7700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 -- 172.21.15.170:0/3210722045 shutdown_connections 2024-09-21T02:17:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.916+0000 7fd7750e7640 1 -- 172.21.15.170:0/3210722045 wait complete. 2024-09-21T02:17:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 Processor -- start 2024-09-21T02:17:13.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 -- start start 2024-09-21T02:17:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 0x7fd770159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700ce1c0 0x7fd77015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7700cf7e0 0x7fd7700b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7701670c0 con 0x7fd7700cb460 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd770166f40 con 0x7fd7700cf7e0 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd770167240 con 0x7fd7700ce1c0 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 0x7fd770159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 0x7fd770159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51306/0 (socket says 172.21.15.170:51306) 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 -- 172.21.15.170:0/2675932977 learned_addr learned my addr 172.21.15.170:0/2675932977 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76659c640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700ce1c0 0x7fd77015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 -- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700ce1c0 msgr2=0x7fd77015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76f59e640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7700cf7e0 0x7fd7700b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700ce1c0 0x7fd77015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 -- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7700cf7e0 msgr2=0x7fd7700b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7700cf7e0 0x7fd7700b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 -- 172.21.15.170:0/2675932977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7701fa550 con 0x7fd7700cb460 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76659c640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700ce1c0 0x7fd77015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd76ed9d640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 0x7fd770159490 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd76000b440 tx=0x7fd76000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760015020 con 0x7fd7700cb460 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7600119a0 con 0x7fd7700cb460 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7701fa830 con 0x7fd7700cb460 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760011df0 con 0x7fd7700cb460 2024-09-21T02:17:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.920+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7701fadf0 con 0x7fd7700cb460 2024-09-21T02:17:13.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.924+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7700ce5d0 con 0x7fd7700cb460 2024-09-21T02:17:13.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.924+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7fd760018020 con 0x7fd7700cb460 2024-09-21T02:17:13.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.924+0000 7fd767fff640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd750073440 0x7fd750075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:13.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.924+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7fd760094950 con 0x7fd7700cb460 2024-09-21T02:17:13.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.924+0000 7fd76659c640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd750073440 0x7fd750075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:13.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.924+0000 7fd76659c640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd750073440 0x7fd750075950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd7700af9f0 tx=0x7fd758009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:13.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:13.928+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd76001f020 con 0x7fd7700cb460 2024-09-21T02:17:14.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.112+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd76000f050 con 0x7fd7700cb460 2024-09-21T02:17:14.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.112+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7701584d0 con 0x7fd7700cb460 2024-09-21T02:17:14.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd767fff640 1 -- 172.21.15.170:0/2675932977 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) ==== 80+0+0 (secure 0 0 0) 0x7fd76000d040 con 0x7fd7700cb460 2024-09-21T02:17:14.118 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd750073440 msgr2=0x7fd750075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd750073440 0x7fd750075950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd7700af9f0 tx=0x7fd758009040 comp rx=0 tx=0).stop 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 msgr2=0x7fd770159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 0x7fd770159490 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd76000b440 tx=0x7fd76000b900 comp rx=0 tx=0).stop 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 shutdown_connections 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd750073440 0x7fd750075950 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7700ce1c0 0x7fd77015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7700cf7e0 0x7fd7700b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 --2- 172.21.15.170:0/2675932977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7700cb460 0x7fd770159490 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 >> 172.21.15.170:0/2675932977 conn(0x7fd7700b1c40 msgr2=0x7fd7700b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 shutdown_connections 2024-09-21T02:17:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.116+0000 7fd7750e7640 1 -- 172.21.15.170:0/2675932977 wait complete. 2024-09-21T02:17:14.142 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:17:14.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:14.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:14.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[40218]: fuse finished with error 0 and tester_r 0 2024-09-21T02:17:14.154 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:17:14.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:14.154 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:17:14.312 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:14.312 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:14.312 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:14.318 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:14.318 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:14.364 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 -- 172.21.15.170:0/1861543877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c070030 msgr2=0x7f308c070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 --2- 172.21.15.170:0/1861543877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c070030 0x7f308c070440 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f308c0b2b80 tx=0x7f307c01c7d0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 -- 172.21.15.170:0/1861543877 shutdown_connections 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 --2- 172.21.15.170:0/1861543877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c070030 0x7f308c070440 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 --2- 172.21.15.170:0/1861543877 >> v2:172.21.15.191:3300/0 conn(0x7f308c15f990 0x7f308c0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 -- 172.21.15.170:0/1861543877 >> 172.21.15.170:0/1861543877 conn(0x7f308c0b1c40 msgr2=0x7f308c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 -- 172.21.15.170:0/1861543877 shutdown_connections 2024-09-21T02:17:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.520+0000 7f3090c9d640 1 -- 172.21.15.170:0/1861543877 wait complete. 2024-09-21T02:17:14.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 Processor -- start 2024-09-21T02:17:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- start start 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f308c070030 0x7f308c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308c0ce4c0 0x7f308c09d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 0x7f308c09bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0ce8d0 con 0x7f308c15f990 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0cb710 con 0x7f308c0ce4c0 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f308c0cea50 con 0x7f308c070030 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 0x7f308c09bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 0x7f308c09bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51334/0 (socket says 172.21.15.170:51334) 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 -- 172.21.15.170:0/965434903 learned_addr learned my addr 172.21.15.170:0/965434903 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3089d9b640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308c0ce4c0 0x7f308c09d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308a59c640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f308c070030 0x7f308c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 -- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f308c070030 msgr2=0x7f308c15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f308c070030 0x7f308c15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 -- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308c0ce4c0 msgr2=0x7f308c09d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308c0ce4c0 0x7f308c09d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 -- 172.21.15.170:0/965434903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c15ef50 con 0x7f308c15f990 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f308ad9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 0x7f308c09bba0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f308400c730 tx=0x7f308400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084003b80 con 0x7f308c15f990 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3084003d40 con 0x7f308c15f990 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c09aaf0 con 0x7f308c15f990 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840052d0 con 0x7f308c15f990 2024-09-21T02:17:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c09b0b0 con 0x7f308c15f990 2024-09-21T02:17:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.524+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c15ecc0 con 0x7f308c15f990 2024-09-21T02:17:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.528+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f3084005680 con 0x7f308c15f990 2024-09-21T02:17:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.528+0000 7f3082ffd640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f306c073590 0x7f306c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.528+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f3084095390 con 0x7f308c15f990 2024-09-21T02:17:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.528+0000 7f308a59c640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f306c073590 0x7f306c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.528+0000 7f308a59c640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f306c073590 0x7f306c075aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f30740058d0 tx=0x7f3074005bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.532+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f308405fee0 con 0x7f308c15f990 2024-09-21T02:17:14.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.716+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f308c070440 con 0x7f308c15f990 2024-09-21T02:17:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.716+0000 7f3082ffd640 1 -- 172.21.15.170:0/965434903 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) ==== 80+0+0 (secure 0 0 0) 0x7f308405f7e0 con 0x7f308c15f990 2024-09-21T02:17:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f306c073590 msgr2=0x7f306c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f306c073590 0x7f306c075aa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f30740058d0 tx=0x7f3074005bd0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 msgr2=0x7f308c09bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 0x7f308c09bba0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f308400c730 tx=0x7f308400cbf0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 shutdown_connections 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f306c073590 0x7f306c075aa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c15f990 0x7f308c09bba0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308c0ce4c0 0x7f308c09d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 --2- 172.21.15.170:0/965434903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f308c070030 0x7f308c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 >> 172.21.15.170:0/965434903 conn(0x7f308c0b1c40 msgr2=0x7f308c155460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 shutdown_connections 2024-09-21T02:17:14.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:14.720+0000 7f3090c9d640 1 -- 172.21.15.170:0/965434903 wait complete. 2024-09-21T02:17:14.743 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:17:14.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:14.743 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:14.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34584]: fuse finished with error 0 and tester_r 0 2024-09-21T02:17:14.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:17:14.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:14.758 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:17:14.893 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:14.893 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:14.893 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:14.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:17:14.899 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:14.900 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:14.900 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:14.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:17:14.906 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:15.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 -- 172.21.15.170:0/82934660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc0b5410 msgr2=0x7fd6dc06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/82934660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc06fa30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009840 tx=0x7fd6d001c820 comp rx=0 tx=0).stop 2024-09-21T02:17:15.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 -- 172.21.15.170:0/82934660 shutdown_connections 2024-09-21T02:17:15.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/82934660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6dc0cd210 0x7fd6dc0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/82934660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc06fa30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 -- 172.21.15.170:0/82934660 >> 172.21.15.170:0/82934660 conn(0x7fd6dc0b1c40 msgr2=0x7fd6dc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:15.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 -- 172.21.15.170:0/82934660 shutdown_connections 2024-09-21T02:17:15.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.100+0000 7fd6e44b3640 1 -- 172.21.15.170:0/82934660 wait complete. 2024-09-21T02:17:15.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 Processor -- start 2024-09-21T02:17:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 -- start start 2024-09-21T02:17:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6dc0b2f40 0x7fd6dc1fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 0x7fd6dc144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc169fd0 con 0x7fd6dc15bd50 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc169e50 con 0x7fd6dc0b5410 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e44b3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd6dc16a150 con 0x7fd6dc0b2f40 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 0x7fd6dc144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 0x7fd6dc144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51340/0 (socket says 172.21.15.170:51340) 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 -- 172.21.15.170:0/1433587812 learned_addr learned my addr 172.21.15.170:0/1433587812 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e224f640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6dc0b2f40 0x7fd6dc1fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e1a4e640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 -- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6dc0b2f40 msgr2=0x7fd6dc1fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6dc0b2f40 0x7fd6dc1fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 -- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6dc0b5410 msgr2=0x7fd6dc15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc15b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 -- 172.21.15.170:0/1433587812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1fd310 con 0x7fd6dc15bd50 2024-09-21T02:17:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e1a4e640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6e2a50640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 0x7fd6dc144bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e940 tx=0x7fd6d400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4004d30 con 0x7fd6dc15bd50 2024-09-21T02:17:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6d400cb20 con 0x7fd6dc15bd50 2024-09-21T02:17:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.104+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4005020 con 0x7fd6dc15bd50 2024-09-21T02:17:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc15c160 con 0x7fd6dc15bd50 2024-09-21T02:17:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc09aa60 con 0x7fd6dc15bd50 2024-09-21T02:17:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6dc06fa30 con 0x7fd6dc15bd50 2024-09-21T02:17:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd6d40051c0 con 0x7fd6dc15bd50 2024-09-21T02:17:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6ceffd640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6c0073590 0x7fd6c0075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7fd6d4095280 con 0x7fd6dc15bd50 2024-09-21T02:17:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6e224f640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6c0073590 0x7fd6c0075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.108+0000 7fd6e224f640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6c0073590 0x7fd6c0075aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00af000 tx=0x7fd6d001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:15.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.112+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6dc15c160 con 0x7fd6dc15bd50 2024-09-21T02:17:15.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.316+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6dc09b3f0 con 0x7fd6dc15bd50 2024-09-21T02:17:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.316+0000 7fd6ceffd640 1 -- 172.21.15.170:0/1433587812 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7fd6dc1fd310 con 0x7fd6dc15bd50 2024-09-21T02:17:15.320 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:15.320 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"btime":"2024-09-21T02:17:04:005205+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:16:50.547545+0000","modified":"2024-09-21T02:16:53.589716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/3667702062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3667702062},{"type":"v1","addr":"172.21.15.191:6837","nonce":3667702062}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5889,"qdb_cluster":[5889]},"id":4}]} 2024-09-21T02:17:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 38 2024-09-21T02:17:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6c0073590 msgr2=0x7fd6c0075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6c0073590 0x7fd6c0075aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00af000 tx=0x7fd6d001b060 comp rx=0 tx=0).stop 2024-09-21T02:17:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 msgr2=0x7fd6dc144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 0x7fd6dc144bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e940 tx=0x7fd6d400ee00 comp rx=0 tx=0).stop 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 shutdown_connections 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6c0073590 0x7fd6c0075aa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6dc15bd50 0x7fd6dc144bf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6dc0b5410 0x7fd6dc15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 --2- 172.21.15.170:0/1433587812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6dc0b2f40 0x7fd6dc1fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 >> 172.21.15.170:0/1433587812 conn(0x7fd6dc0b1c40 msgr2=0x7fd6dc158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 shutdown_connections 2024-09-21T02:17:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.320+0000 7fd6e44b3640 1 -- 172.21.15.170:0/1433587812 wait complete. 2024-09-21T02:17:15.342 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f7365f640 1 -- 172.21.15.170:0/102652951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58004d50 msgr2=0x7f6f58005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f7365f640 1 --2- 172.21.15.170:0/102652951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58004d50 0x7f6f58005200 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009eb0 tx=0x7f6f6401d5d0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f71bfc640 1 -- 172.21.15.170:0/102652951 reap_dead start 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f7365f640 1 -- 172.21.15.170:0/102652951 shutdown_connections 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f7365f640 1 -- 172.21.15.170:0/102652951 >> 172.21.15.170:0/102652951 conn(0x7f6f6c09a9c0 msgr2=0x7f6f6c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f7365f640 1 -- 172.21.15.170:0/102652951 shutdown_connections 2024-09-21T02:17:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.484+0000 7f6f7365f640 1 -- 172.21.15.170:0/102652951 wait complete. 2024-09-21T02:17:15.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 Processor -- start 2024-09-21T02:17:15.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- start start 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f6c15a490 0x7f6f6c070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f6c159bd0 0x7f6f6c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 0x7f6f6c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c164da0 con 0x7f6f6c159fe0 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0ccac0 con 0x7f6f6c159bd0 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c164f20 con 0x7f6f6c15a490 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 0x7f6f6c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 0x7f6f6c0b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59748/0 (socket says 172.21.15.170:59748) 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 -- 172.21.15.170:0/2918474607 learned_addr learned my addr 172.21.15.170:0/2918474607 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f713fb640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f6c15a490 0x7f6f6c070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f70bfa640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f6c159bd0 0x7f6f6c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 -- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f6c15a490 msgr2=0x7f6f6c070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f6c15a490 0x7f6f6c070b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 -- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f6c159bd0 msgr2=0x7f6f6c0b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f6c159bd0 0x7f6f6c0b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 -- 172.21.15.170:0/2918474607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c09cb70 con 0x7f6f6c159fe0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f71bfc640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 0x7f6f6c0b1690 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6f6401de70 tx=0x7f6f64004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64009ab0 con 0x7f6f6c159fe0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f640a9510 con 0x7f6f6c159fe0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c15f220 con 0x7f6f6c159fe0 2024-09-21T02:17:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64009200 con 0x7f6f6c159fe0 2024-09-21T02:17:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c09cdf0 con 0x7f6f6c159fe0 2024-09-21T02:17:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c0cc060 con 0x7f6f6c159fe0 2024-09-21T02:17:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.488+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6f640b9030 con 0x7f6f6c159fe0 2024-09-21T02:17:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.492+0000 7f6f61ffb640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6f50073560 0x7f6f50075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.492+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f6f6412f8a0 con 0x7f6f6c159fe0 2024-09-21T02:17:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.492+0000 7f6f713fb640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6f50073560 0x7f6f50075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.492+0000 7f6f713fb640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6f50073560 0x7f6f50075a70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c15eab0 tx=0x7f6f68006ba0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:15.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.496+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6f6c09cb70 con 0x7f6f6c159fe0 2024-09-21T02:17:15.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.696+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f6c0ce9d0 con 0x7f6f6c159fe0 2024-09-21T02:17:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.696+0000 7f6f61ffb640 1 -- 172.21.15.170:0/2918474607 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7f6f6412f020 con 0x7f6f6c159fe0 2024-09-21T02:17:15.701 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:15.701 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"btime":"2024-09-21T02:17:04:005205+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:16:50.547545+0000","modified":"2024-09-21T02:16:53.589716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/3667702062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3667702062},{"type":"v1","addr":"172.21.15.191:6837","nonce":3667702062}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5889,"qdb_cluster":[5889]},"id":4}]} 2024-09-21T02:17:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 38 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6f50073560 msgr2=0x7f6f50075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6f50073560 0x7f6f50075a70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c15eab0 tx=0x7f6f68006ba0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 msgr2=0x7f6f6c0b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 0x7f6f6c0b1690 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6f6401de70 tx=0x7f6f64004f30 comp rx=0 tx=0).stop 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 shutdown_connections 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6f50073560 0x7f6f50075a70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f6c159fe0 0x7f6f6c0b1690 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f6c159bd0 0x7f6f6c0b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 --2- 172.21.15.170:0/2918474607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f6c15a490 0x7f6f6c070b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 >> 172.21.15.170:0/2918474607 conn(0x7f6f6c09a9c0 msgr2=0x7f6f6c0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 shutdown_connections 2024-09-21T02:17:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.700+0000 7f6f7365f640 1 -- 172.21.15.170:0/2918474607 wait complete. 2024-09-21T02:17:15.722 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:17:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 -- 172.21.15.170:0/2359727708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 msgr2=0x7fcea40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 --2- 172.21.15.170:0/2359727708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea40ce5d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009bc0 tx=0x7fce9c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 -- 172.21.15.170:0/2359727708 shutdown_connections 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 --2- 172.21.15.170:0/2359727708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea40ce5d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 --2- 172.21.15.170:0/2359727708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea40cb460 0x7fcea40cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 -- 172.21.15.170:0/2359727708 >> 172.21.15.170:0/2359727708 conn(0x7fcea40b1c40 msgr2=0x7fcea40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 -- 172.21.15.170:0/2359727708 shutdown_connections 2024-09-21T02:17:15.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.864+0000 7fceab1bd640 1 -- 172.21.15.170:0/2359727708 wait complete. 2024-09-21T02:17:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 Processor -- start 2024-09-21T02:17:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- start start 2024-09-21T02:17:15.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea40cb460 0x7fcea40b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea40cf7e0 0x7fcea415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41690d0 con 0x7fcea40ce1c0 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4168f50 con 0x7fcea40cf7e0 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcea4169250 con 0x7fcea40cb460 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea415dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59762/0 (socket says 172.21.15.170:59762) 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 -- 172.21.15.170:0/1780417107 learned_addr learned my addr 172.21.15.170:0/1780417107 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea8f59640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea40cb460 0x7fcea40b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea975a640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea40cf7e0 0x7fcea415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 -- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea40cb460 msgr2=0x7fcea40b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea40cb460 0x7fcea40b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 -- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea40cf7e0 msgr2=0x7fcea415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea40cf7e0 0x7fcea415e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 -- 172.21.15.170:0/1780417107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea406a8a0 con 0x7fcea40ce1c0 2024-09-21T02:17:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea3fff640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea415dcc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fce980058d0 tx=0x7fce9800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98017070 con 0x7fcea40ce1c0 2024-09-21T02:17:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce98009d20 con 0x7fcea40ce1c0 2024-09-21T02:17:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce980106c0 con 0x7fcea40ce1c0 2024-09-21T02:17:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea41f7fd0 con 0x7fcea40ce1c0 2024-09-21T02:17:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41faad0 con 0x7fcea40ce1c0 2024-09-21T02:17:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea40cacd0 con 0x7fcea40ce1c0 2024-09-21T02:17:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.868+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fce9801a070 con 0x7fcea40ce1c0 2024-09-21T02:17:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.872+0000 7fcea17fa640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce90073590 0x7fce90075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.872+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7fce98094b00 con 0x7fcea40ce1c0 2024-09-21T02:17:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.872+0000 7fcea8f59640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce90073590 0x7fce90075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.872+0000 7fcea8f59640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce90073590 0x7fce90075aa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce94001230 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:15.876+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fce9805f550 con 0x7fcea40ce1c0 2024-09-21T02:17:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.048+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcea40b22a0 con 0x7fcea40ce1c0 2024-09-21T02:17:16.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.048+0000 7fcea17fa640 1 -- 172.21.15.170:0/1780417107 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16868 (secure 0 0 0) 0x7fce9805ee50 con 0x7fcea40ce1c0 2024-09-21T02:17:16.051 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:16.051 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":61,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:07.037605+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:16:48.250782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T02:16:49.517893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.1e","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:17:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce90073590 msgr2=0x7fce90075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce90073590 0x7fce90075aa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce94001230 tx=0x7fce94009040 comp rx=0 tx=0).stop 2024-09-21T02:17:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 msgr2=0x7fcea415dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea415dcc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fce980058d0 tx=0x7fce9800dde0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 shutdown_connections 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fce90073590 0x7fce90075aa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea40cf7e0 0x7fcea415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ce1c0 0x7fcea415dcc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 --2- 172.21.15.170:0/1780417107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea40cb460 0x7fcea40b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 >> 172.21.15.170:0/1780417107 conn(0x7fcea40b1c40 msgr2=0x7fcea4156770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 shutdown_connections 2024-09-21T02:17:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.052+0000 7fceab1bd640 1 -- 172.21.15.170:0/1780417107 wait complete. 2024-09-21T02:17:16.072 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:17:16.072 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:16.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb42565640 1 -- 172.21.15.170:0/2832870824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb24004d50 msgr2=0x7fdb240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb42565640 1 --2- 172.21.15.170:0/2832870824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb24004d50 0x7fdb240051c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0cdc20 tx=0x7fdb3401d2c0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb40b02640 1 -- 172.21.15.170:0/2832870824 reap_dead start 2024-09-21T02:17:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb42565640 1 -- 172.21.15.170:0/2832870824 shutdown_connections 2024-09-21T02:17:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb42565640 1 -- 172.21.15.170:0/2832870824 >> 172.21.15.170:0/2832870824 conn(0x7fdb3c0b1c40 msgr2=0x7fdb3c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb42565640 1 -- 172.21.15.170:0/2832870824 shutdown_connections 2024-09-21T02:17:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.220+0000 7fdb42565640 1 -- 172.21.15.170:0/2832870824 wait complete. 2024-09-21T02:17:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 Processor -- start 2024-09-21T02:17:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 -- start start 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 0x7fdb3c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb3c158820 0x7fdb3c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb3c15a1d0 0x7fdb3c0c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c0cb500 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c074f00 con 0x7fdb3c15a1d0 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c0cb680 con 0x7fdb3c158820 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 0x7fdb3c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 0x7fdb3c0c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59780/0 (socket says 172.21.15.170:59780) 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 -- 172.21.15.170:0/2516415622 learned_addr learned my addr 172.21.15.170:0/2516415622 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb40b02640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb3c15a1d0 0x7fdb3c0c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 -- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb3c158820 msgr2=0x7fdb3c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3b7fe640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb3c158820 0x7fdb3c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb3c158820 0x7fdb3c159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 -- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb3c15a1d0 msgr2=0x7fdb3c0c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb3c15a1d0 0x7fdb3c0c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 -- 172.21.15.170:0/2516415622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c0ad130 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb40b02640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb3c15a1d0 0x7fdb3c0c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb3bfff640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 0x7fdb3c0c9b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00fa00 tx=0x7fdb2c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00bcb0 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb2c004030 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c004460 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c0ad410 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.224+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c0ad9d0 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.228+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3c0ce680 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.228+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fdb2c003600 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.228+0000 7fdb38ff9640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdb28073560 0x7fdb28075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.228+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7fdb2c095610 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.228+0000 7fdb3b7fe640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdb28073560 0x7fdb28075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.228+0000 7fdb3b7fe640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdb28073560 0x7fdb28075a70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0c87b0 tx=0x7fdb3000e420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:16.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.232+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdb2c05fca0 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.428+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb3c0ce2a0 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.428+0000 7fdb38ff9640 1 -- 172.21.15.170:0/2516415622 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7fdb2c0646d0 con 0x7fdb3c1f64c0 2024-09-21T02:17:16.432 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:16.432 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"btime":"2024-09-21T02:17:04:005205+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:16:50.547545+0000","modified":"2024-09-21T02:16:53.589716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/3667702062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3667702062},{"type":"v1","addr":"172.21.15.191:6837","nonce":3667702062}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5889,"qdb_cluster":[5889]},"id":4}]} 2024-09-21T02:17:16.432 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 38 2024-09-21T02:17:16.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdb28073560 msgr2=0x7fdb28075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdb28073560 0x7fdb28075a70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0c87b0 tx=0x7fdb3000e420 comp rx=0 tx=0).stop 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 msgr2=0x7fdb3c0c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 0x7fdb3c0c9b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00fa00 tx=0x7fdb2c00fec0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 shutdown_connections 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fdb28073560 0x7fdb28075a70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb3c158820 0x7fdb3c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb3c15a1d0 0x7fdb3c0c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 --2- 172.21.15.170:0/2516415622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb3c1f64c0 0x7fdb3c0c9b50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 >> 172.21.15.170:0/2516415622 conn(0x7fdb3c0b1c40 msgr2=0x7fdb3c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 shutdown_connections 2024-09-21T02:17:16.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.432+0000 7fdb42565640 1 -- 172.21.15.170:0/2516415622 wait complete. 2024-09-21T02:17:16.453 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 -- 172.21.15.170:0/2962850756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 msgr2=0x7f807415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 --2- 172.21.15.170:0/2962850756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 0x7f807415c850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8064009900 tx=0x7f806401c8b0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 -- 172.21.15.170:0/2962850756 shutdown_connections 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 --2- 172.21.15.170:0/2962850756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80740b3e30 0x7f80740b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 --2- 172.21.15.170:0/2962850756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 0x7f807415c850 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 -- 172.21.15.170:0/2962850756 >> 172.21.15.170:0/2962850756 conn(0x7f80740cee30 msgr2=0x7f80740cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 -- 172.21.15.170:0/2962850756 shutdown_connections 2024-09-21T02:17:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 -- 172.21.15.170:0/2962850756 wait complete. 2024-09-21T02:17:16.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 Processor -- start 2024-09-21T02:17:16.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.608+0000 7f8079afa640 1 -- start start 2024-09-21T02:17:16.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 0x7f8074151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80740b3e30 0x7f8074154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 0x7f8074154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f807416a470 con 0x7f8074072900 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f807416a2f0 con 0x7f80740b7ab0 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f807416a5f0 con 0x7f80740b3e30 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 0x7f8074151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 0x7f8074151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59786/0 (socket says 172.21.15.170:59786) 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 -- 172.21.15.170:0/1126791693 learned_addr learned my addr 172.21.15.170:0/1126791693 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8072ffd640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80740b3e30 0x7f8074154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8073fff640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 0x7f8074154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 -- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80740b3e30 msgr2=0x7f8074154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80740b3e30 0x7f8074154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 -- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 msgr2=0x7f8074154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 0x7f8074154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 -- 172.21.15.170:0/1126791693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80740cbf80 con 0x7f8074072900 2024-09-21T02:17:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f80737fe640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 0x7f8074151830 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f80640b0040 tx=0x7f8064005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640b2040 con 0x7f8074072900 2024-09-21T02:17:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80740cc260 con 0x7f8074072900 2024-09-21T02:17:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8064005310 con 0x7f8074072900 2024-09-21T02:17:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640046d0 con 0x7f8074072900 2024-09-21T02:17:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807415ab00 con 0x7f8074072900 2024-09-21T02:17:16.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.612+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80740b4240 con 0x7f8074072900 2024-09-21T02:17:16.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.616+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8064004890 con 0x7f8074072900 2024-09-21T02:17:16.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.616+0000 7f805bfff640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8060073590 0x7f8060075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.616+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f80640bd070 con 0x7f8074072900 2024-09-21T02:17:16.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.616+0000 7f8072ffd640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8060073590 0x7f8060075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.616+0000 7f8072ffd640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8060073590 0x7f8060075aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f807415c280 tx=0x7f80680091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:16.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.620+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80640164d0 con 0x7f8074072900 2024-09-21T02:17:16.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.812+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80740d1f70 con 0x7f8074072900 2024-09-21T02:17:16.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.812+0000 7f805bfff640 1 -- 172.21.15.170:0/1126791693 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5229 (secure 0 0 0) 0x7f80640f95e0 con 0x7f8074072900 2024-09-21T02:17:16.815 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:16.816 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"btime":"2024-09-21T02:17:04:005205+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:16:50.547545+0000","modified":"2024-09-21T02:16:53.589716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5889},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5889":{"gid":5889,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/3667702062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3667702062},{"type":"v1","addr":"172.21.15.191:6837","nonce":3667702062}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5889,"qdb_cluster":[5889]},"id":4}]} 2024-09-21T02:17:16.816 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 38 2024-09-21T02:17:16.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8060073590 msgr2=0x7f8060075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8060073590 0x7f8060075aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f807415c280 tx=0x7f80680091e0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 msgr2=0x7f8074151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 0x7f8074151830 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f80640b0040 tx=0x7f8064005de0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 shutdown_connections 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8060073590 0x7f8060075aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80740b3e30 0x7f8074154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80740b7ab0 0x7f8074154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 --2- 172.21.15.170:0/1126791693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8074072900 0x7f8074151830 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 >> 172.21.15.170:0/1126791693 conn(0x7f80740cee30 msgr2=0x7f80740b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 shutdown_connections 2024-09-21T02:17:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.816+0000 7f8079afa640 1 -- 172.21.15.170:0/1126791693 wait complete. 2024-09-21T02:17:16.836 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:17:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 -- 172.21.15.170:0/1345524517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cf7e0 msgr2=0x7f6760070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 --2- 172.21.15.170:0/1345524517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cf7e0 0x7f6760070340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6750009900 tx=0x7f675001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 -- 172.21.15.170:0/1345524517 shutdown_connections 2024-09-21T02:17:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 --2- 172.21.15.170:0/1345524517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67600cb460 0x7f67600cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 --2- 172.21.15.170:0/1345524517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cf7e0 0x7f6760070340 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 -- 172.21.15.170:0/1345524517 >> 172.21.15.170:0/1345524517 conn(0x7f67600b1c40 msgr2=0x7f67600b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 -- 172.21.15.170:0/1345524517 shutdown_connections 2024-09-21T02:17:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.984+0000 7f6765d39640 1 -- 172.21.15.170:0/1345524517 wait complete. 2024-09-21T02:17:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 Processor -- start 2024-09-21T02:17:16.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 -- start start 2024-09-21T02:17:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cb460 0x7f67600c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 0x7f6760158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67600cf7e0 0x7f67600c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f67601690d0 con 0x7f67600cc8c0 2024-09-21T02:17:16.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6760168f50 con 0x7f67600cb460 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f6765d39640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6760169250 con 0x7f67600cf7e0 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 0x7f6760158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 0x7f6760158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59798/0 (socket says 172.21.15.170:59798) 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 -- 172.21.15.170:0/3603243106 learned_addr learned my addr 172.21.15.170:0/3603243106 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675f7fe640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cb460 0x7f67600c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675ffff640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67600cf7e0 0x7f67600c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 -- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67600cf7e0 msgr2=0x7f67600c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67600cf7e0 0x7f67600c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 -- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cb460 msgr2=0x7f67600c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cb460 0x7f67600c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 -- 172.21.15.170:0/3603243106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676009dc70 con 0x7f67600cc8c0 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.988+0000 7f675effd640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 0x7f6760158570 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f675400e7a0 tx=0x7f675400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:16.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675400ccc0 con 0x7f67600cc8c0 2024-09-21T02:17:16.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6754010430 con 0x7f67600cc8c0 2024-09-21T02:17:16.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67600c46d0 con 0x7f67600cc8c0 2024-09-21T02:17:16.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754010860 con 0x7f67600cc8c0 2024-09-21T02:17:16.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67600c4c90 con 0x7f67600cc8c0 2024-09-21T02:17:16.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67600cb870 con 0x7f67600cc8c0 2024-09-21T02:17:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6754010a00 con 0x7f67600cc8c0 2024-09-21T02:17:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6747fff640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f674c073560 0x7f674c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f6754095310 con 0x7f67600cc8c0 2024-09-21T02:17:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f675f7fe640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f674c073560 0x7f674c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.992+0000 7f675f7fe640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f674c073560 0x7f674c075a70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f67500af000 tx=0x7f675001b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:17.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:16.996+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f675405f700 con 0x7f67600cc8c0 2024-09-21T02:17:17.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.172+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6760159bb0 con 0x7f67600cc8c0 2024-09-21T02:17:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.176+0000 7f6747fff640 1 -- 172.21.15.170:0/3603243106 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16868 (secure 0 0 0) 0x7f6754064130 con 0x7f67600cc8c0 2024-09-21T02:17:17.179 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:17.179 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":61,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:07.037605+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:16:48.250782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T02:16:49.517893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.1e","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:17:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.176+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f674c073560 msgr2=0x7f674c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.176+0000 7f6765d39640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f674c073560 0x7f674c075a70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f67500af000 tx=0x7f675001b120 comp rx=0 tx=0).stop 2024-09-21T02:17:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.176+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 msgr2=0x7f6760158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.176+0000 7f6765d39640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 0x7f6760158570 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f675400e7a0 tx=0x7f675400ec60 comp rx=0 tx=0).stop 2024-09-21T02:17:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 shutdown_connections 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f674c073560 0x7f674c075a70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67600cf7e0 0x7f67600c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67600cc8c0 0x7f6760158570 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 --2- 172.21.15.170:0/3603243106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67600cb460 0x7f67600c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 >> 172.21.15.170:0/3603243106 conn(0x7f67600b1c40 msgr2=0x7f67600cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 shutdown_connections 2024-09-21T02:17:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.180+0000 7f6765d39640 1 -- 172.21.15.170:0/3603243106 wait complete. 2024-09-21T02:17:17.200 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:17:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.352+0000 7f864d546640 1 -- 172.21.15.170:0/1216418257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8630003a40 msgr2=0x7f8630003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.352+0000 7f864d546640 1 --2- 172.21.15.170:0/1216418257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8630003a40 0x7f8630003eb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f863800c260 tx=0x7f863801cf40 comp rx=0 tx=0).stop 2024-09-21T02:17:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.352+0000 7f86477fe640 1 -- 172.21.15.170:0/1216418257 reap_dead start 2024-09-21T02:17:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.352+0000 7f864d546640 1 -- 172.21.15.170:0/1216418257 shutdown_connections 2024-09-21T02:17:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.352+0000 7f864d546640 1 -- 172.21.15.170:0/1216418257 >> 172.21.15.170:0/1216418257 conn(0x7f86480711c0 msgr2=0x7f86480715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 -- 172.21.15.170:0/1216418257 shutdown_connections 2024-09-21T02:17:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 -- 172.21.15.170:0/1216418257 wait complete. 2024-09-21T02:17:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 Processor -- start 2024-09-21T02:17:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 -- start start 2024-09-21T02:17:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8630003a40 0x7f8648152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 0x7f8648163a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86480b3700 0x7f86480b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8648168800 con 0x7f8648153aa0 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8648168680 con 0x7f86480b3700 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f864d546640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8648168980 con 0x7f8630003a40 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 0x7f8648163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 0x7f8648163a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59814/0 (socket says 172.21.15.170:59814) 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 -- 172.21.15.170:0/1535812615 learned_addr learned my addr 172.21.15.170:0/1535812615 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f8646ffd640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8630003a40 0x7f8648152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86477fe640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86480b3700 0x7f86480b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 -- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8630003a40 msgr2=0x7f8648152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8630003a40 0x7f8648152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 -- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86480b3700 msgr2=0x7f86480b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86480b3700 0x7f86480b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 -- 172.21.15.170:0/1535812615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481541a0 con 0x7f8648153aa0 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f8646ffd640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8630003a40 0x7f8648152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.356+0000 7f86467fc640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 0x7f8648163a00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f863c004f10 tx=0x7f863c0015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c015020 con 0x7f8648153aa0 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f863c011960 con 0x7f8648153aa0 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c011d90 con 0x7f8648153aa0 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481612e0 con 0x7f8648153aa0 2024-09-21T02:17:17.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481618a0 con 0x7f8648153aa0 2024-09-21T02:17:17.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86480d0c30 con 0x7f8648153aa0 2024-09-21T02:17:17.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f863c018020 con 0x7f8648153aa0 2024-09-21T02:17:17.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f862f7fe640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8634073560 0x7f8634075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f863c095350 con 0x7f8648153aa0 2024-09-21T02:17:17.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.360+0000 7f8646ffd640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8634073560 0x7f8634075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.364+0000 7f8646ffd640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8634073560 0x7f8634075a70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86380ab000 tx=0x7f86380a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:17.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.364+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f863c01f020 con 0x7f8648153aa0 2024-09-21T02:17:17.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.576+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f86480b4cc0 con 0x7f8648153aa0 2024-09-21T02:17:17.580 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:17:17.576+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:17:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.580+0000 7f862f7fe640 1 -- 172.21.15.170:0/1535812615 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7f863c05fe10 con 0x7f8648153aa0 2024-09-21T02:17:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:17:17.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.580+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8634073560 msgr2=0x7f8634075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.580+0000 7f864d546640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8634073560 0x7f8634075a70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86380ab000 tx=0x7f86380a7040 comp rx=0 tx=0).stop 2024-09-21T02:17:17.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 msgr2=0x7f8648163a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 0x7f8648163a00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f863c004f10 tx=0x7f863c0015e0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 shutdown_connections 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8634073560 0x7f8634075a70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86480b3700 0x7f86480b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8648153aa0 0x7f8648163a00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 --2- 172.21.15.170:0/1535812615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8630003a40 0x7f8648152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 >> 172.21.15.170:0/1535812615 conn(0x7f86480711c0 msgr2=0x7f864815b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 shutdown_connections 2024-09-21T02:17:17.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.584+0000 7f864d546640 1 -- 172.21.15.170:0/1535812615 wait complete. 2024-09-21T02:17:17.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:17:17.698 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:17:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- 172.21.15.170:0/3803740090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c0d4c60 msgr2=0x7f4b5c0d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/3803740090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c0d4c60 0x7f4b5c0d2450 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009840 tx=0x7f4b4c01c820 comp rx=0 tx=0).stop 2024-09-21T02:17:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- 172.21.15.170:0/3803740090 shutdown_connections 2024-09-21T02:17:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/3803740090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b5c0b66e0 0x7f4b5c06e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/3803740090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c0d4c60 0x7f4b5c0d2450 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- 172.21.15.170:0/3803740090 >> 172.21.15.170:0/3803740090 conn(0x7f4b5c09d8c0 msgr2=0x7f4b5c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- 172.21.15.170:0/3803740090 shutdown_connections 2024-09-21T02:17:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- 172.21.15.170:0/3803740090 wait complete. 2024-09-21T02:17:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 Processor -- start 2024-09-21T02:17:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- start start 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b5c0b9660 0x7f4b5c157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b5c0d4c60 0x7f4b5c1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 0x7f4b5c161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c16c090 con 0x7f4b5c159bb0 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c16bf10 con 0x7f4b5c0d4c60 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b61d7c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c16c210 con 0x7f4b5c0b9660 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 0x7f4b5c161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 0x7f4b5c161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59824/0 (socket says 172.21.15.170:59824) 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 -- 172.21.15.170:0/2589375890 learned_addr learned my addr 172.21.15.170:0/2589375890 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5b7fe640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b5c0b9660 0x7f4b5c157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b52ffd640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b5c0d4c60 0x7f4b5c1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 -- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b5c0b9660 msgr2=0x7f4b5c157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b5c0b9660 0x7f4b5c157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 -- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b5c0d4c60 msgr2=0x7f4b5c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b5c0d4c60 0x7f4b5c1614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.756+0000 7f4b5bfff640 1 -- 172.21.15.170:0/2589375890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c15e2b0 con 0x7f4b5c159bb0 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b5bfff640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 0x7f4b5c161a00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e940 tx=0x7f4b5400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54004d30 con 0x7f4b5c159bb0 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b5400cb20 con 0x7f4b5c159bb0 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54005020 con 0x7f4b5c159bb0 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c158410 con 0x7f4b5c159bb0 2024-09-21T02:17:17.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1fc4e0 con 0x7f4b5c159bb0 2024-09-21T02:17:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c0b6e80 con 0x7f4b5c159bb0 2024-09-21T02:17:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f4b540051c0 con 0x7f4b5c159bb0 2024-09-21T02:17:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b58ff9640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4b3c073560 0x7f4b3c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f4b54095900 con 0x7f4b5c159bb0 2024-09-21T02:17:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b5b7fe640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4b3c073560 0x7f4b3c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:17.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.760+0000 7f4b5b7fe640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4b3c073560 0x7f4b3c075a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0af000 tx=0x7f4b4c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:17.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.764+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4b54016030 con 0x7f4b5c159bb0 2024-09-21T02:17:17.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:17.964+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4b5c1fbf90 con 0x7f4b5c159bb0 2024-09-21T02:17:18.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b58ff9640 1 -- 172.21.15.170:0/2589375890 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7f4b54060370 con 0x7f4b5c159bb0 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4b3c073560 msgr2=0x7f4b3c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4b3c073560 0x7f4b3c075a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0af000 tx=0x7f4b4c01b060 comp rx=0 tx=0).stop 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 msgr2=0x7f4b5c161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 0x7f4b5c161a00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e940 tx=0x7f4b5400ee00 comp rx=0 tx=0).stop 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 shutdown_connections 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4b3c073560 0x7f4b3c075a70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c159bb0 0x7f4b5c161a00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b5c0d4c60 0x7f4b5c1614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 --2- 172.21.15.170:0/2589375890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b5c0b9660 0x7f4b5c157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 >> 172.21.15.170:0/2589375890 conn(0x7f4b5c09d8c0 msgr2=0x7f4b5c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:18.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 shutdown_connections 2024-09-21T02:17:18.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.584+0000 7f4b61d7c640 1 -- 172.21.15.170:0/2589375890 wait complete. 2024-09-21T02:17:18.614 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305bd65640 1 -- 172.21.15.170:0/1475469014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3040004d50 msgr2=0x7f3040005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305bd65640 1 --2- 172.21.15.170:0/1475469014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3040004d50 0x7f3040005160 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f30540d14a0 tx=0x7f304c01d5d0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305a302640 1 -- 172.21.15.170:0/1475469014 reap_dead start 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305bd65640 1 -- 172.21.15.170:0/1475469014 shutdown_connections 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305bd65640 1 -- 172.21.15.170:0/1475469014 >> 172.21.15.170:0/1475469014 conn(0x7f30540cef80 msgr2=0x7f30540cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305bd65640 1 -- 172.21.15.170:0/1475469014 shutdown_connections 2024-09-21T02:17:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.776+0000 7f305bd65640 1 -- 172.21.15.170:0/1475469014 wait complete. 2024-09-21T02:17:18.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 Processor -- start 2024-09-21T02:17:18.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 -- start start 2024-09-21T02:17:18.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 0x7f30540b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:18.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f305415bb50 0x7f305415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f305406df10 0x7f305406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f30540d0b30 con 0x7f305406f530 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30540d09b0 con 0x7f305415bb50 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305bd65640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f30540b8050 con 0x7f305406df10 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 0x7f30540b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 0x7f30540b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59842/0 (socket says 172.21.15.170:59842) 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 -- 172.21.15.170:0/812131414 learned_addr learned my addr 172.21.15.170:0/812131414 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f305a302640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f305406df10 0x7f305406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059300640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f305415bb50 0x7f305415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 -- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f305406df10 msgr2=0x7f305406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f305406df10 0x7f305406e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 -- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f305415bb50 msgr2=0x7f305415bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:18.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f305415bb50 0x7f305415bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 -- 172.21.15.170:0/812131414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305415c4a0 con 0x7f305406f530 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059300640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f305415bb50 0x7f305415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f3059b01640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 0x7f30540b3cf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3050006de0 tx=0x7f3050004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050015020 con 0x7f305406f530 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.780+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30500109d0 con 0x7f305406f530 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305406fb50 con 0x7f305406f530 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050010e20 con 0x7f305406f530 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30540b34f0 con 0x7f305406f530 2024-09-21T02:17:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3054071ba0 con 0x7f305406f530 2024-09-21T02:17:18.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f3050018020 con 0x7f305406f530 2024-09-21T02:17:18.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f304a7fc640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3038073560 0x7f3038075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:18.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5853+0+0 (secure 0 0 0) 0x7f3050095ae0 con 0x7f305406f530 2024-09-21T02:17:18.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f3059300640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3038073560 0x7f3038075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:18.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.784+0000 7f3059300640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3038073560 0x7f3038075a70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f305406dc80 tx=0x7f30440091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:18.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.788+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f305001f020 con 0x7f305406f530 2024-09-21T02:17:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:18.964+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30540c9200 con 0x7f305406f530 2024-09-21T02:17:19.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.592+0000 7f304a7fc640 1 -- 172.21.15.170:0/812131414 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) ==== 181+0+0 (secure 0 0 0) 0x7f305415c4a0 con 0x7f305406f530 2024-09-21T02:17:19.594 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:17:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3038073560 msgr2=0x7f3038075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3038073560 0x7f3038075a70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f305406dc80 tx=0x7f30440091e0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 msgr2=0x7f30540b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 0x7f30540b3cf0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3050006de0 tx=0x7f3050004ac0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 shutdown_connections 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3038073560 0x7f3038075a70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f305406df10 0x7f305406e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f305415bb50 0x7f305415bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 --2- 172.21.15.170:0/812131414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f305406f530 0x7f30540b3cf0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 >> 172.21.15.170:0/812131414 conn(0x7f30540cef80 msgr2=0x7f30540afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 shutdown_connections 2024-09-21T02:17:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.596+0000 7f305bd65640 1 -- 172.21.15.170:0/812131414 wait complete. 2024-09-21T02:17:19.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 -- 172.21.15.170:0/1424371517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96240d1240 msgr2=0x7f96240d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1424371517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96240d1240 0x7f96240d2290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f961c009840 tx=0x7f961c01c790 comp rx=0 tx=0).stop 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 -- 172.21.15.170:0/1424371517 shutdown_connections 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1424371517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96240d1240 0x7f96240d2290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1424371517 >> v2:172.21.15.170:3300/0 conn(0x7f96240b69d0 0x7f96240d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 -- 172.21.15.170:0/1424371517 >> 172.21.15.170:0/1424371517 conn(0x7f962408bf50 msgr2=0x7f962409d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 -- 172.21.15.170:0/1424371517 shutdown_connections 2024-09-21T02:17:19.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.792+0000 7f962bdc7640 1 -- 172.21.15.170:0/1424371517 wait complete. 2024-09-21T02:17:19.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 Processor -- start 2024-09-21T02:17:19.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 -- start start 2024-09-21T02:17:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f962406e2e0 0x7f96240b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96240b69d0 0x7f96240b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 0x7f96241612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:19.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f962416c370 con 0x7f96240d1240 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f962416c1f0 con 0x7f96240b69d0 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f962416c4f0 con 0x7f962406e2e0 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 0x7f96241612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 0x7f96241612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59872/0 (socket says 172.21.15.170:59872) 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 -- 172.21.15.170:0/1538789511 learned_addr learned my addr 172.21.15.170:0/1538789511 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f9629b63640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f962406e2e0 0x7f96240b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f9629362640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96240b69d0 0x7f96240b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 -- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f962406e2e0 msgr2=0x7f96240b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f962406e2e0 0x7f96240b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 -- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96240b69d0 msgr2=0x7f96240b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96240b69d0 0x7f96240b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 -- 172.21.15.170:0/1538789511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96241f9d10 con 0x7f96240d1240 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962a364640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 0x7f96241612a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b0040 tx=0x7f961c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0b2040 con 0x7f96240d1240 2024-09-21T02:17:19.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f961c004040 con 0x7f96240d1240 2024-09-21T02:17:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9624160200 con 0x7f96240d1240 2024-09-21T02:17:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c004470 con 0x7f96240d1240 2024-09-21T02:17:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.796+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96241607c0 con 0x7f96240d1240 2024-09-21T02:17:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.800+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96240d0d00 con 0x7f96240d1240 2024-09-21T02:17:19.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.800+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f961c0bd030 con 0x7f96240d1240 2024-09-21T02:17:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.800+0000 7f96167fc640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9608073560 0x7f9608075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.800+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5415+0+0 (secure 0 0 0) 0x7f961c12e010 con 0x7f96240d1240 2024-09-21T02:17:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.800+0000 7f9629b63640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9608073560 0x7f9608075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:19.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.800+0000 7f9629b63640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9608073560 0x7f9608075a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9618001400 tx=0x7f9618009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:19.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.804+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f961c0c4060 con 0x7f96240d1240 2024-09-21T02:17:19.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:19.980+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f96240b8900 con 0x7f96240d1240 2024-09-21T02:17:20.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.600+0000 7f96167fc640 1 -- 172.21.15.170:0/1538789511 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) ==== 169+0+0 (secure 0 0 0) 0x7f961c016410 con 0x7f96240d1240 2024-09-21T02:17:20.605 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:17:20.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9608073560 msgr2=0x7f9608075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9608073560 0x7f9608075a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9618001400 tx=0x7f9618009040 comp rx=0 tx=0).stop 2024-09-21T02:17:20.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 msgr2=0x7f96241612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 0x7f96241612a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b0040 tx=0x7f961c0056e0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 shutdown_connections 2024-09-21T02:17:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f9608073560 0x7f9608075a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96240d1240 0x7f96241612a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96240b69d0 0x7f96240b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 --2- 172.21.15.170:0/1538789511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f962406e2e0 0x7f96240b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 >> 172.21.15.170:0/1538789511 conn(0x7f962408bf50 msgr2=0x7f96240d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 shutdown_connections 2024-09-21T02:17:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.608+0000 7f962bdc7640 1 -- 172.21.15.170:0/1538789511 wait complete. 2024-09-21T02:17:20.637 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- 172.21.15.170:0/596951771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd70015f990 msgr2=0x7fd700070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 --2- 172.21.15.170:0/596951771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd70015f990 0x7fd700070030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009900 tx=0x7fd6fc01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- 172.21.15.170:0/596951771 shutdown_connections 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 --2- 172.21.15.170:0/596951771 >> v2:172.21.15.191:3300/0 conn(0x7fd700070570 0x7fd7000ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 --2- 172.21.15.170:0/596951771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd70015f990 0x7fd700070030 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- 172.21.15.170:0/596951771 >> 172.21.15.170:0/596951771 conn(0x7fd7000b1c40 msgr2=0x7fd7000b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- 172.21.15.170:0/596951771 shutdown_connections 2024-09-21T02:17:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- 172.21.15.170:0/596951771 wait complete. 2024-09-21T02:17:20.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 Processor -- start 2024-09-21T02:17:20.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- start start 2024-09-21T02:17:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700070570 0x7fd7001577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 0x7fd700156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd70015f990 0x7fd70009c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd700165fe0 con 0x7fd7000cea00 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000cb760 con 0x7fd70015f990 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd707a52640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd700166160 con 0x7fd700070570 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd704fed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 0x7fd700156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd704fed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 0x7fd700156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59884/0 (socket says 172.21.15.170:59884) 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.800+0000 7fd704fed640 1 -- 172.21.15.170:0/3613949087 learned_addr learned my addr 172.21.15.170:0/3613949087 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd7057ee640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700070570 0x7fd7001577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd705fef640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd70015f990 0x7fd70009c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd704fed640 1 -- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700070570 msgr2=0x7fd7001577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd704fed640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700070570 0x7fd7001577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd704fed640 1 -- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd70015f990 msgr2=0x7fd70009c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd704fed640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd70015f990 0x7fd70009c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd704fed640 1 -- 172.21.15.170:0/3613949087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7001583d0 con 0x7fd7000cea00 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd705fef640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd70015f990 0x7fd70009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd704fed640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 0x7fd700156ee0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e7a0 tx=0x7fd6f000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000ccc0 con 0x7fd7000cea00 2024-09-21T02:17:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6f000ce60 con 0x7fd7000cea00 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7001586b0 con 0x7fd7000cea00 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0005260 con 0x7fd7000cea00 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70009c1b0 con 0x7fd7000cea00 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd700158140 con 0x7fd7000cea00 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd6f0005400 con 0x7fd7000cea00 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd6f5ffb640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6e4073560 0x7fd6e4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7fd6f0094e70 con 0x7fd7000cea00 2024-09-21T02:17:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.804+0000 7fd7057ee640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6e4073560 0x7fd6e4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.808+0000 7fd7057ee640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6e4073560 0x7fd6e4075a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0af000 tx=0x7fd6fc01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.808+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6f005fc90 con 0x7fd7000cea00 2024-09-21T02:17:20.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.972+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fd7000cd8a0 con 0x7fd7000cea00 2024-09-21T02:17:20.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd6f5ffb640 1 -- 172.21.15.170:0/3613949087 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7fd6f005f590 con 0x7fd7000cea00 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6e4073560 msgr2=0x7fd6e4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6e4073560 0x7fd6e4075a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0af000 tx=0x7fd6fc01b120 comp rx=0 tx=0).stop 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 msgr2=0x7fd700156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 0x7fd700156ee0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e7a0 tx=0x7fd6f000ec60 comp rx=0 tx=0).stop 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 shutdown_connections 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd6e4073560 0x7fd6e4075a70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd70015f990 0x7fd70009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7000cea00 0x7fd700156ee0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 --2- 172.21.15.170:0/3613949087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd700070570 0x7fd7001577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 >> 172.21.15.170:0/3613949087 conn(0x7fd7000b1c40 msgr2=0x7fd70006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 shutdown_connections 2024-09-21T02:17:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:20.980+0000 7fd707a52640 1 -- 172.21.15.170:0/3613949087 wait complete. 2024-09-21T02:17:21.003 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-21T02:17:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/1702519537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b1230 msgr2=0x7f3b880b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/1702519537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b1230 0x7f3b880b1640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009c80 tx=0x7f3b8001cb30 comp rx=0 tx=0).stop 2024-09-21T02:17:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/1702519537 shutdown_connections 2024-09-21T02:17:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/1702519537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b1230 0x7f3b880b1640 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/1702519537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880b3fa0 0x7f3b880cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/1702519537 >> 172.21.15.170:0/1702519537 conn(0x7f3b8808bf50 msgr2=0x7f3b8809d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:21.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/1702519537 shutdown_connections 2024-09-21T02:17:21.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.148+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/1702519537 wait complete. 2024-09-21T02:17:21.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 Processor -- start 2024-09-21T02:17:21.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- start start 2024-09-21T02:17:21.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b880b1230 0x7f3b881576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 0x7f3b8815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880d0e50 0x7f3b8815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8816c0c0 con 0x7f3b880b3fa0 2024-09-21T02:17:21.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8816bf40 con 0x7f3b880b1230 2024-09-21T02:17:21.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8816c240 con 0x7f3b880d0e50 2024-09-21T02:17:21.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 0x7f3b8815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 0x7f3b8815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59900/0 (socket says 172.21.15.170:59900) 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 -- 172.21.15.170:0/67447805 learned_addr learned my addr 172.21.15.170:0/67447805 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d8b8640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b880b1230 0x7f3b881576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8e0b9640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880d0e50 0x7f3b8815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 -- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880d0e50 msgr2=0x7f3b8815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880d0e50 0x7f3b8815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 -- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b880b1230 msgr2=0x7f3b881576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b880b1230 0x7f3b881576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 -- 172.21.15.170:0/67447805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b88157c30 con 0x7f3b880b3fa0 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8e0b9640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880d0e50 0x7f3b8815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8d0b7640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 0x7f3b8815b090 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b410 tx=0x7f3b7800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78015030 con 0x7f3b880b3fa0 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b78004020 con 0x7f3b880b3fa0 2024-09-21T02:17:21.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b88157e20 con 0x7f3b880b3fa0 2024-09-21T02:17:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780044b0 con 0x7f3b880b3fa0 2024-09-21T02:17:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881f93a0 con 0x7f3b880b3fa0 2024-09-21T02:17:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.152+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b881f9cd0 con 0x7f3b880b3fa0 2024-09-21T02:17:21.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.156+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f3b78018070 con 0x7f3b880b3fa0 2024-09-21T02:17:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.156+0000 7f3b7e7fc640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b6c073560 0x7f3b6c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.156+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f3b78094190 con 0x7f3b880b3fa0 2024-09-21T02:17:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.156+0000 7f3b8d8b8640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b6c073560 0x7f3b6c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.156+0000 7f3b8d8b8640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b6c073560 0x7f3b6c075a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005fd0 tx=0x7f3b84005d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:21.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.160+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3b7805f020 con 0x7f3b880b3fa0 2024-09-21T02:17:21.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.376+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f3b880b1640 con 0x7f3b880b3fa0 2024-09-21T02:17:21.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.604+0000 7f3b7e7fc640 1 -- 172.21.15.170:0/67447805 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v210) ==== 161+0+0 (secure 0 0 0) 0x7f3b7805e920 con 0x7f3b880b3fa0 2024-09-21T02:17:21.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b6c073560 msgr2=0x7f3b6c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b6c073560 0x7f3b6c075a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005fd0 tx=0x7f3b84005d10 comp rx=0 tx=0).stop 2024-09-21T02:17:21.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 msgr2=0x7f3b8815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 0x7f3b8815b090 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b410 tx=0x7f3b7800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 shutdown_connections 2024-09-21T02:17:21.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3b6c073560 0x7f3b6c075a70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880d0e50 0x7f3b8815b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b880b3fa0 0x7f3b8815b090 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 --2- 172.21.15.170:0/67447805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b880b1230 0x7f3b881576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 >> 172.21.15.170:0/67447805 conn(0x7f3b8808bf50 msgr2=0x7f3b880b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:21.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 shutdown_connections 2024-09-21T02:17:21.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.608+0000 7f3b8fb1c640 1 -- 172.21.15.170:0/67447805 wait complete. 2024-09-21T02:17:21.632 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-21T02:17:21.632 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:17:21.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 -- 172.21.15.170:0/383611905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 msgr2=0x7ffba0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/383611905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffba0005160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4073f30 tx=0x7ffbb001d590 comp rx=0 tx=0).stop 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbb997c640 1 -- 172.21.15.170:0/383611905 reap_dead start 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 -- 172.21.15.170:0/383611905 shutdown_connections 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 -- 172.21.15.170:0/383611905 >> 172.21.15.170:0/383611905 conn(0x7ffbb409a9c0 msgr2=0x7ffbb40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 -- 172.21.15.170:0/383611905 shutdown_connections 2024-09-21T02:17:21.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 -- 172.21.15.170:0/383611905 wait complete. 2024-09-21T02:17:21.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 Processor -- start 2024-09-21T02:17:21.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.784+0000 7ffbbb3df640 1 -- start start 2024-09-21T02:17:21.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffbb40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbb40af760 0x7ffbb40afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbb40cee40 0x7ffbb40cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4160510 con 0x7ffba0004d50 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4160390 con 0x7ffbb40cee40 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb4160690 con 0x7ffbb40af760 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffbb40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffbb40c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59918/0 (socket says 172.21.15.170:59918) 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 -- 172.21.15.170:0/1163385406 learned_addr learned my addr 172.21.15.170:0/1163385406 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb997c640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbb40cee40 0x7ffbb40cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb897a640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbb40af760 0x7ffbb40afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 -- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbb40af760 msgr2=0x7ffbb40afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbb40af760 0x7ffbb40afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 -- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbb40cee40 msgr2=0x7ffbb40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbb40cee40 0x7ffbb40cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 -- 172.21.15.170:0/1163385406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b00b0 con 0x7ffba0004d50 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb897a640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbb40af760 0x7ffbb40afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbb917b640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffbb40c97d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffbac007fd0 tx=0x7ffbac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac017070 con 0x7ffba0004d50 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbac005280 con 0x7ffba0004d50 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb40ca1b0 con 0x7ffba0004d50 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00a8a0 con 0x7ffba0004d50 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb40ca770 con 0x7ffba0004d50 2024-09-21T02:17:21.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb40b1730 con 0x7ffba0004d50 2024-09-21T02:17:21.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7ffbac01a070 con 0x7ffba0004d50 2024-09-21T02:17:21.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.788+0000 7ffba9ffb640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffb98073560 0x7ffb98075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:21.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.792+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7ffbac093b60 con 0x7ffba0004d50 2024-09-21T02:17:21.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.792+0000 7ffbb897a640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffb98073560 0x7ffb98075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:21.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.792+0000 7ffbb897a640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffb98073560 0x7ffb98075a70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffba4005d10 tx=0x7ffba4005c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:21.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.796+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffbb40b00b0 con 0x7ffba0004d50 2024-09-21T02:17:21.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.964+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbb40b5d10 con 0x7ffba0004d50 2024-09-21T02:17:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.964+0000 7ffba9ffb640 1 -- 172.21.15.170:0/1163385406 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+13082 (secure 0 0 0) 0x7ffbac05e960 con 0x7ffba0004d50 2024-09-21T02:17:21.968 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:21.968 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":64,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:20.596181+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/3667702062":"2024-09-22T02:17:17.581078+0000","172.21.15.191:6836/3667702062":"2024-09-22T02:17:17.581078+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:17:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffb98073560 msgr2=0x7ffb98075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffb98073560 0x7ffb98075a70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffba4005d10 tx=0x7ffba4005c60 comp rx=0 tx=0).stop 2024-09-21T02:17:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 msgr2=0x7ffbb40c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffbb40c97d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffbac007fd0 tx=0x7ffbac00cbb0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 shutdown_connections 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ffb98073560 0x7ffb98075a70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbb40af760 0x7ffbb40afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbb40cee40 0x7ffbb40cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 --2- 172.21.15.170:0/1163385406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffba0004d50 0x7ffbb40c97d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 >> 172.21.15.170:0/1163385406 conn(0x7ffbb409a9c0 msgr2=0x7ffbb40716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 shutdown_connections 2024-09-21T02:17:21.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:21.968+0000 7ffbbb3df640 1 -- 172.21.15.170:0/1163385406 wait complete. 2024-09-21T02:17:21.989 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:17:21.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 -- 172.21.15.170:0/2804192688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 msgr2=0x7fa93c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2804192688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c0cb280 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0b61b0 tx=0x7fa93001c800 comp rx=0 tx=0).stop 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 -- 172.21.15.170:0/2804192688 shutdown_connections 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2804192688 >> v2:172.21.15.191:3300/0 conn(0x7fa93c074680 0x7fa93c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2804192688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c0cb280 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 -- 172.21.15.170:0/2804192688 >> 172.21.15.170:0/2804192688 conn(0x7fa93c0d0170 msgr2=0x7fa93c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 -- 172.21.15.170:0/2804192688 shutdown_connections 2024-09-21T02:17:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.144+0000 7fa941c5a640 1 -- 172.21.15.170:0/2804192688 wait complete. 2024-09-21T02:17:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 Processor -- start 2024-09-21T02:17:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 -- start start 2024-09-21T02:17:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa93c074680 0x7fa93c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa93c0cf7d0 0x7fa93c0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c168ec0 con 0x7fa93c074270 2024-09-21T02:17:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa93c0cc2d0 con 0x7fa93c0cf7d0 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa93c169040 con 0x7fa93c074680 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59924/0 (socket says 172.21.15.170:59924) 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 -- 172.21.15.170:0/2103598438 learned_addr learned my addr 172.21.15.170:0/2103598438 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93affd640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa93c074680 0x7fa93c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93bfff640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa93c0cf7d0 0x7fa93c0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 -- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa93c074680 msgr2=0x7fa93c1511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa93c074680 0x7fa93c1511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 -- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa93c0cf7d0 msgr2=0x7fa93c0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa93c0cf7d0 0x7fa93c0c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 -- 172.21.15.170:0/2103598438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93c0b2530 con 0x7fa93c074270 2024-09-21T02:17:22.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa93b7fe640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c150cb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c400 tx=0x7fa92c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:22.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c011790 con 0x7fa93c074270 2024-09-21T02:17:22.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa92c017070 con 0x7fa93c074270 2024-09-21T02:17:22.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93c156150 con 0x7fa93c074270 2024-09-21T02:17:22.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c011cc0 con 0x7fa93c074270 2024-09-21T02:17:22.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.148+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93c156710 con 0x7fa93c074270 2024-09-21T02:17:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.152+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93c073b60 con 0x7fa93c074270 2024-09-21T02:17:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.152+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fa92c01c460 con 0x7fa93c074270 2024-09-21T02:17:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.152+0000 7fa923fff640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa928073560 0x7fa928075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.152+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7fa92c0942e0 con 0x7fa93c074270 2024-09-21T02:17:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.152+0000 7fa93affd640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa928073560 0x7fa928075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.152+0000 7fa93affd640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa928073560 0x7fa928075a70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9300af000 tx=0x7fa93001b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:22.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.156+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa92c05f170 con 0x7fa93c074270 2024-09-21T02:17:22.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.328+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa93c0b3ff0 con 0x7fa93c074270 2024-09-21T02:17:22.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.328+0000 7fa923fff640 1 -- 172.21.15.170:0/2103598438 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7fa92c05ea70 con 0x7fa93c074270 2024-09-21T02:17:22.331 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa928073560 msgr2=0x7fa928075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa928073560 0x7fa928075a70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9300af000 tx=0x7fa93001b040 comp rx=0 tx=0).stop 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 msgr2=0x7fa93c150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c150cb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c400 tx=0x7fa92c00c8c0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 shutdown_connections 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fa928073560 0x7fa928075a70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa93c074680 0x7fa93c1511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa93c0cf7d0 0x7fa93c0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 --2- 172.21.15.170:0/2103598438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa93c074270 0x7fa93c150cb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 >> 172.21.15.170:0/2103598438 conn(0x7fa93c0d0170 msgr2=0x7fa93c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:22.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 shutdown_connections 2024-09-21T02:17:22.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.332+0000 7fa941c5a640 1 -- 172.21.15.170:0/2103598438 wait complete. 2024-09-21T02:17:22.353 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- 172.21.15.170:0/3453992285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc104003a40 msgr2=0x7fc104003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 --2- 172.21.15.170:0/3453992285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc104003a40 0x7fc104003eb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc1140016f0 tx=0x7fc11401c760 comp rx=0 tx=0).stop 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc11bfff640 1 -- 172.21.15.170:0/3453992285 reap_dead start 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- 172.21.15.170:0/3453992285 shutdown_connections 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- 172.21.15.170:0/3453992285 >> 172.21.15.170:0/3453992285 conn(0x7fc11c09a9c0 msgr2=0x7fc11c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- 172.21.15.170:0/3453992285 shutdown_connections 2024-09-21T02:17:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- 172.21.15.170:0/3453992285 wait complete. 2024-09-21T02:17:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 Processor -- start 2024-09-21T02:17:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- start start 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc104003a40 0x7fc11c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 0x7fc11c0b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc11c0b2f50 0x7fc11c09b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c168ca0 con 0x7fc11c1567a0 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c168b20 con 0x7fc104003a40 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc122160640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c168e20 con 0x7fc11c0b2f50 2024-09-21T02:17:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc11affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 0x7fc11c0b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc11affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 0x7fc11c0b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59940/0 (socket says 172.21.15.170:59940) 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc11affd640 1 -- 172.21.15.170:0/3862123370 learned_addr learned my addr 172.21.15.170:0/3862123370 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.500+0000 7fc11bfff640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc11c0b2f50 0x7fc11c09b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11b7fe640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc104003a40 0x7fc11c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11affd640 1 -- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc11c0b2f50 msgr2=0x7fc11c09b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11affd640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc11c0b2f50 0x7fc11c09b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11affd640 1 -- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc104003a40 msgr2=0x7fc11c156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11affd640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc104003a40 0x7fc11c156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11affd640 1 -- 172.21.15.170:0/3862123370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c0b3360 con 0x7fc11c1567a0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11b7fe640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc104003a40 0x7fc11c156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11affd640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 0x7fc11c0b2a10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc110001dc0 tx=0x7fc11000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110018070 con 0x7fc11c1567a0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc110003b90 con 0x7fc11c1567a0 2024-09-21T02:17:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c156580 con 0x7fc11c1567a0 2024-09-21T02:17:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11000b840 con 0x7fc11c1567a0 2024-09-21T02:17:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c1f70f0 con 0x7fc11c1567a0 2024-09-21T02:17:22.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c0cd0f0 con 0x7fc11c1567a0 2024-09-21T02:17:22.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fc11001b070 con 0x7fc11c1567a0 2024-09-21T02:17:22.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc103fff640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc108073560 0x7fc108075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:22.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7fc110093f30 con 0x7fc11c1567a0 2024-09-21T02:17:22.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11b7fe640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc108073560 0x7fc108075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:22.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.504+0000 7fc11b7fe640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc108073560 0x7fc108075a70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc1140ac000 tx=0x7fc114019560 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:22.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.508+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc11c0b3360 con 0x7fc11c1567a0 2024-09-21T02:17:22.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:22.744+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fc11c1f6ba0 con 0x7fc11c1567a0 2024-09-21T02:17:23.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.616+0000 7fc103fff640 1 -- 172.21.15.170:0/3862123370 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v212) ==== 134+0+0 (secure 0 0 0) 0x7fc11c0d1cb0 con 0x7fc11c1567a0 2024-09-21T02:17:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc108073560 msgr2=0x7fc108075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc108073560 0x7fc108075a70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc1140ac000 tx=0x7fc114019560 comp rx=0 tx=0).stop 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 msgr2=0x7fc11c0b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 0x7fc11c0b2a10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc110001dc0 tx=0x7fc11000df80 comp rx=0 tx=0).stop 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 shutdown_connections 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc108073560 0x7fc108075a70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc11c0b2f50 0x7fc11c09b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc11c1567a0 0x7fc11c0b2a10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 --2- 172.21.15.170:0/3862123370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc104003a40 0x7fc11c156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 >> 172.21.15.170:0/3862123370 conn(0x7fc11c09a9c0 msgr2=0x7fc11c0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 shutdown_connections 2024-09-21T02:17:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.620+0000 7fc122160640 1 -- 172.21.15.170:0/3862123370 wait complete. 2024-09-21T02:17:23.647 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 -- 172.21.15.170:0/424264759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff79c004810 msgr2=0x7ff79c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/424264759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff79c004810 0x7ff79c004c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4160550 tx=0x7ff7a801d250 comp rx=0 tx=0).stop 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7b8b77640 1 -- 172.21.15.170:0/424264759 reap_dead start 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 -- 172.21.15.170:0/424264759 shutdown_connections 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 -- 172.21.15.170:0/424264759 >> 172.21.15.170:0/424264759 conn(0x7ff7b40b1c40 msgr2=0x7ff7b40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 -- 172.21.15.170:0/424264759 shutdown_connections 2024-09-21T02:17:23.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 -- 172.21.15.170:0/424264759 wait complete. 2024-09-21T02:17:23.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 Processor -- start 2024-09-21T02:17:23.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.804+0000 7ff7ba5da640 1 -- start start 2024-09-21T02:17:23.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff79c004810 0x7ff7b40ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:23.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 0x7ff7b4158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:23.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b40b3a40 0x7ff7b40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:23.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40c3980 con 0x7ff7b41f43f0 2024-09-21T02:17:23.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40c3800 con 0x7ff79c004810 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b40cf760 con 0x7ff7b40b3a40 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 0x7ff7b4158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 0x7ff7b4158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59948/0 (socket says 172.21.15.170:59948) 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 -- 172.21.15.170:0/3025766843 learned_addr learned my addr 172.21.15.170:0/3025766843 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b3fff640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff79c004810 0x7ff7b40ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b8b77640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b40b3a40 0x7ff7b40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 -- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b40b3a40 msgr2=0x7ff7b40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b40b3a40 0x7ff7b40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 -- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff79c004810 msgr2=0x7ff7b40ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff79c004810 0x7ff7b40ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 -- 172.21.15.170:0/3025766843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b40b4390 con 0x7ff7b41f43f0 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b8b77640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b40b3a40 0x7ff7b40b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b37fe640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 0x7ff7b4158f00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009a80 tx=0x7ff7a8005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:23.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80b1040 con 0x7ff7b41f43f0 2024-09-21T02:17:23.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7a8004110 con 0x7ff7b41f43f0 2024-09-21T02:17:23.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b409aaf0 con 0x7ff7b41f43f0 2024-09-21T02:17:23.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80ac7b0 con 0x7ff7b41f43f0 2024-09-21T02:17:23.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b409b070 con 0x7ff7b41f43f0 2024-09-21T02:17:23.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b40b3430 con 0x7ff7b41f43f0 2024-09-21T02:17:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.808+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7ff7a80046d0 con 0x7ff7b41f43f0 2024-09-21T02:17:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.812+0000 7ff7b0ff9640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7a0073590 0x7ff7a0075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.812+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7ff7a812e830 con 0x7ff7b41f43f0 2024-09-21T02:17:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.812+0000 7ff7b3fff640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7a0073590 0x7ff7a0075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:23.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.812+0000 7ff7b3fff640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7a0073590 0x7ff7a0075aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001a40 tx=0x7ff7a4001120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:23.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.816+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff7b40b4390 con 0x7ff7b41f43f0 2024-09-21T02:17:23.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.988+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff7b406fb90 con 0x7ff7b41f43f0 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.988+0000 7ff7b0ff9640 1 -- 172.21.15.170:0/3025766843 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7ff7a8016f60 con 0x7ff7b41f43f0 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:17:23.993 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:17:23.994 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:17:23.995 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7a0073590 msgr2=0x7ff7a0075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7a0073590 0x7ff7a0075aa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001a40 tx=0x7ff7a4001120 comp rx=0 tx=0).stop 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 msgr2=0x7ff7b4158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 0x7ff7b4158f00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009a80 tx=0x7ff7a8005600 comp rx=0 tx=0).stop 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 shutdown_connections 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7ff7a0073590 0x7ff7a0075aa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7b40b3a40 0x7ff7b40b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7b41f43f0 0x7ff7b4158f00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 --2- 172.21.15.170:0/3025766843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff79c004810 0x7ff7b40ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 >> 172.21.15.170:0/3025766843 conn(0x7ff7b40b1c40 msgr2=0x7ff7b415c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:23.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 shutdown_connections 2024-09-21T02:17:23.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:23.992+0000 7ff7ba5da640 1 -- 172.21.15.170:0/3025766843 wait complete. 2024-09-21T02:17:24.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:17:24.150 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:17:24.152 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:17:24.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 -- 172.21.15.170:0/893449453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c0d4c20 msgr2=0x7f1a2c1684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 --2- 172.21.15.170:0/893449453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c0d4c20 0x7f1a2c1684e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009900 tx=0x7f1a1c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 -- 172.21.15.170:0/893449453 shutdown_connections 2024-09-21T02:17:24.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 --2- 172.21.15.170:0/893449453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c0d4c20 0x7f1a2c1684e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 --2- 172.21.15.170:0/893449453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0d17c0 0x7f1a2c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 -- 172.21.15.170:0/893449453 >> 172.21.15.170:0/893449453 conn(0x7f1a2c08bf50 msgr2=0x7f1a2c09d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:24.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 -- 172.21.15.170:0/893449453 shutdown_connections 2024-09-21T02:17:24.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.168+0000 7f1a32b44640 1 -- 172.21.15.170:0/893449453 wait complete. 2024-09-21T02:17:24.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 Processor -- start 2024-09-21T02:17:24.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- start start 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0d17c0 0x7f1a2c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0d1bd0 0x7f1a2c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 0x7f1a2c0b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c16bff0 con 0x7f1a2c15a300 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c16be70 con 0x7f1a2c0d1bd0 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2c16c170 con 0x7f1a2c0d17c0 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 0x7f1a2c0b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 0x7f1a2c0b73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59974/0 (socket says 172.21.15.170:59974) 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 -- 172.21.15.170:0/2717445706 learned_addr learned my addr 172.21.15.170:0/2717445706 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a2bfff640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0d1bd0 0x7f1a2c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a308e0640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0d17c0 0x7f1a2c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 -- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0d17c0 msgr2=0x7f1a2c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0d17c0 0x7f1a2c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 -- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0d1bd0 msgr2=0x7f1a2c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0d1bd0 0x7f1a2c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 -- 172.21.15.170:0/2717445706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c0764c0 con 0x7f1a2c15a300 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a310e1640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 0x7f1a2c0b73b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b0040 tx=0x7f1a1c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0b2040 con 0x7f1a2c15a300 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a1c004320 con 0x7f1a2c15a300 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0052d0 con 0x7f1a2c15a300 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c0a2470 con 0x7f1a2c15a300 2024-09-21T02:17:24.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c0cdef0 con 0x7f1a2c15a300 2024-09-21T02:17:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.172+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c0b3a10 con 0x7f1a2c15a300 2024-09-21T02:17:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.176+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f1a1c005470 con 0x7f1a2c15a300 2024-09-21T02:17:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.176+0000 7f1a297fa640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a180734d0 0x7f1a180759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.176+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f1a1c12e3a0 con 0x7f1a2c15a300 2024-09-21T02:17:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.176+0000 7f1a308e0640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a180734d0 0x7f1a180759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.176+0000 7f1a308e0640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a180734d0 0x7f1a180759e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009ec0 tx=0x7f1a24009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:24.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.180+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a1c0bd060 con 0x7f1a2c15a300 2024-09-21T02:17:24.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.348+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1a2c15c2f0 con 0x7f1a2c15a300 2024-09-21T02:17:24.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.352+0000 7f1a297fa640 1 -- 172.21.15.170:0/2717445706 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f1a1c0164d0 con 0x7f1a2c15a300 2024-09-21T02:17:24.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a180734d0 msgr2=0x7f1a180759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a180734d0 0x7f1a180759e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009ec0 tx=0x7f1a24009500 comp rx=0 tx=0).stop 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 msgr2=0x7f1a2c0b73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 0x7f1a2c0b73b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b0040 tx=0x7f1a1c005df0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 shutdown_connections 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1a180734d0 0x7f1a180759e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a2c15a300 0x7f1a2c0b73b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a2c0d1bd0 0x7f1a2c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 --2- 172.21.15.170:0/2717445706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a2c0d17c0 0x7f1a2c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 >> 172.21.15.170:0/2717445706 conn(0x7f1a2c08bf50 msgr2=0x7f1a2c0b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 shutdown_connections 2024-09-21T02:17:24.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.356+0000 7f1a32b44640 1 -- 172.21.15.170:0/2717445706 wait complete. 2024-09-21T02:17:24.379 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:17:24.379 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:17:24.446 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:17:24.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:17:24.529 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:24.529 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:24.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:17:24.535 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:24.535 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:24.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:17:24.541 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:24.541 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:24.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:17:24.581 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:24.581 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:24.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:17:24.588 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:17:24.588 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:24.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 -- 172.21.15.170:0/3353820123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c280b25f0 msgr2=0x7f5c280cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 --2- 172.21.15.170:0/3353820123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c280b25f0 0x7f5c280cf950 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5c280d1240 tx=0x7f5c1801c8a0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 -- 172.21.15.170:0/3353820123 shutdown_connections 2024-09-21T02:17:24.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 --2- 172.21.15.170:0/3353820123 >> v2:172.21.15.191:3300/0 conn(0x7f5c2815faf0 0x7f5c2815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 --2- 172.21.15.170:0/3353820123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c280b25f0 0x7f5c280cf950 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 -- 172.21.15.170:0/3353820123 >> 172.21.15.170:0/3353820123 conn(0x7f5c2809a9c0 msgr2=0x7f5c280d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:24.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 -- 172.21.15.170:0/3353820123 shutdown_connections 2024-09-21T02:17:24.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.780+0000 7f5c2e279640 1 -- 172.21.15.170:0/3353820123 wait complete. 2024-09-21T02:17:24.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 Processor -- start 2024-09-21T02:17:24.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 -- start start 2024-09-21T02:17:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c280b25f0 0x7f5c28157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 0x7f5c2815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c28160440 0x7f5c2815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281694b0 con 0x7f5c2815faf0 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28169330 con 0x7f5c280b25f0 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5c28169630 con 0x7f5c28160440 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 0x7f5c2815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 0x7f5c2815a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59998/0 (socket says 172.21.15.170:59998) 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 -- 172.21.15.170:0/718040614 learned_addr learned my addr 172.21.15.170:0/718040614 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2c816640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c28160440 0x7f5c2815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c27fff640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c280b25f0 0x7f5c28157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 -- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c28160440 msgr2=0x7f5c2815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c28160440 0x7f5c2815aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 -- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c280b25f0 msgr2=0x7f5c28157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c280b25f0 0x7f5c28157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 -- 172.21.15.170:0/718040614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c281fd750 con 0x7f5c2815faf0 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c277fe640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 0x7f5c2815a580 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000caa0 tx=0x7f5c2000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20007dc0 con 0x7f5c2815faf0 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c20004c20 con 0x7f5c2815faf0 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c281f93d0 con 0x7f5c2815faf0 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20005280 con 0x7f5c2815faf0 2024-09-21T02:17:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.784+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c281f9990 con 0x7f5c2815faf0 2024-09-21T02:17:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.788+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c281fd4c0 con 0x7f5c2815faf0 2024-09-21T02:17:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.788+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5c20005420 con 0x7f5c2815faf0 2024-09-21T02:17:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.788+0000 7f5c24ff9640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5c08073560 0x7f5c08075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.788+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f5c200951b0 con 0x7f5c2815faf0 2024-09-21T02:17:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.788+0000 7f5c27fff640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5c08073560 0x7f5c08075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:24.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.788+0000 7f5c27fff640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5c08073560 0x7f5c08075a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c180af000 tx=0x7f5c1801b0e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.792+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5c20014070 con 0x7f5c2815faf0 2024-09-21T02:17:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.992+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c280cf950 con 0x7f5c2815faf0 2024-09-21T02:17:24.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.992+0000 7f5c24ff9640 1 -- 172.21.15.170:0/718040614 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+3917 (secure 0 0 0) 0x7f5c2005feb0 con 0x7f5c2815faf0 2024-09-21T02:17:24.996 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:24.996 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":40,"btime":"2024-09-21T02:17:18:584134+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/190253315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":190253315},{"type":"v1","addr":"172.21.15.170:6834","nonce":190253315}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5009,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1954806063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1954806063},{"type":"v1","addr":"172.21.15.191:6835","nonce":1954806063}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5150,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4145551669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4145551669},{"type":"v1","addr":"172.21.15.191:6837","nonce":4145551669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/105636451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":105636451},{"type":"v1","addr":"172.21.15.170:6835","nonce":105636451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-09-21T02:17:24.996 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 40 2024-09-21T02:17:24.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5c08073560 msgr2=0x7f5c08075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5c08073560 0x7f5c08075a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c180af000 tx=0x7f5c1801b0e0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 msgr2=0x7f5c2815a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:24.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 0x7f5c2815a580 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000caa0 tx=0x7f5c2000cf60 comp rx=0 tx=0).stop 2024-09-21T02:17:24.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 shutdown_connections 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5c08073560 0x7f5c08075a70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c28160440 0x7f5c2815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c2815faf0 0x7f5c2815a580 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 --2- 172.21.15.170:0/718040614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c280b25f0 0x7f5c28157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 >> 172.21.15.170:0/718040614 conn(0x7f5c2809a9c0 msgr2=0x7f5c2815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 shutdown_connections 2024-09-21T02:17:24.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:24.996+0000 7f5c2e279640 1 -- 172.21.15.170:0/718040614 wait complete. 2024-09-21T02:17:25.017 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:17:25.017 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:17:25.017 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:17:25.017 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:17:25.017 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:17:25.017 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:17:25.018 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:17:25.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:17:25.018 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:17:25.018 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:17:25.018 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:17:25.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:17:25.018 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:17:25.018 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:17:25.019 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:17:25.019 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:17:25.035 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:17:25.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:17:25.036 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:17:25.036 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:17:25.038 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:17:25.038 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:17:25.040 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:17:25.040 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:17:25.078 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:17:25.078 DEBUG:teuthology.parallel:result is None 2024-09-21T02:17:25.079 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:17:25.079 DEBUG:teuthology.parallel:result is None 2024-09-21T02:17:25.084 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:17:25.084 DEBUG:teuthology.parallel:result is None 2024-09-21T02:17:25.085 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:17:25.085 DEBUG:teuthology.parallel:result is None 2024-09-21T02:17:25.085 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:17:25.164 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:17:25.169 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:17:25.180 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:17:25.182 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:17:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 -- 172.21.15.170:0/2271053876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 msgr2=0x7f2f740cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 --2- 172.21.15.170:0/2271053876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f740cb200 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009c80 tx=0x7f2f6c01cd70 comp rx=0 tx=0).stop 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 -- 172.21.15.170:0/2271053876 shutdown_connections 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 --2- 172.21.15.170:0/2271053876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f740cb200 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 --2- 172.21.15.170:0/2271053876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f7415f990 0x7f2f74070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 -- 172.21.15.170:0/2271053876 >> 172.21.15.170:0/2271053876 conn(0x7f2f740b1c40 msgr2=0x7f2f740b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 -- 172.21.15.170:0/2271053876 shutdown_connections 2024-09-21T02:17:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.228+0000 7f2f79d27640 1 -- 172.21.15.170:0/2271053876 wait complete. 2024-09-21T02:17:25.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 Processor -- start 2024-09-21T02:17:25.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 -- start start 2024-09-21T02:17:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f74070570 0x7f2f7409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f741f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7415f990 0x7f2f7414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f740ccb70 con 0x7f2f740cadf0 2024-09-21T02:17:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2f740cccf0 con 0x7f2f7415f990 2024-09-21T02:17:25.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2f740cce70 con 0x7f2f74070570 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f741f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f741f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60060/0 (socket says 172.21.15.170:60060) 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 -- 172.21.15.170:0/828070770 learned_addr learned my addr 172.21.15.170:0/828070770 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f737fe640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f74070570 0x7f2f7409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f73fff640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7415f990 0x7f2f7414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 -- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f74070570 msgr2=0x7f2f7409a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f74070570 0x7f2f7409a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 -- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7415f990 msgr2=0x7f2f7414d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7415f990 0x7f2f7414d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 -- 172.21.15.170:0/828070770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741f93e0 con 0x7f2f740cadf0 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f73fff640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7415f990 0x7f2f7414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f72ffd640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f741f8ea0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c3c0 tx=0x7f2f6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:25.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68017070 con 0x7f2f740cadf0 2024-09-21T02:17:25.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f680053c0 con 0x7f2f740cadf0 2024-09-21T02:17:25.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68003860 con 0x7f2f740cadf0 2024-09-21T02:17:25.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741f9e20 con 0x7f2f740cadf0 2024-09-21T02:17:25.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.232+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f741f9b40 con 0x7f2f740cadf0 2024-09-21T02:17:25.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.236+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f741f95b0 con 0x7f2f740cadf0 2024-09-21T02:17:25.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.236+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2f6801a070 con 0x7f2f740cadf0 2024-09-21T02:17:25.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.236+0000 7f2f5bfff640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2f60073560 0x7f2f60075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.236+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4943+0+0 (secure 0 0 0) 0x7f2f68093810 con 0x7f2f740cadf0 2024-09-21T02:17:25.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.236+0000 7f2f737fe640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2f60073560 0x7f2f60075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.236+0000 7f2f737fe640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2f60073560 0x7f2f60075a70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009b40 tx=0x7f2f640091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:25.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.240+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2f740cb740 con 0x7f2f740cadf0 2024-09-21T02:17:25.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.412+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2f740cb200 con 0x7f2f740cadf0 2024-09-21T02:17:25.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.636+0000 7f2f5bfff640 1 -- 172.21.15.170:0/828070770 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) ==== 97+0+0 (secure 0 0 0) 0x7f2f6805dfc0 con 0x7f2f740cadf0 2024-09-21T02:17:25.641 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2f60073560 msgr2=0x7f2f60075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2f60073560 0x7f2f60075a70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009b40 tx=0x7f2f640091e0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 msgr2=0x7f2f741f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f741f8ea0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c3c0 tx=0x7f2f6800c880 comp rx=0 tx=0).stop 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 shutdown_connections 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2f60073560 0x7f2f60075a70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f7415f990 0x7f2f7414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f740cadf0 0x7f2f741f8ea0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 --2- 172.21.15.170:0/828070770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f74070570 0x7f2f7409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 >> 172.21.15.170:0/828070770 conn(0x7f2f740b1c40 msgr2=0x7f2f740d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 shutdown_connections 2024-09-21T02:17:25.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.640+0000 7f2f79d27640 1 -- 172.21.15.170:0/828070770 wait complete. 2024-09-21T02:17:25.666 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:17:25.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 -- 172.21.15.170:0/2917497780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cb460 msgr2=0x7fc2980cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 --2- 172.21.15.170:0/2917497780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cb460 0x7fc2980cb870 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc288009840 tx=0x7fc28801c820 comp rx=0 tx=0).stop 2024-09-21T02:17:25.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 -- 172.21.15.170:0/2917497780 shutdown_connections 2024-09-21T02:17:25.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 --2- 172.21.15.170:0/2917497780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cb460 0x7fc2980cb870 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 --2- 172.21.15.170:0/2917497780 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2980cf7e0 0x7fc298070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 -- 172.21.15.170:0/2917497780 >> 172.21.15.170:0/2917497780 conn(0x7fc2980b1c40 msgr2=0x7fc2980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:25.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 -- 172.21.15.170:0/2917497780 shutdown_connections 2024-09-21T02:17:25.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 -- 172.21.15.170:0/2917497780 wait complete. 2024-09-21T02:17:25.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 Processor -- start 2024-09-21T02:17:25.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.816+0000 7fc2a0538640 1 -- start start 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2980cb460 0x7fc29815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 0x7fc29814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2980cf7e0 0x7fc29815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc298169120 con 0x7fc2980cc8c0 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc298168fa0 con 0x7fc2980cf7e0 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc2981692a0 con 0x7fc2980cb460 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 0x7fc29814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 0x7fc29814b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47350/0 (socket says 172.21.15.170:47350) 2024-09-21T02:17:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 -- 172.21.15.170:0/3326416792 learned_addr learned my addr 172.21.15.170:0/3326416792 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29e2d4640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2980cb460 0x7fc29815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29ead5640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2980cf7e0 0x7fc29815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 -- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2980cb460 msgr2=0x7fc29815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2980cb460 0x7fc29815bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 -- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2980cf7e0 msgr2=0x7fc29815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2980cf7e0 0x7fc29815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 -- 172.21.15.170:0/3326416792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29815c8e0 con 0x7fc2980cc8c0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc29dad3640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 0x7fc29814b430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc28801aed0 tx=0x7fc288004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2880bb070 con 0x7fc2980cc8c0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc288005030 con 0x7fc2980cc8c0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29815cbc0 con 0x7fc2980cc8c0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2880b2040 con 0x7fc2980cc8c0 2024-09-21T02:17:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29809c0e0 con 0x7fc2980cc8c0 2024-09-21T02:17:25.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29815c650 con 0x7fc2980cc8c0 2024-09-21T02:17:25.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fc2880be070 con 0x7fc2980cc8c0 2024-09-21T02:17:25.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc28effd640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc27c073560 0x7fc27c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:25.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.820+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7fc28812e090 con 0x7fc2980cc8c0 2024-09-21T02:17:25.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.824+0000 7fc29e2d4640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc27c073560 0x7fc27c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:25.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.824+0000 7fc29e2d4640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc27c073560 0x7fc27c075a70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc294005fd0 tx=0x7fc294005d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:25.824+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc288016410 con 0x7fc2980cc8c0 2024-09-21T02:17:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.044+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc298070340 con 0x7fc2980cc8c0 2024-09-21T02:17:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.044+0000 7fc28effd640 1 -- 172.21.15.170:0/3326416792 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5617 (secure 0 0 0) 0x7fc2880f8f70 con 0x7fc2980cc8c0 2024-09-21T02:17:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:17:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:17:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:17:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:17:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:17:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:17:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.alice", 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCoLO5mVA40BBAA9j6DCK0ZfbzWtDD9jy8NDg==", 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol41760", 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:26.063 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:17:26.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc27c073560 msgr2=0x7fc27c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc27c073560 0x7fc27c075a70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc294005fd0 tx=0x7fc294005d10 comp rx=0 tx=0).stop 2024-09-21T02:17:26.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 msgr2=0x7fc29814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 0x7fc29814b430 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc28801aed0 tx=0x7fc288004040 comp rx=0 tx=0).stop 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 shutdown_connections 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fc27c073560 0x7fc27c075a70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2980cf7e0 0x7fc29815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2980cc8c0 0x7fc29814b430 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 --2- 172.21.15.170:0/3326416792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2980cb460 0x7fc29815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 >> 172.21.15.170:0/3326416792 conn(0x7fc2980b1c40 msgr2=0x7fc2980cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 shutdown_connections 2024-09-21T02:17:26.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.048+0000 7fc2a0538640 1 -- 172.21.15.170:0/3326416792 wait complete. 2024-09-21T02:17:26.070 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-21T02:17:26.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.224+0000 7f740a50c640 1 -- 172.21.15.170:0/4038627017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 msgr2=0x7f73ec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.224+0000 7f740a50c640 1 --2- 172.21.15.170:0/4038627017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f73ec004c20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f73f80016f0 tx=0x7f73f801c7b0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.224+0000 7f7408aa9640 1 -- 172.21.15.170:0/4038627017 reap_dead start 2024-09-21T02:17:26.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- 172.21.15.170:0/4038627017 shutdown_connections 2024-09-21T02:17:26.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- 172.21.15.170:0/4038627017 >> 172.21.15.170:0/4038627017 conn(0x7f74040b1c40 msgr2=0x7f74040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:26.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- 172.21.15.170:0/4038627017 shutdown_connections 2024-09-21T02:17:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- 172.21.15.170:0/4038627017 wait complete. 2024-09-21T02:17:26.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 Processor -- start 2024-09-21T02:17:26.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- start start 2024-09-21T02:17:26.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f74041f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f740415a2f0 0x7f740415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f740415a700 0x7f740415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7404164cb0 con 0x7f73ec004810 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7404164b30 con 0x7f740415a700 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f740a50c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7404164e30 con 0x7f740415a2f0 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f74041f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f74041f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47366/0 (socket says 172.21.15.170:47366) 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 -- 172.21.15.170:0/2397909628 learned_addr learned my addr 172.21.15.170:0/2397909628 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f74037fe640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f740415a2f0 0x7f740415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 -- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f740415a2f0 msgr2=0x7f740415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f740415a2f0 0x7f740415e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7408aa9640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f740415a700 0x7f740415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 -- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f740415a700 msgr2=0x7f740415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f740415a700 0x7f740415e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 -- 172.21.15.170:0/2397909628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74041f3780 con 0x7f73ec004810 2024-09-21T02:17:26.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.228+0000 7f7403fff640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f74041f27b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f73f400f390 tx=0x7f73f400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f401a070 con 0x7f73ec004810 2024-09-21T02:17:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73f40084e0 con 0x7f73ec004810 2024-09-21T02:17:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74041f31c0 con 0x7f73ec004810 2024-09-21T02:17:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4006980 con 0x7f73ec004810 2024-09-21T02:17:26.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74041f3a60 con 0x7f73ec004810 2024-09-21T02:17:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74040cb870 con 0x7f73ec004810 2024-09-21T02:17:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f73f401d070 con 0x7f73ec004810 2024-09-21T02:17:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f7400ff9640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f73f0073560 0x7f73f0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7f73f4096770 con 0x7f73ec004810 2024-09-21T02:17:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f74037fe640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f73f0073560 0x7f73f0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.232+0000 7f74037fe640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f73f0073560 0x7f73f0075a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f73f80016c0 tx=0x7f73f80195a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:26.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.236+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f73f40615c0 con 0x7f73ec004810 2024-09-21T02:17:26.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.472+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f74040b43e0 con 0x7f73ec004810 2024-09-21T02:17:26.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f7400ff9640 1 -- 172.21.15.170:0/2397909628 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v23) ==== 82+0+0 (secure 0 0 0) 0x7f73f4060ec0 con 0x7f73ec004810 2024-09-21T02:17:26.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f73f0073560 msgr2=0x7f73f0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f73f0073560 0x7f73f0075a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f73f80016c0 tx=0x7f73f80195a0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 msgr2=0x7f74041f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f74041f27b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f73f400f390 tx=0x7f73f400f850 comp rx=0 tx=0).stop 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 shutdown_connections 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f73f0073560 0x7f73f0075a70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f740415a2f0 0x7f740415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f740415a700 0x7f740415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 --2- 172.21.15.170:0/2397909628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73ec004810 0x7f74041f27b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 >> 172.21.15.170:0/2397909628 conn(0x7f74040b1c40 msgr2=0x7f74040b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 shutdown_connections 2024-09-21T02:17:26.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.476+0000 7f740a50c640 1 -- 172.21.15.170:0/2397909628 wait complete. 2024-09-21T02:17:26.501 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:17:26.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 -- 172.21.15.170:0/1820572968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 msgr2=0x7f00440ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1820572968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f00440ce920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f002c009840 tx=0x7f002c01c790 comp rx=0 tx=0).stop 2024-09-21T02:17:26.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 -- 172.21.15.170:0/1820572968 shutdown_connections 2024-09-21T02:17:26.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1820572968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f00440ce920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 -- 172.21.15.170:0/1820572968 >> 172.21.15.170:0/1820572968 conn(0x7f00440cd3c0 msgr2=0x7f00440cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:26.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 -- 172.21.15.170:0/1820572968 shutdown_connections 2024-09-21T02:17:26.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 -- 172.21.15.170:0/1820572968 wait complete. 2024-09-21T02:17:26.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 Processor -- start 2024-09-21T02:17:26.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.648+0000 7f0049ce2640 1 -- start start 2024-09-21T02:17:26.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 0x7f0044073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f0044158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440cee60 0x7f00441fc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0044169bf0 con 0x7f0044071bd0 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0044169a70 con 0x7f00440ce510 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0044169d70 con 0x7f00440cee60 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 0x7f0044073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 0x7f0044073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47376/0 (socket says 172.21.15.170:47376) 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 -- 172.21.15.170:0/1045252044 learned_addr learned my addr 172.21.15.170:0/1045252044 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0043fff640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440cee60 0x7f00441fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0042ffd640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f0044158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 -- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440cee60 msgr2=0x7f00441fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440cee60 0x7f00441fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 -- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 msgr2=0x7f0044158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f0044158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 -- 172.21.15.170:0/1045252044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004415a5c0 con 0x7f0044071bd0 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0042ffd640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f0044158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f00437fe640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 0x7f0044073680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c3c0 tx=0x7f003c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:26.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c011790 con 0x7f0044071bd0 2024-09-21T02:17:26.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f003c017070 con 0x7f0044071bd0 2024-09-21T02:17:26.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c011d20 con 0x7f0044071bd0 2024-09-21T02:17:26.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004415a8a0 con 0x7f0044071bd0 2024-09-21T02:17:26.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441fc0e0 con 0x7f0044071bd0 2024-09-21T02:17:26.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00440ce920 con 0x7f0044071bd0 2024-09-21T02:17:26.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f003c01c460 con 0x7f0044071bd0 2024-09-21T02:17:26.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.652+0000 7f003bfff640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0024073560 0x7f0024075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:26.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.656+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7f003c0941e0 con 0x7f0044071bd0 2024-09-21T02:17:26.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.656+0000 7f0042ffd640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0024073560 0x7f0024075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:26.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.656+0000 7f0042ffd640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0024073560 0x7f0024075a70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f002c0af000 tx=0x7f002c01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.660+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f003c05f0c0 con 0x7f0044071bd0 2024-09-21T02:17:26.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.820+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f00440cdbc0 con 0x7f0044071bd0 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.820+0000 7f003bfff640 1 -- 172.21.15.170:0/1045252044 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7f003c05e9c0 con 0x7f0044071bd0 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:26.823 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:17:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0024073560 msgr2=0x7f0024075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0024073560 0x7f0024075a70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f002c0af000 tx=0x7f002c01afd0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 msgr2=0x7f0044073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 0x7f0044073680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c3c0 tx=0x7f003c00c880 comp rx=0 tx=0).stop 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 shutdown_connections 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f0024073560 0x7f0024075a70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440cee60 0x7f00441fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00440ce510 0x7f0044158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 --2- 172.21.15.170:0/1045252044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044071bd0 0x7f0044073680 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 >> 172.21.15.170:0/1045252044 conn(0x7f00440cd3c0 msgr2=0x7f004414efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 shutdown_connections 2024-09-21T02:17:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:26.824+0000 7f0049ce2640 1 -- 172.21.15.170:0/1045252044 wait complete. 2024-09-21T02:17:26.843 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:17:26.844 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:17:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.000+0000 7fe16e34d640 1 -- 172.21.15.170:0/1648479502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680cf5c0 msgr2=0x7fe1680cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.000+0000 7fe16e34d640 1 --2- 172.21.15.170:0/1648479502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680cf5c0 0x7fe1680cf9d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c01c6d0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- 172.21.15.170:0/1648479502 shutdown_connections 2024-09-21T02:17:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 --2- 172.21.15.170:0/1648479502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680cf5c0 0x7fe1680cf9d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 --2- 172.21.15.170:0/1648479502 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe16815b740 0x7fe1680b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- 172.21.15.170:0/1648479502 >> 172.21.15.170:0/1648479502 conn(0x7fe1680b2d50 msgr2=0x7fe1680b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:27.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- 172.21.15.170:0/1648479502 shutdown_connections 2024-09-21T02:17:27.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- 172.21.15.170:0/1648479502 wait complete. 2024-09-21T02:17:27.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 Processor -- start 2024-09-21T02:17:27.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- start start 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 0x7fe168159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1680cf5c0 0x7fe16815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe16815b740 0x7fe16814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe168169010 con 0x7fe1680b0af0 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe168168e90 con 0x7fe16815b740 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16e34d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe168169190 con 0x7fe1680cf5c0 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 0x7fe168159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 0x7fe168159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47388/0 (socket says 172.21.15.170:47388) 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 -- 172.21.15.170:0/2091223775 learned_addr learned my addr 172.21.15.170:0/2091223775 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe16c8ea640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe16815b740 0x7fe16814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 -- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1680cf5c0 msgr2=0x7fe16815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1680cf5c0 0x7fe16815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 -- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe16815b740 msgr2=0x7fe16814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe16815b740 0x7fe16814e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 -- 172.21.15.170:0/2091223775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681f9480 con 0x7fe1680b0af0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.004+0000 7fe167fff640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 0x7fe168159f20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe15800cd10 tx=0x7fe15800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158004b50 con 0x7fe1680b0af0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe158004de0 con 0x7fe1680b0af0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681f9760 con 0x7fe1680b0af0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158005290 con 0x7fe1680b0af0 2024-09-21T02:17:27.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681f9d20 con 0x7fe1680b0af0 2024-09-21T02:17:27.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1681fa100 con 0x7fe1680b0af0 2024-09-21T02:17:27.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe158005430 con 0x7fe1680b0af0 2024-09-21T02:17:27.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe164ff9640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe154073560 0x7fe154075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4857+0+0 (secure 0 0 0) 0x7fe158094f20 con 0x7fe1680b0af0 2024-09-21T02:17:27.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe1677fe640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe154073560 0x7fe154075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.008+0000 7fe1677fe640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe154073560 0x7fe154075a70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0af000 tx=0x7fe15c019470 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:27.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.012+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe158014070 con 0x7fe1680b0af0 2024-09-21T02:17:27.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.188+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe1680cf9d0 con 0x7fe1680b0af0 2024-09-21T02:17:27.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.652+0000 7fe164ff9640 1 -- 172.21.15.170:0/2091223775 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7fe15805fc70 con 0x7fe1680b0af0 2024-09-21T02:17:27.655 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:17:27.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe154073560 msgr2=0x7fe154075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe154073560 0x7fe154075a70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0af000 tx=0x7fe15c019470 comp rx=0 tx=0).stop 2024-09-21T02:17:27.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 msgr2=0x7fe168159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 0x7fe168159f20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe15800cd10 tx=0x7fe15800a620 comp rx=0 tx=0).stop 2024-09-21T02:17:27.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 shutdown_connections 2024-09-21T02:17:27.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe154073560 0x7fe154075a70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1680cf5c0 0x7fe16815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe16815b740 0x7fe16814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 --2- 172.21.15.170:0/2091223775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1680b0af0 0x7fe168159f20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 >> 172.21.15.170:0/2091223775 conn(0x7fe1680b2d50 msgr2=0x7fe1681516e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 shutdown_connections 2024-09-21T02:17:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.656+0000 7fe16e34d640 1 -- 172.21.15.170:0/2091223775 wait complete. 2024-09-21T02:17:27.688 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 -- 172.21.15.170:0/3888486067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00d0210 msgr2=0x7f2bb00cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 --2- 172.21.15.170:0/3888486067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00d0210 0x7f2bb00cd020 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0076b50 tx=0x7f2bac01c8a0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 -- 172.21.15.170:0/3888486067 shutdown_connections 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 --2- 172.21.15.170:0/3888486067 >> v2:172.21.15.191:3300/0 conn(0x7f2bb0160ec0 0x7f2bb01612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 --2- 172.21.15.170:0/3888486067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb00b75f0 0x7f2bb00b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 --2- 172.21.15.170:0/3888486067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00d0210 0x7f2bb00cd020 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 -- 172.21.15.170:0/3888486067 >> 172.21.15.170:0/3888486067 conn(0x7f2bb00cede0 msgr2=0x7f2bb00cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 -- 172.21.15.170:0/3888486067 shutdown_connections 2024-09-21T02:17:27.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 -- 172.21.15.170:0/3888486067 wait complete. 2024-09-21T02:17:27.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 Processor -- start 2024-09-21T02:17:27.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.852+0000 7f2bb852e640 1 -- start start 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 0x7f2bb015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb00d0210 0x7f2bb00cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb0160ec0 0x7f2bb00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb00b2860 con 0x7f2bb00b75f0 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb00b26e0 con 0x7f2bb00d0210 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb00b29e0 con 0x7f2bb0160ec0 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 0x7f2bb015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 0x7f2bb015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47416/0 (socket says 172.21.15.170:47416) 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 -- 172.21.15.170:0/943107595 learned_addr learned my addr 172.21.15.170:0/943107595 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:27.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb5ac9640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb00d0210 0x7f2bb00cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb6acb640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb0160ec0 0x7f2bb00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 -- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb0160ec0 msgr2=0x7f2bb00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb0160ec0 0x7f2bb00cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 -- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb00d0210 msgr2=0x7f2bb00cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb00d0210 0x7f2bb00cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 -- 172.21.15.170:0/943107595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb015c250 con 0x7f2bb00b75f0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb6acb640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb0160ec0 0x7f2bb00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb62ca640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 0x7f2bb015f150 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2bac000c10 tx=0x7f2bac005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0b2040 con 0x7f2bb00b75f0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bac0bb070 con 0x7f2bb00b75f0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb00cc620 con 0x7f2bb00b75f0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac003600 con 0x7f2bb00b75f0 2024-09-21T02:17:27.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb00cb3a0 con 0x7f2bb00b75f0 2024-09-21T02:17:27.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.856+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb00cd020 con 0x7f2bb00b75f0 2024-09-21T02:17:27.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.860+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2bac0037a0 con 0x7f2bb00b75f0 2024-09-21T02:17:27.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.860+0000 7f2ba6ffd640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2b94073560 0x7f2b94075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:27.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.860+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5227+0+0 (secure 0 0 0) 0x7f2bac12f6e0 con 0x7f2bb00b75f0 2024-09-21T02:17:27.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.860+0000 7f2bb5ac9640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2b94073560 0x7f2b94075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:27.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.860+0000 7f2bb5ac9640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2b94073560 0x7f2b94075a70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00c5d90 tx=0x7f2ba0005ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:27.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:27.864+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2bac016490 con 0x7f2bb00b75f0 2024-09-21T02:17:28.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.036+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2bb00cf290 con 0x7f2bb00b75f0 2024-09-21T02:17:28.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.664+0000 7f2ba6ffd640 1 -- 172.21.15.170:0/943107595 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) ==== 170+0+0 (secure 0 0 0) 0x7f2bac0fa580 con 0x7f2bb00b75f0 2024-09-21T02:17:28.667 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:17:28.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2b94073560 msgr2=0x7f2b94075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:28.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2b94073560 0x7f2b94075a70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00c5d90 tx=0x7f2ba0005ca0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 msgr2=0x7f2bb015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:28.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 0x7f2bb015f150 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2bac000c10 tx=0x7f2bac005da0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 shutdown_connections 2024-09-21T02:17:28.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2b94073560 0x7f2b94075a70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2bb0160ec0 0x7f2bb00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2bb00d0210 0x7f2bb00cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 --2- 172.21.15.170:0/943107595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bb00b75f0 0x7f2bb015f150 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 >> 172.21.15.170:0/943107595 conn(0x7f2bb00cede0 msgr2=0x7f2bb00b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:28.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 shutdown_connections 2024-09-21T02:17:28.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.668+0000 7f2bb852e640 1 -- 172.21.15.170:0/943107595 wait complete. 2024-09-21T02:17:28.710 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:17:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 -- 172.21.15.170:0/588920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 msgr2=0x7fe9880cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 --2- 172.21.15.170:0/588920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe9880cb990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe980009bc0 tx=0x7fe98001cba0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 -- 172.21.15.170:0/588920603 shutdown_connections 2024-09-21T02:17:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 --2- 172.21.15.170:0/588920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe9880cb990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 -- 172.21.15.170:0/588920603 >> 172.21.15.170:0/588920603 conn(0x7fe9880d0170 msgr2=0x7fe9880d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 -- 172.21.15.170:0/588920603 shutdown_connections 2024-09-21T02:17:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 -- 172.21.15.170:0/588920603 wait complete. 2024-09-21T02:17:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 Processor -- start 2024-09-21T02:17:28.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.860+0000 7fe98cb78640 1 -- start start 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe988074270 0x7fe9880b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe988148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9880cf7d0 0x7fe9880ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe98816a260 con 0x7fe988074680 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe98816a0e0 con 0x7fe988074270 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe98816a3e0 con 0x7fe9880cf7d0 2024-09-21T02:17:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe988148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe988148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47436/0 (socket says 172.21.15.170:47436) 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 -- 172.21.15.170:0/832679244 learned_addr learned my addr 172.21.15.170:0/832679244 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98659c640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe988074270 0x7fe9880b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 -- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9880cf7d0 msgr2=0x7fe9880ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9880cf7d0 0x7fe9880ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 -- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe988074270 msgr2=0x7fe9880b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe988074270 0x7fe9880b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 -- 172.21.15.170:0/832679244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98815e620 con 0x7fe988074680 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe985d9b640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe988148680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe97800e940 tx=0x7fe97800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978001a80 con 0x7fe988074680 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe978001d10 con 0x7fe988074680 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9780116b0 con 0x7fe988074680 2024-09-21T02:17:28.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98815e900 con 0x7fe988074680 2024-09-21T02:17:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe988148ed0 con 0x7fe988074680 2024-09-21T02:17:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.864+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe988073b60 con 0x7fe988074680 2024-09-21T02:17:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.868+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe97801a400 con 0x7fe988074680 2024-09-21T02:17:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.868+0000 7fe976ffd640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe968073590 0x7fe968075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.868+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5609+0+0 (secure 0 0 0) 0x7fe978097370 con 0x7fe988074680 2024-09-21T02:17:28.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.868+0000 7fe98659c640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe968073590 0x7fe968075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:28.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.868+0000 7fe98659c640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe968073590 0x7fe968075aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00a750 tx=0x7fe97c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:28.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:28.872+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe978029080 con 0x7fe988074680 2024-09-21T02:17:29.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.080+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe988155610 con 0x7fe988074680 2024-09-21T02:17:29.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.700+0000 7fe976ffd640 1 -- 172.21.15.170:0/832679244 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v43) ==== 174+0+0 (secure 0 0 0) 0x7fe978016030 con 0x7fe988074680 2024-09-21T02:17:29.709 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 10 and data pool 11 2024-09-21T02:17:29.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe968073590 msgr2=0x7fe968075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:29.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe968073590 0x7fe968075aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00a750 tx=0x7fe97c008040 comp rx=0 tx=0).stop 2024-09-21T02:17:29.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 msgr2=0x7fe988148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:29.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe988148680 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe97800e940 tx=0x7fe97800ee00 comp rx=0 tx=0).stop 2024-09-21T02:17:29.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 shutdown_connections 2024-09-21T02:17:29.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe968073590 0x7fe968075aa0 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00a750 tx=0x7fe97c008040 comp rx=0 tx=0).stop 2024-09-21T02:17:29.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9880cf7d0 0x7fe9880ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe988074680 0x7fe988148680 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 --2- 172.21.15.170:0/832679244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe988074270 0x7fe9880b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 >> 172.21.15.170:0/832679244 conn(0x7fe9880d0170 msgr2=0x7fe988154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:29.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 shutdown_connections 2024-09-21T02:17:29.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.724+0000 7fe98cb78640 1 -- 172.21.15.170:0/832679244 wait complete. 2024-09-21T02:17:29.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 -- 172.21.15.170:0/381317241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580cb500 msgr2=0x7fe2580ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 --2- 172.21.15.170:0/381317241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580cb500 0x7fe2580ce8d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe25000acd0 tx=0x7fe25001def0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 -- 172.21.15.170:0/381317241 shutdown_connections 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 --2- 172.21.15.170:0/381317241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580cb500 0x7fe2580ce8d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 -- 172.21.15.170:0/381317241 >> 172.21.15.170:0/381317241 conn(0x7fe2580b1c40 msgr2=0x7fe2580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 -- 172.21.15.170:0/381317241 shutdown_connections 2024-09-21T02:17:29.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 -- 172.21.15.170:0/381317241 wait complete. 2024-09-21T02:17:29.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 Processor -- start 2024-09-21T02:17:29.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.900+0000 7fe25d51e640 1 -- start start 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 0x7fe258151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25815f990 0x7fe2580c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe258159df0 0x7fe2580c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe258169390 con 0x7fe2580ce4c0 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe258169210 con 0x7fe258159df0 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe258169510 con 0x7fe25815f990 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 0x7fe258151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:29.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 0x7fe258151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47462/0 (socket says 172.21.15.170:47462) 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 -- 172.21.15.170:0/404313009 learned_addr learned my addr 172.21.15.170:0/404313009 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe24ffff640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25815f990 0x7fe2580c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe2577fe640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe258159df0 0x7fe2580c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 -- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25815f990 msgr2=0x7fe2580c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25815f990 0x7fe2580c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 -- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe258159df0 msgr2=0x7fe2580c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe258159df0 0x7fe2580c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 -- 172.21.15.170:0/404313009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25815a560 con 0x7fe2580ce4c0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe256ffd640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 0x7fe258151ae0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe24800b440 tx=0x7fe24800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248015020 con 0x7fe2580ce4c0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe248011980 con 0x7fe2580ce4c0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25815a840 con 0x7fe2580ce4c0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248011dd0 con 0x7fe2580ce4c0 2024-09-21T02:17:29.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25809cc40 con 0x7fe2580ce4c0 2024-09-21T02:17:29.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25809cea0 con 0x7fe2580ce4c0 2024-09-21T02:17:29.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.904+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe248018020 con 0x7fe2580ce4c0 2024-09-21T02:17:29.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.908+0000 7fe24f7fe640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe238073560 0x7fe238075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:29.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.908+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fe2480949f0 con 0x7fe2580ce4c0 2024-09-21T02:17:29.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.908+0000 7fe24ffff640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe238073560 0x7fe238075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:29.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.908+0000 7fe24ffff640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe238073560 0x7fe238075a70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe25806a970 tx=0x7fe240009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:29.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:29.912+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe24801f020 con 0x7fe2580ce4c0 2024-09-21T02:17:30.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.084+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe25809b500 con 0x7fe2580ce4c0 2024-09-21T02:17:30.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe24f7fe640 1 -- 172.21.15.170:0/404313009 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16729 (secure 0 0 0) 0x7fe24800d040 con 0x7fe2580ce4c0 2024-09-21T02:17:30.090 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:30.090 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":71,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:29.085799+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:17:27.202813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T02:17:28.049434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:17:30.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe238073560 msgr2=0x7fe238075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe238073560 0x7fe238075a70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe25806a970 tx=0x7fe240009040 comp rx=0 tx=0).stop 2024-09-21T02:17:30.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 msgr2=0x7fe258151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 0x7fe258151ae0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe24800b440 tx=0x7fe24800b900 comp rx=0 tx=0).stop 2024-09-21T02:17:30.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 shutdown_connections 2024-09-21T02:17:30.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe238073560 0x7fe238075a70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.088+0000 7fe25d51e640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25815f990 0x7fe2580c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.092+0000 7fe25d51e640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe258159df0 0x7fe2580c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.092+0000 7fe25d51e640 1 --2- 172.21.15.170:0/404313009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2580ce4c0 0x7fe258151ae0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.092+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 >> 172.21.15.170:0/404313009 conn(0x7fe2580b1c40 msgr2=0x7fe25815b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:30.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.092+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 shutdown_connections 2024-09-21T02:17:30.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.092+0000 7fe25d51e640 1 -- 172.21.15.170:0/404313009 wait complete. 2024-09-21T02:17:30.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 -- 172.21.15.170:0/3227537359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40b4540 msgr2=0x7f45f415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/3227537359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40b4540 0x7f45f415c850 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f45f4075080 tx=0x7f45e401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 -- 172.21.15.170:0/3227537359 shutdown_connections 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/3227537359 >> v2:172.21.15.191:3300/0 conn(0x7f45f40d0aa0 0x7f45f40d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/3227537359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40b4540 0x7f45f415c850 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 -- 172.21.15.170:0/3227537359 >> 172.21.15.170:0/3227537359 conn(0x7f45f40cee30 msgr2=0x7f45f40cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 -- 172.21.15.170:0/3227537359 shutdown_connections 2024-09-21T02:17:30.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 -- 172.21.15.170:0/3227537359 wait complete. 2024-09-21T02:17:30.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 Processor -- start 2024-09-21T02:17:30.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.272+0000 7f45fb4b0640 1 -- start start 2024-09-21T02:17:30.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45f40b7ab0 0x7f45f415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45f40d0aa0 0x7f45f40c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 0x7f45f40c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45f416a470 con 0x7f45f40ca440 2024-09-21T02:17:30.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f45f416a2f0 con 0x7f45f40b7ab0 2024-09-21T02:17:30.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f45f416a5f0 con 0x7f45f40d0aa0 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 0x7f45f40c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 0x7f45f40c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47472/0 (socket says 172.21.15.170:47472) 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 -- 172.21.15.170:0/1668349248 learned_addr learned my addr 172.21.15.170:0/1668349248 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f924c640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45f40b7ab0 0x7f45f415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f8a4b640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45f40d0aa0 0x7f45f40c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 -- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45f40d0aa0 msgr2=0x7f45f40c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45f40d0aa0 0x7f45f40c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 -- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45f40b7ab0 msgr2=0x7f45f415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45f40b7ab0 0x7f45f415d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 -- 172.21.15.170:0/1668349248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f415e170 con 0x7f45f40ca440 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f924c640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45f40b7ab0 0x7f45f415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45f9a4d640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 0x7f45f40c6370 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00cad0 tx=0x7f45ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00ae20 con 0x7f45f40ca440 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ec003980 con 0x7f45f40ca440 2024-09-21T02:17:30.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f415db80 con 0x7f45f40ca440 2024-09-21T02:17:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003df0 con 0x7f45f40ca440 2024-09-21T02:17:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f40c8ef0 con 0x7f45f40ca440 2024-09-21T02:17:30.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.276+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f415c850 con 0x7f45f40ca440 2024-09-21T02:17:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.280+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f45ec004030 con 0x7f45f40ca440 2024-09-21T02:17:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.280+0000 7f45e9ffb640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45d8073560 0x7f45d8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.280+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f45ec0951d0 con 0x7f45f40ca440 2024-09-21T02:17:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.280+0000 7f45f924c640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45d8073560 0x7f45d8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.280+0000 7f45f924c640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45d8073560 0x7f45d8075a70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f45f40d0480 tx=0x7f45f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:30.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.284+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f45ec014070 con 0x7f45f40ca440 2024-09-21T02:17:30.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.452+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45f40d2070 con 0x7f45f40ca440 2024-09-21T02:17:30.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45e9ffb640 1 -- 172.21.15.170:0/1668349248 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16729 (secure 0 0 0) 0x7f45ec05fc60 con 0x7f45f40ca440 2024-09-21T02:17:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:30.457 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":71,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:29.085799+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:17:27.202813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T02:17:28.049434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45d8073560 msgr2=0x7f45d8075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45d8073560 0x7f45d8075a70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f45f40d0480 tx=0x7f45f0009040 comp rx=0 tx=0).stop 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 msgr2=0x7f45f40c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 0x7f45f40c6370 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00cad0 tx=0x7f45ec00cf90 comp rx=0 tx=0).stop 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 shutdown_connections 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f45d8073560 0x7f45d8075a70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45f40d0aa0 0x7f45f40c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45f40ca440 0x7f45f40c6370 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 --2- 172.21.15.170:0/1668349248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45f40b7ab0 0x7f45f415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 >> 172.21.15.170:0/1668349248 conn(0x7f45f40cee30 msgr2=0x7f45f40b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 shutdown_connections 2024-09-21T02:17:30.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.456+0000 7f45fb4b0640 1 -- 172.21.15.170:0/1668349248 wait complete. 2024-09-21T02:17:30.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 -- 172.21.15.170:0/1168146815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7368004810 msgr2=0x7f7368004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 --2- 172.21.15.170:0/1168146815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7368004810 0x7f7368004c20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f73640016f0 tx=0x7f736401c790 comp rx=0 tx=0).stop 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 -- 172.21.15.170:0/1168146815 shutdown_connections 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 --2- 172.21.15.170:0/1168146815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7368004810 0x7f7368004c20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 -- 172.21.15.170:0/1168146815 >> 172.21.15.170:0/1168146815 conn(0x7f737c08dca0 msgr2=0x7f737c09d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 -- 172.21.15.170:0/1168146815 shutdown_connections 2024-09-21T02:17:30.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 -- 172.21.15.170:0/1168146815 wait complete. 2024-09-21T02:17:30.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 Processor -- start 2024-09-21T02:17:30.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.636+0000 7f73821a0640 1 -- start start 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7368004810 0x7f737c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 0x7f737c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c0b5030 0x7f737c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f737c0d06c0 con 0x7f737c15d6f0 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f737c06ec10 con 0x7f737c0b5030 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f737c0d0840 con 0x7f7368004810 2024-09-21T02:17:30.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 0x7f737c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 0x7f737c0b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47488/0 (socket says 172.21.15.170:47488) 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 -- 172.21.15.170:0/2020142943 learned_addr learned my addr 172.21.15.170:0/2020142943 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f737b7fe640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7368004810 0x7f737c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f737bfff640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c0b5030 0x7f737c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 -- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7368004810 msgr2=0x7f737c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7368004810 0x7f737c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 -- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c0b5030 msgr2=0x7f737c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c0b5030 0x7f737c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 -- 172.21.15.170:0/2020142943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c15c930 con 0x7f737c15d6f0 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f737b7fe640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7368004810 0x7f737c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f737bfff640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c0b5030 0x7f737c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7373fff640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 0x7f737c0b6240 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f736401b060 tx=0x7f7364003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640b9070 con 0x7f737c15d6f0 2024-09-21T02:17:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73640af040 con 0x7f737c15d6f0 2024-09-21T02:17:30.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c15cc10 con 0x7f737c15d6f0 2024-09-21T02:17:30.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640ae070 con 0x7f737c15d6f0 2024-09-21T02:17:30.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c0b59f0 con 0x7f737c15d6f0 2024-09-21T02:17:30.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c15af90 con 0x7f737c15d6f0 2024-09-21T02:17:30.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.640+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f7364006330 con 0x7f737c15d6f0 2024-09-21T02:17:30.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.644+0000 7f7378ff9640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f735c073580 0x7f735c075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:30.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.644+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f736412dfd0 con 0x7f737c15d6f0 2024-09-21T02:17:30.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.644+0000 7f737b7fe640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f735c073580 0x7f735c075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:30.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.644+0000 7f737b7fe640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f735c073580 0x7f735c075a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f737c075850 tx=0x7f736c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:30.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.648+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f73640c2080 con 0x7f737c15d6f0 2024-09-21T02:17:30.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:30.844+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f737c0d0da0 con 0x7f737c15d6f0 2024-09-21T02:17:31.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.732+0000 7f7378ff9640 1 -- 172.21.15.170:0/2020142943 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7f73640164f0 con 0x7f737c15d6f0 2024-09-21T02:17:31.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f735c073580 msgr2=0x7f735c075a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:31.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f735c073580 0x7f735c075a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f737c075850 tx=0x7f736c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 msgr2=0x7f737c0b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:31.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 0x7f737c0b6240 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f736401b060 tx=0x7f7364003be0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 shutdown_connections 2024-09-21T02:17:31.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f735c073580 0x7f735c075a90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c0b5030 0x7f737c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c15d6f0 0x7f737c0b6240 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 --2- 172.21.15.170:0/2020142943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7368004810 0x7f737c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 >> 172.21.15.170:0/2020142943 conn(0x7f737c08dca0 msgr2=0x7f737c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:31.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 shutdown_connections 2024-09-21T02:17:31.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.736+0000 7f73821a0640 1 -- 172.21.15.170:0/2020142943 wait complete. 2024-09-21T02:17:31.763 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:17:31.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:17:31.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 -- 172.21.15.170:0/1490465138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b0074270 msgr2=0x7f89b00cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:31.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/1490465138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b0074270 0x7f89b00cb280 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f89b00b61b0 tx=0x7f899801c800 comp rx=0 tx=0).stop 2024-09-21T02:17:31.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 -- 172.21.15.170:0/1490465138 shutdown_connections 2024-09-21T02:17:31.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/1490465138 >> v2:172.21.15.191:3300/0 conn(0x7f89b0074680 0x7f89b00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/1490465138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b0074270 0x7f89b00cb280 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 -- 172.21.15.170:0/1490465138 >> 172.21.15.170:0/1490465138 conn(0x7f89b00d0170 msgr2=0x7f89b00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:31.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 -- 172.21.15.170:0/1490465138 shutdown_connections 2024-09-21T02:17:31.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 -- 172.21.15.170:0/1490465138 wait complete. 2024-09-21T02:17:31.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 Processor -- start 2024-09-21T02:17:31.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.916+0000 7f89b4b7b640 1 -- start start 2024-09-21T02:17:31.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0074270 0x7f89b0158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89b0074680 0x7f89b01fc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 0x7f89b014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0167030 con 0x7f89b00cf7d0 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0166eb0 con 0x7f89b0074270 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f89b01671b0 con 0x7f89b0074680 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 0x7f89b014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 0x7f89b014a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47516/0 (socket says 172.21.15.170:47516) 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 -- 172.21.15.170:0/3236893771 learned_addr learned my addr 172.21.15.170:0/3236893771 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89a7fff640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89b0074680 0x7f89b01fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89ae59c640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0074270 0x7f89b0158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 -- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89b0074680 msgr2=0x7f89b01fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:31.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89b0074680 0x7f89b01fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 -- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0074270 msgr2=0x7f89b0158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0074270 0x7f89b0158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 -- 172.21.15.170:0/3236893771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b01593f0 con 0x7f89b00cf7d0 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89a7fff640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89b0074680 0x7f89b01fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89ae59c640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0074270 0x7f89b0158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89aed9d640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 0x7f89b014a4d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f89a800cd10 tx=0x7f89a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a800f450 con 0x7f89b00cf7d0 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a800f700 con 0x7f89b00cf7d0 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b0159870 con 0x7f89b00cf7d0 2024-09-21T02:17:31.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b0159e30 con 0x7f89b00cf7d0 2024-09-21T02:17:31.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a800fb50 con 0x7f89b00cf7d0 2024-09-21T02:17:31.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b01595c0 con 0x7f89b00cf7d0 2024-09-21T02:17:31.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.920+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f89a800fcf0 con 0x7f89b00cf7d0 2024-09-21T02:17:31.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.924+0000 7f89a6ffd640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8990073560 0x7f8990075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:31.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.924+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f89a8095340 con 0x7f89b00cf7d0 2024-09-21T02:17:31.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.924+0000 7f89ae59c640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8990073560 0x7f8990075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:31.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.924+0000 7f89ae59c640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8990073560 0x7f8990075a70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f89a0001630 tx=0x7f89a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:31.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:31.928+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f89a8014030 con 0x7f89b00cf7d0 2024-09-21T02:17:32.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.140+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f89b0073b60 con 0x7f89b00cf7d0 2024-09-21T02:17:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.744+0000 7f89a6ffd640 1 -- 172.21.15.170:0/3236893771 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7f89a805fef0 con 0x7f89b00cf7d0 2024-09-21T02:17:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8990073560 msgr2=0x7f8990075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8990073560 0x7f8990075a70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f89a0001630 tx=0x7f89a0009040 comp rx=0 tx=0).stop 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 msgr2=0x7f89b014a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 0x7f89b014a4d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f89a800cd10 tx=0x7f89a800a620 comp rx=0 tx=0).stop 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 shutdown_connections 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f8990073560 0x7f8990075a70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89b0074680 0x7f89b01fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89b00cf7d0 0x7f89b014a4d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 --2- 172.21.15.170:0/3236893771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0074270 0x7f89b0158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 >> 172.21.15.170:0/3236893771 conn(0x7f89b00d0170 msgr2=0x7f89b00adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 shutdown_connections 2024-09-21T02:17:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.748+0000 7f89b4b7b640 1 -- 172.21.15.170:0/3236893771 wait complete. 2024-09-21T02:17:32.774 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:17:32.775 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.928+0000 7f4283109640 1 -- 172.21.15.170:0/3905761262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 msgr2=0x7f427c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.928+0000 7f4283109640 1 --2- 172.21.15.170:0/3905761262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c070180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4274009bc0 tx=0x7f427401cba0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- 172.21.15.170:0/3905761262 shutdown_connections 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 --2- 172.21.15.170:0/3905761262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c070180 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- 172.21.15.170:0/3905761262 >> 172.21.15.170:0/3905761262 conn(0x7f427c0cd3c0 msgr2=0x7f427c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- 172.21.15.170:0/3905761262 shutdown_connections 2024-09-21T02:17:32.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- 172.21.15.170:0/3905761262 wait complete. 2024-09-21T02:17:32.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 Processor -- start 2024-09-21T02:17:32.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- start start 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f427c0706c0 0x7f427c159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f427c0cc0a0 0x7f427c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f427c16b280 con 0x7f427c074340 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f427c16b100 con 0x7f427c0706c0 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4283109640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f427c16b400 con 0x7f427c0cc0a0 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:32.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c14b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47524/0 (socket says 172.21.15.170:47524) 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 -- 172.21.15.170:0/2013421340 learned_addr learned my addr 172.21.15.170:0/2013421340 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f42816a6640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f427c0cc0a0 0x7f427c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4280ea5640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f427c0706c0 0x7f427c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 -- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f427c0cc0a0 msgr2=0x7f427c0af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f427c0cc0a0 0x7f427c0af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 -- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f427c0706c0 msgr2=0x7f427c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f427c0706c0 0x7f427c159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f427bfff640 1 -- 172.21.15.170:0/2013421340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c14bde0 con 0x7f427c074340 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f4280ea5640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f427c0706c0 0x7f427c159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.932+0000 7f42816a6640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f427c0cc0a0 0x7f427c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f427bfff640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c14b8a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f426400c730 tx=0x7f426400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264016020 con 0x7f427c074340 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f426400a950 con 0x7f427c074340 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c153d90 con 0x7f427c074340 2024-09-21T02:17:32.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426400ada0 con 0x7f427c074340 2024-09-21T02:17:32.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c154350 con 0x7f427c074340 2024-09-21T02:17:32.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c153b00 con 0x7f427c074340 2024-09-21T02:17:32.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f4264019020 con 0x7f427c074340 2024-09-21T02:17:32.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f42797fa640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f425c073590 0x7f425c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:32.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f4264094640 con 0x7f427c074340 2024-09-21T02:17:32.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f4280ea5640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f425c073590 0x7f425c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:32.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.936+0000 7f4280ea5640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f425c073590 0x7f425c075aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f42740af000 tx=0x7f427401b3e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:32.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:32.940+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4264020020 con 0x7f427c074340 2024-09-21T02:17:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.152+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f427c070180 con 0x7f427c074340 2024-09-21T02:17:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.152+0000 7f42797fa640 1 -- 172.21.15.170:0/2013421340 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7f427c070ad0 con 0x7f427c074340 2024-09-21T02:17:33.155 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:33.155 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":47,"btime":"2024-09-21T02:17:32:738667+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:17:33.155 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 47 2024-09-21T02:17:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f425c073590 msgr2=0x7f425c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f425c073590 0x7f425c075aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f42740af000 tx=0x7f427401b3e0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 msgr2=0x7f427c14b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c14b8a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f426400c730 tx=0x7f426400cbf0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 shutdown_connections 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f425c073590 0x7f425c075aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f427c0cc0a0 0x7f427c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c074340 0x7f427c14b8a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 --2- 172.21.15.170:0/2013421340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f427c0706c0 0x7f427c159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 >> 172.21.15.170:0/2013421340 conn(0x7f427c0cd3c0 msgr2=0x7f427c0b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 shutdown_connections 2024-09-21T02:17:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.156+0000 7f4283109640 1 -- 172.21.15.170:0/2013421340 wait complete. 2024-09-21T02:17:33.177 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:17:33.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 -- 172.21.15.170:0/2920137879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b2f40 msgr2=0x7fab600b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 --2- 172.21.15.170:0/2920137879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b2f40 0x7fab600b3350 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fab600b3a10 tx=0x7fab5001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 -- 172.21.15.170:0/2920137879 shutdown_connections 2024-09-21T02:17:33.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 --2- 172.21.15.170:0/2920137879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b2f40 0x7fab600b3350 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 --2- 172.21.15.170:0/2920137879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab600b5410 0x7fab6006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 -- 172.21.15.170:0/2920137879 >> 172.21.15.170:0/2920137879 conn(0x7fab600b1c40 msgr2=0x7fab600b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:33.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 -- 172.21.15.170:0/2920137879 shutdown_connections 2024-09-21T02:17:33.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.328+0000 7fab66553640 1 -- 172.21.15.170:0/2920137879 wait complete. 2024-09-21T02:17:33.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 Processor -- start 2024-09-21T02:17:33.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 -- start start 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab600b2f40 0x7fab60156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 0x7fab6014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab600ccb00 0x7fab6009b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab60169fd0 con 0x7fab600b5410 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fab60169e50 con 0x7fab600b2f40 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fab6016a150 con 0x7fab600ccb00 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 0x7fab6014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 0x7fab6014e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47536/0 (socket says 172.21.15.170:47536) 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 -- 172.21.15.170:0/3859074650 learned_addr learned my addr 172.21.15.170:0/3859074650 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab64af0640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab600ccb00 0x7fab6009b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5ffff640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab600b2f40 0x7fab60156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 -- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab600ccb00 msgr2=0x7fab6009b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab600ccb00 0x7fab6009b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 -- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab600b2f40 msgr2=0x7fab60156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab600b2f40 0x7fab60156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 -- 172.21.15.170:0/3859074650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6014e5c0 con 0x7fab600b5410 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5f7fe640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 0x7fab6014e080 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fab500b0040 tx=0x7fab50005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500bb070 con 0x7fab600b5410 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab500a5c10 con 0x7fab600b5410 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6009b910 con 0x7fab600b5410 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500b2040 con 0x7fab600b5410 2024-09-21T02:17:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.332+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab60155f20 con 0x7fab600b5410 2024-09-21T02:17:33.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.336+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab601559b0 con 0x7fab600b5410 2024-09-21T02:17:33.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.336+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fab500be070 con 0x7fab600b5410 2024-09-21T02:17:33.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.336+0000 7fab5cff9640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab4c073590 0x7fab4c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.336+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fab5012e2a0 con 0x7fab600b5410 2024-09-21T02:17:33.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.336+0000 7fab5ffff640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab4c073590 0x7fab4c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.336+0000 7fab5ffff640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab4c073590 0x7fab4c075aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fab540058d0 tx=0x7fab54005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:33.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.340+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fab500164d0 con 0x7fab600b5410 2024-09-21T02:17:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.500+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab600b3350 con 0x7fab600b5410 2024-09-21T02:17:33.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.504+0000 7fab5cff9640 1 -- 172.21.15.170:0/3859074650 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16716 (secure 0 0 0) 0x7fab500f8cb0 con 0x7fab600b5410 2024-09-21T02:17:33.507 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:33.507 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":72,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:30.671864+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:17:27.202813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T02:17:28.049434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:17:33.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab4c073590 msgr2=0x7fab4c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab4c073590 0x7fab4c075aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fab540058d0 tx=0x7fab54005860 comp rx=0 tx=0).stop 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 msgr2=0x7fab6014e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 0x7fab6014e080 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fab500b0040 tx=0x7fab50005e00 comp rx=0 tx=0).stop 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 shutdown_connections 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fab4c073590 0x7fab4c075aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab600ccb00 0x7fab6009b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab600b5410 0x7fab6014e080 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 --2- 172.21.15.170:0/3859074650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab600b2f40 0x7fab60156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 >> 172.21.15.170:0/3859074650 conn(0x7fab600b1c40 msgr2=0x7fab6015c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 shutdown_connections 2024-09-21T02:17:33.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.508+0000 7fab66553640 1 -- 172.21.15.170:0/3859074650 wait complete. 2024-09-21T02:17:33.528 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:17:33.528 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 -- 172.21.15.170:0/2515194848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300ce1c0 msgr2=0x7fef300ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 --2- 172.21.15.170:0/2515194848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300ce1c0 0x7fef300ce5d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef28009bc0 tx=0x7fef2801cba0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 -- 172.21.15.170:0/2515194848 shutdown_connections 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 --2- 172.21.15.170:0/2515194848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300ce1c0 0x7fef300ce5d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 --2- 172.21.15.170:0/2515194848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef300cb460 0x7fef300cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 -- 172.21.15.170:0/2515194848 >> 172.21.15.170:0/2515194848 conn(0x7fef300b1c40 msgr2=0x7fef300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 -- 172.21.15.170:0/2515194848 shutdown_connections 2024-09-21T02:17:33.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.664+0000 7fef35ad0640 1 -- 172.21.15.170:0/2515194848 wait complete. 2024-09-21T02:17:33.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 Processor -- start 2024-09-21T02:17:33.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- start start 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef300cb460 0x7fef301fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef300ce1c0 0x7fef3014bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 0x7fef3006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef301690d0 con 0x7fef300cf7e0 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fef30168f50 con 0x7fef300ce1c0 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fef30169250 con 0x7fef300cb460 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 0x7fef3006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 0x7fef3006eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47546/0 (socket says 172.21.15.170:47546) 2024-09-21T02:17:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 -- 172.21.15.170:0/3090365890 learned_addr learned my addr 172.21.15.170:0/3090365890 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 -- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef300cb460 msgr2=0x7fef301fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2f7fe640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef300cb460 0x7fef301fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2effd640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef300ce1c0 0x7fef3014bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef300cb460 0x7fef301fc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 -- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef300ce1c0 msgr2=0x7fef3014bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef300ce1c0 0x7fef3014bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 -- 172.21.15.170:0/3090365890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef301fcc80 con 0x7fef300cf7e0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef2ffff640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 0x7fef3006eeb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fef2801a7d0 tx=0x7fef28005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280bc070 con 0x7fef300cf7e0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef28004200 con 0x7fef300cf7e0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef301fcf60 con 0x7fef300cf7e0 2024-09-21T02:17:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280b2040 con 0x7fef300cf7e0 2024-09-21T02:17:33.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef30157990 con 0x7fef300cf7e0 2024-09-21T02:17:33.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.668+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef300cb870 con 0x7fef300cf7e0 2024-09-21T02:17:33.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.672+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fef280bf070 con 0x7fef300cf7e0 2024-09-21T02:17:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.672+0000 7fef17fff640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef1c073560 0x7fef1c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.672+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fef2812e720 con 0x7fef300cf7e0 2024-09-21T02:17:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.672+0000 7fef2f7fe640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef1c073560 0x7fef1c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.672+0000 7fef2f7fe640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef1c073560 0x7fef1c075a70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fef20001570 tx=0x7fef20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:33.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.676+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fef28016790 con 0x7fef300cf7e0 2024-09-21T02:17:33.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.840+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef300cad50 con 0x7fef1c073560 2024-09-21T02:17:33.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.852+0000 7fef17fff640 1 -- 172.21.15.170:0/3090365890 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245894 (secure 0 0 0) 0x7fef300cad50 con 0x7fef1c073560 2024-09-21T02:17:33.857 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:33.859 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:17:33.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef1c073560 msgr2=0x7fef1c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef1c073560 0x7fef1c075a70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fef20001570 tx=0x7fef20009040 comp rx=0 tx=0).stop 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 msgr2=0x7fef3006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 0x7fef3006eeb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fef2801a7d0 tx=0x7fef28005b10 comp rx=0 tx=0).stop 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 shutdown_connections 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fef1c073560 0x7fef1c075a70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef300cf7e0 0x7fef3006eeb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef300ce1c0 0x7fef3014bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 --2- 172.21.15.170:0/3090365890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef300cb460 0x7fef301fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 >> 172.21.15.170:0/3090365890 conn(0x7fef300b1c40 msgr2=0x7fef301554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 shutdown_connections 2024-09-21T02:17:33.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:33.856+0000 7fef35ad0640 1 -- 172.21.15.170:0/3090365890 wait complete. 2024-09-21T02:17:33.877 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-21T02:17:32.961564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269708,"kb_used_data":7680,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754705012,"statfs":{"total":773094113280,"available":772817932288,"internally_reserved":0,"allocated":7864320,"data_stored":3962224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.303810"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679110+0000","last_change":"2024-09-21T02:17:28.679110+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679110+0000","last_undegraded":"2024-09-21T02:17:28.679110+0000","last_fullsized":"2024-09-21T02:17:28.679110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682492+0000","last_change":"2024-09-21T02:17:28.682492+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682492+0000","last_undegraded":"2024-09-21T02:17:28.682492+0000","last_fullsized":"2024-09-21T02:17:28.682492+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689700+0000","last_change":"2024-09-21T02:17:28.689700+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689700+0000","last_undegraded":"2024-09-21T02:17:28.689700+0000","last_fullsized":"2024-09-21T02:17:28.689700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690481+0000","last_change":"2024-09-21T02:17:28.690481+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690481+0000","last_undegraded":"2024-09-21T02:17:28.690481+0000","last_fullsized":"2024-09-21T02:17:28.690481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.675278+0000","last_change":"2024-09-21T02:17:28.675278+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.675278+0000","last_undegraded":"2024-09-21T02:17:28.675278+0000","last_fullsized":"2024-09-21T02:17:28.675278+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679207+0000","last_change":"2024-09-21T02:17:28.679207+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679207+0000","last_undegraded":"2024-09-21T02:17:28.679207+0000","last_fullsized":"2024-09-21T02:17:28.679207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689631+0000","last_change":"2024-09-21T02:17:28.689631+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689631+0000","last_undegraded":"2024-09-21T02:17:28.689631+0000","last_fullsized":"2024-09-21T02:17:28.689631+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682248+0000","last_change":"2024-09-21T02:17:28.682248+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682248+0000","last_undegraded":"2024-09-21T02:17:28.682248+0000","last_fullsized":"2024-09-21T02:17:28.682248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686730+0000","last_change":"2024-09-21T02:17:28.686730+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686730+0000","last_undegraded":"2024-09-21T02:17:28.686730+0000","last_fullsized":"2024-09-21T02:17:28.686730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687831+0000","last_change":"2024-09-21T02:17:28.687831+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687831+0000","last_undegraded":"2024-09-21T02:17:28.687831+0000","last_fullsized":"2024-09-21T02:17:28.687831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678223+0000","last_change":"2024-09-21T02:17:28.678223+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678223+0000","last_undegraded":"2024-09-21T02:17:28.678223+0000","last_fullsized":"2024-09-21T02:17:28.678223+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687804+0000","last_change":"2024-09-21T02:17:28.687804+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687804+0000","last_undegraded":"2024-09-21T02:17:28.687804+0000","last_fullsized":"2024-09-21T02:17:28.687804+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681761+0000","last_change":"2024-09-21T02:17:28.681761+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681761+0000","last_undegraded":"2024-09-21T02:17:28.681761+0000","last_fullsized":"2024-09-21T02:17:28.681761+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677586+0000","last_change":"2024-09-21T02:17:28.677586+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677586+0000","last_undegraded":"2024-09-21T02:17:28.677586+0000","last_fullsized":"2024-09-21T02:17:28.677586+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680007+0000","last_change":"2024-09-21T02:17:28.680007+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680007+0000","last_undegraded":"2024-09-21T02:17:28.680007+0000","last_fullsized":"2024-09-21T02:17:28.680007+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679210+0000","last_change":"2024-09-21T02:17:28.679210+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679210+0000","last_undegraded":"2024-09-21T02:17:28.679210+0000","last_fullsized":"2024-09-21T02:17:28.679210+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.675097+0000","last_change":"2024-09-21T02:17:28.675097+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.675097+0000","last_undegraded":"2024-09-21T02:17:28.675097+0000","last_fullsized":"2024-09-21T02:17:28.675097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682671+0000","last_change":"2024-09-21T02:17:28.682671+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682671+0000","last_undegraded":"2024-09-21T02:17:28.682671+0000","last_fullsized":"2024-09-21T02:17:28.682671+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678167+0000","last_change":"2024-09-21T02:17:28.678167+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678167+0000","last_undegraded":"2024-09-21T02:17:28.678167+0000","last_fullsized":"2024-09-21T02:17:28.678167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689162+0000","last_change":"2024-09-21T02:17:28.689162+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689162+0000","last_undegraded":"2024-09-21T02:17:28.689162+0000","last_fullsized":"2024-09-21T02:17:28.689162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677094+0000","last_change":"2024-09-21T02:17:28.677094+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677094+0000","last_undegraded":"2024-09-21T02:17:28.677094+0000","last_fullsized":"2024-09-21T02:17:28.677094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686054+0000","last_change":"2024-09-21T02:17:28.686054+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686054+0000","last_undegraded":"2024-09-21T02:17:28.686054+0000","last_fullsized":"2024-09-21T02:17:28.686054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679394+0000","last_change":"2024-09-21T02:17:28.679394+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679394+0000","last_undegraded":"2024-09-21T02:17:28.679394+0000","last_fullsized":"2024-09-21T02:17:28.679394+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676363+0000","last_change":"2024-09-21T02:17:28.676363+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676363+0000","last_undegraded":"2024-09-21T02:17:28.676363+0000","last_fullsized":"2024-09-21T02:17:28.676363+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681111+0000","last_change":"2024-09-21T02:17:28.681111+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681111+0000","last_undegraded":"2024-09-21T02:17:28.681111+0000","last_fullsized":"2024-09-21T02:17:28.681111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685715+0000","last_change":"2024-09-21T02:17:28.685715+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685715+0000","last_undegraded":"2024-09-21T02:17:28.685715+0000","last_fullsized":"2024-09-21T02:17:28.685715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678448+0000","last_change":"2024-09-21T02:17:28.678448+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678448+0000","last_undegraded":"2024-09-21T02:17:28.678448+0000","last_fullsized":"2024-09-21T02:17:28.678448+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680042+0000","last_change":"2024-09-21T02:17:28.680042+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680042+0000","last_undegraded":"2024-09-21T02:17:28.680042+0000","last_fullsized":"2024-09-21T02:17:28.680042+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677092+0000","last_change":"2024-09-21T02:17:28.677092+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677092+0000","last_undegraded":"2024-09-21T02:17:28.677092+0000","last_fullsized":"2024-09-21T02:17:28.677092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678656+0000","last_change":"2024-09-21T02:17:28.678656+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678656+0000","last_undegraded":"2024-09-21T02:17:28.678656+0000","last_fullsized":"2024-09-21T02:17:28.678656+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.703566+0000","last_change":"2024-09-21T02:17:28.701807+0000","last_active":"2024-09-21T02:17:28.703566+0000","last_peered":"2024-09-21T02:17:28.703566+0000","last_clean":"2024-09-21T02:17:28.703566+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:28.703566+0000","last_undegraded":"2024-09-21T02:17:28.703566+0000","last_fullsized":"2024-09-21T02:17:28.703566+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.377682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684585+0000","last_change":"2024-09-21T02:17:28.684585+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684585+0000","last_undegraded":"2024-09-21T02:17:28.684585+0000","last_fullsized":"2024-09-21T02:17:28.684585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.716213+0000","last_change":"2024-09-21T02:17:28.709949+0000","last_active":"2024-09-21T02:17:28.716213+0000","last_peered":"2024-09-21T02:17:28.716213+0000","last_clean":"2024-09-21T02:17:28.716213+0000","last_became_active":"2024-09-21T02:17:28.709165+0000","last_became_peered":"2024-09-21T02:17:28.709165+0000","last_unstale":"2024-09-21T02:17:28.716213+0000","last_undegraded":"2024-09-21T02:17:28.716213+0000","last_fullsized":"2024-09-21T02:17:28.716213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:25.745781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680994+0000","last_change":"2024-09-21T02:17:28.680994+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680994+0000","last_undegraded":"2024-09-21T02:17:28.680994+0000","last_fullsized":"2024-09-21T02:17:28.680994+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.695050+0000","last_change":"2024-09-21T02:17:28.694169+0000","last_active":"2024-09-21T02:17:28.695050+0000","last_peered":"2024-09-21T02:17:28.695050+0000","last_clean":"2024-09-21T02:17:28.695050+0000","last_became_active":"2024-09-21T02:17:28.693855+0000","last_became_peered":"2024-09-21T02:17:28.693855+0000","last_unstale":"2024-09-21T02:17:28.695050+0000","last_undegraded":"2024-09-21T02:17:28.695050+0000","last_fullsized":"2024-09-21T02:17:28.695050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:59.351666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685956+0000","last_change":"2024-09-21T02:17:28.685956+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685956+0000","last_undegraded":"2024-09-21T02:17:28.685956+0000","last_fullsized":"2024-09-21T02:17:28.685956+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.705049+0000","last_change":"2024-09-21T02:17:28.704350+0000","last_active":"2024-09-21T02:17:28.705049+0000","last_peered":"2024-09-21T02:17:28.705049+0000","last_clean":"2024-09-21T02:17:28.705049+0000","last_became_active":"2024-09-21T02:17:28.703930+0000","last_became_peered":"2024-09-21T02:17:28.703930+0000","last_unstale":"2024-09-21T02:17:28.705049+0000","last_undegraded":"2024-09-21T02:17:28.705049+0000","last_fullsized":"2024-09-21T02:17:28.705049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:28.293800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677902+0000","last_change":"2024-09-21T02:17:28.677902+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677902+0000","last_undegraded":"2024-09-21T02:17:28.677902+0000","last_fullsized":"2024-09-21T02:17:28.677902+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.670252+0000","last_change":"2024-09-21T02:17:27.670252+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.670252+0000","last_undegraded":"2024-09-21T02:17:27.670252+0000","last_fullsized":"2024-09-21T02:17:27.670252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680022+0000","last_change":"2024-09-21T02:17:28.680022+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680022+0000","last_undegraded":"2024-09-21T02:17:28.680022+0000","last_fullsized":"2024-09-21T02:17:28.680022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702082+0000","last_change":"2024-09-21T02:17:28.700727+0000","last_active":"2024-09-21T02:17:28.702082+0000","last_peered":"2024-09-21T02:17:28.702082+0000","last_clean":"2024-09-21T02:17:28.702082+0000","last_became_active":"2024-09-21T02:17:28.700382+0000","last_became_peered":"2024-09-21T02:17:28.700382+0000","last_unstale":"2024-09-21T02:17:28.702082+0000","last_undegraded":"2024-09-21T02:17:28.702082+0000","last_fullsized":"2024-09-21T02:17:28.702082+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:47.409072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.694503+0000","last_change":"2024-09-21T02:17:28.694503+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.694503+0000","last_undegraded":"2024-09-21T02:17:28.694503+0000","last_fullsized":"2024-09-21T02:17:28.694503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.706889+0000","last_change":"2024-09-21T02:17:28.704257+0000","last_active":"2024-09-21T02:17:28.706889+0000","last_peered":"2024-09-21T02:17:28.706889+0000","last_clean":"2024-09-21T02:17:28.706889+0000","last_became_active":"2024-09-21T02:17:28.703356+0000","last_became_peered":"2024-09-21T02:17:28.703356+0000","last_unstale":"2024-09-21T02:17:28.706889+0000","last_undegraded":"2024-09-21T02:17:28.706889+0000","last_fullsized":"2024-09-21T02:17:28.706889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.637435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667720+0000","last_change":"2024-09-21T02:17:27.667720+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667720+0000","last_undegraded":"2024-09-21T02:17:27.667720+0000","last_fullsized":"2024-09-21T02:17:27.667720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686873+0000","last_change":"2024-09-21T02:17:28.686873+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686873+0000","last_undegraded":"2024-09-21T02:17:28.686873+0000","last_fullsized":"2024-09-21T02:17:28.686873+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.709826+0000","last_change":"2024-09-21T02:17:28.709078+0000","last_active":"2024-09-21T02:17:28.709826+0000","last_peered":"2024-09-21T02:17:28.709826+0000","last_clean":"2024-09-21T02:17:28.709826+0000","last_became_active":"2024-09-21T02:17:28.708778+0000","last_became_peered":"2024-09-21T02:17:28.708778+0000","last_unstale":"2024-09-21T02:17:28.709826+0000","last_undegraded":"2024-09-21T02:17:28.709826+0000","last_fullsized":"2024-09-21T02:17:28.709826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:47.678448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682103+0000","last_change":"2024-09-21T02:17:28.682103+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682103+0000","last_undegraded":"2024-09-21T02:17:28.682103+0000","last_fullsized":"2024-09-21T02:17:28.682103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698567+0000","last_change":"2024-09-21T02:17:28.697107+0000","last_active":"2024-09-21T02:17:28.698567+0000","last_peered":"2024-09-21T02:17:28.698567+0000","last_clean":"2024-09-21T02:17:28.698567+0000","last_became_active":"2024-09-21T02:17:28.696766+0000","last_became_peered":"2024-09-21T02:17:28.696766+0000","last_unstale":"2024-09-21T02:17:28.698567+0000","last_undegraded":"2024-09-21T02:17:28.698567+0000","last_fullsized":"2024-09-21T02:17:28.698567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:37.937427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680209+0000","last_change":"2024-09-21T02:17:28.680209+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680209+0000","last_undegraded":"2024-09-21T02:17:28.680209+0000","last_fullsized":"2024-09-21T02:17:28.680209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.671648+0000","last_change":"2024-09-21T02:17:27.671648+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.671648+0000","last_undegraded":"2024-09-21T02:17:27.671648+0000","last_fullsized":"2024-09-21T02:17:27.671648+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.674892+0000","last_change":"2024-09-21T02:17:28.674892+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.674892+0000","last_undegraded":"2024-09-21T02:17:28.674892+0000","last_fullsized":"2024-09-21T02:17:28.674892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.716794+0000","last_change":"2024-09-21T02:17:28.711641+0000","last_active":"2024-09-21T02:17:28.716794+0000","last_peered":"2024-09-21T02:17:28.716794+0000","last_clean":"2024-09-21T02:17:28.716794+0000","last_became_active":"2024-09-21T02:17:28.710785+0000","last_became_peered":"2024-09-21T02:17:28.710785+0000","last_unstale":"2024-09-21T02:17:28.716794+0000","last_undegraded":"2024-09-21T02:17:28.716794+0000","last_fullsized":"2024-09-21T02:17:28.716794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:10.915960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676986+0000","last_change":"2024-09-21T02:17:28.676986+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676986+0000","last_undegraded":"2024-09-21T02:17:28.676986+0000","last_fullsized":"2024-09-21T02:17:28.676986+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.707351+0000","last_change":"2024-09-21T02:17:28.705017+0000","last_active":"2024-09-21T02:17:28.707351+0000","last_peered":"2024-09-21T02:17:28.707351+0000","last_clean":"2024-09-21T02:17:28.707351+0000","last_became_active":"2024-09-21T02:17:28.704183+0000","last_became_peered":"2024-09-21T02:17:28.704183+0000","last_unstale":"2024-09-21T02:17:28.707351+0000","last_undegraded":"2024-09-21T02:17:28.707351+0000","last_fullsized":"2024-09-21T02:17:28.707351+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:01.558053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687837+0000","last_change":"2024-09-21T02:17:28.687837+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687837+0000","last_undegraded":"2024-09-21T02:17:28.687837+0000","last_fullsized":"2024-09-21T02:17:28.687837+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.696201+0000","last_change":"2024-09-21T02:17:28.695228+0000","last_active":"2024-09-21T02:17:28.696201+0000","last_peered":"2024-09-21T02:17:28.696201+0000","last_clean":"2024-09-21T02:17:28.696201+0000","last_became_active":"2024-09-21T02:17:28.694899+0000","last_became_peered":"2024-09-21T02:17:28.694899+0000","last_unstale":"2024-09-21T02:17:28.696201+0000","last_undegraded":"2024-09-21T02:17:28.696201+0000","last_fullsized":"2024-09-21T02:17:28.696201+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:21.720911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676604+0000","last_change":"2024-09-21T02:17:28.676604+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676604+0000","last_undegraded":"2024-09-21T02:17:28.676604+0000","last_fullsized":"2024-09-21T02:17:28.676604+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.715839+0000","last_change":"2024-09-21T02:17:28.715260+0000","last_active":"2024-09-21T02:17:28.715839+0000","last_peered":"2024-09-21T02:17:28.715839+0000","last_clean":"2024-09-21T02:17:28.715839+0000","last_became_active":"2024-09-21T02:17:28.715003+0000","last_became_peered":"2024-09-21T02:17:28.715003+0000","last_unstale":"2024-09-21T02:17:28.715839+0000","last_undegraded":"2024-09-21T02:17:28.715839+0000","last_fullsized":"2024-09-21T02:17:28.715839+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:40.200566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690971+0000","last_change":"2024-09-21T02:17:28.690971+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690971+0000","last_undegraded":"2024-09-21T02:17:28.690971+0000","last_fullsized":"2024-09-21T02:17:28.690971+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.697964+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:28.697964+0000","last_peered":"2024-09-21T02:17:28.697964+0000","last_clean":"2024-09-21T02:17:28.697964+0000","last_became_active":"2024-09-21T02:17:28.696779+0000","last_became_peered":"2024-09-21T02:17:28.696779+0000","last_unstale":"2024-09-21T02:17:28.697964+0000","last_undegraded":"2024-09-21T02:17:28.697964+0000","last_fullsized":"2024-09-21T02:17:28.697964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:57.291183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681255+0000","last_change":"2024-09-21T02:17:28.681255+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681255+0000","last_undegraded":"2024-09-21T02:17:28.681255+0000","last_fullsized":"2024-09-21T02:17:28.681255+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698993+0000","last_change":"2024-09-21T02:17:28.697982+0000","last_active":"2024-09-21T02:17:28.698993+0000","last_peered":"2024-09-21T02:17:28.698993+0000","last_clean":"2024-09-21T02:17:28.698993+0000","last_became_active":"2024-09-21T02:17:28.697661+0000","last_became_peered":"2024-09-21T02:17:28.697661+0000","last_unstale":"2024-09-21T02:17:28.698993+0000","last_undegraded":"2024-09-21T02:17:28.698993+0000","last_fullsized":"2024-09-21T02:17:28.698993+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:44.914475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680404+0000","last_change":"2024-09-21T02:17:28.680404+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680404+0000","last_undegraded":"2024-09-21T02:17:28.680404+0000","last_fullsized":"2024-09-21T02:17:28.680404+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.693106+0000","last_change":"2024-09-21T02:17:28.691325+0000","last_active":"2024-09-21T02:17:28.693106+0000","last_peered":"2024-09-21T02:17:28.693106+0000","last_clean":"2024-09-21T02:17:28.693106+0000","last_became_active":"2024-09-21T02:17:28.691006+0000","last_became_peered":"2024-09-21T02:17:28.691006+0000","last_unstale":"2024-09-21T02:17:28.693106+0000","last_undegraded":"2024-09-21T02:17:28.693106+0000","last_fullsized":"2024-09-21T02:17:28.693106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:30.202022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684275+0000","last_change":"2024-09-21T02:17:28.684275+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684275+0000","last_undegraded":"2024-09-21T02:17:28.684275+0000","last_fullsized":"2024-09-21T02:17:28.684275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.695625+0000","last_change":"2024-09-21T02:17:28.694528+0000","last_active":"2024-09-21T02:17:28.695625+0000","last_peered":"2024-09-21T02:17:28.695625+0000","last_clean":"2024-09-21T02:17:28.695625+0000","last_became_active":"2024-09-21T02:17:28.694184+0000","last_became_peered":"2024-09-21T02:17:28.694184+0000","last_unstale":"2024-09-21T02:17:28.695625+0000","last_undegraded":"2024-09-21T02:17:28.695625+0000","last_fullsized":"2024-09-21T02:17:28.695625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:38.024426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684205+0000","last_change":"2024-09-21T02:17:28.684205+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684205+0000","last_undegraded":"2024-09-21T02:17:28.684205+0000","last_fullsized":"2024-09-21T02:17:28.684205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.699813+0000","last_change":"2024-09-21T02:17:28.695543+0000","last_active":"2024-09-21T02:17:28.699813+0000","last_peered":"2024-09-21T02:17:28.699813+0000","last_clean":"2024-09-21T02:17:28.699813+0000","last_became_active":"2024-09-21T02:17:28.694827+0000","last_became_peered":"2024-09-21T02:17:28.694827+0000","last_unstale":"2024-09-21T02:17:28.699813+0000","last_undegraded":"2024-09-21T02:17:28.699813+0000","last_fullsized":"2024-09-21T02:17:28.699813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:13.363085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690155+0000","last_change":"2024-09-21T02:17:28.690155+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690155+0000","last_undegraded":"2024-09-21T02:17:28.690155+0000","last_fullsized":"2024-09-21T02:17:28.690155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.703540+0000","last_change":"2024-09-21T02:17:28.702105+0000","last_active":"2024-09-21T02:17:28.703540+0000","last_peered":"2024-09-21T02:17:28.703540+0000","last_clean":"2024-09-21T02:17:28.703540+0000","last_became_active":"2024-09-21T02:17:28.701785+0000","last_became_peered":"2024-09-21T02:17:28.701785+0000","last_unstale":"2024-09-21T02:17:28.703540+0000","last_undegraded":"2024-09-21T02:17:28.703540+0000","last_fullsized":"2024-09-21T02:17:28.703540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:04.860266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.691292+0000","last_change":"2024-09-21T02:17:28.691292+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.691292+0000","last_undegraded":"2024-09-21T02:17:28.691292+0000","last_fullsized":"2024-09-21T02:17:28.691292+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702589+0000","last_change":"2024-09-21T02:17:28.700846+0000","last_active":"2024-09-21T02:17:28.702589+0000","last_peered":"2024-09-21T02:17:28.702589+0000","last_clean":"2024-09-21T02:17:28.702589+0000","last_became_active":"2024-09-21T02:17:28.700490+0000","last_became_peered":"2024-09-21T02:17:28.700490+0000","last_unstale":"2024-09-21T02:17:28.702589+0000","last_undegraded":"2024-09-21T02:17:28.702589+0000","last_fullsized":"2024-09-21T02:17:28.702589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:37.333804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.715131+0000","last_change":"2024-09-21T02:17:28.706151+0000","last_active":"2024-09-21T02:17:28.715131+0000","last_peered":"2024-09-21T02:17:28.715131+0000","last_clean":"2024-09-21T02:17:28.715131+0000","last_became_active":"2024-09-21T02:17:28.705306+0000","last_became_peered":"2024-09-21T02:17:28.705306+0000","last_unstale":"2024-09-21T02:17:28.715131+0000","last_undegraded":"2024-09-21T02:17:28.715131+0000","last_fullsized":"2024-09-21T02:17:28.715131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:50.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678875+0000","last_change":"2024-09-21T02:17:28.678875+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678875+0000","last_undegraded":"2024-09-21T02:17:28.678875+0000","last_fullsized":"2024-09-21T02:17:28.678875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698755+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:28.698755+0000","last_peered":"2024-09-21T02:17:28.698755+0000","last_clean":"2024-09-21T02:17:28.698755+0000","last_became_active":"2024-09-21T02:17:28.696772+0000","last_became_peered":"2024-09-21T02:17:28.696772+0000","last_unstale":"2024-09-21T02:17:28.698755+0000","last_undegraded":"2024-09-21T02:17:28.698755+0000","last_fullsized":"2024-09-21T02:17:28.698755+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:56.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685181+0000","last_change":"2024-09-21T02:17:28.685181+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685181+0000","last_undegraded":"2024-09-21T02:17:28.685181+0000","last_fullsized":"2024-09-21T02:17:28.685181+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.693796+0000","last_change":"2024-09-21T02:17:28.692883+0000","last_active":"2024-09-21T02:17:28.693796+0000","last_peered":"2024-09-21T02:17:28.693796+0000","last_clean":"2024-09-21T02:17:28.693796+0000","last_became_active":"2024-09-21T02:17:28.692516+0000","last_became_peered":"2024-09-21T02:17:28.692516+0000","last_unstale":"2024-09-21T02:17:28.693796+0000","last_undegraded":"2024-09-21T02:17:28.693796+0000","last_fullsized":"2024-09-21T02:17:28.693796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:46.327406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680004+0000","last_change":"2024-09-21T02:17:28.680004+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680004+0000","last_undegraded":"2024-09-21T02:17:28.680004+0000","last_fullsized":"2024-09-21T02:17:28.680004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.670227+0000","last_change":"2024-09-21T02:17:27.670227+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.670227+0000","last_undegraded":"2024-09-21T02:17:27.670227+0000","last_fullsized":"2024-09-21T02:17:27.670227+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685323+0000","last_change":"2024-09-21T02:17:28.685323+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685323+0000","last_undegraded":"2024-09-21T02:17:28.685323+0000","last_fullsized":"2024-09-21T02:17:28.685323+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667752+0000","last_change":"2024-09-21T02:17:27.667752+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667752+0000","last_undegraded":"2024-09-21T02:17:27.667752+0000","last_fullsized":"2024-09-21T02:17:27.667752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676870+0000","last_change":"2024-09-21T02:17:28.676870+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676870+0000","last_undegraded":"2024-09-21T02:17:28.676870+0000","last_fullsized":"2024-09-21T02:17:28.676870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667737+0000","last_change":"2024-09-21T02:17:27.667737+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667737+0000","last_undegraded":"2024-09-21T02:17:27.667737+0000","last_fullsized":"2024-09-21T02:17:27.667737+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678057+0000","last_change":"2024-09-21T02:17:28.678057+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678057+0000","last_undegraded":"2024-09-21T02:17:28.678057+0000","last_fullsized":"2024-09-21T02:17:28.678057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.706342+0000","last_change":"2024-09-21T02:17:28.703261+0000","last_active":"2024-09-21T02:17:28.706342+0000","last_peered":"2024-09-21T02:17:28.706342+0000","last_clean":"2024-09-21T02:17:28.706342+0000","last_became_active":"2024-09-21T02:17:28.702239+0000","last_became_peered":"2024-09-21T02:17:28.702239+0000","last_unstale":"2024-09-21T02:17:28.706342+0000","last_undegraded":"2024-09-21T02:17:28.706342+0000","last_fullsized":"2024-09-21T02:17:28.706342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689896+0000","last_change":"2024-09-21T02:17:28.689896+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689896+0000","last_undegraded":"2024-09-21T02:17:28.689896+0000","last_fullsized":"2024-09-21T02:17:28.689896+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.687693+0000","last_change":"2024-09-21T02:17:28.686833+0000","last_active":"2024-09-21T02:17:28.687693+0000","last_peered":"2024-09-21T02:17:28.687693+0000","last_clean":"2024-09-21T02:17:28.687693+0000","last_became_active":"2024-09-21T02:17:28.686533+0000","last_became_peered":"2024-09-21T02:17:28.686533+0000","last_unstale":"2024-09-21T02:17:28.687693+0000","last_undegraded":"2024-09-21T02:17:28.687693+0000","last_fullsized":"2024-09-21T02:17:28.687693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.923032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677868+0000","last_change":"2024-09-21T02:17:28.677868+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677868+0000","last_undegraded":"2024-09-21T02:17:28.677868+0000","last_fullsized":"2024-09-21T02:17:28.677868+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.710276+0000","last_change":"2024-09-21T02:17:28.709392+0000","last_active":"2024-09-21T02:17:28.710276+0000","last_peered":"2024-09-21T02:17:28.710276+0000","last_clean":"2024-09-21T02:17:28.710276+0000","last_became_active":"2024-09-21T02:17:28.708868+0000","last_became_peered":"2024-09-21T02:17:28.708868+0000","last_unstale":"2024-09-21T02:17:28.710276+0000","last_undegraded":"2024-09-21T02:17:28.710276+0000","last_fullsized":"2024-09-21T02:17:28.710276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:04.119473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678082+0000","last_change":"2024-09-21T02:17:28.678082+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678082+0000","last_undegraded":"2024-09-21T02:17:28.678082+0000","last_fullsized":"2024-09-21T02:17:28.678082+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702334+0000","last_change":"2024-09-21T02:17:28.701648+0000","last_active":"2024-09-21T02:17:28.702334+0000","last_peered":"2024-09-21T02:17:28.702334+0000","last_clean":"2024-09-21T02:17:28.702334+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:28.702334+0000","last_undegraded":"2024-09-21T02:17:28.702334+0000","last_fullsized":"2024-09-21T02:17:28.702334+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:12.548515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":196,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.688710+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:17:28.688710+0000","last_peered":"2024-09-21T02:17:28.688710+0000","last_clean":"2024-09-21T02:17:28.688710+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:17:28.688710+0000","last_undegraded":"2024-09-21T02:17:28.688710+0000","last_fullsized":"2024-09-21T02:17:28.688710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":352330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1425408,"data_stored":936431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":843776,"data_stored":352330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1437696,"data_stored":942698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":819200,"data_stored":340246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:17:33.881 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 -- 172.21.15.170:0/3051743422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b22b0 msgr2=0x7f57200b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 --2- 172.21.15.170:0/3051743422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b22b0 0x7f57200b1730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5720073a30 tx=0x7f571001cce0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 -- 172.21.15.170:0/3051743422 shutdown_connections 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 --2- 172.21.15.170:0/3051743422 >> v2:172.21.15.191:3300/0 conn(0x7f572006d510 0x7f5720160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 --2- 172.21.15.170:0/3051743422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b22b0 0x7f57200b1730 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 -- 172.21.15.170:0/3051743422 >> 172.21.15.170:0/3051743422 conn(0x7f572009a9c0 msgr2=0x7f57200d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 -- 172.21.15.170:0/3051743422 shutdown_connections 2024-09-21T02:17:34.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.068+0000 7f5726d59640 1 -- 172.21.15.170:0/3051743422 wait complete. 2024-09-21T02:17:34.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 Processor -- start 2024-09-21T02:17:34.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- start start 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57200b22b0 0x7f5720156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 0x7f57200c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57201573c0 0x7f57200c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f572015c700 con 0x7f57200b6930 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f572015c580 con 0x7f57201573c0 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f572015c880 con 0x7f57200b22b0 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 0x7f57200c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 0x7f57200c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47566/0 (socket says 172.21.15.170:47566) 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 -- 172.21.15.170:0/352542189 learned_addr learned my addr 172.21.15.170:0/352542189 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:34.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5724af5640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57200b22b0 0x7f5720156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 -- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57200b22b0 msgr2=0x7f5720156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57200b22b0 0x7f5720156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f57252f6640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57201573c0 0x7f57200c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 -- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57201573c0 msgr2=0x7f57200c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57201573c0 0x7f57200c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 -- 172.21.15.170:0/352542189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57200ae730 con 0x7f57200b6930 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571ffff640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 0x7f57200c67e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f571400b3e0 tx=0x7f571400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714015020 con 0x7f57200b6930 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5714011940 con 0x7f57200b6930 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57200aea10 con 0x7f57200b6930 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714011d90 con 0x7f57200b6930 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57200af4d0 con 0x7f57200b6930 2024-09-21T02:17:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.072+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57200ae4a0 con 0x7f57200b6930 2024-09-21T02:17:34.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.076+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5714018020 con 0x7f57200b6930 2024-09-21T02:17:34.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.076+0000 7f571d7fa640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f570c073590 0x7f570c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.076+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f5714094820 con 0x7f57200b6930 2024-09-21T02:17:34.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.076+0000 7f5724af5640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f570c073590 0x7f570c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.076+0000 7f5724af5640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f570c073590 0x7f570c075aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57100af000 tx=0x7f571001b520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:34.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.080+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f571401f020 con 0x7f57200b6930 2024-09-21T02:17:34.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.248+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5720075140 con 0x7f570c073590 2024-09-21T02:17:34.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.260+0000 7f571d7fa640 1 -- 172.21.15.170:0/352542189 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245894 (secure 0 0 0) 0x7f5720075140 con 0x7f570c073590 2024-09-21T02:17:34.266 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:34.267 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:17:34.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f570c073590 msgr2=0x7f570c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f570c073590 0x7f570c075aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57100af000 tx=0x7f571001b520 comp rx=0 tx=0).stop 2024-09-21T02:17:34.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 msgr2=0x7f57200c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 0x7f57200c67e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f571400b3e0 tx=0x7f571400b8a0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 shutdown_connections 2024-09-21T02:17:34.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f570c073590 0x7f570c075aa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57201573c0 0x7f57200c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57200b6930 0x7f57200c67e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 --2- 172.21.15.170:0/352542189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57200b22b0 0x7f5720156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 >> 172.21.15.170:0/352542189 conn(0x7f572009a9c0 msgr2=0x7f57200716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:34.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 shutdown_connections 2024-09-21T02:17:34.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.264+0000 7f5726d59640 1 -- 172.21.15.170:0/352542189 wait complete. 2024-09-21T02:17:34.286 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-21T02:17:32.961564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269708,"kb_used_data":7680,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754705012,"statfs":{"total":773094113280,"available":772817932288,"internally_reserved":0,"allocated":7864320,"data_stored":3962224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.303810"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679110+0000","last_change":"2024-09-21T02:17:28.679110+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679110+0000","last_undegraded":"2024-09-21T02:17:28.679110+0000","last_fullsized":"2024-09-21T02:17:28.679110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682492+0000","last_change":"2024-09-21T02:17:28.682492+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682492+0000","last_undegraded":"2024-09-21T02:17:28.682492+0000","last_fullsized":"2024-09-21T02:17:28.682492+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689700+0000","last_change":"2024-09-21T02:17:28.689700+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689700+0000","last_undegraded":"2024-09-21T02:17:28.689700+0000","last_fullsized":"2024-09-21T02:17:28.689700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690481+0000","last_change":"2024-09-21T02:17:28.690481+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690481+0000","last_undegraded":"2024-09-21T02:17:28.690481+0000","last_fullsized":"2024-09-21T02:17:28.690481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.675278+0000","last_change":"2024-09-21T02:17:28.675278+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.675278+0000","last_undegraded":"2024-09-21T02:17:28.675278+0000","last_fullsized":"2024-09-21T02:17:28.675278+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679207+0000","last_change":"2024-09-21T02:17:28.679207+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679207+0000","last_undegraded":"2024-09-21T02:17:28.679207+0000","last_fullsized":"2024-09-21T02:17:28.679207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689631+0000","last_change":"2024-09-21T02:17:28.689631+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689631+0000","last_undegraded":"2024-09-21T02:17:28.689631+0000","last_fullsized":"2024-09-21T02:17:28.689631+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682248+0000","last_change":"2024-09-21T02:17:28.682248+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682248+0000","last_undegraded":"2024-09-21T02:17:28.682248+0000","last_fullsized":"2024-09-21T02:17:28.682248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686730+0000","last_change":"2024-09-21T02:17:28.686730+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686730+0000","last_undegraded":"2024-09-21T02:17:28.686730+0000","last_fullsized":"2024-09-21T02:17:28.686730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687831+0000","last_change":"2024-09-21T02:17:28.687831+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687831+0000","last_undegraded":"2024-09-21T02:17:28.687831+0000","last_fullsized":"2024-09-21T02:17:28.687831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678223+0000","last_change":"2024-09-21T02:17:28.678223+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678223+0000","last_undegraded":"2024-09-21T02:17:28.678223+0000","last_fullsized":"2024-09-21T02:17:28.678223+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687804+0000","last_change":"2024-09-21T02:17:28.687804+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687804+0000","last_undegraded":"2024-09-21T02:17:28.687804+0000","last_fullsized":"2024-09-21T02:17:28.687804+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681761+0000","last_change":"2024-09-21T02:17:28.681761+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681761+0000","last_undegraded":"2024-09-21T02:17:28.681761+0000","last_fullsized":"2024-09-21T02:17:28.681761+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677586+0000","last_change":"2024-09-21T02:17:28.677586+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677586+0000","last_undegraded":"2024-09-21T02:17:28.677586+0000","last_fullsized":"2024-09-21T02:17:28.677586+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680007+0000","last_change":"2024-09-21T02:17:28.680007+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680007+0000","last_undegraded":"2024-09-21T02:17:28.680007+0000","last_fullsized":"2024-09-21T02:17:28.680007+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679210+0000","last_change":"2024-09-21T02:17:28.679210+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679210+0000","last_undegraded":"2024-09-21T02:17:28.679210+0000","last_fullsized":"2024-09-21T02:17:28.679210+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.675097+0000","last_change":"2024-09-21T02:17:28.675097+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.675097+0000","last_undegraded":"2024-09-21T02:17:28.675097+0000","last_fullsized":"2024-09-21T02:17:28.675097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682671+0000","last_change":"2024-09-21T02:17:28.682671+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682671+0000","last_undegraded":"2024-09-21T02:17:28.682671+0000","last_fullsized":"2024-09-21T02:17:28.682671+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678167+0000","last_change":"2024-09-21T02:17:28.678167+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678167+0000","last_undegraded":"2024-09-21T02:17:28.678167+0000","last_fullsized":"2024-09-21T02:17:28.678167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689162+0000","last_change":"2024-09-21T02:17:28.689162+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689162+0000","last_undegraded":"2024-09-21T02:17:28.689162+0000","last_fullsized":"2024-09-21T02:17:28.689162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677094+0000","last_change":"2024-09-21T02:17:28.677094+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677094+0000","last_undegraded":"2024-09-21T02:17:28.677094+0000","last_fullsized":"2024-09-21T02:17:28.677094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686054+0000","last_change":"2024-09-21T02:17:28.686054+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686054+0000","last_undegraded":"2024-09-21T02:17:28.686054+0000","last_fullsized":"2024-09-21T02:17:28.686054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679394+0000","last_change":"2024-09-21T02:17:28.679394+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679394+0000","last_undegraded":"2024-09-21T02:17:28.679394+0000","last_fullsized":"2024-09-21T02:17:28.679394+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676363+0000","last_change":"2024-09-21T02:17:28.676363+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676363+0000","last_undegraded":"2024-09-21T02:17:28.676363+0000","last_fullsized":"2024-09-21T02:17:28.676363+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681111+0000","last_change":"2024-09-21T02:17:28.681111+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681111+0000","last_undegraded":"2024-09-21T02:17:28.681111+0000","last_fullsized":"2024-09-21T02:17:28.681111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685715+0000","last_change":"2024-09-21T02:17:28.685715+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685715+0000","last_undegraded":"2024-09-21T02:17:28.685715+0000","last_fullsized":"2024-09-21T02:17:28.685715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678448+0000","last_change":"2024-09-21T02:17:28.678448+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678448+0000","last_undegraded":"2024-09-21T02:17:28.678448+0000","last_fullsized":"2024-09-21T02:17:28.678448+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680042+0000","last_change":"2024-09-21T02:17:28.680042+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680042+0000","last_undegraded":"2024-09-21T02:17:28.680042+0000","last_fullsized":"2024-09-21T02:17:28.680042+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677092+0000","last_change":"2024-09-21T02:17:28.677092+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677092+0000","last_undegraded":"2024-09-21T02:17:28.677092+0000","last_fullsized":"2024-09-21T02:17:28.677092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678656+0000","last_change":"2024-09-21T02:17:28.678656+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678656+0000","last_undegraded":"2024-09-21T02:17:28.678656+0000","last_fullsized":"2024-09-21T02:17:28.678656+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.703566+0000","last_change":"2024-09-21T02:17:28.701807+0000","last_active":"2024-09-21T02:17:28.703566+0000","last_peered":"2024-09-21T02:17:28.703566+0000","last_clean":"2024-09-21T02:17:28.703566+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:28.703566+0000","last_undegraded":"2024-09-21T02:17:28.703566+0000","last_fullsized":"2024-09-21T02:17:28.703566+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.377682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684585+0000","last_change":"2024-09-21T02:17:28.684585+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684585+0000","last_undegraded":"2024-09-21T02:17:28.684585+0000","last_fullsized":"2024-09-21T02:17:28.684585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.716213+0000","last_change":"2024-09-21T02:17:28.709949+0000","last_active":"2024-09-21T02:17:28.716213+0000","last_peered":"2024-09-21T02:17:28.716213+0000","last_clean":"2024-09-21T02:17:28.716213+0000","last_became_active":"2024-09-21T02:17:28.709165+0000","last_became_peered":"2024-09-21T02:17:28.709165+0000","last_unstale":"2024-09-21T02:17:28.716213+0000","last_undegraded":"2024-09-21T02:17:28.716213+0000","last_fullsized":"2024-09-21T02:17:28.716213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:25.745781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680994+0000","last_change":"2024-09-21T02:17:28.680994+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680994+0000","last_undegraded":"2024-09-21T02:17:28.680994+0000","last_fullsized":"2024-09-21T02:17:28.680994+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.695050+0000","last_change":"2024-09-21T02:17:28.694169+0000","last_active":"2024-09-21T02:17:28.695050+0000","last_peered":"2024-09-21T02:17:28.695050+0000","last_clean":"2024-09-21T02:17:28.695050+0000","last_became_active":"2024-09-21T02:17:28.693855+0000","last_became_peered":"2024-09-21T02:17:28.693855+0000","last_unstale":"2024-09-21T02:17:28.695050+0000","last_undegraded":"2024-09-21T02:17:28.695050+0000","last_fullsized":"2024-09-21T02:17:28.695050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:59.351666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685956+0000","last_change":"2024-09-21T02:17:28.685956+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685956+0000","last_undegraded":"2024-09-21T02:17:28.685956+0000","last_fullsized":"2024-09-21T02:17:28.685956+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.705049+0000","last_change":"2024-09-21T02:17:28.704350+0000","last_active":"2024-09-21T02:17:28.705049+0000","last_peered":"2024-09-21T02:17:28.705049+0000","last_clean":"2024-09-21T02:17:28.705049+0000","last_became_active":"2024-09-21T02:17:28.703930+0000","last_became_peered":"2024-09-21T02:17:28.703930+0000","last_unstale":"2024-09-21T02:17:28.705049+0000","last_undegraded":"2024-09-21T02:17:28.705049+0000","last_fullsized":"2024-09-21T02:17:28.705049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:28.293800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677902+0000","last_change":"2024-09-21T02:17:28.677902+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677902+0000","last_undegraded":"2024-09-21T02:17:28.677902+0000","last_fullsized":"2024-09-21T02:17:28.677902+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.670252+0000","last_change":"2024-09-21T02:17:27.670252+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.670252+0000","last_undegraded":"2024-09-21T02:17:27.670252+0000","last_fullsized":"2024-09-21T02:17:27.670252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680022+0000","last_change":"2024-09-21T02:17:28.680022+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680022+0000","last_undegraded":"2024-09-21T02:17:28.680022+0000","last_fullsized":"2024-09-21T02:17:28.680022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702082+0000","last_change":"2024-09-21T02:17:28.700727+0000","last_active":"2024-09-21T02:17:28.702082+0000","last_peered":"2024-09-21T02:17:28.702082+0000","last_clean":"2024-09-21T02:17:28.702082+0000","last_became_active":"2024-09-21T02:17:28.700382+0000","last_became_peered":"2024-09-21T02:17:28.700382+0000","last_unstale":"2024-09-21T02:17:28.702082+0000","last_undegraded":"2024-09-21T02:17:28.702082+0000","last_fullsized":"2024-09-21T02:17:28.702082+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:47.409072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.694503+0000","last_change":"2024-09-21T02:17:28.694503+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.694503+0000","last_undegraded":"2024-09-21T02:17:28.694503+0000","last_fullsized":"2024-09-21T02:17:28.694503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.706889+0000","last_change":"2024-09-21T02:17:28.704257+0000","last_active":"2024-09-21T02:17:28.706889+0000","last_peered":"2024-09-21T02:17:28.706889+0000","last_clean":"2024-09-21T02:17:28.706889+0000","last_became_active":"2024-09-21T02:17:28.703356+0000","last_became_peered":"2024-09-21T02:17:28.703356+0000","last_unstale":"2024-09-21T02:17:28.706889+0000","last_undegraded":"2024-09-21T02:17:28.706889+0000","last_fullsized":"2024-09-21T02:17:28.706889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.637435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667720+0000","last_change":"2024-09-21T02:17:27.667720+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667720+0000","last_undegraded":"2024-09-21T02:17:27.667720+0000","last_fullsized":"2024-09-21T02:17:27.667720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686873+0000","last_change":"2024-09-21T02:17:28.686873+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686873+0000","last_undegraded":"2024-09-21T02:17:28.686873+0000","last_fullsized":"2024-09-21T02:17:28.686873+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.709826+0000","last_change":"2024-09-21T02:17:28.709078+0000","last_active":"2024-09-21T02:17:28.709826+0000","last_peered":"2024-09-21T02:17:28.709826+0000","last_clean":"2024-09-21T02:17:28.709826+0000","last_became_active":"2024-09-21T02:17:28.708778+0000","last_became_peered":"2024-09-21T02:17:28.708778+0000","last_unstale":"2024-09-21T02:17:28.709826+0000","last_undegraded":"2024-09-21T02:17:28.709826+0000","last_fullsized":"2024-09-21T02:17:28.709826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:47.678448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682103+0000","last_change":"2024-09-21T02:17:28.682103+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682103+0000","last_undegraded":"2024-09-21T02:17:28.682103+0000","last_fullsized":"2024-09-21T02:17:28.682103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698567+0000","last_change":"2024-09-21T02:17:28.697107+0000","last_active":"2024-09-21T02:17:28.698567+0000","last_peered":"2024-09-21T02:17:28.698567+0000","last_clean":"2024-09-21T02:17:28.698567+0000","last_became_active":"2024-09-21T02:17:28.696766+0000","last_became_peered":"2024-09-21T02:17:28.696766+0000","last_unstale":"2024-09-21T02:17:28.698567+0000","last_undegraded":"2024-09-21T02:17:28.698567+0000","last_fullsized":"2024-09-21T02:17:28.698567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:37.937427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680209+0000","last_change":"2024-09-21T02:17:28.680209+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680209+0000","last_undegraded":"2024-09-21T02:17:28.680209+0000","last_fullsized":"2024-09-21T02:17:28.680209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.671648+0000","last_change":"2024-09-21T02:17:27.671648+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.671648+0000","last_undegraded":"2024-09-21T02:17:27.671648+0000","last_fullsized":"2024-09-21T02:17:27.671648+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.674892+0000","last_change":"2024-09-21T02:17:28.674892+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.674892+0000","last_undegraded":"2024-09-21T02:17:28.674892+0000","last_fullsized":"2024-09-21T02:17:28.674892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.716794+0000","last_change":"2024-09-21T02:17:28.711641+0000","last_active":"2024-09-21T02:17:28.716794+0000","last_peered":"2024-09-21T02:17:28.716794+0000","last_clean":"2024-09-21T02:17:28.716794+0000","last_became_active":"2024-09-21T02:17:28.710785+0000","last_became_peered":"2024-09-21T02:17:28.710785+0000","last_unstale":"2024-09-21T02:17:28.716794+0000","last_undegraded":"2024-09-21T02:17:28.716794+0000","last_fullsized":"2024-09-21T02:17:28.716794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:10.915960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676986+0000","last_change":"2024-09-21T02:17:28.676986+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676986+0000","last_undegraded":"2024-09-21T02:17:28.676986+0000","last_fullsized":"2024-09-21T02:17:28.676986+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.707351+0000","last_change":"2024-09-21T02:17:28.705017+0000","last_active":"2024-09-21T02:17:28.707351+0000","last_peered":"2024-09-21T02:17:28.707351+0000","last_clean":"2024-09-21T02:17:28.707351+0000","last_became_active":"2024-09-21T02:17:28.704183+0000","last_became_peered":"2024-09-21T02:17:28.704183+0000","last_unstale":"2024-09-21T02:17:28.707351+0000","last_undegraded":"2024-09-21T02:17:28.707351+0000","last_fullsized":"2024-09-21T02:17:28.707351+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:01.558053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687837+0000","last_change":"2024-09-21T02:17:28.687837+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687837+0000","last_undegraded":"2024-09-21T02:17:28.687837+0000","last_fullsized":"2024-09-21T02:17:28.687837+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.696201+0000","last_change":"2024-09-21T02:17:28.695228+0000","last_active":"2024-09-21T02:17:28.696201+0000","last_peered":"2024-09-21T02:17:28.696201+0000","last_clean":"2024-09-21T02:17:28.696201+0000","last_became_active":"2024-09-21T02:17:28.694899+0000","last_became_peered":"2024-09-21T02:17:28.694899+0000","last_unstale":"2024-09-21T02:17:28.696201+0000","last_undegraded":"2024-09-21T02:17:28.696201+0000","last_fullsized":"2024-09-21T02:17:28.696201+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:21.720911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676604+0000","last_change":"2024-09-21T02:17:28.676604+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676604+0000","last_undegraded":"2024-09-21T02:17:28.676604+0000","last_fullsized":"2024-09-21T02:17:28.676604+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.715839+0000","last_change":"2024-09-21T02:17:28.715260+0000","last_active":"2024-09-21T02:17:28.715839+0000","last_peered":"2024-09-21T02:17:28.715839+0000","last_clean":"2024-09-21T02:17:28.715839+0000","last_became_active":"2024-09-21T02:17:28.715003+0000","last_became_peered":"2024-09-21T02:17:28.715003+0000","last_unstale":"2024-09-21T02:17:28.715839+0000","last_undegraded":"2024-09-21T02:17:28.715839+0000","last_fullsized":"2024-09-21T02:17:28.715839+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:40.200566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690971+0000","last_change":"2024-09-21T02:17:28.690971+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690971+0000","last_undegraded":"2024-09-21T02:17:28.690971+0000","last_fullsized":"2024-09-21T02:17:28.690971+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.697964+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:28.697964+0000","last_peered":"2024-09-21T02:17:28.697964+0000","last_clean":"2024-09-21T02:17:28.697964+0000","last_became_active":"2024-09-21T02:17:28.696779+0000","last_became_peered":"2024-09-21T02:17:28.696779+0000","last_unstale":"2024-09-21T02:17:28.697964+0000","last_undegraded":"2024-09-21T02:17:28.697964+0000","last_fullsized":"2024-09-21T02:17:28.697964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:57.291183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681255+0000","last_change":"2024-09-21T02:17:28.681255+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681255+0000","last_undegraded":"2024-09-21T02:17:28.681255+0000","last_fullsized":"2024-09-21T02:17:28.681255+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698993+0000","last_change":"2024-09-21T02:17:28.697982+0000","last_active":"2024-09-21T02:17:28.698993+0000","last_peered":"2024-09-21T02:17:28.698993+0000","last_clean":"2024-09-21T02:17:28.698993+0000","last_became_active":"2024-09-21T02:17:28.697661+0000","last_became_peered":"2024-09-21T02:17:28.697661+0000","last_unstale":"2024-09-21T02:17:28.698993+0000","last_undegraded":"2024-09-21T02:17:28.698993+0000","last_fullsized":"2024-09-21T02:17:28.698993+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:44.914475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680404+0000","last_change":"2024-09-21T02:17:28.680404+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680404+0000","last_undegraded":"2024-09-21T02:17:28.680404+0000","last_fullsized":"2024-09-21T02:17:28.680404+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.693106+0000","last_change":"2024-09-21T02:17:28.691325+0000","last_active":"2024-09-21T02:17:28.693106+0000","last_peered":"2024-09-21T02:17:28.693106+0000","last_clean":"2024-09-21T02:17:28.693106+0000","last_became_active":"2024-09-21T02:17:28.691006+0000","last_became_peered":"2024-09-21T02:17:28.691006+0000","last_unstale":"2024-09-21T02:17:28.693106+0000","last_undegraded":"2024-09-21T02:17:28.693106+0000","last_fullsized":"2024-09-21T02:17:28.693106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:30.202022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684275+0000","last_change":"2024-09-21T02:17:28.684275+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684275+0000","last_undegraded":"2024-09-21T02:17:28.684275+0000","last_fullsized":"2024-09-21T02:17:28.684275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.695625+0000","last_change":"2024-09-21T02:17:28.694528+0000","last_active":"2024-09-21T02:17:28.695625+0000","last_peered":"2024-09-21T02:17:28.695625+0000","last_clean":"2024-09-21T02:17:28.695625+0000","last_became_active":"2024-09-21T02:17:28.694184+0000","last_became_peered":"2024-09-21T02:17:28.694184+0000","last_unstale":"2024-09-21T02:17:28.695625+0000","last_undegraded":"2024-09-21T02:17:28.695625+0000","last_fullsized":"2024-09-21T02:17:28.695625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:38.024426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684205+0000","last_change":"2024-09-21T02:17:28.684205+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684205+0000","last_undegraded":"2024-09-21T02:17:28.684205+0000","last_fullsized":"2024-09-21T02:17:28.684205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.699813+0000","last_change":"2024-09-21T02:17:28.695543+0000","last_active":"2024-09-21T02:17:28.699813+0000","last_peered":"2024-09-21T02:17:28.699813+0000","last_clean":"2024-09-21T02:17:28.699813+0000","last_became_active":"2024-09-21T02:17:28.694827+0000","last_became_peered":"2024-09-21T02:17:28.694827+0000","last_unstale":"2024-09-21T02:17:28.699813+0000","last_undegraded":"2024-09-21T02:17:28.699813+0000","last_fullsized":"2024-09-21T02:17:28.699813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:13.363085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690155+0000","last_change":"2024-09-21T02:17:28.690155+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690155+0000","last_undegraded":"2024-09-21T02:17:28.690155+0000","last_fullsized":"2024-09-21T02:17:28.690155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.703540+0000","last_change":"2024-09-21T02:17:28.702105+0000","last_active":"2024-09-21T02:17:28.703540+0000","last_peered":"2024-09-21T02:17:28.703540+0000","last_clean":"2024-09-21T02:17:28.703540+0000","last_became_active":"2024-09-21T02:17:28.701785+0000","last_became_peered":"2024-09-21T02:17:28.701785+0000","last_unstale":"2024-09-21T02:17:28.703540+0000","last_undegraded":"2024-09-21T02:17:28.703540+0000","last_fullsized":"2024-09-21T02:17:28.703540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:04.860266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.691292+0000","last_change":"2024-09-21T02:17:28.691292+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.691292+0000","last_undegraded":"2024-09-21T02:17:28.691292+0000","last_fullsized":"2024-09-21T02:17:28.691292+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702589+0000","last_change":"2024-09-21T02:17:28.700846+0000","last_active":"2024-09-21T02:17:28.702589+0000","last_peered":"2024-09-21T02:17:28.702589+0000","last_clean":"2024-09-21T02:17:28.702589+0000","last_became_active":"2024-09-21T02:17:28.700490+0000","last_became_peered":"2024-09-21T02:17:28.700490+0000","last_unstale":"2024-09-21T02:17:28.702589+0000","last_undegraded":"2024-09-21T02:17:28.702589+0000","last_fullsized":"2024-09-21T02:17:28.702589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:37.333804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.715131+0000","last_change":"2024-09-21T02:17:28.706151+0000","last_active":"2024-09-21T02:17:28.715131+0000","last_peered":"2024-09-21T02:17:28.715131+0000","last_clean":"2024-09-21T02:17:28.715131+0000","last_became_active":"2024-09-21T02:17:28.705306+0000","last_became_peered":"2024-09-21T02:17:28.705306+0000","last_unstale":"2024-09-21T02:17:28.715131+0000","last_undegraded":"2024-09-21T02:17:28.715131+0000","last_fullsized":"2024-09-21T02:17:28.715131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:50.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678875+0000","last_change":"2024-09-21T02:17:28.678875+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678875+0000","last_undegraded":"2024-09-21T02:17:28.678875+0000","last_fullsized":"2024-09-21T02:17:28.678875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698755+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:28.698755+0000","last_peered":"2024-09-21T02:17:28.698755+0000","last_clean":"2024-09-21T02:17:28.698755+0000","last_became_active":"2024-09-21T02:17:28.696772+0000","last_became_peered":"2024-09-21T02:17:28.696772+0000","last_unstale":"2024-09-21T02:17:28.698755+0000","last_undegraded":"2024-09-21T02:17:28.698755+0000","last_fullsized":"2024-09-21T02:17:28.698755+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:56.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685181+0000","last_change":"2024-09-21T02:17:28.685181+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685181+0000","last_undegraded":"2024-09-21T02:17:28.685181+0000","last_fullsized":"2024-09-21T02:17:28.685181+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.693796+0000","last_change":"2024-09-21T02:17:28.692883+0000","last_active":"2024-09-21T02:17:28.693796+0000","last_peered":"2024-09-21T02:17:28.693796+0000","last_clean":"2024-09-21T02:17:28.693796+0000","last_became_active":"2024-09-21T02:17:28.692516+0000","last_became_peered":"2024-09-21T02:17:28.692516+0000","last_unstale":"2024-09-21T02:17:28.693796+0000","last_undegraded":"2024-09-21T02:17:28.693796+0000","last_fullsized":"2024-09-21T02:17:28.693796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:46.327406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680004+0000","last_change":"2024-09-21T02:17:28.680004+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680004+0000","last_undegraded":"2024-09-21T02:17:28.680004+0000","last_fullsized":"2024-09-21T02:17:28.680004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.670227+0000","last_change":"2024-09-21T02:17:27.670227+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.670227+0000","last_undegraded":"2024-09-21T02:17:27.670227+0000","last_fullsized":"2024-09-21T02:17:27.670227+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685323+0000","last_change":"2024-09-21T02:17:28.685323+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685323+0000","last_undegraded":"2024-09-21T02:17:28.685323+0000","last_fullsized":"2024-09-21T02:17:28.685323+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667752+0000","last_change":"2024-09-21T02:17:27.667752+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667752+0000","last_undegraded":"2024-09-21T02:17:27.667752+0000","last_fullsized":"2024-09-21T02:17:27.667752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676870+0000","last_change":"2024-09-21T02:17:28.676870+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676870+0000","last_undegraded":"2024-09-21T02:17:28.676870+0000","last_fullsized":"2024-09-21T02:17:28.676870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667737+0000","last_change":"2024-09-21T02:17:27.667737+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667737+0000","last_undegraded":"2024-09-21T02:17:27.667737+0000","last_fullsized":"2024-09-21T02:17:27.667737+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678057+0000","last_change":"2024-09-21T02:17:28.678057+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678057+0000","last_undegraded":"2024-09-21T02:17:28.678057+0000","last_fullsized":"2024-09-21T02:17:28.678057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.706342+0000","last_change":"2024-09-21T02:17:28.703261+0000","last_active":"2024-09-21T02:17:28.706342+0000","last_peered":"2024-09-21T02:17:28.706342+0000","last_clean":"2024-09-21T02:17:28.706342+0000","last_became_active":"2024-09-21T02:17:28.702239+0000","last_became_peered":"2024-09-21T02:17:28.702239+0000","last_unstale":"2024-09-21T02:17:28.706342+0000","last_undegraded":"2024-09-21T02:17:28.706342+0000","last_fullsized":"2024-09-21T02:17:28.706342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689896+0000","last_change":"2024-09-21T02:17:28.689896+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689896+0000","last_undegraded":"2024-09-21T02:17:28.689896+0000","last_fullsized":"2024-09-21T02:17:28.689896+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.687693+0000","last_change":"2024-09-21T02:17:28.686833+0000","last_active":"2024-09-21T02:17:28.687693+0000","last_peered":"2024-09-21T02:17:28.687693+0000","last_clean":"2024-09-21T02:17:28.687693+0000","last_became_active":"2024-09-21T02:17:28.686533+0000","last_became_peered":"2024-09-21T02:17:28.686533+0000","last_unstale":"2024-09-21T02:17:28.687693+0000","last_undegraded":"2024-09-21T02:17:28.687693+0000","last_fullsized":"2024-09-21T02:17:28.687693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.923032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677868+0000","last_change":"2024-09-21T02:17:28.677868+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677868+0000","last_undegraded":"2024-09-21T02:17:28.677868+0000","last_fullsized":"2024-09-21T02:17:28.677868+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.710276+0000","last_change":"2024-09-21T02:17:28.709392+0000","last_active":"2024-09-21T02:17:28.710276+0000","last_peered":"2024-09-21T02:17:28.710276+0000","last_clean":"2024-09-21T02:17:28.710276+0000","last_became_active":"2024-09-21T02:17:28.708868+0000","last_became_peered":"2024-09-21T02:17:28.708868+0000","last_unstale":"2024-09-21T02:17:28.710276+0000","last_undegraded":"2024-09-21T02:17:28.710276+0000","last_fullsized":"2024-09-21T02:17:28.710276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:04.119473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678082+0000","last_change":"2024-09-21T02:17:28.678082+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678082+0000","last_undegraded":"2024-09-21T02:17:28.678082+0000","last_fullsized":"2024-09-21T02:17:28.678082+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702334+0000","last_change":"2024-09-21T02:17:28.701648+0000","last_active":"2024-09-21T02:17:28.702334+0000","last_peered":"2024-09-21T02:17:28.702334+0000","last_clean":"2024-09-21T02:17:28.702334+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:28.702334+0000","last_undegraded":"2024-09-21T02:17:28.702334+0000","last_fullsized":"2024-09-21T02:17:28.702334+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:12.548515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":196,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.688710+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:17:28.688710+0000","last_peered":"2024-09-21T02:17:28.688710+0000","last_clean":"2024-09-21T02:17:28.688710+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:17:28.688710+0000","last_undegraded":"2024-09-21T02:17:28.688710+0000","last_fullsized":"2024-09-21T02:17:28.688710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":352330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1425408,"data_stored":936431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":843776,"data_stored":352330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1437696,"data_stored":942698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":819200,"data_stored":340246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:17:34.289 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 -- 172.21.15.170:0/387620190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 msgr2=0x7f23900cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 --2- 172.21.15.170:0/387620190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23900cca40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2390074ec0 tx=0x7f238001c890 comp rx=0 tx=0).stop 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 -- 172.21.15.170:0/387620190 shutdown_connections 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 --2- 172.21.15.170:0/387620190 >> v2:172.21.15.191:3300/0 conn(0x7f23900ccf80 0x7f23900cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 --2- 172.21.15.170:0/387620190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23900cca40 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 -- 172.21.15.170:0/387620190 >> 172.21.15.170:0/387620190 conn(0x7f23900711c0 msgr2=0x7f23900715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 -- 172.21.15.170:0/387620190 shutdown_connections 2024-09-21T02:17:34.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.476+0000 7f2395a78640 1 -- 172.21.15.170:0/387620190 wait complete. 2024-09-21T02:17:34.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 Processor -- start 2024-09-21T02:17:34.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- start start 2024-09-21T02:17:34.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23900ccf80 0x7f2390163140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23901658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2390160f60 0x7f23901567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2390169f10 con 0x7f2390160b50 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2390169d90 con 0x7f2390160f60 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f239016a090 con 0x7f23900ccf80 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23901658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23901658e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47584/0 (socket says 172.21.15.170:47584) 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 -- 172.21.15.170:0/2703065385 learned_addr learned my addr 172.21.15.170:0/2703065385 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f238f7fe640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23900ccf80 0x7f2390163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f238ffff640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2390160f60 0x7f23901567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 -- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23900ccf80 msgr2=0x7f2390163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23900ccf80 0x7f2390163140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 -- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2390160f60 msgr2=0x7f23901567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2390160f60 0x7f23901567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 -- 172.21.15.170:0/2703065385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2390165e20 con 0x7f2390160b50 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2387fff640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23901658e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f237800c9a0 tx=0x7f237800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378004220 con 0x7f2390160b50 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23780079b0 con 0x7f2390160b50 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780044d0 con 0x7f2390160b50 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2390163b20 con 0x7f2390160b50 2024-09-21T02:17:34.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901f9220 con 0x7f2390160b50 2024-09-21T02:17:34.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.480+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23900cec70 con 0x7f2390160b50 2024-09-21T02:17:34.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.484+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2378005020 con 0x7f2390160b50 2024-09-21T02:17:34.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.484+0000 7f238cff9640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2370073560 0x7f2370075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.484+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f2378094e40 con 0x7f2390160b50 2024-09-21T02:17:34.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.484+0000 7f238f7fe640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2370073560 0x7f2370075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.484+0000 7f238f7fe640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2370073560 0x7f2370075a70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f23800af000 tx=0x7f238001b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:34.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.488+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2378014030 con 0x7f2390160b50 2024-09-21T02:17:34.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.644+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23901638b0 con 0x7f2370073560 2024-09-21T02:17:34.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.656+0000 7f238cff9640 1 -- 172.21.15.170:0/2703065385 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245894 (secure 0 0 0) 0x7f23901638b0 con 0x7f2370073560 2024-09-21T02:17:34.660 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:34.661 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:17:34.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2370073560 msgr2=0x7f2370075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2370073560 0x7f2370075a70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f23800af000 tx=0x7f238001b0d0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 msgr2=0x7f23901658e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23901658e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f237800c9a0 tx=0x7f237800ce60 comp rx=0 tx=0).stop 2024-09-21T02:17:34.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 shutdown_connections 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2370073560 0x7f2370075a70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2390160f60 0x7f23901567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2390160b50 0x7f23901658e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 --2- 172.21.15.170:0/2703065385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23900ccf80 0x7f2390163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 >> 172.21.15.170:0/2703065385 conn(0x7f23900711c0 msgr2=0x7f23900b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 shutdown_connections 2024-09-21T02:17:34.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.660+0000 7f2395a78640 1 -- 172.21.15.170:0/2703065385 wait complete. 2024-09-21T02:17:34.680 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-21T02:17:32.961564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269708,"kb_used_data":7680,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754705012,"statfs":{"total":773094113280,"available":772817932288,"internally_reserved":0,"allocated":7864320,"data_stored":3962224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.303810"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679110+0000","last_change":"2024-09-21T02:17:28.679110+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679110+0000","last_undegraded":"2024-09-21T02:17:28.679110+0000","last_fullsized":"2024-09-21T02:17:28.679110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682492+0000","last_change":"2024-09-21T02:17:28.682492+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682492+0000","last_undegraded":"2024-09-21T02:17:28.682492+0000","last_fullsized":"2024-09-21T02:17:28.682492+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689700+0000","last_change":"2024-09-21T02:17:28.689700+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689700+0000","last_undegraded":"2024-09-21T02:17:28.689700+0000","last_fullsized":"2024-09-21T02:17:28.689700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690481+0000","last_change":"2024-09-21T02:17:28.690481+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690481+0000","last_undegraded":"2024-09-21T02:17:28.690481+0000","last_fullsized":"2024-09-21T02:17:28.690481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.675278+0000","last_change":"2024-09-21T02:17:28.675278+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.675278+0000","last_undegraded":"2024-09-21T02:17:28.675278+0000","last_fullsized":"2024-09-21T02:17:28.675278+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679207+0000","last_change":"2024-09-21T02:17:28.679207+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679207+0000","last_undegraded":"2024-09-21T02:17:28.679207+0000","last_fullsized":"2024-09-21T02:17:28.679207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689631+0000","last_change":"2024-09-21T02:17:28.689631+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689631+0000","last_undegraded":"2024-09-21T02:17:28.689631+0000","last_fullsized":"2024-09-21T02:17:28.689631+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682248+0000","last_change":"2024-09-21T02:17:28.682248+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682248+0000","last_undegraded":"2024-09-21T02:17:28.682248+0000","last_fullsized":"2024-09-21T02:17:28.682248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686730+0000","last_change":"2024-09-21T02:17:28.686730+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686730+0000","last_undegraded":"2024-09-21T02:17:28.686730+0000","last_fullsized":"2024-09-21T02:17:28.686730+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687831+0000","last_change":"2024-09-21T02:17:28.687831+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687831+0000","last_undegraded":"2024-09-21T02:17:28.687831+0000","last_fullsized":"2024-09-21T02:17:28.687831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678223+0000","last_change":"2024-09-21T02:17:28.678223+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678223+0000","last_undegraded":"2024-09-21T02:17:28.678223+0000","last_fullsized":"2024-09-21T02:17:28.678223+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687804+0000","last_change":"2024-09-21T02:17:28.687804+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687804+0000","last_undegraded":"2024-09-21T02:17:28.687804+0000","last_fullsized":"2024-09-21T02:17:28.687804+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681761+0000","last_change":"2024-09-21T02:17:28.681761+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681761+0000","last_undegraded":"2024-09-21T02:17:28.681761+0000","last_fullsized":"2024-09-21T02:17:28.681761+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677586+0000","last_change":"2024-09-21T02:17:28.677586+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677586+0000","last_undegraded":"2024-09-21T02:17:28.677586+0000","last_fullsized":"2024-09-21T02:17:28.677586+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680007+0000","last_change":"2024-09-21T02:17:28.680007+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680007+0000","last_undegraded":"2024-09-21T02:17:28.680007+0000","last_fullsized":"2024-09-21T02:17:28.680007+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679210+0000","last_change":"2024-09-21T02:17:28.679210+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679210+0000","last_undegraded":"2024-09-21T02:17:28.679210+0000","last_fullsized":"2024-09-21T02:17:28.679210+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.675097+0000","last_change":"2024-09-21T02:17:28.675097+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.675097+0000","last_undegraded":"2024-09-21T02:17:28.675097+0000","last_fullsized":"2024-09-21T02:17:28.675097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682671+0000","last_change":"2024-09-21T02:17:28.682671+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682671+0000","last_undegraded":"2024-09-21T02:17:28.682671+0000","last_fullsized":"2024-09-21T02:17:28.682671+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678167+0000","last_change":"2024-09-21T02:17:28.678167+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678167+0000","last_undegraded":"2024-09-21T02:17:28.678167+0000","last_fullsized":"2024-09-21T02:17:28.678167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689162+0000","last_change":"2024-09-21T02:17:28.689162+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689162+0000","last_undegraded":"2024-09-21T02:17:28.689162+0000","last_fullsized":"2024-09-21T02:17:28.689162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677094+0000","last_change":"2024-09-21T02:17:28.677094+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677094+0000","last_undegraded":"2024-09-21T02:17:28.677094+0000","last_fullsized":"2024-09-21T02:17:28.677094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686054+0000","last_change":"2024-09-21T02:17:28.686054+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686054+0000","last_undegraded":"2024-09-21T02:17:28.686054+0000","last_fullsized":"2024-09-21T02:17:28.686054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.679394+0000","last_change":"2024-09-21T02:17:28.679394+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.679394+0000","last_undegraded":"2024-09-21T02:17:28.679394+0000","last_fullsized":"2024-09-21T02:17:28.679394+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676363+0000","last_change":"2024-09-21T02:17:28.676363+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676363+0000","last_undegraded":"2024-09-21T02:17:28.676363+0000","last_fullsized":"2024-09-21T02:17:28.676363+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681111+0000","last_change":"2024-09-21T02:17:28.681111+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681111+0000","last_undegraded":"2024-09-21T02:17:28.681111+0000","last_fullsized":"2024-09-21T02:17:28.681111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685715+0000","last_change":"2024-09-21T02:17:28.685715+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685715+0000","last_undegraded":"2024-09-21T02:17:28.685715+0000","last_fullsized":"2024-09-21T02:17:28.685715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678448+0000","last_change":"2024-09-21T02:17:28.678448+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678448+0000","last_undegraded":"2024-09-21T02:17:28.678448+0000","last_fullsized":"2024-09-21T02:17:28.678448+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680042+0000","last_change":"2024-09-21T02:17:28.680042+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680042+0000","last_undegraded":"2024-09-21T02:17:28.680042+0000","last_fullsized":"2024-09-21T02:17:28.680042+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677092+0000","last_change":"2024-09-21T02:17:28.677092+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677092+0000","last_undegraded":"2024-09-21T02:17:28.677092+0000","last_fullsized":"2024-09-21T02:17:28.677092+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678656+0000","last_change":"2024-09-21T02:17:28.678656+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678656+0000","last_undegraded":"2024-09-21T02:17:28.678656+0000","last_fullsized":"2024-09-21T02:17:28.678656+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.703566+0000","last_change":"2024-09-21T02:17:28.701807+0000","last_active":"2024-09-21T02:17:28.703566+0000","last_peered":"2024-09-21T02:17:28.703566+0000","last_clean":"2024-09-21T02:17:28.703566+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:28.703566+0000","last_undegraded":"2024-09-21T02:17:28.703566+0000","last_fullsized":"2024-09-21T02:17:28.703566+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:42.377682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684585+0000","last_change":"2024-09-21T02:17:28.684585+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684585+0000","last_undegraded":"2024-09-21T02:17:28.684585+0000","last_fullsized":"2024-09-21T02:17:28.684585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.716213+0000","last_change":"2024-09-21T02:17:28.709949+0000","last_active":"2024-09-21T02:17:28.716213+0000","last_peered":"2024-09-21T02:17:28.716213+0000","last_clean":"2024-09-21T02:17:28.716213+0000","last_became_active":"2024-09-21T02:17:28.709165+0000","last_became_peered":"2024-09-21T02:17:28.709165+0000","last_unstale":"2024-09-21T02:17:28.716213+0000","last_undegraded":"2024-09-21T02:17:28.716213+0000","last_fullsized":"2024-09-21T02:17:28.716213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:35:25.745781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680994+0000","last_change":"2024-09-21T02:17:28.680994+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680994+0000","last_undegraded":"2024-09-21T02:17:28.680994+0000","last_fullsized":"2024-09-21T02:17:28.680994+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.695050+0000","last_change":"2024-09-21T02:17:28.694169+0000","last_active":"2024-09-21T02:17:28.695050+0000","last_peered":"2024-09-21T02:17:28.695050+0000","last_clean":"2024-09-21T02:17:28.695050+0000","last_became_active":"2024-09-21T02:17:28.693855+0000","last_became_peered":"2024-09-21T02:17:28.693855+0000","last_unstale":"2024-09-21T02:17:28.695050+0000","last_undegraded":"2024-09-21T02:17:28.695050+0000","last_fullsized":"2024-09-21T02:17:28.695050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:59.351666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685956+0000","last_change":"2024-09-21T02:17:28.685956+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685956+0000","last_undegraded":"2024-09-21T02:17:28.685956+0000","last_fullsized":"2024-09-21T02:17:28.685956+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.705049+0000","last_change":"2024-09-21T02:17:28.704350+0000","last_active":"2024-09-21T02:17:28.705049+0000","last_peered":"2024-09-21T02:17:28.705049+0000","last_clean":"2024-09-21T02:17:28.705049+0000","last_became_active":"2024-09-21T02:17:28.703930+0000","last_became_peered":"2024-09-21T02:17:28.703930+0000","last_unstale":"2024-09-21T02:17:28.705049+0000","last_undegraded":"2024-09-21T02:17:28.705049+0000","last_fullsized":"2024-09-21T02:17:28.705049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:28.293800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677902+0000","last_change":"2024-09-21T02:17:28.677902+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677902+0000","last_undegraded":"2024-09-21T02:17:28.677902+0000","last_fullsized":"2024-09-21T02:17:28.677902+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.670252+0000","last_change":"2024-09-21T02:17:27.670252+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.670252+0000","last_undegraded":"2024-09-21T02:17:27.670252+0000","last_fullsized":"2024-09-21T02:17:27.670252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680022+0000","last_change":"2024-09-21T02:17:28.680022+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680022+0000","last_undegraded":"2024-09-21T02:17:28.680022+0000","last_fullsized":"2024-09-21T02:17:28.680022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702082+0000","last_change":"2024-09-21T02:17:28.700727+0000","last_active":"2024-09-21T02:17:28.702082+0000","last_peered":"2024-09-21T02:17:28.702082+0000","last_clean":"2024-09-21T02:17:28.702082+0000","last_became_active":"2024-09-21T02:17:28.700382+0000","last_became_peered":"2024-09-21T02:17:28.700382+0000","last_unstale":"2024-09-21T02:17:28.702082+0000","last_undegraded":"2024-09-21T02:17:28.702082+0000","last_fullsized":"2024-09-21T02:17:28.702082+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:47.409072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.694503+0000","last_change":"2024-09-21T02:17:28.694503+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.694503+0000","last_undegraded":"2024-09-21T02:17:28.694503+0000","last_fullsized":"2024-09-21T02:17:28.694503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.706889+0000","last_change":"2024-09-21T02:17:28.704257+0000","last_active":"2024-09-21T02:17:28.706889+0000","last_peered":"2024-09-21T02:17:28.706889+0000","last_clean":"2024-09-21T02:17:28.706889+0000","last_became_active":"2024-09-21T02:17:28.703356+0000","last_became_peered":"2024-09-21T02:17:28.703356+0000","last_unstale":"2024-09-21T02:17:28.706889+0000","last_undegraded":"2024-09-21T02:17:28.706889+0000","last_fullsized":"2024-09-21T02:17:28.706889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:32.637435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667720+0000","last_change":"2024-09-21T02:17:27.667720+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667720+0000","last_undegraded":"2024-09-21T02:17:27.667720+0000","last_fullsized":"2024-09-21T02:17:27.667720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.686873+0000","last_change":"2024-09-21T02:17:28.686873+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.686873+0000","last_undegraded":"2024-09-21T02:17:28.686873+0000","last_fullsized":"2024-09-21T02:17:28.686873+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.709826+0000","last_change":"2024-09-21T02:17:28.709078+0000","last_active":"2024-09-21T02:17:28.709826+0000","last_peered":"2024-09-21T02:17:28.709826+0000","last_clean":"2024-09-21T02:17:28.709826+0000","last_became_active":"2024-09-21T02:17:28.708778+0000","last_became_peered":"2024-09-21T02:17:28.708778+0000","last_unstale":"2024-09-21T02:17:28.709826+0000","last_undegraded":"2024-09-21T02:17:28.709826+0000","last_fullsized":"2024-09-21T02:17:28.709826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:47.678448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.682103+0000","last_change":"2024-09-21T02:17:28.682103+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.682103+0000","last_undegraded":"2024-09-21T02:17:28.682103+0000","last_fullsized":"2024-09-21T02:17:28.682103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698567+0000","last_change":"2024-09-21T02:17:28.697107+0000","last_active":"2024-09-21T02:17:28.698567+0000","last_peered":"2024-09-21T02:17:28.698567+0000","last_clean":"2024-09-21T02:17:28.698567+0000","last_became_active":"2024-09-21T02:17:28.696766+0000","last_became_peered":"2024-09-21T02:17:28.696766+0000","last_unstale":"2024-09-21T02:17:28.698567+0000","last_undegraded":"2024-09-21T02:17:28.698567+0000","last_fullsized":"2024-09-21T02:17:28.698567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:37.937427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680209+0000","last_change":"2024-09-21T02:17:28.680209+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680209+0000","last_undegraded":"2024-09-21T02:17:28.680209+0000","last_fullsized":"2024-09-21T02:17:28.680209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.671648+0000","last_change":"2024-09-21T02:17:27.671648+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.671648+0000","last_undegraded":"2024-09-21T02:17:27.671648+0000","last_fullsized":"2024-09-21T02:17:27.671648+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.674892+0000","last_change":"2024-09-21T02:17:28.674892+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.674892+0000","last_undegraded":"2024-09-21T02:17:28.674892+0000","last_fullsized":"2024-09-21T02:17:28.674892+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.716794+0000","last_change":"2024-09-21T02:17:28.711641+0000","last_active":"2024-09-21T02:17:28.716794+0000","last_peered":"2024-09-21T02:17:28.716794+0000","last_clean":"2024-09-21T02:17:28.716794+0000","last_became_active":"2024-09-21T02:17:28.710785+0000","last_became_peered":"2024-09-21T02:17:28.710785+0000","last_unstale":"2024-09-21T02:17:28.716794+0000","last_undegraded":"2024-09-21T02:17:28.716794+0000","last_fullsized":"2024-09-21T02:17:28.716794+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:10.915960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676986+0000","last_change":"2024-09-21T02:17:28.676986+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676986+0000","last_undegraded":"2024-09-21T02:17:28.676986+0000","last_fullsized":"2024-09-21T02:17:28.676986+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.707351+0000","last_change":"2024-09-21T02:17:28.705017+0000","last_active":"2024-09-21T02:17:28.707351+0000","last_peered":"2024-09-21T02:17:28.707351+0000","last_clean":"2024-09-21T02:17:28.707351+0000","last_became_active":"2024-09-21T02:17:28.704183+0000","last_became_peered":"2024-09-21T02:17:28.704183+0000","last_unstale":"2024-09-21T02:17:28.707351+0000","last_undegraded":"2024-09-21T02:17:28.707351+0000","last_fullsized":"2024-09-21T02:17:28.707351+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:01.558053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.687837+0000","last_change":"2024-09-21T02:17:28.687837+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.687837+0000","last_undegraded":"2024-09-21T02:17:28.687837+0000","last_fullsized":"2024-09-21T02:17:28.687837+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.696201+0000","last_change":"2024-09-21T02:17:28.695228+0000","last_active":"2024-09-21T02:17:28.696201+0000","last_peered":"2024-09-21T02:17:28.696201+0000","last_clean":"2024-09-21T02:17:28.696201+0000","last_became_active":"2024-09-21T02:17:28.694899+0000","last_became_peered":"2024-09-21T02:17:28.694899+0000","last_unstale":"2024-09-21T02:17:28.696201+0000","last_undegraded":"2024-09-21T02:17:28.696201+0000","last_fullsized":"2024-09-21T02:17:28.696201+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:21.720911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676604+0000","last_change":"2024-09-21T02:17:28.676604+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676604+0000","last_undegraded":"2024-09-21T02:17:28.676604+0000","last_fullsized":"2024-09-21T02:17:28.676604+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.715839+0000","last_change":"2024-09-21T02:17:28.715260+0000","last_active":"2024-09-21T02:17:28.715839+0000","last_peered":"2024-09-21T02:17:28.715839+0000","last_clean":"2024-09-21T02:17:28.715839+0000","last_became_active":"2024-09-21T02:17:28.715003+0000","last_became_peered":"2024-09-21T02:17:28.715003+0000","last_unstale":"2024-09-21T02:17:28.715839+0000","last_undegraded":"2024-09-21T02:17:28.715839+0000","last_fullsized":"2024-09-21T02:17:28.715839+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:40.200566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690971+0000","last_change":"2024-09-21T02:17:28.690971+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690971+0000","last_undegraded":"2024-09-21T02:17:28.690971+0000","last_fullsized":"2024-09-21T02:17:28.690971+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.697964+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:28.697964+0000","last_peered":"2024-09-21T02:17:28.697964+0000","last_clean":"2024-09-21T02:17:28.697964+0000","last_became_active":"2024-09-21T02:17:28.696779+0000","last_became_peered":"2024-09-21T02:17:28.696779+0000","last_unstale":"2024-09-21T02:17:28.697964+0000","last_undegraded":"2024-09-21T02:17:28.697964+0000","last_fullsized":"2024-09-21T02:17:28.697964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:57.291183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.681255+0000","last_change":"2024-09-21T02:17:28.681255+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.681255+0000","last_undegraded":"2024-09-21T02:17:28.681255+0000","last_fullsized":"2024-09-21T02:17:28.681255+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698993+0000","last_change":"2024-09-21T02:17:28.697982+0000","last_active":"2024-09-21T02:17:28.698993+0000","last_peered":"2024-09-21T02:17:28.698993+0000","last_clean":"2024-09-21T02:17:28.698993+0000","last_became_active":"2024-09-21T02:17:28.697661+0000","last_became_peered":"2024-09-21T02:17:28.697661+0000","last_unstale":"2024-09-21T02:17:28.698993+0000","last_undegraded":"2024-09-21T02:17:28.698993+0000","last_fullsized":"2024-09-21T02:17:28.698993+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:44.914475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680404+0000","last_change":"2024-09-21T02:17:28.680404+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680404+0000","last_undegraded":"2024-09-21T02:17:28.680404+0000","last_fullsized":"2024-09-21T02:17:28.680404+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.693106+0000","last_change":"2024-09-21T02:17:28.691325+0000","last_active":"2024-09-21T02:17:28.693106+0000","last_peered":"2024-09-21T02:17:28.693106+0000","last_clean":"2024-09-21T02:17:28.693106+0000","last_became_active":"2024-09-21T02:17:28.691006+0000","last_became_peered":"2024-09-21T02:17:28.691006+0000","last_unstale":"2024-09-21T02:17:28.693106+0000","last_undegraded":"2024-09-21T02:17:28.693106+0000","last_fullsized":"2024-09-21T02:17:28.693106+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:30.202022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684275+0000","last_change":"2024-09-21T02:17:28.684275+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684275+0000","last_undegraded":"2024-09-21T02:17:28.684275+0000","last_fullsized":"2024-09-21T02:17:28.684275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.695625+0000","last_change":"2024-09-21T02:17:28.694528+0000","last_active":"2024-09-21T02:17:28.695625+0000","last_peered":"2024-09-21T02:17:28.695625+0000","last_clean":"2024-09-21T02:17:28.695625+0000","last_became_active":"2024-09-21T02:17:28.694184+0000","last_became_peered":"2024-09-21T02:17:28.694184+0000","last_unstale":"2024-09-21T02:17:28.695625+0000","last_undegraded":"2024-09-21T02:17:28.695625+0000","last_fullsized":"2024-09-21T02:17:28.695625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:38.024426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.684205+0000","last_change":"2024-09-21T02:17:28.684205+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.684205+0000","last_undegraded":"2024-09-21T02:17:28.684205+0000","last_fullsized":"2024-09-21T02:17:28.684205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.699813+0000","last_change":"2024-09-21T02:17:28.695543+0000","last_active":"2024-09-21T02:17:28.699813+0000","last_peered":"2024-09-21T02:17:28.699813+0000","last_clean":"2024-09-21T02:17:28.699813+0000","last_became_active":"2024-09-21T02:17:28.694827+0000","last_became_peered":"2024-09-21T02:17:28.694827+0000","last_unstale":"2024-09-21T02:17:28.699813+0000","last_undegraded":"2024-09-21T02:17:28.699813+0000","last_fullsized":"2024-09-21T02:17:28.699813+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:13.363085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.690155+0000","last_change":"2024-09-21T02:17:28.690155+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.690155+0000","last_undegraded":"2024-09-21T02:17:28.690155+0000","last_fullsized":"2024-09-21T02:17:28.690155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.703540+0000","last_change":"2024-09-21T02:17:28.702105+0000","last_active":"2024-09-21T02:17:28.703540+0000","last_peered":"2024-09-21T02:17:28.703540+0000","last_clean":"2024-09-21T02:17:28.703540+0000","last_became_active":"2024-09-21T02:17:28.701785+0000","last_became_peered":"2024-09-21T02:17:28.701785+0000","last_unstale":"2024-09-21T02:17:28.703540+0000","last_undegraded":"2024-09-21T02:17:28.703540+0000","last_fullsized":"2024-09-21T02:17:28.703540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:04.860266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.691292+0000","last_change":"2024-09-21T02:17:28.691292+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.691292+0000","last_undegraded":"2024-09-21T02:17:28.691292+0000","last_fullsized":"2024-09-21T02:17:28.691292+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702589+0000","last_change":"2024-09-21T02:17:28.700846+0000","last_active":"2024-09-21T02:17:28.702589+0000","last_peered":"2024-09-21T02:17:28.702589+0000","last_clean":"2024-09-21T02:17:28.702589+0000","last_became_active":"2024-09-21T02:17:28.700490+0000","last_became_peered":"2024-09-21T02:17:28.700490+0000","last_unstale":"2024-09-21T02:17:28.702589+0000","last_undegraded":"2024-09-21T02:17:28.702589+0000","last_fullsized":"2024-09-21T02:17:28.702589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:25:37.333804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T02:17:28.656677+0000","last_change":"2024-09-21T02:17:28.656677+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.656677+0000","last_undegraded":"2024-09-21T02:17:28.656677+0000","last_fullsized":"2024-09-21T02:17:28.656677+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.715131+0000","last_change":"2024-09-21T02:17:28.706151+0000","last_active":"2024-09-21T02:17:28.715131+0000","last_peered":"2024-09-21T02:17:28.715131+0000","last_clean":"2024-09-21T02:17:28.715131+0000","last_became_active":"2024-09-21T02:17:28.705306+0000","last_became_peered":"2024-09-21T02:17:28.705306+0000","last_unstale":"2024-09-21T02:17:28.715131+0000","last_undegraded":"2024-09-21T02:17:28.715131+0000","last_fullsized":"2024-09-21T02:17:28.715131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:51:50.360448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678875+0000","last_change":"2024-09-21T02:17:28.678875+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678875+0000","last_undegraded":"2024-09-21T02:17:28.678875+0000","last_fullsized":"2024-09-21T02:17:28.678875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.698755+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:28.698755+0000","last_peered":"2024-09-21T02:17:28.698755+0000","last_clean":"2024-09-21T02:17:28.698755+0000","last_became_active":"2024-09-21T02:17:28.696772+0000","last_became_peered":"2024-09-21T02:17:28.696772+0000","last_unstale":"2024-09-21T02:17:28.698755+0000","last_undegraded":"2024-09-21T02:17:28.698755+0000","last_fullsized":"2024-09-21T02:17:28.698755+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:56.277562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685181+0000","last_change":"2024-09-21T02:17:28.685181+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685181+0000","last_undegraded":"2024-09-21T02:17:28.685181+0000","last_fullsized":"2024-09-21T02:17:28.685181+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.693796+0000","last_change":"2024-09-21T02:17:28.692883+0000","last_active":"2024-09-21T02:17:28.693796+0000","last_peered":"2024-09-21T02:17:28.693796+0000","last_clean":"2024-09-21T02:17:28.693796+0000","last_became_active":"2024-09-21T02:17:28.692516+0000","last_became_peered":"2024-09-21T02:17:28.692516+0000","last_unstale":"2024-09-21T02:17:28.693796+0000","last_undegraded":"2024-09-21T02:17:28.693796+0000","last_fullsized":"2024-09-21T02:17:28.693796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:46.327406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.680004+0000","last_change":"2024-09-21T02:17:28.680004+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.680004+0000","last_undegraded":"2024-09-21T02:17:28.680004+0000","last_fullsized":"2024-09-21T02:17:28.680004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.670227+0000","last_change":"2024-09-21T02:17:27.670227+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.670227+0000","last_undegraded":"2024-09-21T02:17:27.670227+0000","last_fullsized":"2024-09-21T02:17:27.670227+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.685323+0000","last_change":"2024-09-21T02:17:28.685323+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.685323+0000","last_undegraded":"2024-09-21T02:17:28.685323+0000","last_fullsized":"2024-09-21T02:17:28.685323+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667752+0000","last_change":"2024-09-21T02:17:27.667752+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667752+0000","last_undegraded":"2024-09-21T02:17:27.667752+0000","last_fullsized":"2024-09-21T02:17:27.667752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.676870+0000","last_change":"2024-09-21T02:17:28.676870+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.676870+0000","last_undegraded":"2024-09-21T02:17:28.676870+0000","last_fullsized":"2024-09-21T02:17:28.676870+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-09-21T02:17:27.667737+0000","last_change":"2024-09-21T02:17:27.667737+0000","last_active":"2024-09-21T02:17:27.644478+0000","last_peered":"2024-09-21T02:17:27.644478+0000","last_clean":"2024-09-21T02:17:27.644478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:27.667737+0000","last_undegraded":"2024-09-21T02:17:27.667737+0000","last_fullsized":"2024-09-21T02:17:27.667737+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678057+0000","last_change":"2024-09-21T02:17:28.678057+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678057+0000","last_undegraded":"2024-09-21T02:17:28.678057+0000","last_fullsized":"2024-09-21T02:17:28.678057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.706342+0000","last_change":"2024-09-21T02:17:28.703261+0000","last_active":"2024-09-21T02:17:28.706342+0000","last_peered":"2024-09-21T02:17:28.706342+0000","last_clean":"2024-09-21T02:17:28.706342+0000","last_became_active":"2024-09-21T02:17:28.702239+0000","last_became_peered":"2024-09-21T02:17:28.702239+0000","last_unstale":"2024-09-21T02:17:28.706342+0000","last_undegraded":"2024-09-21T02:17:28.706342+0000","last_fullsized":"2024-09-21T02:17:28.706342+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:26.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.689896+0000","last_change":"2024-09-21T02:17:28.689896+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.689896+0000","last_undegraded":"2024-09-21T02:17:28.689896+0000","last_fullsized":"2024-09-21T02:17:28.689896+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.687693+0000","last_change":"2024-09-21T02:17:28.686833+0000","last_active":"2024-09-21T02:17:28.687693+0000","last_peered":"2024-09-21T02:17:28.687693+0000","last_clean":"2024-09-21T02:17:28.687693+0000","last_became_active":"2024-09-21T02:17:28.686533+0000","last_became_peered":"2024-09-21T02:17:28.686533+0000","last_unstale":"2024-09-21T02:17:28.687693+0000","last_undegraded":"2024-09-21T02:17:28.687693+0000","last_fullsized":"2024-09-21T02:17:28.687693+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:09.923032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.677868+0000","last_change":"2024-09-21T02:17:28.677868+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.677868+0000","last_undegraded":"2024-09-21T02:17:28.677868+0000","last_fullsized":"2024-09-21T02:17:28.677868+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.710276+0000","last_change":"2024-09-21T02:17:28.709392+0000","last_active":"2024-09-21T02:17:28.710276+0000","last_peered":"2024-09-21T02:17:28.710276+0000","last_clean":"2024-09-21T02:17:28.710276+0000","last_became_active":"2024-09-21T02:17:28.708868+0000","last_became_peered":"2024-09-21T02:17:28.708868+0000","last_unstale":"2024-09-21T02:17:28.710276+0000","last_undegraded":"2024-09-21T02:17:28.710276+0000","last_fullsized":"2024-09-21T02:17:28.710276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:04.119473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-09-21T02:17:28.678082+0000","last_change":"2024-09-21T02:17:28.678082+0000","last_active":"2024-09-21T02:17:28.656677+0000","last_peered":"2024-09-21T02:17:28.656677+0000","last_clean":"2024-09-21T02:17:28.656677+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:17:28.678082+0000","last_undegraded":"2024-09-21T02:17:28.678082+0000","last_fullsized":"2024-09-21T02:17:28.678082+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.702334+0000","last_change":"2024-09-21T02:17:28.701648+0000","last_active":"2024-09-21T02:17:28.702334+0000","last_peered":"2024-09-21T02:17:28.702334+0000","last_clean":"2024-09-21T02:17:28.702334+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:28.702334+0000","last_undegraded":"2024-09-21T02:17:28.702334+0000","last_fullsized":"2024-09-21T02:17:28.702334+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:12.548515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":196,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-21T02:17:28.688710+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:17:28.688710+0000","last_peered":"2024-09-21T02:17:28.688710+0000","last_clean":"2024-09-21T02:17:28.688710+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:17:28.688710+0000","last_undegraded":"2024-09-21T02:17:28.688710+0000","last_fullsized":"2024-09-21T02:17:28.688710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":352330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1425408,"data_stored":936431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36176,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335664,"statfs":{"total":96636764160,"available":96599719936,"internally_reserved":0,"allocated":843776,"data_stored":352330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1437696,"data_stored":942698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36160,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":831488,"data_stored":346063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":819200,"data_stored":340246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3f is not active+clean 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.679110+0000', 'last_change': '2024-09-21T02:17:28.679110+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.679110+0000', 'last_undegraded': '2024-09-21T02:17:28.679110+0000', 'last_fullsized': '2024-09-21T02:17:28.679110+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3e is not active+clean 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.682492+0000', 'last_change': '2024-09-21T02:17:28.682492+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.682492+0000', 'last_undegraded': '2024-09-21T02:17:28.682492+0000', 'last_fullsized': '2024-09-21T02:17:28.682492+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3d is not active+clean 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.689700+0000', 'last_change': '2024-09-21T02:17:28.689700+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.689700+0000', 'last_undegraded': '2024-09-21T02:17:28.689700+0000', 'last_fullsized': '2024-09-21T02:17:28.689700+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.690481+0000', 'last_change': '2024-09-21T02:17:28.690481+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.690481+0000', 'last_undegraded': '2024-09-21T02:17:28.690481+0000', 'last_fullsized': '2024-09-21T02:17:28.690481+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.684 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3b is not active+clean 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.675278+0000', 'last_change': '2024-09-21T02:17:28.675278+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.675278+0000', 'last_undegraded': '2024-09-21T02:17:28.675278+0000', 'last_fullsized': '2024-09-21T02:17:28.675278+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3a is not active+clean 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.679207+0000', 'last_change': '2024-09-21T02:17:28.679207+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.679207+0000', 'last_undegraded': '2024-09-21T02:17:28.679207+0000', 'last_fullsized': '2024-09-21T02:17:28.679207+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:PG 11.39 is not active+clean 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.689631+0000', 'last_change': '2024-09-21T02:17:28.689631+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.689631+0000', 'last_undegraded': '2024-09-21T02:17:28.689631+0000', 'last_fullsized': '2024-09-21T02:17:28.689631+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:PG 11.38 is not active+clean 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.682248+0000', 'last_change': '2024-09-21T02:17:28.682248+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.682248+0000', 'last_undegraded': '2024-09-21T02:17:28.682248+0000', 'last_fullsized': '2024-09-21T02:17:28.682248+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:PG 11.37 is not active+clean 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.686730+0000', 'last_change': '2024-09-21T02:17:28.686730+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.686730+0000', 'last_undegraded': '2024-09-21T02:17:28.686730+0000', 'last_fullsized': '2024-09-21T02:17:28.686730+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:PG 11.36 is not active+clean 2024-09-21T02:17:34.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.687831+0000', 'last_change': '2024-09-21T02:17:28.687831+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.687831+0000', 'last_undegraded': '2024-09-21T02:17:28.687831+0000', 'last_fullsized': '2024-09-21T02:17:28.687831+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:PG 11.35 is not active+clean 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678223+0000', 'last_change': '2024-09-21T02:17:28.678223+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678223+0000', 'last_undegraded': '2024-09-21T02:17:28.678223+0000', 'last_fullsized': '2024-09-21T02:17:28.678223+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:17:28.656677+0000', 'last_change': '2024-09-21T02:17:28.656677+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.656677+0000', 'last_undegraded': '2024-09-21T02:17:28.656677+0000', 'last_fullsized': '2024-09-21T02:17:28.656677+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:PG 11.33 is not active+clean 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.687804+0000', 'last_change': '2024-09-21T02:17:28.687804+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.687804+0000', 'last_undegraded': '2024-09-21T02:17:28.687804+0000', 'last_fullsized': '2024-09-21T02:17:28.687804+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:PG 11.32 is not active+clean 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:17:28.656677+0000', 'last_change': '2024-09-21T02:17:28.656677+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.656677+0000', 'last_undegraded': '2024-09-21T02:17:28.656677+0000', 'last_fullsized': '2024-09-21T02:17:28.656677+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:PG 11.31 is not active+clean 2024-09-21T02:17:34.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:17:28.656677+0000', 'last_change': '2024-09-21T02:17:28.656677+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.656677+0000', 'last_undegraded': '2024-09-21T02:17:28.656677+0000', 'last_fullsized': '2024-09-21T02:17:28.656677+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:PG 11.30 is not active+clean 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.681761+0000', 'last_change': '2024-09-21T02:17:28.681761+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.681761+0000', 'last_undegraded': '2024-09-21T02:17:28.681761+0000', 'last_fullsized': '2024-09-21T02:17:28.681761+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2f is not active+clean 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.677586+0000', 'last_change': '2024-09-21T02:17:28.677586+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.677586+0000', 'last_undegraded': '2024-09-21T02:17:28.677586+0000', 'last_fullsized': '2024-09-21T02:17:28.677586+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680007+0000', 'last_change': '2024-09-21T02:17:28.680007+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680007+0000', 'last_undegraded': '2024-09-21T02:17:28.680007+0000', 'last_fullsized': '2024-09-21T02:17:28.680007+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2d is not active+clean 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.679210+0000', 'last_change': '2024-09-21T02:17:28.679210+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.679210+0000', 'last_undegraded': '2024-09-21T02:17:28.679210+0000', 'last_fullsized': '2024-09-21T02:17:28.679210+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2c is not active+clean 2024-09-21T02:17:34.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.675097+0000', 'last_change': '2024-09-21T02:17:28.675097+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.675097+0000', 'last_undegraded': '2024-09-21T02:17:28.675097+0000', 'last_fullsized': '2024-09-21T02:17:28.675097+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2b is not active+clean 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.682671+0000', 'last_change': '2024-09-21T02:17:28.682671+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.682671+0000', 'last_undegraded': '2024-09-21T02:17:28.682671+0000', 'last_fullsized': '2024-09-21T02:17:28.682671+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2a is not active+clean 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678167+0000', 'last_change': '2024-09-21T02:17:28.678167+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678167+0000', 'last_undegraded': '2024-09-21T02:17:28.678167+0000', 'last_fullsized': '2024-09-21T02:17:28.678167+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:PG 11.29 is not active+clean 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.689162+0000', 'last_change': '2024-09-21T02:17:28.689162+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.689162+0000', 'last_undegraded': '2024-09-21T02:17:28.689162+0000', 'last_fullsized': '2024-09-21T02:17:28.689162+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:PG 11.28 is not active+clean 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.677094+0000', 'last_change': '2024-09-21T02:17:28.677094+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.677094+0000', 'last_undegraded': '2024-09-21T02:17:28.677094+0000', 'last_fullsized': '2024-09-21T02:17:28.677094+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:PG 11.27 is not active+clean 2024-09-21T02:17:34.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.686054+0000', 'last_change': '2024-09-21T02:17:28.686054+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.686054+0000', 'last_undegraded': '2024-09-21T02:17:28.686054+0000', 'last_fullsized': '2024-09-21T02:17:28.686054+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:PG 11.26 is not active+clean 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.679394+0000', 'last_change': '2024-09-21T02:17:28.679394+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.679394+0000', 'last_undegraded': '2024-09-21T02:17:28.679394+0000', 'last_fullsized': '2024-09-21T02:17:28.679394+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:PG 11.25 is not active+clean 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.676363+0000', 'last_change': '2024-09-21T02:17:28.676363+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.676363+0000', 'last_undegraded': '2024-09-21T02:17:28.676363+0000', 'last_fullsized': '2024-09-21T02:17:28.676363+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:PG 11.24 is not active+clean 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.681111+0000', 'last_change': '2024-09-21T02:17:28.681111+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.681111+0000', 'last_undegraded': '2024-09-21T02:17:28.681111+0000', 'last_fullsized': '2024-09-21T02:17:28.681111+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:PG 11.23 is not active+clean 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.685715+0000', 'last_change': '2024-09-21T02:17:28.685715+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.685715+0000', 'last_undegraded': '2024-09-21T02:17:28.685715+0000', 'last_fullsized': '2024-09-21T02:17:28.685715+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:PG 11.22 is not active+clean 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678448+0000', 'last_change': '2024-09-21T02:17:28.678448+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678448+0000', 'last_undegraded': '2024-09-21T02:17:28.678448+0000', 'last_fullsized': '2024-09-21T02:17:28.678448+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.689 INFO:tasks.ceph.ceph_manager.ceph:PG 11.21 is not active+clean 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680042+0000', 'last_change': '2024-09-21T02:17:28.680042+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680042+0000', 'last_undegraded': '2024-09-21T02:17:28.680042+0000', 'last_fullsized': '2024-09-21T02:17:28.680042+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:PG 11.20 is not active+clean 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.677092+0000', 'last_change': '2024-09-21T02:17:28.677092+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.677092+0000', 'last_undegraded': '2024-09-21T02:17:28.677092+0000', 'last_fullsized': '2024-09-21T02:17:28.677092+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1e is not active+clean 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678656+0000', 'last_change': '2024-09-21T02:17:28.678656+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678656+0000', 'last_undegraded': '2024-09-21T02:17:28.678656+0000', 'last_fullsized': '2024-09-21T02:17:28.678656+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1f is not active+clean 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.684585+0000', 'last_change': '2024-09-21T02:17:28.684585+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.684585+0000', 'last_undegraded': '2024-09-21T02:17:28.684585+0000', 'last_fullsized': '2024-09-21T02:17:28.684585+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1c is not active+clean 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680994+0000', 'last_change': '2024-09-21T02:17:28.680994+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680994+0000', 'last_undegraded': '2024-09-21T02:17:28.680994+0000', 'last_fullsized': '2024-09-21T02:17:28.680994+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1d is not active+clean 2024-09-21T02:17:34.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.685956+0000', 'last_change': '2024-09-21T02:17:28.685956+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.685956+0000', 'last_undegraded': '2024-09-21T02:17:28.685956+0000', 'last_fullsized': '2024-09-21T02:17:28.685956+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1a is not active+clean 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.677902+0000', 'last_change': '2024-09-21T02:17:28.677902+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.677902+0000', 'last_undegraded': '2024-09-21T02:17:28.677902+0000', 'last_fullsized': '2024-09-21T02:17:28.677902+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:27.670252+0000', 'last_change': '2024-09-21T02:17:27.670252+0000', 'last_active': '2024-09-21T02:17:27.644478+0000', 'last_peered': '2024-09-21T02:17:27.644478+0000', 'last_clean': '2024-09-21T02:17:27.644478+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:27.670252+0000', 'last_undegraded': '2024-09-21T02:17:27.670252+0000', 'last_fullsized': '2024-09-21T02:17:27.670252+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1b is not active+clean 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680022+0000', 'last_change': '2024-09-21T02:17:28.680022+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680022+0000', 'last_undegraded': '2024-09-21T02:17:28.680022+0000', 'last_fullsized': '2024-09-21T02:17:28.680022+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:PG 11.18 is not active+clean 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.694503+0000', 'last_change': '2024-09-21T02:17:28.694503+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.694503+0000', 'last_undegraded': '2024-09-21T02:17:28.694503+0000', 'last_fullsized': '2024-09-21T02:17:28.694503+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:PG 11.19 is not active+clean 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:17:28.656677+0000', 'last_change': '2024-09-21T02:17:28.656677+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.656677+0000', 'last_undegraded': '2024-09-21T02:17:28.656677+0000', 'last_fullsized': '2024-09-21T02:17:28.656677+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:PG 10.18 is not active+clean 2024-09-21T02:17:34.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:27.667720+0000', 'last_change': '2024-09-21T02:17:27.667720+0000', 'last_active': '2024-09-21T02:17:27.644478+0000', 'last_peered': '2024-09-21T02:17:27.644478+0000', 'last_clean': '2024-09-21T02:17:27.644478+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:27.667720+0000', 'last_undegraded': '2024-09-21T02:17:27.667720+0000', 'last_fullsized': '2024-09-21T02:17:27.667720+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:PG 11.16 is not active+clean 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.686873+0000', 'last_change': '2024-09-21T02:17:28.686873+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.686873+0000', 'last_undegraded': '2024-09-21T02:17:28.686873+0000', 'last_fullsized': '2024-09-21T02:17:28.686873+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:PG 11.17 is not active+clean 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.682103+0000', 'last_change': '2024-09-21T02:17:28.682103+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.682103+0000', 'last_undegraded': '2024-09-21T02:17:28.682103+0000', 'last_fullsized': '2024-09-21T02:17:28.682103+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:PG 11.14 is not active+clean 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680209+0000', 'last_change': '2024-09-21T02:17:28.680209+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680209+0000', 'last_undegraded': '2024-09-21T02:17:28.680209+0000', 'last_fullsized': '2024-09-21T02:17:28.680209+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:27.671648+0000', 'last_change': '2024-09-21T02:17:27.671648+0000', 'last_active': '2024-09-21T02:17:27.644478+0000', 'last_peered': '2024-09-21T02:17:27.644478+0000', 'last_clean': '2024-09-21T02:17:27.644478+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:27.671648+0000', 'last_undegraded': '2024-09-21T02:17:27.671648+0000', 'last_fullsized': '2024-09-21T02:17:27.671648+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:17:34.692 INFO:tasks.ceph.ceph_manager.ceph:PG 11.15 is not active+clean 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.674892+0000', 'last_change': '2024-09-21T02:17:28.674892+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.674892+0000', 'last_undegraded': '2024-09-21T02:17:28.674892+0000', 'last_fullsized': '2024-09-21T02:17:28.674892+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:PG 11.12 is not active+clean 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.676986+0000', 'last_change': '2024-09-21T02:17:28.676986+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.676986+0000', 'last_undegraded': '2024-09-21T02:17:28.676986+0000', 'last_fullsized': '2024-09-21T02:17:28.676986+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:PG 11.13 is not active+clean 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.687837+0000', 'last_change': '2024-09-21T02:17:28.687837+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.687837+0000', 'last_undegraded': '2024-09-21T02:17:28.687837+0000', 'last_fullsized': '2024-09-21T02:17:28.687837+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:PG 11.10 is not active+clean 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.676604+0000', 'last_change': '2024-09-21T02:17:28.676604+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.676604+0000', 'last_undegraded': '2024-09-21T02:17:28.676604+0000', 'last_fullsized': '2024-09-21T02:17:28.676604+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:PG 11.11 is not active+clean 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.690971+0000', 'last_change': '2024-09-21T02:17:28.690971+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.690971+0000', 'last_undegraded': '2024-09-21T02:17:28.690971+0000', 'last_fullsized': '2024-09-21T02:17:28.690971+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:PG 11.e is not active+clean 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.681255+0000', 'last_change': '2024-09-21T02:17:28.681255+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.681255+0000', 'last_undegraded': '2024-09-21T02:17:28.681255+0000', 'last_fullsized': '2024-09-21T02:17:28.681255+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.693 INFO:tasks.ceph.ceph_manager.ceph:PG 11.f is not active+clean 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680404+0000', 'last_change': '2024-09-21T02:17:28.680404+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680404+0000', 'last_undegraded': '2024-09-21T02:17:28.680404+0000', 'last_fullsized': '2024-09-21T02:17:28.680404+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.684275+0000', 'last_change': '2024-09-21T02:17:28.684275+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.684275+0000', 'last_undegraded': '2024-09-21T02:17:28.684275+0000', 'last_fullsized': '2024-09-21T02:17:28.684275+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:PG 11.d is not active+clean 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.684205+0000', 'last_change': '2024-09-21T02:17:28.684205+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.684205+0000', 'last_undegraded': '2024-09-21T02:17:28.684205+0000', 'last_fullsized': '2024-09-21T02:17:28.684205+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:PG 11.b is not active+clean 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.690155+0000', 'last_change': '2024-09-21T02:17:28.690155+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.690155+0000', 'last_undegraded': '2024-09-21T02:17:28.690155+0000', 'last_fullsized': '2024-09-21T02:17:28.690155+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:PG 11.8 is not active+clean 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.691292+0000', 'last_change': '2024-09-21T02:17:28.691292+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.691292+0000', 'last_undegraded': '2024-09-21T02:17:28.691292+0000', 'last_fullsized': '2024-09-21T02:17:28.691292+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:PG 11.9 is not active+clean 2024-09-21T02:17:34.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T02:17:28.656677+0000', 'last_change': '2024-09-21T02:17:28.656677+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.656677+0000', 'last_undegraded': '2024-09-21T02:17:28.656677+0000', 'last_fullsized': '2024-09-21T02:17:28.656677+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 11.6 is not active+clean 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678875+0000', 'last_change': '2024-09-21T02:17:28.678875+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678875+0000', 'last_undegraded': '2024-09-21T02:17:28.678875+0000', 'last_fullsized': '2024-09-21T02:17:28.678875+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 11.7 is not active+clean 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.685181+0000', 'last_change': '2024-09-21T02:17:28.685181+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.685181+0000', 'last_undegraded': '2024-09-21T02:17:28.685181+0000', 'last_fullsized': '2024-09-21T02:17:28.685181+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 11.4 is not active+clean 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.680004+0000', 'last_change': '2024-09-21T02:17:28.680004+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.680004+0000', 'last_undegraded': '2024-09-21T02:17:28.680004+0000', 'last_fullsized': '2024-09-21T02:17:28.680004+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 10.5 is not active+clean 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:27.670227+0000', 'last_change': '2024-09-21T02:17:27.670227+0000', 'last_active': '2024-09-21T02:17:27.644478+0000', 'last_peered': '2024-09-21T02:17:27.644478+0000', 'last_clean': '2024-09-21T02:17:27.644478+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:27.670227+0000', 'last_undegraded': '2024-09-21T02:17:27.670227+0000', 'last_fullsized': '2024-09-21T02:17:27.670227+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 11.5 is not active+clean 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.685323+0000', 'last_change': '2024-09-21T02:17:28.685323+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.685323+0000', 'last_undegraded': '2024-09-21T02:17:28.685323+0000', 'last_fullsized': '2024-09-21T02:17:28.685323+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:27.667752+0000', 'last_change': '2024-09-21T02:17:27.667752+0000', 'last_active': '2024-09-21T02:17:27.644478+0000', 'last_peered': '2024-09-21T02:17:27.644478+0000', 'last_clean': '2024-09-21T02:17:27.644478+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:27.667752+0000', 'last_undegraded': '2024-09-21T02:17:27.667752+0000', 'last_fullsized': '2024-09-21T02:17:27.667752+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:17:34.695 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2 is not active+clean 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.676870+0000', 'last_change': '2024-09-21T02:17:28.676870+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.676870+0000', 'last_undegraded': '2024-09-21T02:17:28.676870+0000', 'last_fullsized': '2024-09-21T02:17:28.676870+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3 is not active+clean 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:27.667737+0000', 'last_change': '2024-09-21T02:17:27.667737+0000', 'last_active': '2024-09-21T02:17:27.644478+0000', 'last_peered': '2024-09-21T02:17:27.644478+0000', 'last_clean': '2024-09-21T02:17:27.644478+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:27.667737+0000', 'last_undegraded': '2024-09-21T02:17:27.667737+0000', 'last_fullsized': '2024-09-21T02:17:27.667737+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:27.644478+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3 is not active+clean 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678057+0000', 'last_change': '2024-09-21T02:17:28.678057+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678057+0000', 'last_undegraded': '2024-09-21T02:17:28.678057+0000', 'last_fullsized': '2024-09-21T02:17:28.678057+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.689896+0000', 'last_change': '2024-09-21T02:17:28.689896+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.689896+0000', 'last_undegraded': '2024-09-21T02:17:28.689896+0000', 'last_fullsized': '2024-09-21T02:17:28.689896+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1 is not active+clean 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.677868+0000', 'last_change': '2024-09-21T02:17:28.677868+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.677868+0000', 'last_undegraded': '2024-09-21T02:17:28.677868+0000', 'last_fullsized': '2024-09-21T02:17:28.677868+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:PG 11.a is not active+clean 2024-09-21T02:17:34.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:17:28.678082+0000', 'last_change': '2024-09-21T02:17:28.678082+0000', 'last_active': '2024-09-21T02:17:28.656677+0000', 'last_peered': '2024-09-21T02:17:28.656677+0000', 'last_clean': '2024-09-21T02:17:28.656677+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:17:28.678082+0000', 'last_undegraded': '2024-09-21T02:17:28.678082+0000', 'last_fullsized': '2024-09-21T02:17:28.678082+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'last_clean_scrub_stamp': '2024-09-21T02:17:28.656677+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:17:34.697 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T02:17:34.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 -- 172.21.15.170:0/4116018076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 msgr2=0x7fd52c0b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 --2- 172.21.15.170:0/4116018076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c0b5ef0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd52c08c7f0 tx=0x7fd52001c890 comp rx=0 tx=0).stop 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 -- 172.21.15.170:0/4116018076 shutdown_connections 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 --2- 172.21.15.170:0/4116018076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c0b5ef0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 --2- 172.21.15.170:0/4116018076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd52c0b9ba0 0x7fd52c0d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 -- 172.21.15.170:0/4116018076 >> 172.21.15.170:0/4116018076 conn(0x7fd52c09d7d0 msgr2=0x7fd52c0d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 -- 172.21.15.170:0/4116018076 shutdown_connections 2024-09-21T02:17:34.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.864+0000 7fd532856640 1 -- 172.21.15.170:0/4116018076 wait complete. 2024-09-21T02:17:34.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 Processor -- start 2024-09-21T02:17:34.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- start start 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd52c06e190 0x7fd52c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd52c0b9ba0 0x7fd52c0c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c16c7f0 con 0x7fd52c06dd80 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c16c670 con 0x7fd52c06e190 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd52c16c970 con 0x7fd52c0b9ba0 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47600/0 (socket says 172.21.15.170:47600) 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 -- 172.21.15.170:0/975257103 learned_addr learned my addr 172.21.15.170:0/975257103 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd530df3640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd52c0b9ba0 0x7fd52c0c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52b7fe640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd52c06e190 0x7fd52c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 -- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd52c0b9ba0 msgr2=0x7fd52c0c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd52c0b9ba0 0x7fd52c0c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 -- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd52c06e190 msgr2=0x7fd52c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:34.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd52c06e190 0x7fd52c0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 -- 172.21.15.170:0/975257103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c1f9670 con 0x7fd52c06dd80 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd52bfff640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c15a5a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c940 tx=0x7fd51c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c004220 con 0x7fd52c06dd80 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd51c0079b0 con 0x7fd52c06dd80 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c1f9950 con 0x7fd52c06dd80 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0044b0 con 0x7fd52c06dd80 2024-09-21T02:17:34.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1f9f10 con 0x7fd52c06dd80 2024-09-21T02:17:34.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.868+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c0b5ef0 con 0x7fd52c06dd80 2024-09-21T02:17:34.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.872+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd51c003600 con 0x7fd52c06dd80 2024-09-21T02:17:34.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.872+0000 7fd528ff9640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd518073560 0x7fd518075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:34.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.872+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fd51c094a40 con 0x7fd52c06dd80 2024-09-21T02:17:34.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.872+0000 7fd52b7fe640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd518073560 0x7fd518075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:34.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.872+0000 7fd52b7fe640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd518073560 0x7fd518075a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd5200af000 tx=0x7fd52001b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:34.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:34.876+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd51c096e20 con 0x7fd52c06dd80 2024-09-21T02:17:35.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.092+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd52c0d4be0 con 0x7fd52c06dd80 2024-09-21T02:17:35.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.092+0000 7fd528ff9640 1 -- 172.21.15.170:0/975257103 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1152 (secure 0 0 0) 0x7fd51c014030 con 0x7fd52c06dd80 2024-09-21T02:17:35.097 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:35.097 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":275,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":72,"num_osds":8,"num_up_osds":8,"osd_up_since":1726884793,"num_in_osds":8,"osd_in_since":1726884783,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":65},{"state_name":"active+clean","count":27},{"state_name":"unknown","count":5}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":276180992,"bytes_avail":772817932288,"bytes_total":773094113280,"unknown_pgs_ratio":0.051546391099691391,"inactive_pgs_ratio":0.67010307312011719},"fsmap":{"epoch":47,"btime":"2024-09-21T02:17:32:738667+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6270}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-21T02:16:02.925716+0000","services":{}},"progress_events":{}} 2024-09-21T02:17:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd518073560 msgr2=0x7fd518075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd518073560 0x7fd518075a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd5200af000 tx=0x7fd52001b0d0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 msgr2=0x7fd52c15a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c15a5a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c940 tx=0x7fd51c00ce00 comp rx=0 tx=0).stop 2024-09-21T02:17:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 shutdown_connections 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd518073560 0x7fd518075a70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd52c0b9ba0 0x7fd52c0c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd52c06e190 0x7fd52c0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 --2- 172.21.15.170:0/975257103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd52c06dd80 0x7fd52c15a5a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 >> 172.21.15.170:0/975257103 conn(0x7fd52c09d7d0 msgr2=0x7fd52c15e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 shutdown_connections 2024-09-21T02:17:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.096+0000 7fd532856640 1 -- 172.21.15.170:0/975257103 wait complete. 2024-09-21T02:17:35.117 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T02:17:35.118 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:17:35.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3510718431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00b7ab0 msgr2=0x7f41c015c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3510718431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00b7ab0 0x7f41c015c850 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009840 tx=0x7f41b401c820 comp rx=0 tx=0).stop 2024-09-21T02:17:35.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3510718431 shutdown_connections 2024-09-21T02:17:35.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3510718431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41c00d0aa0 0x7f41c00d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3510718431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00b7ab0 0x7f41c015c850 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3510718431 >> 172.21.15.170:0/3510718431 conn(0x7f41c00cee30 msgr2=0x7f41c00cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:35.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3510718431 shutdown_connections 2024-09-21T02:17:35.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.268+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3510718431 wait complete. 2024-09-21T02:17:35.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 Processor -- start 2024-09-21T02:17:35.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 -- start start 2024-09-21T02:17:35.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c00b3e30 0x7f41c0158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41c00b7ab0 0x7f41c0157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 0x7f41c0155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41c016a560 con 0x7f41c00d0aa0 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41c016a3e0 con 0x7f41c00b3e30 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f41c016a6e0 con 0x7f41c00b7ab0 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 0x7f41c0155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 0x7f41c0155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47610/0 (socket says 172.21.15.170:47610) 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 -- 172.21.15.170:0/3512732003 learned_addr learned my addr 172.21.15.170:0/3512732003 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41bffff640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41c00b7ab0 0x7f41c0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c4993640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c00b3e30 0x7f41c0158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 -- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41c00b7ab0 msgr2=0x7f41c0157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41c00b7ab0 0x7f41c0157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 -- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c00b3e30 msgr2=0x7f41c0158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c00b3e30 0x7f41c0158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 -- 172.21.15.170:0/3512732003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c0157a80 con 0x7f41c00d0aa0 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c4993640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c00b3e30 0x7f41c0158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c5194640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 0x7f41c0155fa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41b800d640 tx=0x7f41b800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8017070 con 0x7f41c00d0aa0 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41b8005310 con 0x7f41c00d0aa0 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c015a250 con 0x7f41c00d0aa0 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80047b0 con 0x7f41c00d0aa0 2024-09-21T02:17:35.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.272+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c0156a60 con 0x7f41c00d0aa0 2024-09-21T02:17:35.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.276+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c0159fc0 con 0x7f41c00d0aa0 2024-09-21T02:17:35.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.276+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f41b8004950 con 0x7f41c00d0aa0 2024-09-21T02:17:35.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.276+0000 7f41bd7fa640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41ac073560 0x7f41ac075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:35.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.276+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f41b80948a0 con 0x7f41c00d0aa0 2024-09-21T02:17:35.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.276+0000 7f41c4993640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41ac073560 0x7f41ac075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:35.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.276+0000 7f41c4993640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41ac073560 0x7f41ac075a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f41b40af000 tx=0x7f41b401b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:35.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.280+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f41b805ec50 con 0x7f41c00d0aa0 2024-09-21T02:17:35.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.444+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41c015c850 con 0x7f41ac073560 2024-09-21T02:17:35.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.456+0000 7f41bd7fa640 1 -- 172.21.15.170:0/3512732003 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252068 (secure 0 0 0) 0x7f41c015c850 con 0x7f41ac073560 2024-09-21T02:17:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:35.462 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:17:35.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41ac073560 msgr2=0x7f41ac075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41ac073560 0x7f41ac075a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f41b40af000 tx=0x7f41b401b060 comp rx=0 tx=0).stop 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 msgr2=0x7f41c0155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 0x7f41c0155fa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41b800d640 tx=0x7f41b800db00 comp rx=0 tx=0).stop 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 shutdown_connections 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f41ac073560 0x7f41ac075a70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41c00b7ab0 0x7f41c0157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c00d0aa0 0x7f41c0155fa0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 --2- 172.21.15.170:0/3512732003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c00b3e30 0x7f41c0158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 >> 172.21.15.170:0/3512732003 conn(0x7f41c00cee30 msgr2=0x7f41c00b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 shutdown_connections 2024-09-21T02:17:35.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:35.460+0000 7f41c6bf7640 1 -- 172.21.15.170:0/3512732003 wait complete. 2024-09-21T02:17:35.482 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-21T02:17:34.962349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274172,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700548,"statfs":{"total":773094113280,"available":772813361152,"internally_reserved":0,"allocated":8200192,"data_stored":4108796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.293143"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718598+0000","last_change":"2024-09-21T02:17:29.705072+0000","last_active":"2024-09-21T02:17:31.718598+0000","last_peered":"2024-09-21T02:17:31.718598+0000","last_clean":"2024-09-21T02:17:31.718598+0000","last_became_active":"2024-09-21T02:17:29.704270+0000","last_became_peered":"2024-09-21T02:17:29.704270+0000","last_unstale":"2024-09-21T02:17:31.718598+0000","last_undegraded":"2024-09-21T02:17:31.718598+0000","last_fullsized":"2024-09-21T02:17:31.718598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:05.137888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725911+0000","last_change":"2024-09-21T02:17:29.722436+0000","last_active":"2024-09-21T02:17:31.725911+0000","last_peered":"2024-09-21T02:17:31.725911+0000","last_clean":"2024-09-21T02:17:31.725911+0000","last_became_active":"2024-09-21T02:17:29.721911+0000","last_became_peered":"2024-09-21T02:17:29.721911+0000","last_unstale":"2024-09-21T02:17:31.725911+0000","last_undegraded":"2024-09-21T02:17:31.725911+0000","last_fullsized":"2024-09-21T02:17:31.725911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:08.591533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725127+0000","last_change":"2024-09-21T02:17:29.715216+0000","last_active":"2024-09-21T02:17:31.725127+0000","last_peered":"2024-09-21T02:17:31.725127+0000","last_clean":"2024-09-21T02:17:31.725127+0000","last_became_active":"2024-09-21T02:17:29.714248+0000","last_became_peered":"2024-09-21T02:17:29.714248+0000","last_unstale":"2024-09-21T02:17:31.725127+0000","last_undegraded":"2024-09-21T02:17:31.725127+0000","last_fullsized":"2024-09-21T02:17:31.725127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:58.647598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725389+0000","last_change":"2024-09-21T02:17:29.722966+0000","last_active":"2024-09-21T02:17:31.725389+0000","last_peered":"2024-09-21T02:17:31.725389+0000","last_clean":"2024-09-21T02:17:31.725389+0000","last_became_active":"2024-09-21T02:17:29.722134+0000","last_became_peered":"2024-09-21T02:17:29.722134+0000","last_unstale":"2024-09-21T02:17:31.725389+0000","last_undegraded":"2024-09-21T02:17:31.725389+0000","last_fullsized":"2024-09-21T02:17:31.725389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:40.427492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678286+0000","last_change":"2024-09-21T02:17:29.707991+0000","last_active":"2024-09-21T02:17:30.678286+0000","last_peered":"2024-09-21T02:17:30.678286+0000","last_clean":"2024-09-21T02:17:30.678286+0000","last_became_active":"2024-09-21T02:17:29.707123+0000","last_became_peered":"2024-09-21T02:17:29.707123+0000","last_unstale":"2024-09-21T02:17:30.678286+0000","last_undegraded":"2024-09-21T02:17:30.678286+0000","last_fullsized":"2024-09-21T02:17:30.678286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:48.692098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.676932+0000","last_change":"2024-09-21T02:17:29.710665+0000","last_active":"2024-09-21T02:17:30.676932+0000","last_peered":"2024-09-21T02:17:30.676932+0000","last_clean":"2024-09-21T02:17:30.676932+0000","last_became_active":"2024-09-21T02:17:29.710314+0000","last_became_peered":"2024-09-21T02:17:29.710314+0000","last_unstale":"2024-09-21T02:17:30.676932+0000","last_undegraded":"2024-09-21T02:17:30.676932+0000","last_fullsized":"2024-09-21T02:17:30.676932+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:08.028888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678051+0000","last_change":"2024-09-21T02:17:29.734987+0000","last_active":"2024-09-21T02:17:30.678051+0000","last_peered":"2024-09-21T02:17:30.678051+0000","last_clean":"2024-09-21T02:17:30.678051+0000","last_became_active":"2024-09-21T02:17:29.734691+0000","last_became_peered":"2024-09-21T02:17:29.734691+0000","last_unstale":"2024-09-21T02:17:30.678051+0000","last_undegraded":"2024-09-21T02:17:30.678051+0000","last_fullsized":"2024-09-21T02:17:30.678051+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:23.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716377+0000","last_change":"2024-09-21T02:17:29.723550+0000","last_active":"2024-09-21T02:17:31.716377+0000","last_peered":"2024-09-21T02:17:31.716377+0000","last_clean":"2024-09-21T02:17:31.716377+0000","last_became_active":"2024-09-21T02:17:29.723296+0000","last_became_peered":"2024-09-21T02:17:29.723296+0000","last_unstale":"2024-09-21T02:17:31.716377+0000","last_undegraded":"2024-09-21T02:17:31.716377+0000","last_fullsized":"2024-09-21T02:17:31.716377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:21.222572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726890+0000","last_change":"2024-09-21T02:17:29.730322+0000","last_active":"2024-09-21T02:17:31.726890+0000","last_peered":"2024-09-21T02:17:31.726890+0000","last_clean":"2024-09-21T02:17:31.726890+0000","last_became_active":"2024-09-21T02:17:29.729459+0000","last_became_peered":"2024-09-21T02:17:29.729459+0000","last_unstale":"2024-09-21T02:17:31.726890+0000","last_undegraded":"2024-09-21T02:17:31.726890+0000","last_fullsized":"2024-09-21T02:17:31.726890+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:53.198595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678312+0000","last_change":"2024-09-21T02:17:29.713253+0000","last_active":"2024-09-21T02:17:30.678312+0000","last_peered":"2024-09-21T02:17:30.678312+0000","last_clean":"2024-09-21T02:17:30.678312+0000","last_became_active":"2024-09-21T02:17:29.712858+0000","last_became_peered":"2024-09-21T02:17:29.712858+0000","last_unstale":"2024-09-21T02:17:30.678312+0000","last_undegraded":"2024-09-21T02:17:30.678312+0000","last_fullsized":"2024-09-21T02:17:30.678312+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:53.691008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718279+0000","last_change":"2024-09-21T02:17:29.725936+0000","last_active":"2024-09-21T02:17:31.718279+0000","last_peered":"2024-09-21T02:17:31.718279+0000","last_clean":"2024-09-21T02:17:31.718279+0000","last_became_active":"2024-09-21T02:17:29.725606+0000","last_became_peered":"2024-09-21T02:17:29.725606+0000","last_unstale":"2024-09-21T02:17:31.718279+0000","last_undegraded":"2024-09-21T02:17:31.718279+0000","last_fullsized":"2024-09-21T02:17:31.718279+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:16.611330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726635+0000","last_change":"2024-09-21T02:17:29.709463+0000","last_active":"2024-09-21T02:17:31.726635+0000","last_peered":"2024-09-21T02:17:31.726635+0000","last_clean":"2024-09-21T02:17:31.726635+0000","last_became_active":"2024-09-21T02:17:29.708660+0000","last_became_peered":"2024-09-21T02:17:29.708660+0000","last_unstale":"2024-09-21T02:17:31.726635+0000","last_undegraded":"2024-09-21T02:17:31.726635+0000","last_fullsized":"2024-09-21T02:17:31.726635+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:06.592776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727598+0000","last_change":"2024-09-21T02:17:29.716785+0000","last_active":"2024-09-21T02:17:31.727598+0000","last_peered":"2024-09-21T02:17:31.727598+0000","last_clean":"2024-09-21T02:17:31.727598+0000","last_became_active":"2024-09-21T02:17:29.715946+0000","last_became_peered":"2024-09-21T02:17:29.715946+0000","last_unstale":"2024-09-21T02:17:31.727598+0000","last_undegraded":"2024-09-21T02:17:31.727598+0000","last_fullsized":"2024-09-21T02:17:31.727598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:09.674115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726451+0000","last_change":"2024-09-21T02:17:29.713586+0000","last_active":"2024-09-21T02:17:31.726451+0000","last_peered":"2024-09-21T02:17:31.726451+0000","last_clean":"2024-09-21T02:17:31.726451+0000","last_became_active":"2024-09-21T02:17:29.712704+0000","last_became_peered":"2024-09-21T02:17:29.712704+0000","last_unstale":"2024-09-21T02:17:31.726451+0000","last_undegraded":"2024-09-21T02:17:31.726451+0000","last_fullsized":"2024-09-21T02:17:31.726451+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:12.371452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727532+0000","last_change":"2024-09-21T02:17:29.698079+0000","last_active":"2024-09-21T02:17:31.727532+0000","last_peered":"2024-09-21T02:17:31.727532+0000","last_clean":"2024-09-21T02:17:31.727532+0000","last_became_active":"2024-09-21T02:17:29.697390+0000","last_became_peered":"2024-09-21T02:17:29.697390+0000","last_unstale":"2024-09-21T02:17:31.727532+0000","last_undegraded":"2024-09-21T02:17:31.727532+0000","last_fullsized":"2024-09-21T02:17:31.727532+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:27.931179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716346+0000","last_change":"2024-09-21T02:17:29.706049+0000","last_active":"2024-09-21T02:17:31.716346+0000","last_peered":"2024-09-21T02:17:31.716346+0000","last_clean":"2024-09-21T02:17:31.716346+0000","last_became_active":"2024-09-21T02:17:29.704654+0000","last_became_peered":"2024-09-21T02:17:29.704654+0000","last_unstale":"2024-09-21T02:17:31.716346+0000","last_undegraded":"2024-09-21T02:17:31.716346+0000","last_fullsized":"2024-09-21T02:17:31.716346+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:13.345487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.680018+0000","last_change":"2024-09-21T02:17:29.705127+0000","last_active":"2024-09-21T02:17:30.680018+0000","last_peered":"2024-09-21T02:17:30.680018+0000","last_clean":"2024-09-21T02:17:30.680018+0000","last_became_active":"2024-09-21T02:17:29.704326+0000","last_became_peered":"2024-09-21T02:17:29.704326+0000","last_unstale":"2024-09-21T02:17:30.680018+0000","last_undegraded":"2024-09-21T02:17:30.680018+0000","last_fullsized":"2024-09-21T02:17:30.680018+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:23.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678981+0000","last_change":"2024-09-21T02:17:29.714580+0000","last_active":"2024-09-21T02:17:30.678981+0000","last_peered":"2024-09-21T02:17:30.678981+0000","last_clean":"2024-09-21T02:17:30.678981+0000","last_became_active":"2024-09-21T02:17:29.713771+0000","last_became_peered":"2024-09-21T02:17:29.713771+0000","last_unstale":"2024-09-21T02:17:30.678981+0000","last_undegraded":"2024-09-21T02:17:30.678981+0000","last_fullsized":"2024-09-21T02:17:30.678981+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:00.843405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718267+0000","last_change":"2024-09-21T02:17:29.719661+0000","last_active":"2024-09-21T02:17:31.718267+0000","last_peered":"2024-09-21T02:17:31.718267+0000","last_clean":"2024-09-21T02:17:31.718267+0000","last_became_active":"2024-09-21T02:17:29.719332+0000","last_became_peered":"2024-09-21T02:17:29.719332+0000","last_unstale":"2024-09-21T02:17:31.718267+0000","last_undegraded":"2024-09-21T02:17:31.718267+0000","last_fullsized":"2024-09-21T02:17:31.718267+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:34.651554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678641+0000","last_change":"2024-09-21T02:17:29.696930+0000","last_active":"2024-09-21T02:17:30.678641+0000","last_peered":"2024-09-21T02:17:30.678641+0000","last_clean":"2024-09-21T02:17:30.678641+0000","last_became_active":"2024-09-21T02:17:29.696052+0000","last_became_peered":"2024-09-21T02:17:29.696052+0000","last_unstale":"2024-09-21T02:17:30.678641+0000","last_undegraded":"2024-09-21T02:17:30.678641+0000","last_fullsized":"2024-09-21T02:17:30.678641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:42.335553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725379+0000","last_change":"2024-09-21T02:17:29.709737+0000","last_active":"2024-09-21T02:17:31.725379+0000","last_peered":"2024-09-21T02:17:31.725379+0000","last_clean":"2024-09-21T02:17:31.725379+0000","last_became_active":"2024-09-21T02:17:29.709143+0000","last_became_peered":"2024-09-21T02:17:29.709143+0000","last_unstale":"2024-09-21T02:17:31.725379+0000","last_undegraded":"2024-09-21T02:17:31.725379+0000","last_fullsized":"2024-09-21T02:17:31.725379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:35.660810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677265+0000","last_change":"2024-09-21T02:17:29.702516+0000","last_active":"2024-09-21T02:17:30.677265+0000","last_peered":"2024-09-21T02:17:30.677265+0000","last_clean":"2024-09-21T02:17:30.677265+0000","last_became_active":"2024-09-21T02:17:29.701715+0000","last_became_peered":"2024-09-21T02:17:29.701715+0000","last_unstale":"2024-09-21T02:17:30.677265+0000","last_undegraded":"2024-09-21T02:17:30.677265+0000","last_fullsized":"2024-09-21T02:17:30.677265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:51.370338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677636+0000","last_change":"2024-09-21T02:17:29.714639+0000","last_active":"2024-09-21T02:17:30.677636+0000","last_peered":"2024-09-21T02:17:30.677636+0000","last_clean":"2024-09-21T02:17:30.677636+0000","last_became_active":"2024-09-21T02:17:29.714192+0000","last_became_peered":"2024-09-21T02:17:29.714192+0000","last_unstale":"2024-09-21T02:17:30.677636+0000","last_undegraded":"2024-09-21T02:17:30.677636+0000","last_fullsized":"2024-09-21T02:17:30.677636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:38.673076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716990+0000","last_change":"2024-09-21T02:17:29.695971+0000","last_active":"2024-09-21T02:17:31.716990+0000","last_peered":"2024-09-21T02:17:31.716990+0000","last_clean":"2024-09-21T02:17:31.716990+0000","last_became_active":"2024-09-21T02:17:29.695059+0000","last_became_peered":"2024-09-21T02:17:29.695059+0000","last_unstale":"2024-09-21T02:17:31.716990+0000","last_undegraded":"2024-09-21T02:17:31.716990+0000","last_fullsized":"2024-09-21T02:17:31.716990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:23.112727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678227+0000","last_change":"2024-09-21T02:17:29.719867+0000","last_active":"2024-09-21T02:17:30.678227+0000","last_peered":"2024-09-21T02:17:30.678227+0000","last_clean":"2024-09-21T02:17:30.678227+0000","last_became_active":"2024-09-21T02:17:29.719531+0000","last_became_peered":"2024-09-21T02:17:29.719531+0000","last_unstale":"2024-09-21T02:17:30.678227+0000","last_undegraded":"2024-09-21T02:17:30.678227+0000","last_fullsized":"2024-09-21T02:17:30.678227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:07.547304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718155+0000","last_change":"2024-09-21T02:17:29.726083+0000","last_active":"2024-09-21T02:17:31.718155+0000","last_peered":"2024-09-21T02:17:31.718155+0000","last_clean":"2024-09-21T02:17:31.718155+0000","last_became_active":"2024-09-21T02:17:29.725741+0000","last_became_peered":"2024-09-21T02:17:29.725741+0000","last_unstale":"2024-09-21T02:17:31.718155+0000","last_undegraded":"2024-09-21T02:17:31.718155+0000","last_fullsized":"2024-09-21T02:17:31.718155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:29.273928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678759+0000","last_change":"2024-09-21T02:17:29.713725+0000","last_active":"2024-09-21T02:17:30.678759+0000","last_peered":"2024-09-21T02:17:30.678759+0000","last_clean":"2024-09-21T02:17:30.678759+0000","last_became_active":"2024-09-21T02:17:29.712865+0000","last_became_peered":"2024-09-21T02:17:29.712865+0000","last_unstale":"2024-09-21T02:17:30.678759+0000","last_undegraded":"2024-09-21T02:17:30.678759+0000","last_fullsized":"2024-09-21T02:17:30.678759+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:48.550151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727003+0000","last_change":"2024-09-21T02:17:29.712269+0000","last_active":"2024-09-21T02:17:31.727003+0000","last_peered":"2024-09-21T02:17:31.727003+0000","last_clean":"2024-09-21T02:17:31.727003+0000","last_became_active":"2024-09-21T02:17:29.711925+0000","last_became_peered":"2024-09-21T02:17:29.711925+0000","last_unstale":"2024-09-21T02:17:31.727003+0000","last_undegraded":"2024-09-21T02:17:31.727003+0000","last_fullsized":"2024-09-21T02:17:31.727003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:14.158723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727094+0000","last_change":"2024-09-21T02:17:29.725994+0000","last_active":"2024-09-21T02:17:31.727094+0000","last_peered":"2024-09-21T02:17:31.727094+0000","last_clean":"2024-09-21T02:17:31.727094+0000","last_became_active":"2024-09-21T02:17:29.725155+0000","last_became_peered":"2024-09-21T02:17:29.725155+0000","last_unstale":"2024-09-21T02:17:31.727094+0000","last_undegraded":"2024-09-21T02:17:31.727094+0000","last_fullsized":"2024-09-21T02:17:31.727094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:49.942186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716722+0000","last_change":"2024-09-21T02:17:29.714852+0000","last_active":"2024-09-21T02:17:31.716722+0000","last_peered":"2024-09-21T02:17:31.716722+0000","last_clean":"2024-09-21T02:17:31.716722+0000","last_became_active":"2024-09-21T02:17:29.711734+0000","last_became_peered":"2024-09-21T02:17:29.711734+0000","last_unstale":"2024-09-21T02:17:31.716722+0000","last_undegraded":"2024-09-21T02:17:31.716722+0000","last_fullsized":"2024-09-21T02:17:31.716722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:12.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725282+0000","last_change":"2024-09-21T02:17:29.715351+0000","last_active":"2024-09-21T02:17:31.725282+0000","last_peered":"2024-09-21T02:17:31.725282+0000","last_clean":"2024-09-21T02:17:31.725282+0000","last_became_active":"2024-09-21T02:17:29.714439+0000","last_became_peered":"2024-09-21T02:17:29.714439+0000","last_unstale":"2024-09-21T02:17:31.725282+0000","last_undegraded":"2024-09-21T02:17:31.725282+0000","last_fullsized":"2024-09-21T02:17:31.725282+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:43.599967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725483+0000","last_change":"2024-09-21T02:17:29.712524+0000","last_active":"2024-09-21T02:17:31.725483+0000","last_peered":"2024-09-21T02:17:31.725483+0000","last_clean":"2024-09-21T02:17:31.725483+0000","last_became_active":"2024-09-21T02:17:29.712065+0000","last_became_peered":"2024-09-21T02:17:29.712065+0000","last_unstale":"2024-09-21T02:17:31.725483+0000","last_undegraded":"2024-09-21T02:17:31.725483+0000","last_fullsized":"2024-09-21T02:17:31.725483+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.762072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718206+0000","last_change":"2024-09-21T02:17:29.705913+0000","last_active":"2024-09-21T02:17:31.718206+0000","last_peered":"2024-09-21T02:17:31.718206+0000","last_clean":"2024-09-21T02:17:31.718206+0000","last_became_active":"2024-09-21T02:17:29.705008+0000","last_became_peered":"2024-09-21T02:17:29.705008+0000","last_unstale":"2024-09-21T02:17:31.718206+0000","last_undegraded":"2024-09-21T02:17:31.718206+0000","last_fullsized":"2024-09-21T02:17:31.718206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:56.717877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":19,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677209+0000","last_change":"2024-09-21T02:17:28.701807+0000","last_active":"2024-09-21T02:17:30.677209+0000","last_peered":"2024-09-21T02:17:30.677209+0000","last_clean":"2024-09-21T02:17:30.677209+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:30.677209+0000","last_undegraded":"2024-09-21T02:17:30.677209+0000","last_fullsized":"2024-09-21T02:17:30.677209+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:03.455639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727556+0000","last_change":"2024-09-21T02:17:29.718308+0000","last_active":"2024-09-21T02:17:31.727556+0000","last_peered":"2024-09-21T02:17:31.727556+0000","last_clean":"2024-09-21T02:17:31.727556+0000","last_became_active":"2024-09-21T02:17:29.717473+0000","last_became_peered":"2024-09-21T02:17:29.717473+0000","last_unstale":"2024-09-21T02:17:31.727556+0000","last_undegraded":"2024-09-21T02:17:31.727556+0000","last_fullsized":"2024-09-21T02:17:31.727556+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:13.001334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679507+0000","last_change":"2024-09-21T02:17:28.709949+0000","last_active":"2024-09-21T02:17:30.679507+0000","last_peered":"2024-09-21T02:17:30.679507+0000","last_clean":"2024-09-21T02:17:30.679507+0000","last_became_active":"2024-09-21T02:17:28.709165+0000","last_became_peered":"2024-09-21T02:17:28.709165+0000","last_unstale":"2024-09-21T02:17:30.679507+0000","last_undegraded":"2024-09-21T02:17:30.679507+0000","last_fullsized":"2024-09-21T02:17:30.679507+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:46.327406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677135+0000","last_change":"2024-09-21T02:17:29.707494+0000","last_active":"2024-09-21T02:17:30.677135+0000","last_peered":"2024-09-21T02:17:30.677135+0000","last_clean":"2024-09-21T02:17:30.677135+0000","last_became_active":"2024-09-21T02:17:29.707074+0000","last_became_peered":"2024-09-21T02:17:29.707074+0000","last_unstale":"2024-09-21T02:17:30.677135+0000","last_undegraded":"2024-09-21T02:17:30.677135+0000","last_fullsized":"2024-09-21T02:17:30.677135+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:31.375745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718732+0000","last_change":"2024-09-21T02:17:28.694169+0000","last_active":"2024-09-21T02:17:31.718732+0000","last_peered":"2024-09-21T02:17:31.718732+0000","last_clean":"2024-09-21T02:17:31.718732+0000","last_became_active":"2024-09-21T02:17:28.693855+0000","last_became_peered":"2024-09-21T02:17:28.693855+0000","last_unstale":"2024-09-21T02:17:31.718732+0000","last_undegraded":"2024-09-21T02:17:31.718732+0000","last_fullsized":"2024-09-21T02:17:31.718732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:12.711273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678191+0000","last_change":"2024-09-21T02:17:29.728498+0000","last_active":"2024-09-21T02:17:30.678191+0000","last_peered":"2024-09-21T02:17:30.678191+0000","last_clean":"2024-09-21T02:17:30.678191+0000","last_became_active":"2024-09-21T02:17:29.728118+0000","last_became_peered":"2024-09-21T02:17:29.728118+0000","last_unstale":"2024-09-21T02:17:30.678191+0000","last_undegraded":"2024-09-21T02:17:30.678191+0000","last_fullsized":"2024-09-21T02:17:30.678191+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:31.420130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677108+0000","last_change":"2024-09-21T02:17:28.704350+0000","last_active":"2024-09-21T02:17:30.677108+0000","last_peered":"2024-09-21T02:17:30.677108+0000","last_clean":"2024-09-21T02:17:30.677108+0000","last_became_active":"2024-09-21T02:17:28.703930+0000","last_became_peered":"2024-09-21T02:17:28.703930+0000","last_unstale":"2024-09-21T02:17:30.677108+0000","last_undegraded":"2024-09-21T02:17:30.677108+0000","last_fullsized":"2024-09-21T02:17:30.677108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:23.938702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716879+0000","last_change":"2024-09-21T02:17:29.731003+0000","last_active":"2024-09-21T02:17:31.716879+0000","last_peered":"2024-09-21T02:17:31.716879+0000","last_clean":"2024-09-21T02:17:31.716879+0000","last_became_active":"2024-09-21T02:17:29.730612+0000","last_became_peered":"2024-09-21T02:17:29.730612+0000","last_unstale":"2024-09-21T02:17:31.716879+0000","last_undegraded":"2024-09-21T02:17:31.716879+0000","last_fullsized":"2024-09-21T02:17:31.716879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:48.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725576+0000","last_change":"2024-09-21T02:17:28.700632+0000","last_active":"2024-09-21T02:17:31.725576+0000","last_peered":"2024-09-21T02:17:31.725576+0000","last_clean":"2024-09-21T02:17:31.725576+0000","last_became_active":"2024-09-21T02:17:28.699770+0000","last_became_peered":"2024-09-21T02:17:28.699770+0000","last_unstale":"2024-09-21T02:17:31.725576+0000","last_undegraded":"2024-09-21T02:17:31.725576+0000","last_fullsized":"2024-09-21T02:17:31.725576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:59.122385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677136+0000","last_change":"2024-09-21T02:17:29.708363+0000","last_active":"2024-09-21T02:17:30.677136+0000","last_peered":"2024-09-21T02:17:30.677136+0000","last_clean":"2024-09-21T02:17:30.677136+0000","last_became_active":"2024-09-21T02:17:29.707848+0000","last_became_peered":"2024-09-21T02:17:29.707848+0000","last_unstale":"2024-09-21T02:17:30.677136+0000","last_undegraded":"2024-09-21T02:17:30.677136+0000","last_fullsized":"2024-09-21T02:17:30.677136+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:27.411987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677587+0000","last_change":"2024-09-21T02:17:28.700727+0000","last_active":"2024-09-21T02:17:30.677587+0000","last_peered":"2024-09-21T02:17:30.677587+0000","last_clean":"2024-09-21T02:17:30.677587+0000","last_became_active":"2024-09-21T02:17:28.700382+0000","last_became_peered":"2024-09-21T02:17:28.700382+0000","last_unstale":"2024-09-21T02:17:30.677587+0000","last_undegraded":"2024-09-21T02:17:30.677587+0000","last_fullsized":"2024-09-21T02:17:30.677587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:48.490309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725936+0000","last_change":"2024-09-21T02:17:29.711888+0000","last_active":"2024-09-21T02:17:31.725936+0000","last_peered":"2024-09-21T02:17:31.725936+0000","last_clean":"2024-09-21T02:17:31.725936+0000","last_became_active":"2024-09-21T02:17:29.710999+0000","last_became_peered":"2024-09-21T02:17:29.710999+0000","last_unstale":"2024-09-21T02:17:31.725936+0000","last_undegraded":"2024-09-21T02:17:31.725936+0000","last_fullsized":"2024-09-21T02:17:31.725936+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:37.541800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727858+0000","last_change":"2024-09-21T02:17:28.704257+0000","last_active":"2024-09-21T02:17:31.727858+0000","last_peered":"2024-09-21T02:17:31.727858+0000","last_clean":"2024-09-21T02:17:31.727858+0000","last_became_active":"2024-09-21T02:17:28.703356+0000","last_became_peered":"2024-09-21T02:17:28.703356+0000","last_unstale":"2024-09-21T02:17:31.727858+0000","last_undegraded":"2024-09-21T02:17:31.727858+0000","last_fullsized":"2024-09-21T02:17:31.727858+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:11.418878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728147+0000","last_change":"2024-09-21T02:17:29.718207+0000","last_active":"2024-09-21T02:17:31.728147+0000","last_peered":"2024-09-21T02:17:31.728147+0000","last_clean":"2024-09-21T02:17:31.728147+0000","last_became_active":"2024-09-21T02:17:29.717239+0000","last_became_peered":"2024-09-21T02:17:29.717239+0000","last_unstale":"2024-09-21T02:17:31.728147+0000","last_undegraded":"2024-09-21T02:17:31.728147+0000","last_fullsized":"2024-09-21T02:17:31.728147+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.334832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.724548+0000","last_change":"2024-09-21T02:17:28.708059+0000","last_active":"2024-09-21T02:17:31.724548+0000","last_peered":"2024-09-21T02:17:31.724548+0000","last_clean":"2024-09-21T02:17:31.724548+0000","last_became_active":"2024-09-21T02:17:28.702097+0000","last_became_peered":"2024-09-21T02:17:28.702097+0000","last_unstale":"2024-09-21T02:17:31.724548+0000","last_undegraded":"2024-09-21T02:17:31.724548+0000","last_fullsized":"2024-09-21T02:17:31.724548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.749317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678559+0000","last_change":"2024-09-21T02:17:29.720124+0000","last_active":"2024-09-21T02:17:30.678559+0000","last_peered":"2024-09-21T02:17:30.678559+0000","last_clean":"2024-09-21T02:17:30.678559+0000","last_became_active":"2024-09-21T02:17:29.719730+0000","last_became_peered":"2024-09-21T02:17:29.719730+0000","last_unstale":"2024-09-21T02:17:30.678559+0000","last_undegraded":"2024-09-21T02:17:30.678559+0000","last_fullsized":"2024-09-21T02:17:30.678559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:47.274507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718121+0000","last_change":"2024-09-21T02:17:28.709078+0000","last_active":"2024-09-21T02:17:31.718121+0000","last_peered":"2024-09-21T02:17:31.718121+0000","last_clean":"2024-09-21T02:17:31.718121+0000","last_became_active":"2024-09-21T02:17:28.708778+0000","last_became_peered":"2024-09-21T02:17:28.708778+0000","last_unstale":"2024-09-21T02:17:31.718121+0000","last_undegraded":"2024-09-21T02:17:31.718121+0000","last_fullsized":"2024-09-21T02:17:31.718121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:05.707161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725997+0000","last_change":"2024-09-21T02:17:29.713765+0000","last_active":"2024-09-21T02:17:31.725997+0000","last_peered":"2024-09-21T02:17:31.725997+0000","last_clean":"2024-09-21T02:17:31.725997+0000","last_became_active":"2024-09-21T02:17:29.713185+0000","last_became_peered":"2024-09-21T02:17:29.713185+0000","last_unstale":"2024-09-21T02:17:31.725997+0000","last_undegraded":"2024-09-21T02:17:31.725997+0000","last_fullsized":"2024-09-21T02:17:31.725997+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:52.019660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727093+0000","last_change":"2024-09-21T02:17:28.697107+0000","last_active":"2024-09-21T02:17:31.727093+0000","last_peered":"2024-09-21T02:17:31.727093+0000","last_clean":"2024-09-21T02:17:31.727093+0000","last_became_active":"2024-09-21T02:17:28.696766+0000","last_became_peered":"2024-09-21T02:17:28.696766+0000","last_unstale":"2024-09-21T02:17:31.727093+0000","last_undegraded":"2024-09-21T02:17:31.727093+0000","last_fullsized":"2024-09-21T02:17:31.727093+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:54.148477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716722+0000","last_change":"2024-09-21T02:17:29.717855+0000","last_active":"2024-09-21T02:17:31.716722+0000","last_peered":"2024-09-21T02:17:31.716722+0000","last_clean":"2024-09-21T02:17:31.716722+0000","last_became_active":"2024-09-21T02:17:29.717526+0000","last_became_peered":"2024-09-21T02:17:29.717526+0000","last_unstale":"2024-09-21T02:17:31.716722+0000","last_undegraded":"2024-09-21T02:17:31.716722+0000","last_fullsized":"2024-09-21T02:17:31.716722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:24.710835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726671+0000","last_change":"2024-09-21T02:17:28.712665+0000","last_active":"2024-09-21T02:17:31.726671+0000","last_peered":"2024-09-21T02:17:31.726671+0000","last_clean":"2024-09-21T02:17:31.726671+0000","last_became_active":"2024-09-21T02:17:28.711793+0000","last_became_peered":"2024-09-21T02:17:28.711793+0000","last_unstale":"2024-09-21T02:17:31.726671+0000","last_undegraded":"2024-09-21T02:17:31.726671+0000","last_fullsized":"2024-09-21T02:17:31.726671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:57.844738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679386+0000","last_change":"2024-09-21T02:17:29.712983+0000","last_active":"2024-09-21T02:17:30.679386+0000","last_peered":"2024-09-21T02:17:30.679386+0000","last_clean":"2024-09-21T02:17:30.679386+0000","last_became_active":"2024-09-21T02:17:29.711783+0000","last_became_peered":"2024-09-21T02:17:29.711783+0000","last_unstale":"2024-09-21T02:17:30.679386+0000","last_undegraded":"2024-09-21T02:17:30.679386+0000","last_fullsized":"2024-09-21T02:17:30.679386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:45.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728909+0000","last_change":"2024-09-21T02:17:28.711641+0000","last_active":"2024-09-21T02:17:31.728909+0000","last_peered":"2024-09-21T02:17:31.728909+0000","last_clean":"2024-09-21T02:17:31.728909+0000","last_became_active":"2024-09-21T02:17:28.710785+0000","last_became_peered":"2024-09-21T02:17:28.710785+0000","last_unstale":"2024-09-21T02:17:31.728909+0000","last_undegraded":"2024-09-21T02:17:31.728909+0000","last_fullsized":"2024-09-21T02:17:31.728909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:46.501854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677245+0000","last_change":"2024-09-21T02:17:29.692488+0000","last_active":"2024-09-21T02:17:30.677245+0000","last_peered":"2024-09-21T02:17:30.677245+0000","last_clean":"2024-09-21T02:17:30.677245+0000","last_became_active":"2024-09-21T02:17:29.691441+0000","last_became_peered":"2024-09-21T02:17:29.691441+0000","last_unstale":"2024-09-21T02:17:30.677245+0000","last_undegraded":"2024-09-21T02:17:30.677245+0000","last_fullsized":"2024-09-21T02:17:30.677245+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:08.591533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728187+0000","last_change":"2024-09-21T02:17:28.705017+0000","last_active":"2024-09-21T02:17:31.728187+0000","last_peered":"2024-09-21T02:17:31.728187+0000","last_clean":"2024-09-21T02:17:31.728187+0000","last_became_active":"2024-09-21T02:17:28.704183+0000","last_became_peered":"2024-09-21T02:17:28.704183+0000","last_unstale":"2024-09-21T02:17:31.728187+0000","last_undegraded":"2024-09-21T02:17:31.728187+0000","last_fullsized":"2024-09-21T02:17:31.728187+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:30.415777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677918+0000","last_change":"2024-09-21T02:17:29.711001+0000","last_active":"2024-09-21T02:17:30.677918+0000","last_peered":"2024-09-21T02:17:30.677918+0000","last_clean":"2024-09-21T02:17:30.677918+0000","last_became_active":"2024-09-21T02:17:29.710543+0000","last_became_peered":"2024-09-21T02:17:29.710543+0000","last_unstale":"2024-09-21T02:17:30.677918+0000","last_undegraded":"2024-09-21T02:17:30.677918+0000","last_fullsized":"2024-09-21T02:17:30.677918+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:19.910922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677715+0000","last_change":"2024-09-21T02:17:28.695228+0000","last_active":"2024-09-21T02:17:30.677715+0000","last_peered":"2024-09-21T02:17:30.677715+0000","last_clean":"2024-09-21T02:17:30.677715+0000","last_became_active":"2024-09-21T02:17:28.694899+0000","last_became_peered":"2024-09-21T02:17:28.694899+0000","last_unstale":"2024-09-21T02:17:30.677715+0000","last_undegraded":"2024-09-21T02:17:30.677715+0000","last_fullsized":"2024-09-21T02:17:30.677715+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:12.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678205+0000","last_change":"2024-09-21T02:17:29.691271+0000","last_active":"2024-09-21T02:17:30.678205+0000","last_peered":"2024-09-21T02:17:30.678205+0000","last_clean":"2024-09-21T02:17:30.678205+0000","last_became_active":"2024-09-21T02:17:29.690416+0000","last_became_peered":"2024-09-21T02:17:29.690416+0000","last_unstale":"2024-09-21T02:17:30.678205+0000","last_undegraded":"2024-09-21T02:17:30.678205+0000","last_fullsized":"2024-09-21T02:17:30.678205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:49.838744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677810+0000","last_change":"2024-09-21T02:17:28.715260+0000","last_active":"2024-09-21T02:17:30.677810+0000","last_peered":"2024-09-21T02:17:30.677810+0000","last_clean":"2024-09-21T02:17:30.677810+0000","last_became_active":"2024-09-21T02:17:28.715003+0000","last_became_peered":"2024-09-21T02:17:28.715003+0000","last_unstale":"2024-09-21T02:17:30.677810+0000","last_undegraded":"2024-09-21T02:17:30.677810+0000","last_fullsized":"2024-09-21T02:17:30.677810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:44.123972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.724913+0000","last_change":"2024-09-21T02:17:29.710382+0000","last_active":"2024-09-21T02:17:31.724913+0000","last_peered":"2024-09-21T02:17:31.724913+0000","last_clean":"2024-09-21T02:17:31.724913+0000","last_became_active":"2024-09-21T02:17:29.709530+0000","last_became_peered":"2024-09-21T02:17:29.709530+0000","last_unstale":"2024-09-21T02:17:31.724913+0000","last_undegraded":"2024-09-21T02:17:31.724913+0000","last_fullsized":"2024-09-21T02:17:31.724913+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:19.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677480+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:30.677480+0000","last_peered":"2024-09-21T02:17:30.677480+0000","last_clean":"2024-09-21T02:17:30.677480+0000","last_became_active":"2024-09-21T02:17:28.696779+0000","last_became_peered":"2024-09-21T02:17:28.696779+0000","last_unstale":"2024-09-21T02:17:30.677480+0000","last_undegraded":"2024-09-21T02:17:30.677480+0000","last_fullsized":"2024-09-21T02:17:30.677480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:39.238508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726578+0000","last_change":"2024-09-21T02:17:29.702103+0000","last_active":"2024-09-21T02:17:31.726578+0000","last_peered":"2024-09-21T02:17:31.726578+0000","last_clean":"2024-09-21T02:17:31.726578+0000","last_became_active":"2024-09-21T02:17:29.699620+0000","last_became_peered":"2024-09-21T02:17:29.699620+0000","last_unstale":"2024-09-21T02:17:31.726578+0000","last_undegraded":"2024-09-21T02:17:31.726578+0000","last_fullsized":"2024-09-21T02:17:31.726578+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:25.741087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718236+0000","last_change":"2024-09-21T02:17:28.697982+0000","last_active":"2024-09-21T02:17:31.718236+0000","last_peered":"2024-09-21T02:17:31.718236+0000","last_clean":"2024-09-21T02:17:31.718236+0000","last_became_active":"2024-09-21T02:17:28.697661+0000","last_became_peered":"2024-09-21T02:17:28.697661+0000","last_unstale":"2024-09-21T02:17:31.718236+0000","last_undegraded":"2024-09-21T02:17:31.718236+0000","last_fullsized":"2024-09-21T02:17:31.718236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:44.421655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727285+0000","last_change":"2024-09-21T02:17:29.694011+0000","last_active":"2024-09-21T02:17:31.727285+0000","last_peered":"2024-09-21T02:17:31.727285+0000","last_clean":"2024-09-21T02:17:31.727285+0000","last_became_active":"2024-09-21T02:17:29.693144+0000","last_became_peered":"2024-09-21T02:17:29.693144+0000","last_unstale":"2024-09-21T02:17:31.727285+0000","last_undegraded":"2024-09-21T02:17:31.727285+0000","last_fullsized":"2024-09-21T02:17:31.727285+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:27.183808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727121+0000","last_change":"2024-09-21T02:17:28.691325+0000","last_active":"2024-09-21T02:17:31.727121+0000","last_peered":"2024-09-21T02:17:31.727121+0000","last_clean":"2024-09-21T02:17:31.727121+0000","last_became_active":"2024-09-21T02:17:28.691006+0000","last_became_peered":"2024-09-21T02:17:28.691006+0000","last_unstale":"2024-09-21T02:17:31.727121+0000","last_undegraded":"2024-09-21T02:17:31.727121+0000","last_fullsized":"2024-09-21T02:17:31.727121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:43.354026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725793+0000","last_change":"2024-09-21T02:17:29.728111+0000","last_active":"2024-09-21T02:17:31.725793+0000","last_peered":"2024-09-21T02:17:31.725793+0000","last_clean":"2024-09-21T02:17:31.725793+0000","last_became_active":"2024-09-21T02:17:29.727378+0000","last_became_peered":"2024-09-21T02:17:29.727378+0000","last_unstale":"2024-09-21T02:17:31.725793+0000","last_undegraded":"2024-09-21T02:17:31.725793+0000","last_fullsized":"2024-09-21T02:17:31.725793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:10.261343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725007+0000","last_change":"2024-09-21T02:17:28.694528+0000","last_active":"2024-09-21T02:17:31.725007+0000","last_peered":"2024-09-21T02:17:31.725007+0000","last_clean":"2024-09-21T02:17:31.725007+0000","last_became_active":"2024-09-21T02:17:28.694184+0000","last_became_peered":"2024-09-21T02:17:28.694184+0000","last_unstale":"2024-09-21T02:17:31.725007+0000","last_undegraded":"2024-09-21T02:17:31.725007+0000","last_fullsized":"2024-09-21T02:17:31.725007+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:33.475198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725797+0000","last_change":"2024-09-21T02:17:29.726128+0000","last_active":"2024-09-21T02:17:31.725797+0000","last_peered":"2024-09-21T02:17:31.725797+0000","last_clean":"2024-09-21T02:17:31.725797+0000","last_became_active":"2024-09-21T02:17:29.725308+0000","last_became_peered":"2024-09-21T02:17:29.725308+0000","last_unstale":"2024-09-21T02:17:31.725797+0000","last_undegraded":"2024-09-21T02:17:31.725797+0000","last_fullsized":"2024-09-21T02:17:31.725797+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:38.627673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677246+0000","last_change":"2024-09-21T02:17:28.695543+0000","last_active":"2024-09-21T02:17:30.677246+0000","last_peered":"2024-09-21T02:17:30.677246+0000","last_clean":"2024-09-21T02:17:30.677246+0000","last_became_active":"2024-09-21T02:17:28.694827+0000","last_became_peered":"2024-09-21T02:17:28.694827+0000","last_unstale":"2024-09-21T02:17:30.677246+0000","last_undegraded":"2024-09-21T02:17:30.677246+0000","last_fullsized":"2024-09-21T02:17:30.677246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.956160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728393+0000","last_change":"2024-09-21T02:17:29.705415+0000","last_active":"2024-09-21T02:17:31.728393+0000","last_peered":"2024-09-21T02:17:31.728393+0000","last_clean":"2024-09-21T02:17:31.728393+0000","last_became_active":"2024-09-21T02:17:29.704569+0000","last_became_peered":"2024-09-21T02:17:29.704569+0000","last_unstale":"2024-09-21T02:17:31.728393+0000","last_undegraded":"2024-09-21T02:17:31.728393+0000","last_fullsized":"2024-09-21T02:17:31.728393+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:41.180647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677389+0000","last_change":"2024-09-21T02:17:28.702105+0000","last_active":"2024-09-21T02:17:30.677389+0000","last_peered":"2024-09-21T02:17:30.677389+0000","last_clean":"2024-09-21T02:17:30.677389+0000","last_became_active":"2024-09-21T02:17:28.701785+0000","last_became_peered":"2024-09-21T02:17:28.701785+0000","last_unstale":"2024-09-21T02:17:30.677389+0000","last_undegraded":"2024-09-21T02:17:30.677389+0000","last_fullsized":"2024-09-21T02:17:30.677389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:40.433891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726589+0000","last_change":"2024-09-21T02:17:29.725985+0000","last_active":"2024-09-21T02:17:31.726589+0000","last_peered":"2024-09-21T02:17:31.726589+0000","last_clean":"2024-09-21T02:17:31.726589+0000","last_became_active":"2024-09-21T02:17:29.725206+0000","last_became_peered":"2024-09-21T02:17:29.725206+0000","last_unstale":"2024-09-21T02:17:31.726589+0000","last_undegraded":"2024-09-21T02:17:31.726589+0000","last_fullsized":"2024-09-21T02:17:31.726589+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:20.552393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725574+0000","last_change":"2024-09-21T02:17:28.700846+0000","last_active":"2024-09-21T02:17:31.725574+0000","last_peered":"2024-09-21T02:17:31.725574+0000","last_clean":"2024-09-21T02:17:31.725574+0000","last_became_active":"2024-09-21T02:17:28.700490+0000","last_became_peered":"2024-09-21T02:17:28.700490+0000","last_unstale":"2024-09-21T02:17:31.725574+0000","last_undegraded":"2024-09-21T02:17:31.725574+0000","last_fullsized":"2024-09-21T02:17:31.725574+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:00.721655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728772+0000","last_change":"2024-09-21T02:17:29.700985+0000","last_active":"2024-09-21T02:17:31.728772+0000","last_peered":"2024-09-21T02:17:31.728772+0000","last_clean":"2024-09-21T02:17:31.728772+0000","last_became_active":"2024-09-21T02:17:29.700170+0000","last_became_peered":"2024-09-21T02:17:29.700170+0000","last_unstale":"2024-09-21T02:17:31.728772+0000","last_undegraded":"2024-09-21T02:17:31.728772+0000","last_fullsized":"2024-09-21T02:17:31.728772+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:26.939620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677822+0000","last_change":"2024-09-21T02:17:28.706151+0000","last_active":"2024-09-21T02:17:30.677822+0000","last_peered":"2024-09-21T02:17:30.677822+0000","last_clean":"2024-09-21T02:17:30.677822+0000","last_became_active":"2024-09-21T02:17:28.705306+0000","last_became_peered":"2024-09-21T02:17:28.705306+0000","last_unstale":"2024-09-21T02:17:30.677822+0000","last_undegraded":"2024-09-21T02:17:30.677822+0000","last_fullsized":"2024-09-21T02:17:30.677822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:21.178808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679360+0000","last_change":"2024-09-21T02:17:29.708739+0000","last_active":"2024-09-21T02:17:30.679360+0000","last_peered":"2024-09-21T02:17:30.679360+0000","last_clean":"2024-09-21T02:17:30.679360+0000","last_became_active":"2024-09-21T02:17:29.707885+0000","last_became_peered":"2024-09-21T02:17:29.707885+0000","last_unstale":"2024-09-21T02:17:30.679360+0000","last_undegraded":"2024-09-21T02:17:30.679360+0000","last_fullsized":"2024-09-21T02:17:30.679360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:18.505607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718585+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:31.718585+0000","last_peered":"2024-09-21T02:17:31.718585+0000","last_clean":"2024-09-21T02:17:31.718585+0000","last_became_active":"2024-09-21T02:17:28.696772+0000","last_became_peered":"2024-09-21T02:17:28.696772+0000","last_unstale":"2024-09-21T02:17:31.718585+0000","last_undegraded":"2024-09-21T02:17:31.718585+0000","last_fullsized":"2024-09-21T02:17:31.718585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:03.422543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678458+0000","last_change":"2024-09-21T02:17:29.718721+0000","last_active":"2024-09-21T02:17:30.678458+0000","last_peered":"2024-09-21T02:17:30.678458+0000","last_clean":"2024-09-21T02:17:30.678458+0000","last_became_active":"2024-09-21T02:17:29.718407+0000","last_became_peered":"2024-09-21T02:17:29.718407+0000","last_unstale":"2024-09-21T02:17:30.678458+0000","last_undegraded":"2024-09-21T02:17:30.678458+0000","last_fullsized":"2024-09-21T02:17:30.678458+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:54.495444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"72'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718770+0000","last_change":"2024-09-21T02:17:28.692883+0000","last_active":"2024-09-21T02:17:31.718770+0000","last_peered":"2024-09-21T02:17:31.718770+0000","last_clean":"2024-09-21T02:17:31.718770+0000","last_became_active":"2024-09-21T02:17:28.692516+0000","last_became_peered":"2024-09-21T02:17:28.692516+0000","last_unstale":"2024-09-21T02:17:31.718770+0000","last_undegraded":"2024-09-21T02:17:31.718770+0000","last_fullsized":"2024-09-21T02:17:31.718770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:57.655155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727660+0000","last_change":"2024-09-21T02:17:29.698220+0000","last_active":"2024-09-21T02:17:31.727660+0000","last_peered":"2024-09-21T02:17:31.727660+0000","last_clean":"2024-09-21T02:17:31.727660+0000","last_became_active":"2024-09-21T02:17:29.697389+0000","last_became_peered":"2024-09-21T02:17:29.697389+0000","last_unstale":"2024-09-21T02:17:31.727660+0000","last_undegraded":"2024-09-21T02:17:31.727660+0000","last_fullsized":"2024-09-21T02:17:31.727660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:23.052204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726913+0000","last_change":"2024-09-21T02:17:28.710748+0000","last_active":"2024-09-21T02:17:31.726913+0000","last_peered":"2024-09-21T02:17:31.726913+0000","last_clean":"2024-09-21T02:17:31.726913+0000","last_became_active":"2024-09-21T02:17:28.709895+0000","last_became_peered":"2024-09-21T02:17:28.709895+0000","last_unstale":"2024-09-21T02:17:31.726913+0000","last_undegraded":"2024-09-21T02:17:31.726913+0000","last_fullsized":"2024-09-21T02:17:31.726913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:58.752112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678433+0000","last_change":"2024-09-21T02:17:29.718562+0000","last_active":"2024-09-21T02:17:30.678433+0000","last_peered":"2024-09-21T02:17:30.678433+0000","last_clean":"2024-09-21T02:17:30.678433+0000","last_became_active":"2024-09-21T02:17:29.718143+0000","last_became_peered":"2024-09-21T02:17:29.718143+0000","last_unstale":"2024-09-21T02:17:30.678433+0000","last_undegraded":"2024-09-21T02:17:30.678433+0000","last_fullsized":"2024-09-21T02:17:30.678433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:41.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726732+0000","last_change":"2024-09-21T02:17:28.712176+0000","last_active":"2024-09-21T02:17:31.726732+0000","last_peered":"2024-09-21T02:17:31.726732+0000","last_clean":"2024-09-21T02:17:31.726732+0000","last_became_active":"2024-09-21T02:17:28.711338+0000","last_became_peered":"2024-09-21T02:17:28.711338+0000","last_unstale":"2024-09-21T02:17:31.726732+0000","last_undegraded":"2024-09-21T02:17:31.726732+0000","last_fullsized":"2024-09-21T02:17:31.726732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:16.031750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679259+0000","last_change":"2024-09-21T02:17:29.701859+0000","last_active":"2024-09-21T02:17:30.679259+0000","last_peered":"2024-09-21T02:17:30.679259+0000","last_clean":"2024-09-21T02:17:30.679259+0000","last_became_active":"2024-09-21T02:17:29.701010+0000","last_became_peered":"2024-09-21T02:17:29.701010+0000","last_unstale":"2024-09-21T02:17:30.679259+0000","last_undegraded":"2024-09-21T02:17:30.679259+0000","last_fullsized":"2024-09-21T02:17:30.679259+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:25.619135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725918+0000","last_change":"2024-09-21T02:17:28.702782+0000","last_active":"2024-09-21T02:17:31.725918+0000","last_peered":"2024-09-21T02:17:31.725918+0000","last_clean":"2024-09-21T02:17:31.725918+0000","last_became_active":"2024-09-21T02:17:28.701746+0000","last_became_peered":"2024-09-21T02:17:28.701746+0000","last_unstale":"2024-09-21T02:17:31.725918+0000","last_undegraded":"2024-09-21T02:17:31.725918+0000","last_fullsized":"2024-09-21T02:17:31.725918+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:23.056880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.717431+0000","last_change":"2024-09-21T02:17:29.716928+0000","last_active":"2024-09-21T02:17:31.717431+0000","last_peered":"2024-09-21T02:17:31.717431+0000","last_clean":"2024-09-21T02:17:31.717431+0000","last_became_active":"2024-09-21T02:17:29.716616+0000","last_became_peered":"2024-09-21T02:17:29.716616+0000","last_unstale":"2024-09-21T02:17:31.717431+0000","last_undegraded":"2024-09-21T02:17:31.717431+0000","last_fullsized":"2024-09-21T02:17:31.717431+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:26:31.195546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679156+0000","last_change":"2024-09-21T02:17:28.703261+0000","last_active":"2024-09-21T02:17:30.679156+0000","last_peered":"2024-09-21T02:17:30.679156+0000","last_clean":"2024-09-21T02:17:30.679156+0000","last_became_active":"2024-09-21T02:17:28.702239+0000","last_became_peered":"2024-09-21T02:17:28.702239+0000","last_unstale":"2024-09-21T02:17:30.679156+0000","last_undegraded":"2024-09-21T02:17:30.679156+0000","last_fullsized":"2024-09-21T02:17:30.679156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:59.351666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727276+0000","last_change":"2024-09-21T02:17:29.716762+0000","last_active":"2024-09-21T02:17:31.727276+0000","last_peered":"2024-09-21T02:17:31.727276+0000","last_clean":"2024-09-21T02:17:31.727276+0000","last_became_active":"2024-09-21T02:17:29.715816+0000","last_became_peered":"2024-09-21T02:17:29.715816+0000","last_unstale":"2024-09-21T02:17:31.727276+0000","last_undegraded":"2024-09-21T02:17:31.727276+0000","last_fullsized":"2024-09-21T02:17:31.727276+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:25.008881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725718+0000","last_change":"2024-09-21T02:17:28.686833+0000","last_active":"2024-09-21T02:17:31.725718+0000","last_peered":"2024-09-21T02:17:31.725718+0000","last_clean":"2024-09-21T02:17:31.725718+0000","last_became_active":"2024-09-21T02:17:28.686533+0000","last_became_peered":"2024-09-21T02:17:28.686533+0000","last_unstale":"2024-09-21T02:17:31.725718+0000","last_undegraded":"2024-09-21T02:17:31.725718+0000","last_fullsized":"2024-09-21T02:17:31.725718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:15.420276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725007+0000","last_change":"2024-09-21T02:17:29.715826+0000","last_active":"2024-09-21T02:17:31.725007+0000","last_peered":"2024-09-21T02:17:31.725007+0000","last_clean":"2024-09-21T02:17:31.725007+0000","last_became_active":"2024-09-21T02:17:29.714951+0000","last_became_peered":"2024-09-21T02:17:29.714951+0000","last_unstale":"2024-09-21T02:17:31.725007+0000","last_undegraded":"2024-09-21T02:17:31.725007+0000","last_fullsized":"2024-09-21T02:17:31.725007+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:34.951731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.676723+0000","last_change":"2024-09-21T02:17:28.709392+0000","last_active":"2024-09-21T02:17:30.676723+0000","last_peered":"2024-09-21T02:17:30.676723+0000","last_clean":"2024-09-21T02:17:30.676723+0000","last_became_active":"2024-09-21T02:17:28.708868+0000","last_became_peered":"2024-09-21T02:17:28.708868+0000","last_unstale":"2024-09-21T02:17:30.676723+0000","last_undegraded":"2024-09-21T02:17:30.676723+0000","last_fullsized":"2024-09-21T02:17:30.676723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:14.391234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678579+0000","last_change":"2024-09-21T02:17:29.709912+0000","last_active":"2024-09-21T02:17:30.678579+0000","last_peered":"2024-09-21T02:17:30.678579+0000","last_clean":"2024-09-21T02:17:30.678579+0000","last_became_active":"2024-09-21T02:17:29.709035+0000","last_became_peered":"2024-09-21T02:17:29.709035+0000","last_unstale":"2024-09-21T02:17:30.678579+0000","last_undegraded":"2024-09-21T02:17:30.678579+0000","last_fullsized":"2024-09-21T02:17:30.678579+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:21.043217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677926+0000","last_change":"2024-09-21T02:17:28.701648+0000","last_active":"2024-09-21T02:17:30.677926+0000","last_peered":"2024-09-21T02:17:30.677926+0000","last_clean":"2024-09-21T02:17:30.677926+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:30.677926+0000","last_undegraded":"2024-09-21T02:17:30.677926+0000","last_fullsized":"2024-09-21T02:17:30.677926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:14.619555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":200,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677624+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:17:30.677624+0000","last_peered":"2024-09-21T02:17:30.677624+0000","last_clean":"2024-09-21T02:17:30.677624+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:17:30.677624+0000","last_undegraded":"2024-09-21T02:17:30.677624+0000","last_fullsized":"2024-09-21T02:17:30.677624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":367017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":884736,"data_stored":367051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1466368,"data_stored":955863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36208,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335632,"statfs":{"total":96636764160,"available":96599687168,"internally_reserved":0,"allocated":880640,"data_stored":365531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":872448,"data_stored":365987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1462272,"data_stored":955773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":872448,"data_stored":365475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":366099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:17:35.636 INFO:tasks.ceph.mds.c.smithi170.stderr:2024-09-21T02:17:35.636+0000 7f2d8f1aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:17:38.486 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 -- 172.21.15.170:0/4168073634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 msgr2=0x7fe73000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/4168073634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe73000b9d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe7480c93a0 tx=0x7fe74001d2c0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 -- 172.21.15.170:0/4168073634 shutdown_connections 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/4168073634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe73000b9d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 -- 172.21.15.170:0/4168073634 >> 172.21.15.170:0/4168073634 conn(0x7fe7480cee30 msgr2=0x7fe7480cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 -- 172.21.15.170:0/4168073634 shutdown_connections 2024-09-21T02:17:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.648+0000 7fe74fe4d640 1 -- 172.21.15.170:0/4168073634 wait complete. 2024-09-21T02:17:38.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 Processor -- start 2024-09-21T02:17:38.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 -- start start 2024-09-21T02:17:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe74815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe748157d90 0x7fe7480b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7480b2b90 0x7fe74806f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe74815cbe0 con 0x7fe730005360 2024-09-21T02:17:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe74815ca60 con 0x7fe7480b2b90 2024-09-21T02:17:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74fe4d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe7480b4240 con 0x7fe748157d90 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe74815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe74815f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46480/0 (socket says 172.21.15.170:46480) 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 -- 172.21.15.170:0/3998519520 learned_addr learned my addr 172.21.15.170:0/3998519520 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74d3e8640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe748157d90 0x7fe7480b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74e3ea640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7480b2b90 0x7fe74806f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 -- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe748157d90 msgr2=0x7fe7480b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe748157d90 0x7fe7480b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 -- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7480b2b90 msgr2=0x7fe74806f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7480b2b90 0x7fe74806f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 -- 172.21.15.170:0/3998519520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74806e020 con 0x7fe730005360 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74d3e8640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe748157d90 0x7fe7480b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.652+0000 7fe74dbe9640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe74815f670 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e790 tx=0x7fe73800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73800eea0 con 0x7fe730005360 2024-09-21T02:17:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe73800cd50 con 0x7fe730005360 2024-09-21T02:17:38.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74806e300 con 0x7fe730005360 2024-09-21T02:17:38.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7380116d0 con 0x7fe730005360 2024-09-21T02:17:38.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74806e8c0 con 0x7fe730005360 2024-09-21T02:17:38.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe748160740 con 0x7fe730005360 2024-09-21T02:17:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe73801a400 con 0x7fe730005360 2024-09-21T02:17:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe7467fc640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe734073560 0x7fe734075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fe738095a20 con 0x7fe730005360 2024-09-21T02:17:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe74d3e8640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe734073560 0x7fe734075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.656+0000 7fe74d3e8640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe734073560 0x7fe734075a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b7330 tx=0x7fe73c006ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:38.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.660+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe738028020 con 0x7fe730005360 2024-09-21T02:17:38.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.832+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe74806a270 con 0x7fe734073560 2024-09-21T02:17:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.844+0000 7fe7467fc640 1 -- 172.21.15.170:0/3998519520 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252068 (secure 0 0 0) 0x7fe73c006370 con 0x7fe734073560 2024-09-21T02:17:38.849 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe734073560 msgr2=0x7fe734075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe734073560 0x7fe734075a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b7330 tx=0x7fe73c006ca0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 msgr2=0x7fe74815f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe74815f670 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe73800e790 tx=0x7fe73800ec50 comp rx=0 tx=0).stop 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 shutdown_connections 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe734073560 0x7fe734075a70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe748157d90 0x7fe7480b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7480b2b90 0x7fe74806f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 --2- 172.21.15.170:0/3998519520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe730005360 0x7fe74815f670 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 >> 172.21.15.170:0/3998519520 conn(0x7fe7480cee30 msgr2=0x7fe7480b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 shutdown_connections 2024-09-21T02:17:38.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:38.848+0000 7fe74fe4d640 1 -- 172.21.15.170:0/3998519520 wait complete. 2024-09-21T02:17:38.871 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-21T02:17:36.963636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274172,"kb_used_data":8008,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754700548,"statfs":{"total":773094113280,"available":772813361152,"internally_reserved":0,"allocated":8200192,"data_stored":4108796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003611"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718598+0000","last_change":"2024-09-21T02:17:29.705072+0000","last_active":"2024-09-21T02:17:31.718598+0000","last_peered":"2024-09-21T02:17:31.718598+0000","last_clean":"2024-09-21T02:17:31.718598+0000","last_became_active":"2024-09-21T02:17:29.704270+0000","last_became_peered":"2024-09-21T02:17:29.704270+0000","last_unstale":"2024-09-21T02:17:31.718598+0000","last_undegraded":"2024-09-21T02:17:31.718598+0000","last_fullsized":"2024-09-21T02:17:31.718598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:05.137888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725911+0000","last_change":"2024-09-21T02:17:29.722436+0000","last_active":"2024-09-21T02:17:31.725911+0000","last_peered":"2024-09-21T02:17:31.725911+0000","last_clean":"2024-09-21T02:17:31.725911+0000","last_became_active":"2024-09-21T02:17:29.721911+0000","last_became_peered":"2024-09-21T02:17:29.721911+0000","last_unstale":"2024-09-21T02:17:31.725911+0000","last_undegraded":"2024-09-21T02:17:31.725911+0000","last_fullsized":"2024-09-21T02:17:31.725911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:08.591533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725127+0000","last_change":"2024-09-21T02:17:29.715216+0000","last_active":"2024-09-21T02:17:31.725127+0000","last_peered":"2024-09-21T02:17:31.725127+0000","last_clean":"2024-09-21T02:17:31.725127+0000","last_became_active":"2024-09-21T02:17:29.714248+0000","last_became_peered":"2024-09-21T02:17:29.714248+0000","last_unstale":"2024-09-21T02:17:31.725127+0000","last_undegraded":"2024-09-21T02:17:31.725127+0000","last_fullsized":"2024-09-21T02:17:31.725127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:58.647598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725389+0000","last_change":"2024-09-21T02:17:29.722966+0000","last_active":"2024-09-21T02:17:31.725389+0000","last_peered":"2024-09-21T02:17:31.725389+0000","last_clean":"2024-09-21T02:17:31.725389+0000","last_became_active":"2024-09-21T02:17:29.722134+0000","last_became_peered":"2024-09-21T02:17:29.722134+0000","last_unstale":"2024-09-21T02:17:31.725389+0000","last_undegraded":"2024-09-21T02:17:31.725389+0000","last_fullsized":"2024-09-21T02:17:31.725389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:40.427492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678286+0000","last_change":"2024-09-21T02:17:29.707991+0000","last_active":"2024-09-21T02:17:30.678286+0000","last_peered":"2024-09-21T02:17:30.678286+0000","last_clean":"2024-09-21T02:17:30.678286+0000","last_became_active":"2024-09-21T02:17:29.707123+0000","last_became_peered":"2024-09-21T02:17:29.707123+0000","last_unstale":"2024-09-21T02:17:30.678286+0000","last_undegraded":"2024-09-21T02:17:30.678286+0000","last_fullsized":"2024-09-21T02:17:30.678286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:48.692098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.676932+0000","last_change":"2024-09-21T02:17:29.710665+0000","last_active":"2024-09-21T02:17:30.676932+0000","last_peered":"2024-09-21T02:17:30.676932+0000","last_clean":"2024-09-21T02:17:30.676932+0000","last_became_active":"2024-09-21T02:17:29.710314+0000","last_became_peered":"2024-09-21T02:17:29.710314+0000","last_unstale":"2024-09-21T02:17:30.676932+0000","last_undegraded":"2024-09-21T02:17:30.676932+0000","last_fullsized":"2024-09-21T02:17:30.676932+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:08.028888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678051+0000","last_change":"2024-09-21T02:17:29.734987+0000","last_active":"2024-09-21T02:17:30.678051+0000","last_peered":"2024-09-21T02:17:30.678051+0000","last_clean":"2024-09-21T02:17:30.678051+0000","last_became_active":"2024-09-21T02:17:29.734691+0000","last_became_peered":"2024-09-21T02:17:29.734691+0000","last_unstale":"2024-09-21T02:17:30.678051+0000","last_undegraded":"2024-09-21T02:17:30.678051+0000","last_fullsized":"2024-09-21T02:17:30.678051+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:23.740669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716377+0000","last_change":"2024-09-21T02:17:29.723550+0000","last_active":"2024-09-21T02:17:31.716377+0000","last_peered":"2024-09-21T02:17:31.716377+0000","last_clean":"2024-09-21T02:17:31.716377+0000","last_became_active":"2024-09-21T02:17:29.723296+0000","last_became_peered":"2024-09-21T02:17:29.723296+0000","last_unstale":"2024-09-21T02:17:31.716377+0000","last_undegraded":"2024-09-21T02:17:31.716377+0000","last_fullsized":"2024-09-21T02:17:31.716377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:21.222572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726890+0000","last_change":"2024-09-21T02:17:29.730322+0000","last_active":"2024-09-21T02:17:31.726890+0000","last_peered":"2024-09-21T02:17:31.726890+0000","last_clean":"2024-09-21T02:17:31.726890+0000","last_became_active":"2024-09-21T02:17:29.729459+0000","last_became_peered":"2024-09-21T02:17:29.729459+0000","last_unstale":"2024-09-21T02:17:31.726890+0000","last_undegraded":"2024-09-21T02:17:31.726890+0000","last_fullsized":"2024-09-21T02:17:31.726890+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:53.198595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678312+0000","last_change":"2024-09-21T02:17:29.713253+0000","last_active":"2024-09-21T02:17:30.678312+0000","last_peered":"2024-09-21T02:17:30.678312+0000","last_clean":"2024-09-21T02:17:30.678312+0000","last_became_active":"2024-09-21T02:17:29.712858+0000","last_became_peered":"2024-09-21T02:17:29.712858+0000","last_unstale":"2024-09-21T02:17:30.678312+0000","last_undegraded":"2024-09-21T02:17:30.678312+0000","last_fullsized":"2024-09-21T02:17:30.678312+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:53.691008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718279+0000","last_change":"2024-09-21T02:17:29.725936+0000","last_active":"2024-09-21T02:17:31.718279+0000","last_peered":"2024-09-21T02:17:31.718279+0000","last_clean":"2024-09-21T02:17:31.718279+0000","last_became_active":"2024-09-21T02:17:29.725606+0000","last_became_peered":"2024-09-21T02:17:29.725606+0000","last_unstale":"2024-09-21T02:17:31.718279+0000","last_undegraded":"2024-09-21T02:17:31.718279+0000","last_fullsized":"2024-09-21T02:17:31.718279+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:16.611330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726635+0000","last_change":"2024-09-21T02:17:29.709463+0000","last_active":"2024-09-21T02:17:31.726635+0000","last_peered":"2024-09-21T02:17:31.726635+0000","last_clean":"2024-09-21T02:17:31.726635+0000","last_became_active":"2024-09-21T02:17:29.708660+0000","last_became_peered":"2024-09-21T02:17:29.708660+0000","last_unstale":"2024-09-21T02:17:31.726635+0000","last_undegraded":"2024-09-21T02:17:31.726635+0000","last_fullsized":"2024-09-21T02:17:31.726635+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:06.592776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727598+0000","last_change":"2024-09-21T02:17:29.716785+0000","last_active":"2024-09-21T02:17:31.727598+0000","last_peered":"2024-09-21T02:17:31.727598+0000","last_clean":"2024-09-21T02:17:31.727598+0000","last_became_active":"2024-09-21T02:17:29.715946+0000","last_became_peered":"2024-09-21T02:17:29.715946+0000","last_unstale":"2024-09-21T02:17:31.727598+0000","last_undegraded":"2024-09-21T02:17:31.727598+0000","last_fullsized":"2024-09-21T02:17:31.727598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:29:09.674115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726451+0000","last_change":"2024-09-21T02:17:29.713586+0000","last_active":"2024-09-21T02:17:31.726451+0000","last_peered":"2024-09-21T02:17:31.726451+0000","last_clean":"2024-09-21T02:17:31.726451+0000","last_became_active":"2024-09-21T02:17:29.712704+0000","last_became_peered":"2024-09-21T02:17:29.712704+0000","last_unstale":"2024-09-21T02:17:31.726451+0000","last_undegraded":"2024-09-21T02:17:31.726451+0000","last_fullsized":"2024-09-21T02:17:31.726451+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:12.371452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727532+0000","last_change":"2024-09-21T02:17:29.698079+0000","last_active":"2024-09-21T02:17:31.727532+0000","last_peered":"2024-09-21T02:17:31.727532+0000","last_clean":"2024-09-21T02:17:31.727532+0000","last_became_active":"2024-09-21T02:17:29.697390+0000","last_became_peered":"2024-09-21T02:17:29.697390+0000","last_unstale":"2024-09-21T02:17:31.727532+0000","last_undegraded":"2024-09-21T02:17:31.727532+0000","last_fullsized":"2024-09-21T02:17:31.727532+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:27.931179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716346+0000","last_change":"2024-09-21T02:17:29.706049+0000","last_active":"2024-09-21T02:17:31.716346+0000","last_peered":"2024-09-21T02:17:31.716346+0000","last_clean":"2024-09-21T02:17:31.716346+0000","last_became_active":"2024-09-21T02:17:29.704654+0000","last_became_peered":"2024-09-21T02:17:29.704654+0000","last_unstale":"2024-09-21T02:17:31.716346+0000","last_undegraded":"2024-09-21T02:17:31.716346+0000","last_fullsized":"2024-09-21T02:17:31.716346+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:13.345487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.680018+0000","last_change":"2024-09-21T02:17:29.705127+0000","last_active":"2024-09-21T02:17:30.680018+0000","last_peered":"2024-09-21T02:17:30.680018+0000","last_clean":"2024-09-21T02:17:30.680018+0000","last_became_active":"2024-09-21T02:17:29.704326+0000","last_became_peered":"2024-09-21T02:17:29.704326+0000","last_unstale":"2024-09-21T02:17:30.680018+0000","last_undegraded":"2024-09-21T02:17:30.680018+0000","last_fullsized":"2024-09-21T02:17:30.680018+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:23.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678981+0000","last_change":"2024-09-21T02:17:29.714580+0000","last_active":"2024-09-21T02:17:30.678981+0000","last_peered":"2024-09-21T02:17:30.678981+0000","last_clean":"2024-09-21T02:17:30.678981+0000","last_became_active":"2024-09-21T02:17:29.713771+0000","last_became_peered":"2024-09-21T02:17:29.713771+0000","last_unstale":"2024-09-21T02:17:30.678981+0000","last_undegraded":"2024-09-21T02:17:30.678981+0000","last_fullsized":"2024-09-21T02:17:30.678981+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:00.843405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718267+0000","last_change":"2024-09-21T02:17:29.719661+0000","last_active":"2024-09-21T02:17:31.718267+0000","last_peered":"2024-09-21T02:17:31.718267+0000","last_clean":"2024-09-21T02:17:31.718267+0000","last_became_active":"2024-09-21T02:17:29.719332+0000","last_became_peered":"2024-09-21T02:17:29.719332+0000","last_unstale":"2024-09-21T02:17:31.718267+0000","last_undegraded":"2024-09-21T02:17:31.718267+0000","last_fullsized":"2024-09-21T02:17:31.718267+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:34.651554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678641+0000","last_change":"2024-09-21T02:17:29.696930+0000","last_active":"2024-09-21T02:17:30.678641+0000","last_peered":"2024-09-21T02:17:30.678641+0000","last_clean":"2024-09-21T02:17:30.678641+0000","last_became_active":"2024-09-21T02:17:29.696052+0000","last_became_peered":"2024-09-21T02:17:29.696052+0000","last_unstale":"2024-09-21T02:17:30.678641+0000","last_undegraded":"2024-09-21T02:17:30.678641+0000","last_fullsized":"2024-09-21T02:17:30.678641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:42.335553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725379+0000","last_change":"2024-09-21T02:17:29.709737+0000","last_active":"2024-09-21T02:17:31.725379+0000","last_peered":"2024-09-21T02:17:31.725379+0000","last_clean":"2024-09-21T02:17:31.725379+0000","last_became_active":"2024-09-21T02:17:29.709143+0000","last_became_peered":"2024-09-21T02:17:29.709143+0000","last_unstale":"2024-09-21T02:17:31.725379+0000","last_undegraded":"2024-09-21T02:17:31.725379+0000","last_fullsized":"2024-09-21T02:17:31.725379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:35.660810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677265+0000","last_change":"2024-09-21T02:17:29.702516+0000","last_active":"2024-09-21T02:17:30.677265+0000","last_peered":"2024-09-21T02:17:30.677265+0000","last_clean":"2024-09-21T02:17:30.677265+0000","last_became_active":"2024-09-21T02:17:29.701715+0000","last_became_peered":"2024-09-21T02:17:29.701715+0000","last_unstale":"2024-09-21T02:17:30.677265+0000","last_undegraded":"2024-09-21T02:17:30.677265+0000","last_fullsized":"2024-09-21T02:17:30.677265+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:51.370338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677636+0000","last_change":"2024-09-21T02:17:29.714639+0000","last_active":"2024-09-21T02:17:30.677636+0000","last_peered":"2024-09-21T02:17:30.677636+0000","last_clean":"2024-09-21T02:17:30.677636+0000","last_became_active":"2024-09-21T02:17:29.714192+0000","last_became_peered":"2024-09-21T02:17:29.714192+0000","last_unstale":"2024-09-21T02:17:30.677636+0000","last_undegraded":"2024-09-21T02:17:30.677636+0000","last_fullsized":"2024-09-21T02:17:30.677636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:38.673076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716990+0000","last_change":"2024-09-21T02:17:29.695971+0000","last_active":"2024-09-21T02:17:31.716990+0000","last_peered":"2024-09-21T02:17:31.716990+0000","last_clean":"2024-09-21T02:17:31.716990+0000","last_became_active":"2024-09-21T02:17:29.695059+0000","last_became_peered":"2024-09-21T02:17:29.695059+0000","last_unstale":"2024-09-21T02:17:31.716990+0000","last_undegraded":"2024-09-21T02:17:31.716990+0000","last_fullsized":"2024-09-21T02:17:31.716990+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:23.112727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678227+0000","last_change":"2024-09-21T02:17:29.719867+0000","last_active":"2024-09-21T02:17:30.678227+0000","last_peered":"2024-09-21T02:17:30.678227+0000","last_clean":"2024-09-21T02:17:30.678227+0000","last_became_active":"2024-09-21T02:17:29.719531+0000","last_became_peered":"2024-09-21T02:17:29.719531+0000","last_unstale":"2024-09-21T02:17:30.678227+0000","last_undegraded":"2024-09-21T02:17:30.678227+0000","last_fullsized":"2024-09-21T02:17:30.678227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:07.547304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718155+0000","last_change":"2024-09-21T02:17:29.726083+0000","last_active":"2024-09-21T02:17:31.718155+0000","last_peered":"2024-09-21T02:17:31.718155+0000","last_clean":"2024-09-21T02:17:31.718155+0000","last_became_active":"2024-09-21T02:17:29.725741+0000","last_became_peered":"2024-09-21T02:17:29.725741+0000","last_unstale":"2024-09-21T02:17:31.718155+0000","last_undegraded":"2024-09-21T02:17:31.718155+0000","last_fullsized":"2024-09-21T02:17:31.718155+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:29.273928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678759+0000","last_change":"2024-09-21T02:17:29.713725+0000","last_active":"2024-09-21T02:17:30.678759+0000","last_peered":"2024-09-21T02:17:30.678759+0000","last_clean":"2024-09-21T02:17:30.678759+0000","last_became_active":"2024-09-21T02:17:29.712865+0000","last_became_peered":"2024-09-21T02:17:29.712865+0000","last_unstale":"2024-09-21T02:17:30.678759+0000","last_undegraded":"2024-09-21T02:17:30.678759+0000","last_fullsized":"2024-09-21T02:17:30.678759+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:48.550151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727003+0000","last_change":"2024-09-21T02:17:29.712269+0000","last_active":"2024-09-21T02:17:31.727003+0000","last_peered":"2024-09-21T02:17:31.727003+0000","last_clean":"2024-09-21T02:17:31.727003+0000","last_became_active":"2024-09-21T02:17:29.711925+0000","last_became_peered":"2024-09-21T02:17:29.711925+0000","last_unstale":"2024-09-21T02:17:31.727003+0000","last_undegraded":"2024-09-21T02:17:31.727003+0000","last_fullsized":"2024-09-21T02:17:31.727003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:06:14.158723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727094+0000","last_change":"2024-09-21T02:17:29.725994+0000","last_active":"2024-09-21T02:17:31.727094+0000","last_peered":"2024-09-21T02:17:31.727094+0000","last_clean":"2024-09-21T02:17:31.727094+0000","last_became_active":"2024-09-21T02:17:29.725155+0000","last_became_peered":"2024-09-21T02:17:29.725155+0000","last_unstale":"2024-09-21T02:17:31.727094+0000","last_undegraded":"2024-09-21T02:17:31.727094+0000","last_fullsized":"2024-09-21T02:17:31.727094+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:49.942186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716722+0000","last_change":"2024-09-21T02:17:29.714852+0000","last_active":"2024-09-21T02:17:31.716722+0000","last_peered":"2024-09-21T02:17:31.716722+0000","last_clean":"2024-09-21T02:17:31.716722+0000","last_became_active":"2024-09-21T02:17:29.711734+0000","last_became_peered":"2024-09-21T02:17:29.711734+0000","last_unstale":"2024-09-21T02:17:31.716722+0000","last_undegraded":"2024-09-21T02:17:31.716722+0000","last_fullsized":"2024-09-21T02:17:31.716722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:17:12.801307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725282+0000","last_change":"2024-09-21T02:17:29.715351+0000","last_active":"2024-09-21T02:17:31.725282+0000","last_peered":"2024-09-21T02:17:31.725282+0000","last_clean":"2024-09-21T02:17:31.725282+0000","last_became_active":"2024-09-21T02:17:29.714439+0000","last_became_peered":"2024-09-21T02:17:29.714439+0000","last_unstale":"2024-09-21T02:17:31.725282+0000","last_undegraded":"2024-09-21T02:17:31.725282+0000","last_fullsized":"2024-09-21T02:17:31.725282+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:43.599967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725483+0000","last_change":"2024-09-21T02:17:29.712524+0000","last_active":"2024-09-21T02:17:31.725483+0000","last_peered":"2024-09-21T02:17:31.725483+0000","last_clean":"2024-09-21T02:17:31.725483+0000","last_became_active":"2024-09-21T02:17:29.712065+0000","last_became_peered":"2024-09-21T02:17:29.712065+0000","last_unstale":"2024-09-21T02:17:31.725483+0000","last_undegraded":"2024-09-21T02:17:31.725483+0000","last_fullsized":"2024-09-21T02:17:31.725483+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:33:22.762072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718206+0000","last_change":"2024-09-21T02:17:29.705913+0000","last_active":"2024-09-21T02:17:31.718206+0000","last_peered":"2024-09-21T02:17:31.718206+0000","last_clean":"2024-09-21T02:17:31.718206+0000","last_became_active":"2024-09-21T02:17:29.705008+0000","last_became_peered":"2024-09-21T02:17:29.705008+0000","last_unstale":"2024-09-21T02:17:31.718206+0000","last_undegraded":"2024-09-21T02:17:31.718206+0000","last_fullsized":"2024-09-21T02:17:31.718206+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:56.717877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":19,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677209+0000","last_change":"2024-09-21T02:17:28.701807+0000","last_active":"2024-09-21T02:17:30.677209+0000","last_peered":"2024-09-21T02:17:30.677209+0000","last_clean":"2024-09-21T02:17:30.677209+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:30.677209+0000","last_undegraded":"2024-09-21T02:17:30.677209+0000","last_fullsized":"2024-09-21T02:17:30.677209+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:03.455639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727556+0000","last_change":"2024-09-21T02:17:29.718308+0000","last_active":"2024-09-21T02:17:31.727556+0000","last_peered":"2024-09-21T02:17:31.727556+0000","last_clean":"2024-09-21T02:17:31.727556+0000","last_became_active":"2024-09-21T02:17:29.717473+0000","last_became_peered":"2024-09-21T02:17:29.717473+0000","last_unstale":"2024-09-21T02:17:31.727556+0000","last_undegraded":"2024-09-21T02:17:31.727556+0000","last_fullsized":"2024-09-21T02:17:31.727556+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:13.001334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679507+0000","last_change":"2024-09-21T02:17:28.709949+0000","last_active":"2024-09-21T02:17:30.679507+0000","last_peered":"2024-09-21T02:17:30.679507+0000","last_clean":"2024-09-21T02:17:30.679507+0000","last_became_active":"2024-09-21T02:17:28.709165+0000","last_became_peered":"2024-09-21T02:17:28.709165+0000","last_unstale":"2024-09-21T02:17:30.679507+0000","last_undegraded":"2024-09-21T02:17:30.679507+0000","last_fullsized":"2024-09-21T02:17:30.679507+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:46.327406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677135+0000","last_change":"2024-09-21T02:17:29.707494+0000","last_active":"2024-09-21T02:17:30.677135+0000","last_peered":"2024-09-21T02:17:30.677135+0000","last_clean":"2024-09-21T02:17:30.677135+0000","last_became_active":"2024-09-21T02:17:29.707074+0000","last_became_peered":"2024-09-21T02:17:29.707074+0000","last_unstale":"2024-09-21T02:17:30.677135+0000","last_undegraded":"2024-09-21T02:17:30.677135+0000","last_fullsized":"2024-09-21T02:17:30.677135+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:50:31.375745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718732+0000","last_change":"2024-09-21T02:17:28.694169+0000","last_active":"2024-09-21T02:17:31.718732+0000","last_peered":"2024-09-21T02:17:31.718732+0000","last_clean":"2024-09-21T02:17:31.718732+0000","last_became_active":"2024-09-21T02:17:28.693855+0000","last_became_peered":"2024-09-21T02:17:28.693855+0000","last_unstale":"2024-09-21T02:17:31.718732+0000","last_undegraded":"2024-09-21T02:17:31.718732+0000","last_fullsized":"2024-09-21T02:17:31.718732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:12.711273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678191+0000","last_change":"2024-09-21T02:17:29.728498+0000","last_active":"2024-09-21T02:17:30.678191+0000","last_peered":"2024-09-21T02:17:30.678191+0000","last_clean":"2024-09-21T02:17:30.678191+0000","last_became_active":"2024-09-21T02:17:29.728118+0000","last_became_peered":"2024-09-21T02:17:29.728118+0000","last_unstale":"2024-09-21T02:17:30.678191+0000","last_undegraded":"2024-09-21T02:17:30.678191+0000","last_fullsized":"2024-09-21T02:17:30.678191+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:31.420130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677108+0000","last_change":"2024-09-21T02:17:28.704350+0000","last_active":"2024-09-21T02:17:30.677108+0000","last_peered":"2024-09-21T02:17:30.677108+0000","last_clean":"2024-09-21T02:17:30.677108+0000","last_became_active":"2024-09-21T02:17:28.703930+0000","last_became_peered":"2024-09-21T02:17:28.703930+0000","last_unstale":"2024-09-21T02:17:30.677108+0000","last_undegraded":"2024-09-21T02:17:30.677108+0000","last_fullsized":"2024-09-21T02:17:30.677108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:23.938702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716879+0000","last_change":"2024-09-21T02:17:29.731003+0000","last_active":"2024-09-21T02:17:31.716879+0000","last_peered":"2024-09-21T02:17:31.716879+0000","last_clean":"2024-09-21T02:17:31.716879+0000","last_became_active":"2024-09-21T02:17:29.730612+0000","last_became_peered":"2024-09-21T02:17:29.730612+0000","last_unstale":"2024-09-21T02:17:31.716879+0000","last_undegraded":"2024-09-21T02:17:31.716879+0000","last_fullsized":"2024-09-21T02:17:31.716879+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:47:48.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725576+0000","last_change":"2024-09-21T02:17:28.700632+0000","last_active":"2024-09-21T02:17:31.725576+0000","last_peered":"2024-09-21T02:17:31.725576+0000","last_clean":"2024-09-21T02:17:31.725576+0000","last_became_active":"2024-09-21T02:17:28.699770+0000","last_became_peered":"2024-09-21T02:17:28.699770+0000","last_unstale":"2024-09-21T02:17:31.725576+0000","last_undegraded":"2024-09-21T02:17:31.725576+0000","last_fullsized":"2024-09-21T02:17:31.725576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:59.122385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677136+0000","last_change":"2024-09-21T02:17:29.708363+0000","last_active":"2024-09-21T02:17:30.677136+0000","last_peered":"2024-09-21T02:17:30.677136+0000","last_clean":"2024-09-21T02:17:30.677136+0000","last_became_active":"2024-09-21T02:17:29.707848+0000","last_became_peered":"2024-09-21T02:17:29.707848+0000","last_unstale":"2024-09-21T02:17:30.677136+0000","last_undegraded":"2024-09-21T02:17:30.677136+0000","last_fullsized":"2024-09-21T02:17:30.677136+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:27.411987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677587+0000","last_change":"2024-09-21T02:17:28.700727+0000","last_active":"2024-09-21T02:17:30.677587+0000","last_peered":"2024-09-21T02:17:30.677587+0000","last_clean":"2024-09-21T02:17:30.677587+0000","last_became_active":"2024-09-21T02:17:28.700382+0000","last_became_peered":"2024-09-21T02:17:28.700382+0000","last_unstale":"2024-09-21T02:17:30.677587+0000","last_undegraded":"2024-09-21T02:17:30.677587+0000","last_fullsized":"2024-09-21T02:17:30.677587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:48.490309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725936+0000","last_change":"2024-09-21T02:17:29.711888+0000","last_active":"2024-09-21T02:17:31.725936+0000","last_peered":"2024-09-21T02:17:31.725936+0000","last_clean":"2024-09-21T02:17:31.725936+0000","last_became_active":"2024-09-21T02:17:29.710999+0000","last_became_peered":"2024-09-21T02:17:29.710999+0000","last_unstale":"2024-09-21T02:17:31.725936+0000","last_undegraded":"2024-09-21T02:17:31.725936+0000","last_fullsized":"2024-09-21T02:17:31.725936+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:37.541800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727858+0000","last_change":"2024-09-21T02:17:28.704257+0000","last_active":"2024-09-21T02:17:31.727858+0000","last_peered":"2024-09-21T02:17:31.727858+0000","last_clean":"2024-09-21T02:17:31.727858+0000","last_became_active":"2024-09-21T02:17:28.703356+0000","last_became_peered":"2024-09-21T02:17:28.703356+0000","last_unstale":"2024-09-21T02:17:31.727858+0000","last_undegraded":"2024-09-21T02:17:31.727858+0000","last_fullsized":"2024-09-21T02:17:31.727858+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:11.418878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728147+0000","last_change":"2024-09-21T02:17:29.718207+0000","last_active":"2024-09-21T02:17:31.728147+0000","last_peered":"2024-09-21T02:17:31.728147+0000","last_clean":"2024-09-21T02:17:31.728147+0000","last_became_active":"2024-09-21T02:17:29.717239+0000","last_became_peered":"2024-09-21T02:17:29.717239+0000","last_unstale":"2024-09-21T02:17:31.728147+0000","last_undegraded":"2024-09-21T02:17:31.728147+0000","last_fullsized":"2024-09-21T02:17:31.728147+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.334832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.724548+0000","last_change":"2024-09-21T02:17:28.708059+0000","last_active":"2024-09-21T02:17:31.724548+0000","last_peered":"2024-09-21T02:17:31.724548+0000","last_clean":"2024-09-21T02:17:31.724548+0000","last_became_active":"2024-09-21T02:17:28.702097+0000","last_became_peered":"2024-09-21T02:17:28.702097+0000","last_unstale":"2024-09-21T02:17:31.724548+0000","last_undegraded":"2024-09-21T02:17:31.724548+0000","last_fullsized":"2024-09-21T02:17:31.724548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:12.749317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678559+0000","last_change":"2024-09-21T02:17:29.720124+0000","last_active":"2024-09-21T02:17:30.678559+0000","last_peered":"2024-09-21T02:17:30.678559+0000","last_clean":"2024-09-21T02:17:30.678559+0000","last_became_active":"2024-09-21T02:17:29.719730+0000","last_became_peered":"2024-09-21T02:17:29.719730+0000","last_unstale":"2024-09-21T02:17:30.678559+0000","last_undegraded":"2024-09-21T02:17:30.678559+0000","last_fullsized":"2024-09-21T02:17:30.678559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:47.274507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718121+0000","last_change":"2024-09-21T02:17:28.709078+0000","last_active":"2024-09-21T02:17:31.718121+0000","last_peered":"2024-09-21T02:17:31.718121+0000","last_clean":"2024-09-21T02:17:31.718121+0000","last_became_active":"2024-09-21T02:17:28.708778+0000","last_became_peered":"2024-09-21T02:17:28.708778+0000","last_unstale":"2024-09-21T02:17:31.718121+0000","last_undegraded":"2024-09-21T02:17:31.718121+0000","last_fullsized":"2024-09-21T02:17:31.718121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:05.707161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725997+0000","last_change":"2024-09-21T02:17:29.713765+0000","last_active":"2024-09-21T02:17:31.725997+0000","last_peered":"2024-09-21T02:17:31.725997+0000","last_clean":"2024-09-21T02:17:31.725997+0000","last_became_active":"2024-09-21T02:17:29.713185+0000","last_became_peered":"2024-09-21T02:17:29.713185+0000","last_unstale":"2024-09-21T02:17:31.725997+0000","last_undegraded":"2024-09-21T02:17:31.725997+0000","last_fullsized":"2024-09-21T02:17:31.725997+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:52.019660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727093+0000","last_change":"2024-09-21T02:17:28.697107+0000","last_active":"2024-09-21T02:17:31.727093+0000","last_peered":"2024-09-21T02:17:31.727093+0000","last_clean":"2024-09-21T02:17:31.727093+0000","last_became_active":"2024-09-21T02:17:28.696766+0000","last_became_peered":"2024-09-21T02:17:28.696766+0000","last_unstale":"2024-09-21T02:17:31.727093+0000","last_undegraded":"2024-09-21T02:17:31.727093+0000","last_fullsized":"2024-09-21T02:17:31.727093+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:54.148477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.716722+0000","last_change":"2024-09-21T02:17:29.717855+0000","last_active":"2024-09-21T02:17:31.716722+0000","last_peered":"2024-09-21T02:17:31.716722+0000","last_clean":"2024-09-21T02:17:31.716722+0000","last_became_active":"2024-09-21T02:17:29.717526+0000","last_became_peered":"2024-09-21T02:17:29.717526+0000","last_unstale":"2024-09-21T02:17:31.716722+0000","last_undegraded":"2024-09-21T02:17:31.716722+0000","last_fullsized":"2024-09-21T02:17:31.716722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:24.710835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726671+0000","last_change":"2024-09-21T02:17:28.712665+0000","last_active":"2024-09-21T02:17:31.726671+0000","last_peered":"2024-09-21T02:17:31.726671+0000","last_clean":"2024-09-21T02:17:31.726671+0000","last_became_active":"2024-09-21T02:17:28.711793+0000","last_became_peered":"2024-09-21T02:17:28.711793+0000","last_unstale":"2024-09-21T02:17:31.726671+0000","last_undegraded":"2024-09-21T02:17:31.726671+0000","last_fullsized":"2024-09-21T02:17:31.726671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:57.844738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679386+0000","last_change":"2024-09-21T02:17:29.712983+0000","last_active":"2024-09-21T02:17:30.679386+0000","last_peered":"2024-09-21T02:17:30.679386+0000","last_clean":"2024-09-21T02:17:30.679386+0000","last_became_active":"2024-09-21T02:17:29.711783+0000","last_became_peered":"2024-09-21T02:17:29.711783+0000","last_unstale":"2024-09-21T02:17:30.679386+0000","last_undegraded":"2024-09-21T02:17:30.679386+0000","last_fullsized":"2024-09-21T02:17:30.679386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:45.653105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728909+0000","last_change":"2024-09-21T02:17:28.711641+0000","last_active":"2024-09-21T02:17:31.728909+0000","last_peered":"2024-09-21T02:17:31.728909+0000","last_clean":"2024-09-21T02:17:31.728909+0000","last_became_active":"2024-09-21T02:17:28.710785+0000","last_became_peered":"2024-09-21T02:17:28.710785+0000","last_unstale":"2024-09-21T02:17:31.728909+0000","last_undegraded":"2024-09-21T02:17:31.728909+0000","last_fullsized":"2024-09-21T02:17:31.728909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:46.501854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677245+0000","last_change":"2024-09-21T02:17:29.692488+0000","last_active":"2024-09-21T02:17:30.677245+0000","last_peered":"2024-09-21T02:17:30.677245+0000","last_clean":"2024-09-21T02:17:30.677245+0000","last_became_active":"2024-09-21T02:17:29.691441+0000","last_became_peered":"2024-09-21T02:17:29.691441+0000","last_unstale":"2024-09-21T02:17:30.677245+0000","last_undegraded":"2024-09-21T02:17:30.677245+0000","last_fullsized":"2024-09-21T02:17:30.677245+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:08.591533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728187+0000","last_change":"2024-09-21T02:17:28.705017+0000","last_active":"2024-09-21T02:17:31.728187+0000","last_peered":"2024-09-21T02:17:31.728187+0000","last_clean":"2024-09-21T02:17:31.728187+0000","last_became_active":"2024-09-21T02:17:28.704183+0000","last_became_peered":"2024-09-21T02:17:28.704183+0000","last_unstale":"2024-09-21T02:17:31.728187+0000","last_undegraded":"2024-09-21T02:17:31.728187+0000","last_fullsized":"2024-09-21T02:17:31.728187+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:30.415777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677918+0000","last_change":"2024-09-21T02:17:29.711001+0000","last_active":"2024-09-21T02:17:30.677918+0000","last_peered":"2024-09-21T02:17:30.677918+0000","last_clean":"2024-09-21T02:17:30.677918+0000","last_became_active":"2024-09-21T02:17:29.710543+0000","last_became_peered":"2024-09-21T02:17:29.710543+0000","last_unstale":"2024-09-21T02:17:30.677918+0000","last_undegraded":"2024-09-21T02:17:30.677918+0000","last_fullsized":"2024-09-21T02:17:30.677918+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:56:19.910922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677715+0000","last_change":"2024-09-21T02:17:28.695228+0000","last_active":"2024-09-21T02:17:30.677715+0000","last_peered":"2024-09-21T02:17:30.677715+0000","last_clean":"2024-09-21T02:17:30.677715+0000","last_became_active":"2024-09-21T02:17:28.694899+0000","last_became_peered":"2024-09-21T02:17:28.694899+0000","last_unstale":"2024-09-21T02:17:30.677715+0000","last_undegraded":"2024-09-21T02:17:30.677715+0000","last_fullsized":"2024-09-21T02:17:30.677715+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:12.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678205+0000","last_change":"2024-09-21T02:17:29.691271+0000","last_active":"2024-09-21T02:17:30.678205+0000","last_peered":"2024-09-21T02:17:30.678205+0000","last_clean":"2024-09-21T02:17:30.678205+0000","last_became_active":"2024-09-21T02:17:29.690416+0000","last_became_peered":"2024-09-21T02:17:29.690416+0000","last_unstale":"2024-09-21T02:17:30.678205+0000","last_undegraded":"2024-09-21T02:17:30.678205+0000","last_fullsized":"2024-09-21T02:17:30.678205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:49.838744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677810+0000","last_change":"2024-09-21T02:17:28.715260+0000","last_active":"2024-09-21T02:17:30.677810+0000","last_peered":"2024-09-21T02:17:30.677810+0000","last_clean":"2024-09-21T02:17:30.677810+0000","last_became_active":"2024-09-21T02:17:28.715003+0000","last_became_peered":"2024-09-21T02:17:28.715003+0000","last_unstale":"2024-09-21T02:17:30.677810+0000","last_undegraded":"2024-09-21T02:17:30.677810+0000","last_fullsized":"2024-09-21T02:17:30.677810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:44.123972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.724913+0000","last_change":"2024-09-21T02:17:29.710382+0000","last_active":"2024-09-21T02:17:31.724913+0000","last_peered":"2024-09-21T02:17:31.724913+0000","last_clean":"2024-09-21T02:17:31.724913+0000","last_became_active":"2024-09-21T02:17:29.709530+0000","last_became_peered":"2024-09-21T02:17:29.709530+0000","last_unstale":"2024-09-21T02:17:31.724913+0000","last_undegraded":"2024-09-21T02:17:31.724913+0000","last_fullsized":"2024-09-21T02:17:31.724913+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:19.636108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677480+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:30.677480+0000","last_peered":"2024-09-21T02:17:30.677480+0000","last_clean":"2024-09-21T02:17:30.677480+0000","last_became_active":"2024-09-21T02:17:28.696779+0000","last_became_peered":"2024-09-21T02:17:28.696779+0000","last_unstale":"2024-09-21T02:17:30.677480+0000","last_undegraded":"2024-09-21T02:17:30.677480+0000","last_fullsized":"2024-09-21T02:17:30.677480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:39.238508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726578+0000","last_change":"2024-09-21T02:17:29.702103+0000","last_active":"2024-09-21T02:17:31.726578+0000","last_peered":"2024-09-21T02:17:31.726578+0000","last_clean":"2024-09-21T02:17:31.726578+0000","last_became_active":"2024-09-21T02:17:29.699620+0000","last_became_peered":"2024-09-21T02:17:29.699620+0000","last_unstale":"2024-09-21T02:17:31.726578+0000","last_undegraded":"2024-09-21T02:17:31.726578+0000","last_fullsized":"2024-09-21T02:17:31.726578+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:25.741087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718236+0000","last_change":"2024-09-21T02:17:28.697982+0000","last_active":"2024-09-21T02:17:31.718236+0000","last_peered":"2024-09-21T02:17:31.718236+0000","last_clean":"2024-09-21T02:17:31.718236+0000","last_became_active":"2024-09-21T02:17:28.697661+0000","last_became_peered":"2024-09-21T02:17:28.697661+0000","last_unstale":"2024-09-21T02:17:31.718236+0000","last_undegraded":"2024-09-21T02:17:31.718236+0000","last_fullsized":"2024-09-21T02:17:31.718236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:44.421655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727285+0000","last_change":"2024-09-21T02:17:29.694011+0000","last_active":"2024-09-21T02:17:31.727285+0000","last_peered":"2024-09-21T02:17:31.727285+0000","last_clean":"2024-09-21T02:17:31.727285+0000","last_became_active":"2024-09-21T02:17:29.693144+0000","last_became_peered":"2024-09-21T02:17:29.693144+0000","last_unstale":"2024-09-21T02:17:31.727285+0000","last_undegraded":"2024-09-21T02:17:31.727285+0000","last_fullsized":"2024-09-21T02:17:31.727285+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:27.183808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727121+0000","last_change":"2024-09-21T02:17:28.691325+0000","last_active":"2024-09-21T02:17:31.727121+0000","last_peered":"2024-09-21T02:17:31.727121+0000","last_clean":"2024-09-21T02:17:31.727121+0000","last_became_active":"2024-09-21T02:17:28.691006+0000","last_became_peered":"2024-09-21T02:17:28.691006+0000","last_unstale":"2024-09-21T02:17:31.727121+0000","last_undegraded":"2024-09-21T02:17:31.727121+0000","last_fullsized":"2024-09-21T02:17:31.727121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:43.354026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725793+0000","last_change":"2024-09-21T02:17:29.728111+0000","last_active":"2024-09-21T02:17:31.725793+0000","last_peered":"2024-09-21T02:17:31.725793+0000","last_clean":"2024-09-21T02:17:31.725793+0000","last_became_active":"2024-09-21T02:17:29.727378+0000","last_became_peered":"2024-09-21T02:17:29.727378+0000","last_unstale":"2024-09-21T02:17:31.725793+0000","last_undegraded":"2024-09-21T02:17:31.725793+0000","last_fullsized":"2024-09-21T02:17:31.725793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:10.261343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725007+0000","last_change":"2024-09-21T02:17:28.694528+0000","last_active":"2024-09-21T02:17:31.725007+0000","last_peered":"2024-09-21T02:17:31.725007+0000","last_clean":"2024-09-21T02:17:31.725007+0000","last_became_active":"2024-09-21T02:17:28.694184+0000","last_became_peered":"2024-09-21T02:17:28.694184+0000","last_unstale":"2024-09-21T02:17:31.725007+0000","last_undegraded":"2024-09-21T02:17:31.725007+0000","last_fullsized":"2024-09-21T02:17:31.725007+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:33.475198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725797+0000","last_change":"2024-09-21T02:17:29.726128+0000","last_active":"2024-09-21T02:17:31.725797+0000","last_peered":"2024-09-21T02:17:31.725797+0000","last_clean":"2024-09-21T02:17:31.725797+0000","last_became_active":"2024-09-21T02:17:29.725308+0000","last_became_peered":"2024-09-21T02:17:29.725308+0000","last_unstale":"2024-09-21T02:17:31.725797+0000","last_undegraded":"2024-09-21T02:17:31.725797+0000","last_fullsized":"2024-09-21T02:17:31.725797+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:38.627673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677246+0000","last_change":"2024-09-21T02:17:28.695543+0000","last_active":"2024-09-21T02:17:30.677246+0000","last_peered":"2024-09-21T02:17:30.677246+0000","last_clean":"2024-09-21T02:17:30.677246+0000","last_became_active":"2024-09-21T02:17:28.694827+0000","last_became_peered":"2024-09-21T02:17:28.694827+0000","last_unstale":"2024-09-21T02:17:30.677246+0000","last_undegraded":"2024-09-21T02:17:30.677246+0000","last_fullsized":"2024-09-21T02:17:30.677246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.956160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728393+0000","last_change":"2024-09-21T02:17:29.705415+0000","last_active":"2024-09-21T02:17:31.728393+0000","last_peered":"2024-09-21T02:17:31.728393+0000","last_clean":"2024-09-21T02:17:31.728393+0000","last_became_active":"2024-09-21T02:17:29.704569+0000","last_became_peered":"2024-09-21T02:17:29.704569+0000","last_unstale":"2024-09-21T02:17:31.728393+0000","last_undegraded":"2024-09-21T02:17:31.728393+0000","last_fullsized":"2024-09-21T02:17:31.728393+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:41.180647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677389+0000","last_change":"2024-09-21T02:17:28.702105+0000","last_active":"2024-09-21T02:17:30.677389+0000","last_peered":"2024-09-21T02:17:30.677389+0000","last_clean":"2024-09-21T02:17:30.677389+0000","last_became_active":"2024-09-21T02:17:28.701785+0000","last_became_peered":"2024-09-21T02:17:28.701785+0000","last_unstale":"2024-09-21T02:17:30.677389+0000","last_undegraded":"2024-09-21T02:17:30.677389+0000","last_fullsized":"2024-09-21T02:17:30.677389+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:40.433891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726589+0000","last_change":"2024-09-21T02:17:29.725985+0000","last_active":"2024-09-21T02:17:31.726589+0000","last_peered":"2024-09-21T02:17:31.726589+0000","last_clean":"2024-09-21T02:17:31.726589+0000","last_became_active":"2024-09-21T02:17:29.725206+0000","last_became_peered":"2024-09-21T02:17:29.725206+0000","last_unstale":"2024-09-21T02:17:31.726589+0000","last_undegraded":"2024-09-21T02:17:31.726589+0000","last_fullsized":"2024-09-21T02:17:31.726589+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:20.552393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725574+0000","last_change":"2024-09-21T02:17:28.700846+0000","last_active":"2024-09-21T02:17:31.725574+0000","last_peered":"2024-09-21T02:17:31.725574+0000","last_clean":"2024-09-21T02:17:31.725574+0000","last_became_active":"2024-09-21T02:17:28.700490+0000","last_became_peered":"2024-09-21T02:17:28.700490+0000","last_unstale":"2024-09-21T02:17:31.725574+0000","last_undegraded":"2024-09-21T02:17:31.725574+0000","last_fullsized":"2024-09-21T02:17:31.725574+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:00.721655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.728772+0000","last_change":"2024-09-21T02:17:29.700985+0000","last_active":"2024-09-21T02:17:31.728772+0000","last_peered":"2024-09-21T02:17:31.728772+0000","last_clean":"2024-09-21T02:17:31.728772+0000","last_became_active":"2024-09-21T02:17:29.700170+0000","last_became_peered":"2024-09-21T02:17:29.700170+0000","last_unstale":"2024-09-21T02:17:31.728772+0000","last_undegraded":"2024-09-21T02:17:31.728772+0000","last_fullsized":"2024-09-21T02:17:31.728772+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:26.939620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677822+0000","last_change":"2024-09-21T02:17:28.706151+0000","last_active":"2024-09-21T02:17:30.677822+0000","last_peered":"2024-09-21T02:17:30.677822+0000","last_clean":"2024-09-21T02:17:30.677822+0000","last_became_active":"2024-09-21T02:17:28.705306+0000","last_became_peered":"2024-09-21T02:17:28.705306+0000","last_unstale":"2024-09-21T02:17:30.677822+0000","last_undegraded":"2024-09-21T02:17:30.677822+0000","last_fullsized":"2024-09-21T02:17:30.677822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:21.178808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679360+0000","last_change":"2024-09-21T02:17:29.708739+0000","last_active":"2024-09-21T02:17:30.679360+0000","last_peered":"2024-09-21T02:17:30.679360+0000","last_clean":"2024-09-21T02:17:30.679360+0000","last_became_active":"2024-09-21T02:17:29.707885+0000","last_became_peered":"2024-09-21T02:17:29.707885+0000","last_unstale":"2024-09-21T02:17:30.679360+0000","last_undegraded":"2024-09-21T02:17:30.679360+0000","last_fullsized":"2024-09-21T02:17:30.679360+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:13:18.505607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718585+0000","last_change":"2024-09-21T02:17:28.697109+0000","last_active":"2024-09-21T02:17:31.718585+0000","last_peered":"2024-09-21T02:17:31.718585+0000","last_clean":"2024-09-21T02:17:31.718585+0000","last_became_active":"2024-09-21T02:17:28.696772+0000","last_became_peered":"2024-09-21T02:17:28.696772+0000","last_unstale":"2024-09-21T02:17:31.718585+0000","last_undegraded":"2024-09-21T02:17:31.718585+0000","last_fullsized":"2024-09-21T02:17:31.718585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:03.422543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678458+0000","last_change":"2024-09-21T02:17:29.718721+0000","last_active":"2024-09-21T02:17:30.678458+0000","last_peered":"2024-09-21T02:17:30.678458+0000","last_clean":"2024-09-21T02:17:30.678458+0000","last_became_active":"2024-09-21T02:17:29.718407+0000","last_became_peered":"2024-09-21T02:17:29.718407+0000","last_unstale":"2024-09-21T02:17:30.678458+0000","last_undegraded":"2024-09-21T02:17:30.678458+0000","last_fullsized":"2024-09-21T02:17:30.678458+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:54.495444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"72'1","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.718770+0000","last_change":"2024-09-21T02:17:28.692883+0000","last_active":"2024-09-21T02:17:31.718770+0000","last_peered":"2024-09-21T02:17:31.718770+0000","last_clean":"2024-09-21T02:17:31.718770+0000","last_became_active":"2024-09-21T02:17:28.692516+0000","last_became_peered":"2024-09-21T02:17:28.692516+0000","last_unstale":"2024-09-21T02:17:31.718770+0000","last_undegraded":"2024-09-21T02:17:31.718770+0000","last_fullsized":"2024-09-21T02:17:31.718770+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:57.655155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727660+0000","last_change":"2024-09-21T02:17:29.698220+0000","last_active":"2024-09-21T02:17:31.727660+0000","last_peered":"2024-09-21T02:17:31.727660+0000","last_clean":"2024-09-21T02:17:31.727660+0000","last_became_active":"2024-09-21T02:17:29.697389+0000","last_became_peered":"2024-09-21T02:17:29.697389+0000","last_unstale":"2024-09-21T02:17:31.727660+0000","last_undegraded":"2024-09-21T02:17:31.727660+0000","last_fullsized":"2024-09-21T02:17:31.727660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:23.052204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726913+0000","last_change":"2024-09-21T02:17:28.710748+0000","last_active":"2024-09-21T02:17:31.726913+0000","last_peered":"2024-09-21T02:17:31.726913+0000","last_clean":"2024-09-21T02:17:31.726913+0000","last_became_active":"2024-09-21T02:17:28.709895+0000","last_became_peered":"2024-09-21T02:17:28.709895+0000","last_unstale":"2024-09-21T02:17:31.726913+0000","last_undegraded":"2024-09-21T02:17:31.726913+0000","last_fullsized":"2024-09-21T02:17:31.726913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:58.752112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678433+0000","last_change":"2024-09-21T02:17:29.718562+0000","last_active":"2024-09-21T02:17:30.678433+0000","last_peered":"2024-09-21T02:17:30.678433+0000","last_clean":"2024-09-21T02:17:30.678433+0000","last_became_active":"2024-09-21T02:17:29.718143+0000","last_became_peered":"2024-09-21T02:17:29.718143+0000","last_unstale":"2024-09-21T02:17:30.678433+0000","last_undegraded":"2024-09-21T02:17:30.678433+0000","last_fullsized":"2024-09-21T02:17:30.678433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:41.579975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.726732+0000","last_change":"2024-09-21T02:17:28.712176+0000","last_active":"2024-09-21T02:17:31.726732+0000","last_peered":"2024-09-21T02:17:31.726732+0000","last_clean":"2024-09-21T02:17:31.726732+0000","last_became_active":"2024-09-21T02:17:28.711338+0000","last_became_peered":"2024-09-21T02:17:28.711338+0000","last_unstale":"2024-09-21T02:17:31.726732+0000","last_undegraded":"2024-09-21T02:17:31.726732+0000","last_fullsized":"2024-09-21T02:17:31.726732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:16.031750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679259+0000","last_change":"2024-09-21T02:17:29.701859+0000","last_active":"2024-09-21T02:17:30.679259+0000","last_peered":"2024-09-21T02:17:30.679259+0000","last_clean":"2024-09-21T02:17:30.679259+0000","last_became_active":"2024-09-21T02:17:29.701010+0000","last_became_peered":"2024-09-21T02:17:29.701010+0000","last_unstale":"2024-09-21T02:17:30.679259+0000","last_undegraded":"2024-09-21T02:17:30.679259+0000","last_fullsized":"2024-09-21T02:17:30.679259+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:25.619135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725918+0000","last_change":"2024-09-21T02:17:28.702782+0000","last_active":"2024-09-21T02:17:31.725918+0000","last_peered":"2024-09-21T02:17:31.725918+0000","last_clean":"2024-09-21T02:17:31.725918+0000","last_became_active":"2024-09-21T02:17:28.701746+0000","last_became_peered":"2024-09-21T02:17:28.701746+0000","last_unstale":"2024-09-21T02:17:31.725918+0000","last_undegraded":"2024-09-21T02:17:31.725918+0000","last_fullsized":"2024-09-21T02:17:31.725918+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:23.056880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.717431+0000","last_change":"2024-09-21T02:17:29.716928+0000","last_active":"2024-09-21T02:17:31.717431+0000","last_peered":"2024-09-21T02:17:31.717431+0000","last_clean":"2024-09-21T02:17:31.717431+0000","last_became_active":"2024-09-21T02:17:29.716616+0000","last_became_peered":"2024-09-21T02:17:29.716616+0000","last_unstale":"2024-09-21T02:17:31.717431+0000","last_undegraded":"2024-09-21T02:17:31.717431+0000","last_fullsized":"2024-09-21T02:17:31.717431+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:26:31.195546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.679156+0000","last_change":"2024-09-21T02:17:28.703261+0000","last_active":"2024-09-21T02:17:30.679156+0000","last_peered":"2024-09-21T02:17:30.679156+0000","last_clean":"2024-09-21T02:17:30.679156+0000","last_became_active":"2024-09-21T02:17:28.702239+0000","last_became_peered":"2024-09-21T02:17:28.702239+0000","last_unstale":"2024-09-21T02:17:30.679156+0000","last_undegraded":"2024-09-21T02:17:30.679156+0000","last_fullsized":"2024-09-21T02:17:30.679156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:59.351666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.727276+0000","last_change":"2024-09-21T02:17:29.716762+0000","last_active":"2024-09-21T02:17:31.727276+0000","last_peered":"2024-09-21T02:17:31.727276+0000","last_clean":"2024-09-21T02:17:31.727276+0000","last_became_active":"2024-09-21T02:17:29.715816+0000","last_became_peered":"2024-09-21T02:17:29.715816+0000","last_unstale":"2024-09-21T02:17:31.727276+0000","last_undegraded":"2024-09-21T02:17:31.727276+0000","last_fullsized":"2024-09-21T02:17:31.727276+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:25.008881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725718+0000","last_change":"2024-09-21T02:17:28.686833+0000","last_active":"2024-09-21T02:17:31.725718+0000","last_peered":"2024-09-21T02:17:31.725718+0000","last_clean":"2024-09-21T02:17:31.725718+0000","last_became_active":"2024-09-21T02:17:28.686533+0000","last_became_peered":"2024-09-21T02:17:28.686533+0000","last_unstale":"2024-09-21T02:17:31.725718+0000","last_undegraded":"2024-09-21T02:17:31.725718+0000","last_fullsized":"2024-09-21T02:17:31.725718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:15.420276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:31.725007+0000","last_change":"2024-09-21T02:17:29.715826+0000","last_active":"2024-09-21T02:17:31.725007+0000","last_peered":"2024-09-21T02:17:31.725007+0000","last_clean":"2024-09-21T02:17:31.725007+0000","last_became_active":"2024-09-21T02:17:29.714951+0000","last_became_peered":"2024-09-21T02:17:29.714951+0000","last_unstale":"2024-09-21T02:17:31.725007+0000","last_undegraded":"2024-09-21T02:17:31.725007+0000","last_fullsized":"2024-09-21T02:17:31.725007+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:34.951731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.676723+0000","last_change":"2024-09-21T02:17:28.709392+0000","last_active":"2024-09-21T02:17:30.676723+0000","last_peered":"2024-09-21T02:17:30.676723+0000","last_clean":"2024-09-21T02:17:30.676723+0000","last_became_active":"2024-09-21T02:17:28.708868+0000","last_became_peered":"2024-09-21T02:17:28.708868+0000","last_unstale":"2024-09-21T02:17:30.676723+0000","last_undegraded":"2024-09-21T02:17:30.676723+0000","last_fullsized":"2024-09-21T02:17:30.676723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:14.391234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.678579+0000","last_change":"2024-09-21T02:17:29.709912+0000","last_active":"2024-09-21T02:17:30.678579+0000","last_peered":"2024-09-21T02:17:30.678579+0000","last_clean":"2024-09-21T02:17:30.678579+0000","last_became_active":"2024-09-21T02:17:29.709035+0000","last_became_peered":"2024-09-21T02:17:29.709035+0000","last_unstale":"2024-09-21T02:17:30.678579+0000","last_undegraded":"2024-09-21T02:17:30.678579+0000","last_fullsized":"2024-09-21T02:17:30.678579+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:28.656677+0000","last_clean_scrub_stamp":"2024-09-21T02:17:28.656677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:21.043217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677926+0000","last_change":"2024-09-21T02:17:28.701648+0000","last_active":"2024-09-21T02:17:30.677926+0000","last_peered":"2024-09-21T02:17:30.677926+0000","last_clean":"2024-09-21T02:17:30.677926+0000","last_became_active":"2024-09-21T02:17:28.701307+0000","last_became_peered":"2024-09-21T02:17:28.701307+0000","last_unstale":"2024-09-21T02:17:30.677926+0000","last_undegraded":"2024-09-21T02:17:30.677926+0000","last_fullsized":"2024-09-21T02:17:30.677926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:17:27.644478+0000","last_clean_scrub_stamp":"2024-09-21T02:17:27.644478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:14.619555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":200,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T02:17:30.677624+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:17:30.677624+0000","last_peered":"2024-09-21T02:17:30.677624+0000","last_clean":"2024-09-21T02:17:30.677624+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:17:30.677624+0000","last_undegraded":"2024-09-21T02:17:30.677624+0000","last_fullsized":"2024-09-21T02:17:30.677624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":367017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":884736,"data_stored":367051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1466368,"data_stored":955863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36208,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335632,"statfs":{"total":96636764160,"available":96599687168,"internally_reserved":0,"allocated":880640,"data_stored":365531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":872448,"data_stored":365987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1462272,"data_stored":955773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335640,"statfs":{"total":96636764160,"available":96599695360,"internally_reserved":0,"allocated":872448,"data_stored":365475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":366099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:17:38.874 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:17:38.875 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 -- 172.21.15.170:0/2085297817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 msgr2=0x7f32640cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 --2- 172.21.15.170:0/2085297817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f32640cee90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f32640cca30 tx=0x7f325c01cb60 comp rx=0 tx=0).stop 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 -- 172.21.15.170:0/2085297817 shutdown_connections 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 --2- 172.21.15.170:0/2085297817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f32640cee90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 --2- 172.21.15.170:0/2085297817 >> v2:172.21.15.191:3300/0 conn(0x7f32640b2fc0 0x7f32640ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 -- 172.21.15.170:0/2085297817 >> 172.21.15.170:0/2085297817 conn(0x7f32640d0170 msgr2=0x7f32640d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 -- 172.21.15.170:0/2085297817 shutdown_connections 2024-09-21T02:17:39.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.076+0000 7f3269f0b640 1 -- 172.21.15.170:0/2085297817 wait complete. 2024-09-21T02:17:39.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 Processor -- start 2024-09-21T02:17:39.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 -- start start 2024-09-21T02:17:39.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f326409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32640b2fc0 0x7f3264152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32640cea80 0x7f32641fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3264169f50 con 0x7f3264070660 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3264169dd0 con 0x7f32640cea80 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f326416a0d0 con 0x7f32640b2fc0 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f326409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f326409a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46500/0 (socket says 172.21.15.170:46500) 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 -- 172.21.15.170:0/662305279 learned_addr learned my addr 172.21.15.170:0/662305279 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 -- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32640b2fc0 msgr2=0x7f3264152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32640b2fc0 0x7f3264152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3263fff640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32640cea80 0x7f32641fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 -- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32640cea80 msgr2=0x7f32641fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32640cea80 0x7f32641fa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 -- 172.21.15.170:0/662305279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326409c6f0 con 0x7f3264070660 2024-09-21T02:17:39.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f32637fe640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f326409a9c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f325400b410 tx=0x7f325400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:39.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254015020 con 0x7f3264070660 2024-09-21T02:17:39.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3254011920 con 0x7f3264070660 2024-09-21T02:17:39.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254011d70 con 0x7f3264070660 2024-09-21T02:17:39.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.080+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32641535b0 con 0x7f3264070660 2024-09-21T02:17:39.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326409c990 con 0x7f3264070660 2024-09-21T02:17:39.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32640cee90 con 0x7f3264070660 2024-09-21T02:17:39.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f3254018020 con 0x7f3264070660 2024-09-21T02:17:39.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f324bfff640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3250073560 0x7f3250075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f32540947b0 con 0x7f3264070660 2024-09-21T02:17:39.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f3262ffd640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3250073560 0x7f3250075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.084+0000 7f3262ffd640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3250073560 0x7f3250075a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32640affe0 tx=0x7f3258007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:39.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.088+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f325401f020 con 0x7f3264070660 2024-09-21T02:17:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.324+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f32640adf30 con 0x7f3264070660 2024-09-21T02:17:39.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.328+0000 7f324bfff640 1 -- 172.21.15.170:0/662305279 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f325405f340 con 0x7f3264070660 2024-09-21T02:17:39.332 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:17:39.332 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:17:39.332 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:17:39.332 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:17:39.332 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:17:39.332 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3250073560 msgr2=0x7f3250075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3250073560 0x7f3250075a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f32640affe0 tx=0x7f3258007a00 comp rx=0 tx=0).stop 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 msgr2=0x7f326409a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f326409a9c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f325400b410 tx=0x7f325400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 shutdown_connections 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f3250073560 0x7f3250075a70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32640b2fc0 0x7f3264152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32640cea80 0x7f32641fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 --2- 172.21.15.170:0/662305279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3264070660 0x7f326409a9c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 >> 172.21.15.170:0/662305279 conn(0x7f32640d0170 msgr2=0x7f3264156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:39.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 shutdown_connections 2024-09-21T02:17:39.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.332+0000 7f3269f0b640 1 -- 172.21.15.170:0/662305279 wait complete. 2024-09-21T02:17:39.354 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/1873862273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1ac004b50 msgr2=0x7fb1ac004fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/1873862273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1ac004b50 0x7fb1ac004fc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009b60 tx=0x7fb1b401d2b0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c2d9d640 1 -- 172.21.15.170:0/1873862273 reap_dead start 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/1873862273 shutdown_connections 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/1873862273 >> 172.21.15.170:0/1873862273 conn(0x7fb1c40b1c40 msgr2=0x7fb1c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/1873862273 shutdown_connections 2024-09-21T02:17:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/1873862273 wait complete. 2024-09-21T02:17:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 Processor -- start 2024-09-21T02:17:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.492+0000 7fb1c8a1d640 1 -- start start 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1ac004b50 0x7fb1c40ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 0x7fb1c415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c41f5600 0x7fb1c40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4167380 con 0x7fb1c415da60 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c4167200 con 0x7fb1c41f5600 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c4167500 con 0x7fb1ac004b50 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 0x7fb1c415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 0x7fb1c415eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46524/0 (socket says 172.21.15.170:46524) 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 -- 172.21.15.170:0/536967674 learned_addr learned my addr 172.21.15.170:0/536967674 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c259c640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1ac004b50 0x7fb1c40ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c2d9d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c41f5600 0x7fb1c40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 -- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1ac004b50 msgr2=0x7fb1c40ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1ac004b50 0x7fb1c40ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 -- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c41f5600 msgr2=0x7fb1c40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c41f5600 0x7fb1c40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 -- 172.21.15.170:0/536967674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c41f9350 con 0x7fb1c415da60 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c2d9d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c41f5600 0x7fb1c40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c259c640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1ac004b50 0x7fb1c40ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c1d9b640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 0x7fb1c415eb90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d7b0 tx=0x7fb1b800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8018070 con 0x7fb1c415da60 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1b80050d0 con 0x7fb1c415da60 2024-09-21T02:17:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c415c000 con 0x7fb1c415da60 2024-09-21T02:17:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800b6c0 con 0x7fb1c415da60 2024-09-21T02:17:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c41f8e20 con 0x7fb1c415da60 2024-09-21T02:17:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c40af850 con 0x7fb1c415da60 2024-09-21T02:17:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.496+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fb1b801b070 con 0x7fb1c415da60 2024-09-21T02:17:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.500+0000 7fb1b2ffd640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb1a4073590 0x7fb1a4075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:39.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.500+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fb1b8093f40 con 0x7fb1c415da60 2024-09-21T02:17:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.500+0000 7fb1c259c640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb1a4073590 0x7fb1a4075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.500+0000 7fb1c259c640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb1a4073590 0x7fb1a4075aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40ae000 tx=0x7fb1b401a050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:39.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.504+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb1b805e410 con 0x7fb1c415da60 2024-09-21T02:17:39.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.704+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c41f90e0 con 0x7fb1c415da60 2024-09-21T02:17:39.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.704+0000 7fb1b2ffd640 1 -- 172.21.15.170:0/536967674 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5237 (secure 0 0 0) 0x7fb1b8062e40 con 0x7fb1c415da60 2024-09-21T02:17:39.707 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:39.707 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":47,"btime":"2024-09-21T02:17:32:738667+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:17:39.708 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 47 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb1a4073590 msgr2=0x7fb1a4075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb1a4073590 0x7fb1a4075aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40ae000 tx=0x7fb1b401a050 comp rx=0 tx=0).stop 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 msgr2=0x7fb1c415eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 0x7fb1c415eb90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d7b0 tx=0x7fb1b800dc70 comp rx=0 tx=0).stop 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 shutdown_connections 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb1a4073590 0x7fb1a4075aa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c41f5600 0x7fb1c40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c415da60 0x7fb1c415eb90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 --2- 172.21.15.170:0/536967674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1ac004b50 0x7fb1c40ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 >> 172.21.15.170:0/536967674 conn(0x7fb1c40b1c40 msgr2=0x7fb1c40b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:39.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 shutdown_connections 2024-09-21T02:17:39.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:39.708+0000 7fb1c8a1d640 1 -- 172.21.15.170:0/536967674 wait complete. 2024-09-21T02:17:40.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:17:29.085709+0000', 'modified': '2024-09-21T02:17:32.143480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6270': {'gid': 6270, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/1298680671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1298680671}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1298680671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6270, 'qdb_cluster': [6270]} 2024-09-21T02:17:40.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:17:40.130 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:17:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bfda83640 1 -- 172.21.15.170:0/2216665932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5be4004da0 msgr2=0x7f5be4005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bfda83640 1 --2- 172.21.15.170:0/2216665932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5be4004da0 0x7f5be4005250 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8073f30 tx=0x7f5be801d250 comp rx=0 tx=0).stop 2024-09-21T02:17:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bf7fff640 1 -- 172.21.15.170:0/2216665932 reap_dead start 2024-09-21T02:17:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bfda83640 1 -- 172.21.15.170:0/2216665932 shutdown_connections 2024-09-21T02:17:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bfda83640 1 -- 172.21.15.170:0/2216665932 >> 172.21.15.170:0/2216665932 conn(0x7f5bf809a9c0 msgr2=0x7f5bf80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bfda83640 1 -- 172.21.15.170:0/2216665932 shutdown_connections 2024-09-21T02:17:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.288+0000 7f5bfda83640 1 -- 172.21.15.170:0/2216665932 wait complete. 2024-09-21T02:17:40.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 Processor -- start 2024-09-21T02:17:40.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 -- start start 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5be4004da0 0x7f5bf8155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 0x7f5bf8159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf8159f40 0x7f5bf80c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80b6cc0 con 0x7f5bf80c6200 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80b6b40 con 0x7f5be4004da0 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf8160aa0 con 0x7f5bf8159f40 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 0x7f5bf8159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 0x7f5bf8159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46542/0 (socket says 172.21.15.170:46542) 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/3847811549 learned_addr learned my addr 172.21.15.170:0/3847811549 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf8159f40 0x7f5bf80c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5be4004da0 0x7f5bf8155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf8159f40 msgr2=0x7f5bf80c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf8159f40 0x7f5bf80c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5be4004da0 msgr2=0x7f5bf8155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5be4004da0 0x7f5bf8155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/3847811549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80c5160 con 0x7f5bf80c6200 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 0x7f5bf8159a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5be000ca70 tx=0x7f5be000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5beffff640 1 -- 172.21.15.170:0/3847811549 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0017070 con 0x7f5bf80c6200 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5beffff640 1 -- 172.21.15.170:0/3847811549 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be0005070 con 0x7f5bf80c6200 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80c5440 con 0x7f5bf80c6200 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5beffff640 1 -- 172.21.15.170:0/3847811549 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00054c0 con 0x7f5bf80c6200 2024-09-21T02:17:40.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.292+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf80c5a00 con 0x7f5bf80c6200 2024-09-21T02:17:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5beffff640 1 -- 172.21.15.170:0/3847811549 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5be001a070 con 0x7f5bf80c6200 2024-09-21T02:17:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5beffff640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5bd8073590 0x7f5bd8075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5beffff640 1 -- 172.21.15.170:0/3847811549 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f5be0094050 con 0x7f5bf80c6200 2024-09-21T02:17:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5bd8073590 0x7f5bd8075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5c04d4e000 1 Processor -- start 2024-09-21T02:17:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5c04d4e000 1 -- start start 2024-09-21T02:17:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.296+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5bd8073590 0x7f5bd8075aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0000c10 tx=0x7f5bf0006b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac379c0 0x558bcad87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 0x558bcad88500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad88a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558bcab7ab40 con 0x558bcac26570 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558bcab69f70 con 0x558bcac379c0 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x558bcab6dd80 con 0x558bcac113c0 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 0x558bcad88500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 0x558bcad88500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46550/0 (socket says 172.21.15.170:46550) 2024-09-21T02:17:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/914416912 learned_addr learned my addr 172.21.15.170:0/914416912 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac379c0 0x558bcad87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad88a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 msgr2=0x558bcad88a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad88a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac379c0 msgr2=0x558bcad87fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac379c0 0x558bcad87fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 -- 172.21.15.170:0/914416912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bcab1d3e0 con 0x558bcac26570 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 0x558bcad88500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5be005e1e0 tx=0x7f5be005a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e487e8919b5b36f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bed7fa640 1 -- 172.21.15.170:0/914416912 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0083020 con 0x558bcac26570 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bed7fa640 1 -- 172.21.15.170:0/914416912 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be005e4e0 con 0x558bcac26570 2024-09-21T02:17:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5bed7fa640 1 -- 172.21.15.170:0/914416912 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0067c40 con 0x558bcac26570 2024-09-21T02:17:40.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- 172.21.15.170:0/914416912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 msgr2=0x558bcad88500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 0x558bcad88500 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5be005e1e0 tx=0x7f5be005a4b0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- 172.21.15.170:0/914416912 shutdown_connections 2024-09-21T02:17:40.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad88a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac26570 0x558bcad88500 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/914416912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac379c0 0x558bcad87fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- 172.21.15.170:0/914416912 >> 172.21.15.170:0/914416912 conn(0x558bca827460 msgr2=0x558bcad6b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- 172.21.15.170:0/914416912 shutdown_connections 2024-09-21T02:17:40.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- 172.21.15.170:0/914416912 wait complete. 2024-09-21T02:17:40.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 -- start start 2024-09-21T02:17:40.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.300+0000 7f5c04d4e000 1 Processor -- start 2024-09-21T02:17:40.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac26570 0x558bcad6aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558bcab10520 con 0x558bcac379c0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558bcab045f0 con 0x558bcac26570 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x558bcab35ee0 con 0x558bcac113c0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 10 client.0 _ll_register_callbacks cb 0x558bcab8b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad6b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46558/0 (socket says 172.21.15.170:46558) 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3115794642 learned_addr learned my addr 172.21.15.170:0/3115794642 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac26570 0x558bcad6aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 msgr2=0x558bcad6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac26570 msgr2=0x558bcad6aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac26570 0x558bcad6aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3115794642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bcab14460 con 0x558bcac379c0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad6b030 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5be801d080 tx=0x7f5be8005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e742b173fa3e95f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 1 -- 172.21.15.170:0/3115794642 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80b1040 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558bcab02110 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 1 -- 172.21.15.170:0/3115794642 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f5be80054f0 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 1 -- 172.21.15.170:0/3115794642 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5be8009730 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 10 client.6447 fetch_fsmap learned FSMap version 47 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x558bcab315b0 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 1 -- 172.21.15.170:0/3115794642 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558bcab02110 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 1 -- 172.21.15.170:0/3115794642 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f5be80b7040 con 0x558bcac379c0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 10 client.6447 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 10 client.6447 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 20 client.6447 populate_metadata read hostname 'smithi170' 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca80c7d0 0x558bcad99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 4 client.6447 mds_command: new command op to 6270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 --> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bcab7ca90 con 0x558bca80c7d0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca80c7d0 0x558bcad99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 10 client.6447 ms_handle_connect on v2:172.21.15.170:6834/1298680671 2024-09-21T02:17:40.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca80c7d0 0x558bcad99580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553fafccbb60164d server_cookie=19d4ae6accbb8814 in_seq=0 out_seq=0 2024-09-21T02:17:40.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 1 -- 172.21.15.170:0/3115794642 <== mds.0 v2:172.21.15.170:6834/1298680671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558bcab7ca90 con 0x558bca80c7d0 2024-09-21T02:17:40.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 10 client.6447 handle_command_reply: tid=0 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 client.6447 shutdown 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 msgr2=0x558bcad6b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad6b030 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5be801d080 tx=0x7f5be8005480 comp rx=0 tx=0).stop 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 shutdown_connections 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca80c7d0 0x558bcad99580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad6b030 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac26570 0x558bcad6aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5bd67fc640 0 client.6447 ms_handle_reset on v2:172.21.15.170:6834/1298680671 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3115794642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac113c0 0x558bcad6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 >> 172.21.15.170:0/3115794642 conn(0x558bca827460 msgr2=0x558bcad3c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 shutdown_connections 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3115794642 wait complete. 2024-09-21T02:17:40.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.304+0000 7f5c04d4e000 20 client.6447 trim_cache size 0 max 16384 2024-09-21T02:17:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 Processor -- start 2024-09-21T02:17:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 -- start start 2024-09-21T02:17:40.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcad9ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcad9f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80c8eb0 con 0x558bcac379c0 2024-09-21T02:17:40.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80c8240 con 0x558bcac113c0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf80c8860 con 0x558bcac26570 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad99580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46562/0 (socket says 172.21.15.170:46562) 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3802055896 learned_addr learned my addr 172.21.15.170:0/3802055896 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcad9ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcad9f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 msgr2=0x558bcad9f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcad9f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 msgr2=0x558bcad9ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcad9ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 -- 172.21.15.170:0/3802055896 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bcab08190 con 0x558bcac379c0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad99580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5be801d290 tx=0x7f5be8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ca90cd02a71522 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5becff9640 1 -- 172.21.15.170:0/3802055896 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80d0070 con 0x558bcac379c0 2024-09-21T02:17:40.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5becff9640 1 -- 172.21.15.170:0/3802055896 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be8000e50 con 0x558bcac379c0 2024-09-21T02:17:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.324+0000 7f5becff9640 1 -- 172.21.15.170:0/3802055896 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80b1040 con 0x558bcac379c0 2024-09-21T02:17:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3802055896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 msgr2=0x558bcad99580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad99580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5be801d290 tx=0x7f5be8004530 comp rx=0 tx=0).stop 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3802055896 shutdown_connections 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcad9f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcad99580 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/3802055896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcad9ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3802055896 >> 172.21.15.170:0/3802055896 conn(0x558bca827460 msgr2=0x558bcad9e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3802055896 shutdown_connections 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/3802055896 wait complete. 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- start start 2024-09-21T02:17:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 Processor -- start 2024-09-21T02:17:40.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcadfe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcadfe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcadfeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558bcab6d090 con 0x558bcac379c0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558bcab79910 con 0x558bcac113c0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x558bcab723b0 con 0x558bcac26570 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 10 client.0 _ll_register_callbacks cb 0x7f5be80af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcadfeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcadfeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46566/0 (socket says 172.21.15.170:46566) 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 -- 172.21.15.170:0/597396572 learned_addr learned my addr 172.21.15.170:0/597396572 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcadfe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcadfe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 -- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 msgr2=0x558bcadfe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcadfe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 -- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 msgr2=0x558bcadfe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcadfe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 -- 172.21.15.170:0/597396572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bcab30ab0 con 0x558bcac379c0 2024-09-21T02:17:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf6ffd640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcadfe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcadfe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bf7fff640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcadfeab0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005ec0 tx=0x7f5be8005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00e5564d9da720a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bd67fc640 1 -- 172.21.15.170:0/597396572 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80b1070 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bd67fc640 1 -- 172.21.15.170:0/597396572 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f5be8004790 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be8000e50 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bd67fc640 1 -- 172.21.15.170:0/597396572 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5be80db050 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 10 client.6453 fetch_fsmap learned FSMap version 47 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x558bcab0f070 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bd67fc640 1 -- 172.21.15.170:0/597396572 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be8000e50 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5bd67fc640 1 -- 172.21.15.170:0/597396572 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f5be8004aa0 con 0x558bcac379c0 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 10 client.6453 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 10 client.6453 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 20 client.6453 populate_metadata read hostname 'smithi170' 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.328+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca7fa830 0x558bcadb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 4 client.6453 mds_command: new command op to 6270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 --> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] -- command(tid 0: {"prefix": "status"}) -- 0x558bcab78540 con 0x558bca7fa830 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca7fa830 0x558bcadb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bf77fe640 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca7fa830 0x558bcadb9e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7e32793c512da3 server_cookie=1cd526e268838433 in_seq=0 out_seq=0 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bd67fc640 10 client.6453 ms_handle_connect on v2:172.21.15.170:6834/1298680671 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bd67fc640 1 -- 172.21.15.170:0/597396572 <== mds.0 v2:172.21.15.170:6834/1298680671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x558bcab78540 con 0x558bca7fa830 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bd67fc640 10 client.6453 handle_command_reply: tid=0 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 client.6453 shutdown 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 msgr2=0x558bcadfeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcadfeab0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5be8005ec0 tx=0x7f5be8005630 comp rx=0 tx=0).stop 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 shutdown_connections 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558bcac26570 0x558bcadfe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:6834/1298680671,v1:172.21.15.170:6835/1298680671] conn(0x558bca7fa830 0x558bcadb9e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558bcac379c0 0x558bcadfeab0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 --2- 172.21.15.170:0/597396572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558bcac113c0 0x558bcadfe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bd67fc640 0 client.6453 ms_handle_reset on v2:172.21.15.170:6834/1298680671 2024-09-21T02:17:40.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 >> 172.21.15.170:0/597396572 conn(0x558bca827460 msgr2=0x558bcad9f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 shutdown_connections 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 1 -- 172.21.15.170:0/597396572 wait complete. 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5c04d4e000 20 client.6453 trim_cache size 0 max 16384 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6270, 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.629558608, 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 47, 2024-09-21T02:17:40.338 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 72, 2024-09-21T02:17:40.339 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 72, 2024-09-21T02:17:40.339 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 15.163061753999999 2024-09-21T02:17:40.339 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:17:40.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5bd8073590 msgr2=0x7f5bd8075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5bd8073590 0x7f5bd8075aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0000c10 tx=0x7f5bf0006b40 comp rx=0 tx=0).stop 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 msgr2=0x7f5bf8159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 0x7f5bf8159a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5be000ca70 tx=0x7f5be000cf30 comp rx=0 tx=0).stop 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 shutdown_connections 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f5bd8073590 0x7f5bd8075aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bf8159f40 0x7f5bf80c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bf80c6200 0x7f5bf8159a00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 --2- 172.21.15.170:0/3847811549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5be4004da0 0x7f5bf8155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 >> 172.21.15.170:0/3847811549 conn(0x7f5bf809a9c0 msgr2=0x7f5bf80ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 shutdown_connections 2024-09-21T02:17:40.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.332+0000 7f5bfda83640 1 -- 172.21.15.170:0/3847811549 wait complete. 2024-09-21T02:17:40.352 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:17:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 -- 172.21.15.170:0/1431357355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01680b5410 msgr2=0x7f016806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1431357355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01680b5410 0x7f016806fa30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0158009840 tx=0x7f015801c820 comp rx=0 tx=0).stop 2024-09-21T02:17:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 -- 172.21.15.170:0/1431357355 shutdown_connections 2024-09-21T02:17:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1431357355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01680b5410 0x7f016806fa30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 -- 172.21.15.170:0/1431357355 >> 172.21.15.170:0/1431357355 conn(0x7f01680b1c40 msgr2=0x7f01680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 -- 172.21.15.170:0/1431357355 shutdown_connections 2024-09-21T02:17:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 -- 172.21.15.170:0/1431357355 wait complete. 2024-09-21T02:17:40.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 Processor -- start 2024-09-21T02:17:40.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.496+0000 7f016f4c5640 1 -- start start 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01680b2f40 0x7f0168148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01680b5410 0x7f01680c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 0x7f01680c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f016816b570 con 0x7f016809cb90 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0168070180 con 0x7f01680b5410 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f016816b6f0 con 0x7f01680b2f40 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 0x7f01680c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 0x7f01680c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46588/0 (socket says 172.21.15.170:46588) 2024-09-21T02:17:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 -- 172.21.15.170:0/1385986459 learned_addr learned my addr 172.21.15.170:0/1385986459 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016d261640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01680b2f40 0x7f0168148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016ca60640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01680b5410 0x7f01680c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 -- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01680b2f40 msgr2=0x7f0168148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01680b2f40 0x7f0168148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 -- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01680b5410 msgr2=0x7f01680c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01680b5410 0x7f01680c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 -- 172.21.15.170:0/1385986459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016809d3f0 con 0x7f016809cb90 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016da62640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 0x7f01680c92e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f016000b970 tx=0x7f016000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160019070 con 0x7f016809cb90 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f016000cc30 con 0x7f016809cb90 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016809d6d0 con 0x7f016809cb90 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600146a0 con 0x7f016809cb90 2024-09-21T02:17:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681f6fa0 con 0x7f016809cb90 2024-09-21T02:17:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.500+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016806fa30 con 0x7f016809cb90 2024-09-21T02:17:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.504+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f016001c070 con 0x7f016809cb90 2024-09-21T02:17:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.504+0000 7f015dffb640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f014c073590 0x7f014c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.504+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f01600945c0 con 0x7f016809cb90 2024-09-21T02:17:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.504+0000 7f016d261640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f014c073590 0x7f014c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:40.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.504+0000 7f016d261640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f014c073590 0x7f014c075aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01580af000 tx=0x7f015801b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.508+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f016005efc0 con 0x7f016809cb90 2024-09-21T02:17:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.704+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f01680b4530 con 0x7f016809cb90 2024-09-21T02:17:40.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.704+0000 7f015dffb640 1 -- 172.21.15.170:0/1385986459 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f016005e8c0 con 0x7f016809cb90 2024-09-21T02:17:40.708 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f014c073590 msgr2=0x7f014c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f014c073590 0x7f014c075aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01580af000 tx=0x7f015801b060 comp rx=0 tx=0).stop 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 msgr2=0x7f01680c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 0x7f01680c92e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f016000b970 tx=0x7f016000be30 comp rx=0 tx=0).stop 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 shutdown_connections 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f014c073590 0x7f014c075aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f016809cb90 0x7f01680c92e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01680b5410 0x7f01680c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 --2- 172.21.15.170:0/1385986459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01680b2f40 0x7f0168148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 >> 172.21.15.170:0/1385986459 conn(0x7f01680b1c40 msgr2=0x7f016809dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 shutdown_connections 2024-09-21T02:17:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40.708+0000 7f016f4c5640 1 -- 172.21.15.170:0/1385986459 wait complete. 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:17:40.729 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:17:40.730 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:17:40.730 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:17:40.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:40.730 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:40.736 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42255sec preferred_lft 42255sec 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:17:40.737 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:40.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:40.738 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:17:40.785 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:17:40.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:40.787 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:17:40.840 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:17:40.840 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:17:40.840 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:17:40.840 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:17:40.840 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:17:40.840 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:17:40.841 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:17:40.841 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:40.841 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:40.841 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:40.841 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:17:40.841 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:17:40.841 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:17:40.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:17:40.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:17:40.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:17:40.841 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:17:40.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:17:40.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:40.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:40.960 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:17:41.014 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:17:41.014 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:17:41.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:17:41.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:41.014 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:17:41.071 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:17:41.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:17:41.071 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:41.118 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:17:41.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:17:41.118 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:41.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:17:41.223 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:41.270 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:17:41.270 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:17:41.270 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:17:41.270 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:41.271 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:41.272 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:41.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:41.273 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:17:41.321 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:17:41.321 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:17:41.366 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:17:41.380 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:41.430 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:17:41.431 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:41.432 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:41.432 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:41.432 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:41.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:41.433 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:17:41.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:17:41.472+0000 7fd9c78c6580 -1 init, newargv = 0x7fd9bc0045f0 newargc=15 2024-09-21T02:17:41.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[41838]: starting ceph client 2024-09-21T02:17:41.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[41838]: starting fuse 2024-09-21T02:17:42.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:17:42.596 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:42.642 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:17:42.643 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:17:42.644 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:42.645 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:17:42.645 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:42.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:42.645 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:17:42.693 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:17:42.694 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:17:42.694 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:17:42.695 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:17:42.696 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:17:42.696 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:17:42.696 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:17:42.818 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.41838.asok 2024-09-21T02:17:42.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:42.830 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41838.asok 2024-09-21T02:17:42.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:42.830 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41838.asok status 2024-09-21T02:17:43.013 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "41838", 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:17:43.014 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6468, 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 6468 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 666530179 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 666530179 2024-09-21T02:17:43.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.6468 192.168.0.1:0/666530179", 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/666530179", 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 47, 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 72, 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:17:43.016 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:17:43.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.031 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:17:43.031 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:43.037 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:17:43.037 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:43.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:43.093 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 -- 172.21.15.170:0/877165129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 msgr2=0x7f19480ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 --2- 172.21.15.170:0/877165129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f19480ce5d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f19480d1160 tx=0x7f194001c800 comp rx=0 tx=0).stop 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 -- 172.21.15.170:0/877165129 shutdown_connections 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 --2- 172.21.15.170:0/877165129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f19480ce5d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 --2- 172.21.15.170:0/877165129 >> v2:172.21.15.191:3300/0 conn(0x7f19480cf7e0 0x7f1948070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 -- 172.21.15.170:0/877165129 >> 172.21.15.170:0/877165129 conn(0x7f19480b1c40 msgr2=0x7f19480b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 -- 172.21.15.170:0/877165129 shutdown_connections 2024-09-21T02:17:43.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.292+0000 7f194e899640 1 -- 172.21.15.170:0/877165129 wait complete. 2024-09-21T02:17:43.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 Processor -- start 2024-09-21T02:17:43.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 -- start start 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19480cb460 0x7f19480b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f1948153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19480cf7e0 0x7f194809d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19481690d0 con 0x7f19480ce1c0 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1948168f50 con 0x7f19480cb460 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1948169250 con 0x7f19480cf7e0 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f1948153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f1948153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46606/0 (socket says 172.21.15.170:46606) 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 -- 172.21.15.170:0/1250293020 learned_addr learned my addr 172.21.15.170:0/1250293020 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194ce36640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19480cf7e0 0x7f194809d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f1947fff640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19480cb460 0x7f19480b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 -- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19480cf7e0 msgr2=0x7f194809d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19480cf7e0 0x7f194809d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 -- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19480cb460 msgr2=0x7f19480b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19480cb460 0x7f19480b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 -- 172.21.15.170:0/1250293020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19480b4ad0 con 0x7f19480ce1c0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f1947fff640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19480cb460 0x7f19480b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f19477fe640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f1948153260 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f193000c490 tx=0x7f193000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930016020 con 0x7f19480ce1c0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1930012940 con 0x7f19480ce1c0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19480b4db0 con 0x7f19480ce1c0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930012d90 con 0x7f19480ce1c0 2024-09-21T02:17:43.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.296+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19480b3220 con 0x7f19480ce1c0 2024-09-21T02:17:43.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.300+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19480ce5d0 con 0x7f19480ce1c0 2024-09-21T02:17:43.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.300+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f1930019020 con 0x7f19480ce1c0 2024-09-21T02:17:43.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.300+0000 7f1944ff9640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1928073590 0x7f1928075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:43.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.300+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f1930094c80 con 0x7f19480ce1c0 2024-09-21T02:17:43.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.300+0000 7f1947fff640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1928073590 0x7f1928075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:43.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.300+0000 7f1947fff640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1928073590 0x7f1928075aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1938009ea0 tx=0x7f1938009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:43.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.304+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1930020020 con 0x7f19480ce1c0 2024-09-21T02:17:43.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.516+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19480ae5e0 con 0x7f19480ce1c0 2024-09-21T02:17:43.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.516+0000 7f1944ff9640 1 -- 172.21.15.170:0/1250293020 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f193005f940 con 0x7f19480ce1c0 2024-09-21T02:17:43.520 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:17:43.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1928073590 msgr2=0x7f1928075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1928073590 0x7f1928075aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1938009ea0 tx=0x7f1938009500 comp rx=0 tx=0).stop 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 msgr2=0x7f1948153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f1948153260 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f193000c490 tx=0x7f193000c950 comp rx=0 tx=0).stop 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 shutdown_connections 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f1928073590 0x7f1928075aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19480cf7e0 0x7f194809d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19480ce1c0 0x7f1948153260 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 --2- 172.21.15.170:0/1250293020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19480cb460 0x7f19480b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 >> 172.21.15.170:0/1250293020 conn(0x7f19480b1c40 msgr2=0x7f19480aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 shutdown_connections 2024-09-21T02:17:43.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:43.520+0000 7f194e899640 1 -- 172.21.15.170:0/1250293020 wait complete. 2024-09-21T02:17:43.541 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:17:43.542 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:17:43.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.542 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:17:43.549 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:17:43.550 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42246sec preferred_lft 42246sec 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:17:43.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:17:43.552 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:17:43.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.553 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:17:43.603 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:17:43.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.605 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:17:43.656 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:17:43.656 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:17:43.656 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:17:43.656 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:17:43.656 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:17:43.657 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:17:43.657 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:17:43.657 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:43.657 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:43.657 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:17:43.657 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:17:43.657 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:17:43.657 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:17:43.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:17:43.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:17:43.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:17:43.657 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:17:43.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:17:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:17:43.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:17:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:43.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.782 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:17:43.788 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:17:43.788 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:17:43.788 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:17:43.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:43.788 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:17:43.843 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:17:43.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:17:43.843 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:43.892 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:17:43.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:17:43.893 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:43.940 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:17:43.998 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:17:44.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:44.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:44.047 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:44.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:44.048 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:17:44.096 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:17:44.096 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:17:44.140 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:17:44.155 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:17:44.204 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:17:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:44.206 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:44.207 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:44.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:44.207 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:17:44.252 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:17:44.250+0000 7f17ea2cf580 -1 init, newargv = 0x7f17cc001280 newargc=15 2024-09-21T02:17:44.252 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34795]: starting ceph client 2024-09-21T02:17:44.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34795]: starting fuse 2024-09-21T02:17:45.357 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:17:45.373 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:17:45.421 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:17:45.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:17:45.423 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:45.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:45.424 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:17:45.472 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:17:45.472 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:17:45.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:17:45.473 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:17:45.474 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:17:45.596 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.34795.asok 2024-09-21T02:17:45.600 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:17:45.609 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34795.asok 2024-09-21T02:17:45.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:45.609 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34795.asok status 2024-09-21T02:17:45.792 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:17:45.792 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:17:45.792 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "34795", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5270, 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5270 2024-09-21T02:17:45.793 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1826585810 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1826585810 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5270 192.168.0.1:0/1826585810", 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/1826585810", 2024-09-21T02:17:45.794 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:17:45.795 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 47, 2024-09-21T02:17:45.795 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 72, 2024-09-21T02:17:45.795 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:17:45.795 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:17:45.795 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:17:45.795 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:17:45.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:45.811 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:17:45.811 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:45.817 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:17:45.817 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:45.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:45.818 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:45.870 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:17:46.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/43030152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80ca7b0 msgr2=0x7fd7c80cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:46.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/43030152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80ca7b0 0x7fd7c80cabc0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0098f0 tx=0x7fd7bc01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:17:46.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/43030152 shutdown_connections 2024-09-21T02:17:46.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/43030152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c806ffc0 0x7fd7c80703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:46.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/43030152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80ca7b0 0x7fd7c80cabc0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:46.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/43030152 >> 172.21.15.170:0/43030152 conn(0x7fd7c80b1c40 msgr2=0x7fd7c80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:46.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/43030152 shutdown_connections 2024-09-21T02:17:46.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/43030152 wait complete. 2024-09-21T02:17:46.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 Processor -- start 2024-09-21T02:17:46.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.040+0000 7fd7cd0cf640 1 -- start start 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c806ffc0 0x7fd7c80c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80ca7b0 0x7fd7c8155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 0x7fd7c815c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8169210 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8169090 con 0x7fd7c806ffc0 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c8169390 con 0x7fd7c80ca7b0 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 0x7fd7c815c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 0x7fd7c815c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41734/0 (socket says 172.21.15.170:41734) 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 -- 172.21.15.170:0/140903605 learned_addr learned my addr 172.21.15.170:0/140903605 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c6d9d640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c806ffc0 0x7fd7c80c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c659c640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80ca7b0 0x7fd7c8155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:46.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 -- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80ca7b0 msgr2=0x7fd7c8155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80ca7b0 0x7fd7c8155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 -- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c806ffc0 msgr2=0x7fd7c80c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c806ffc0 0x7fd7c80c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 -- 172.21.15.170:0/140903605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c809db70 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c6d9d640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c806ffc0 0x7fd7c80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7c759e640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 0x7fd7c815c900 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c760 tx=0x7fd7c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0016020 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c000ce10 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c809de50 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0003be0 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c815c3e0 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c80703d0 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.044+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd7c000a850 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.048+0000 7fd7af7fe640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7b4073560 0x7fd7b4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:46.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.048+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fd7c00942d0 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.048+0000 7fd7c6d9d640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7b4073560 0x7fd7b4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:46.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.048+0000 7fd7c6d9d640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7b4073560 0x7fd7b4075a70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0af000 tx=0x7fd7bc01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:46.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.052+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd7c0020020 con 0x7fd7c80cc7e0 2024-09-21T02:17:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:46.256+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd7c809d900 con 0x7fd7c80cc7e0 2024-09-21T02:17:47.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.044+0000 7fd7af7fe640 1 -- 172.21.15.170:0/140903605 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) ==== 136+0+0 (secure 0 0 0) 0x7fd7c8070910 con 0x7fd7c80cc7e0 2024-09-21T02:17:47.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7b4073560 msgr2=0x7fd7b4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7b4073560 0x7fd7b4075a70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0af000 tx=0x7fd7bc01b120 comp rx=0 tx=0).stop 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 msgr2=0x7fd7c815c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 0x7fd7c815c900 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c760 tx=0x7fd7c000cc20 comp rx=0 tx=0).stop 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 shutdown_connections 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fd7b4073560 0x7fd7b4075a70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80ca7b0 0x7fd7c8155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c80cc7e0 0x7fd7c815c900 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 --2- 172.21.15.170:0/140903605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c806ffc0 0x7fd7c80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 >> 172.21.15.170:0/140903605 conn(0x7fd7c80b1c40 msgr2=0x7fd7c80ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.048+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 shutdown_connections 2024-09-21T02:17:47.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.052+0000 7fd7cd0cf640 1 -- 172.21.15.170:0/140903605 wait complete. 2024-09-21T02:17:47.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 -- 172.21.15.170:0/2390241300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c0ccf80 msgr2=0x7fe52c0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2390241300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c0ccf80 0x7fe52c0cec70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0cf870 tx=0x7fe52001c870 comp rx=0 tx=0).stop 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 -- 172.21.15.170:0/2390241300 shutdown_connections 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2390241300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe52c160f60 0x7fe52c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2390241300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c0ccf80 0x7fe52c0cec70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2390241300 >> v2:172.21.15.191:3300/0 conn(0x7fe52c160b50 0x7fe52c0cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 -- 172.21.15.170:0/2390241300 >> 172.21.15.170:0/2390241300 conn(0x7fe52c0711c0 msgr2=0x7fe52c0715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 -- 172.21.15.170:0/2390241300 shutdown_connections 2024-09-21T02:17:47.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.236+0000 7fe531b8f640 1 -- 172.21.15.170:0/2390241300 wait complete. 2024-09-21T02:17:47.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 Processor -- start 2024-09-21T02:17:47.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 -- start start 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe52c160b50 0x7fe52c165ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 0x7fe52c1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe52c15b7f0 0x7fe52c15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c0b4310 con 0x7fe52c160f60 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c0b4190 con 0x7fe52c160b50 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe52c0b4490 con 0x7fe52c15b7f0 2024-09-21T02:17:47.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 0x7fe52c1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 0x7fe52c1531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41746/0 (socket says 172.21.15.170:41746) 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 -- 172.21.15.170:0/2312288302 learned_addr learned my addr 172.21.15.170:0/2312288302 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52bfff640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe52c15b7f0 0x7fe52c15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52b7fe640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe52c160b50 0x7fe52c165ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 -- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe52c15b7f0 msgr2=0x7fe52c15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe52c15b7f0 0x7fe52c15bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 -- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe52c160b50 msgr2=0x7fe52c165ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe52c160b50 0x7fe52c165ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 -- 172.21.15.170:0/2312288302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c15c140 con 0x7fe52c160f60 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe52affd640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 0x7fe52c1531c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe5200b0040 tx=0x7fe5200a5850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200b2040 con 0x7fe52c160f60 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5200042b0 con 0x7fe52c160f60 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c1f8c60 con 0x7fe52c160f60 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520003600 con 0x7fe52c160f60 2024-09-21T02:17:47.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.240+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c155380 con 0x7fe52c160f60 2024-09-21T02:17:47.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.244+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c1550f0 con 0x7fe52c160f60 2024-09-21T02:17:47.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.244+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe5200bf020 con 0x7fe52c160f60 2024-09-21T02:17:47.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.244+0000 7fe513fff640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe518073550 0x7fe518075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.244+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fe52012efc0 con 0x7fe52c160f60 2024-09-21T02:17:47.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.244+0000 7fe52b7fe640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe518073550 0x7fe518075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.244+0000 7fe52b7fe640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe518073550 0x7fe518075a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005fd0 tx=0x7fe51c005dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:47.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.248+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe52c15c140 con 0x7fe52c160f60 2024-09-21T02:17:47.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.440+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe52c149080 con 0x7fe518073550 2024-09-21T02:17:47.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe513fff640 1 -- 172.21.15.170:0/2312288302 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe51c00e9a0 con 0x7fe518073550 2024-09-21T02:17:47.445 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:17:47.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:47.445 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:17:47.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:47.445 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe518073550 msgr2=0x7fe518075a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe518073550 0x7fe518075a60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005fd0 tx=0x7fe51c005dc0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 msgr2=0x7fe52c1531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 0x7fe52c1531c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe5200b0040 tx=0x7fe5200a5850 comp rx=0 tx=0).stop 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 shutdown_connections 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fe518073550 0x7fe518075a60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe52c15b7f0 0x7fe52c15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe52c160f60 0x7fe52c1531c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 --2- 172.21.15.170:0/2312288302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe52c160b50 0x7fe52c165ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 >> 172.21.15.170:0/2312288302 conn(0x7fe52c0711c0 msgr2=0x7fe52c0b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 shutdown_connections 2024-09-21T02:17:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.444+0000 7fe531b8f640 1 -- 172.21.15.170:0/2312288302 wait complete. 2024-09-21T02:17:47.465 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 -- 172.21.15.170:0/2699186045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 msgr2=0x7f6dc0073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/2699186045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc0073b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00b5b60 tx=0x7f6db001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 -- 172.21.15.170:0/2699186045 shutdown_connections 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/2699186045 >> v2:172.21.15.191:3300/0 conn(0x7f6dc00740a0 0x7f6dc00744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/2699186045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc0073b60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 -- 172.21.15.170:0/2699186045 >> 172.21.15.170:0/2699186045 conn(0x7f6dc00d0170 msgr2=0x7f6dc00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 -- 172.21.15.170:0/2699186045 shutdown_connections 2024-09-21T02:17:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.612+0000 7f6dc50c3640 1 -- 172.21.15.170:0/2699186045 wait complete. 2024-09-21T02:17:47.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 Processor -- start 2024-09-21T02:17:47.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- start start 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc00740a0 0x7f6dc015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc00ce070 0x7f6dc00b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc00b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc0168ec0 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc00b5cd0 con 0x7f6dc00ce070 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc0169040 con 0x7f6dc00740a0 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc00b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc00b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41758/0 (socket says 172.21.15.170:41758) 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 -- 172.21.15.170:0/3619265456 learned_addr learned my addr 172.21.15.170:0/3619265456 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbed9d640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc00740a0 0x7f6dc015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 -- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc00740a0 msgr2=0x7f6dc015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc00740a0 0x7f6dc015aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 -- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc00ce070 msgr2=0x7f6dc00b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6db7fff640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc00ce070 0x7f6dc00b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc00ce070 0x7f6dc00b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 -- 172.21.15.170:0/3619265456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc0151e40 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dbf59e640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc00b0ad0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c760 tx=0x7f6db800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8003b80 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6db8003d20 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc00b1010 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800aa50 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc0152590 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.616+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc0159420 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.620+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6db800abf0 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.620+0000 7f6db77fe640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6da0073590 0x7f6da0075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:47.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.620+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f6db80950d0 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.620+0000 7f6dbed9d640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6da0073590 0x7f6da0075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:47.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.620+0000 7f6dbed9d640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6da0073590 0x7f6da0075aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6db00af000 tx=0x7f6db001b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:47.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.624+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6db805fbf0 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.792+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6dc0152010 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6db77fe640 1 -- 172.21.15.170:0/3619265456 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f6db805f4f0 con 0x7f6dc00cf7d0 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6da0073590 msgr2=0x7f6da0075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6da0073590 0x7f6da0075aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6db00af000 tx=0x7f6db001b100 comp rx=0 tx=0).stop 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 msgr2=0x7f6dc00b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc00b0ad0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c760 tx=0x7f6db800cc20 comp rx=0 tx=0).stop 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 shutdown_connections 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f6da0073590 0x7f6da0075aa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dc00cf7d0 0x7f6dc00b0ad0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dc00ce070 0x7f6dc00b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 --2- 172.21.15.170:0/3619265456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dc00740a0 0x7f6dc015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 >> 172.21.15.170:0/3619265456 conn(0x7f6dc00d0170 msgr2=0x7f6dc0075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 shutdown_connections 2024-09-21T02:17:47.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:47.796+0000 7f6dc50c3640 1 -- 172.21.15.170:0/3619265456 wait complete. 2024-09-21T02:17:47.818 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:47.818 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:47.823 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 -- 172.21.15.170:0/720298428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 msgr2=0x7f718c0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 --2- 172.21.15.170:0/720298428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c0ce8d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f718c070ca0 tx=0x7f718401cd50 comp rx=0 tx=0).stop 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 -- 172.21.15.170:0/720298428 shutdown_connections 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 --2- 172.21.15.170:0/720298428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c0ce8d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 --2- 172.21.15.170:0/720298428 >> v2:172.21.15.191:3300/0 conn(0x7f718c070030 0x7f718c070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 --2- 172.21.15.170:0/720298428 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f718c15f990 0x7f718c0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 -- 172.21.15.170:0/720298428 >> 172.21.15.170:0/720298428 conn(0x7f718c0b1c40 msgr2=0x7f718c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 -- 172.21.15.170:0/720298428 shutdown_connections 2024-09-21T02:17:48.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.008+0000 7f7192f16640 1 -- 172.21.15.170:0/720298428 wait complete. 2024-09-21T02:17:48.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 Processor -- start 2024-09-21T02:17:48.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- start start 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f718c070030 0x7f718c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f718c15f990 0x7f718c0b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f718c168170 con 0x7f718c0ce4c0 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f718c167ff0 con 0x7f718c070030 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f718c1682f0 con 0x7f718c15f990 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41770/0 (socket says 172.21.15.170:41770) 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 -- 172.21.15.170:0/401668430 learned_addr learned my addr 172.21.15.170:0/401668430 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f71914b3640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f718c15f990 0x7f718c0b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7190cb2640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f718c070030 0x7f718c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 -- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f718c15f990 msgr2=0x7f718c0b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f718c15f990 0x7f718c0b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 -- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f718c070030 msgr2=0x7f718c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f718c070030 0x7f718c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 -- 172.21.15.170:0/401668430 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c152f40 con 0x7f718c0ce4c0 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7190cb2640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f718c070030 0x7f718c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f718bfff640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c152410 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f718000c970 tx=0x7f718000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180007bb0 con 0x7f718c0ce4c0 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7180007e40 con 0x7f718c0ce4c0 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c1651b0 con 0x7f718c0ce4c0 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718000f6b0 con 0x7f718c0ce4c0 2024-09-21T02:17:48.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c165770 con 0x7f718c0ce4c0 2024-09-21T02:17:48.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.012+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c164fa0 con 0x7f718c0ce4c0 2024-09-21T02:17:48.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.016+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f7180018400 con 0x7f718c0ce4c0 2024-09-21T02:17:48.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.016+0000 7f71897fa640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7178073590 0x7f7178075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.016+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f7180095070 con 0x7f718c0ce4c0 2024-09-21T02:17:48.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.016+0000 7f7190cb2640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7178073590 0x7f7178075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.016+0000 7f7190cb2640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7178073590 0x7f7178075aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f717c0058d0 tx=0x7f717c005ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.020+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7180014030 con 0x7f718c0ce4c0 2024-09-21T02:17:48.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f718c0ce8d0 con 0x7f718c0ce4c0 2024-09-21T02:17:48.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f71897fa640 1 -- 172.21.15.170:0/401668430 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7f718005fa70 con 0x7f718c0ce4c0 2024-09-21T02:17:48.201 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:17:48.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7178073590 msgr2=0x7f7178075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7178073590 0x7f7178075aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f717c0058d0 tx=0x7f717c005ca0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 msgr2=0x7f718c152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c152410 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f718000c970 tx=0x7f718000ce30 comp rx=0 tx=0).stop 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 shutdown_connections 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7178073590 0x7f7178075aa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f718c15f990 0x7f718c0b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f718c0ce4c0 0x7f718c152410 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 --2- 172.21.15.170:0/401668430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f718c070030 0x7f718c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 >> 172.21.15.170:0/401668430 conn(0x7f718c0b1c40 msgr2=0x7f718c15b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 shutdown_connections 2024-09-21T02:17:48.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.200+0000 7f7192f16640 1 -- 172.21.15.170:0/401668430 wait complete. 2024-09-21T02:17:48.222 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:48.222 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:48.227 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:17:48.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 -- 172.21.15.170:0/1397698971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d3630 msgr2=0x7f79380d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 --2- 172.21.15.170:0/1397698971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d3630 0x7f79380d0d80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f793808c690 tx=0x7f792801c7e0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 -- 172.21.15.170:0/1397698971 shutdown_connections 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 --2- 172.21.15.170:0/1397698971 >> v2:172.21.15.191:3300/0 conn(0x7f79380b6330 0x7f79380b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 --2- 172.21.15.170:0/1397698971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79380d12c0 0x7f79380b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 --2- 172.21.15.170:0/1397698971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d3630 0x7f79380d0d80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 -- 172.21.15.170:0/1397698971 >> 172.21.15.170:0/1397698971 conn(0x7f793809d830 msgr2=0x7f793808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 -- 172.21.15.170:0/1397698971 shutdown_connections 2024-09-21T02:17:48.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.404+0000 7f793e4b3640 1 -- 172.21.15.170:0/1397698971 wait complete. 2024-09-21T02:17:48.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 Processor -- start 2024-09-21T02:17:48.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- start start 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79380b6330 0x7f7938164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 0x7f793815cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79380d3630 0x7f79381f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cf6d0 con 0x7f79380d12c0 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cf550 con 0x7f79380b6330 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f79380cf850 con 0x7f79380d3630 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 0x7f793815cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 0x7f793815cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41800/0 (socket says 172.21.15.170:41800) 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 -- 172.21.15.170:0/3775422676 learned_addr learned my addr 172.21.15.170:0/3775422676 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f7937fff640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79380b6330 0x7f7938164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793ca50640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79380d3630 0x7f79381f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 -- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79380d3630 msgr2=0x7f79381f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79380d3630 0x7f79381f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 -- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79380b6330 msgr2=0x7f7938164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79380b6330 0x7f7938164ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 -- 172.21.15.170:0/3775422676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7938167530 con 0x7f79380d12c0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f7937fff640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79380b6330 0x7f7938164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f792f7fe640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 0x7f793815cbf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f793809de10 tx=0x7f792000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920004230 con 0x7f79380d12c0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79200044c0 con 0x7f79380d12c0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381677d0 con 0x7f79380d12c0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920007bc0 con 0x7f79380d12c0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79380b1500 con 0x7f79380d12c0 2024-09-21T02:17:48.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.408+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79380b5df0 con 0x7f79380d12c0 2024-09-21T02:17:48.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.412+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f7920005020 con 0x7f79380d12c0 2024-09-21T02:17:48.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.412+0000 7f79357fa640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7918073590 0x7f7918075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:48.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.412+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f7920094f10 con 0x7f79380d12c0 2024-09-21T02:17:48.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.412+0000 7f7937fff640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7918073590 0x7f7918075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.412+0000 7f7937fff640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7918073590 0x7f7918075aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f79280af000 tx=0x7f792801b020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:48.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.416+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7920014070 con 0x7f79380d12c0 2024-09-21T02:17:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.584+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7938167cc0 con 0x7f79380d12c0 2024-09-21T02:17:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.584+0000 7f79357fa640 1 -- 172.21.15.170:0/3775422676 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7f792005fb30 con 0x7f79380d12c0 2024-09-21T02:17:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:17:48.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7918073590 msgr2=0x7f7918075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7918073590 0x7f7918075aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f79280af000 tx=0x7f792801b020 comp rx=0 tx=0).stop 2024-09-21T02:17:48.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 msgr2=0x7f793815cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:48.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 0x7f793815cbf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f793809de10 tx=0x7f792000cdd0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 shutdown_connections 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f7918073590 0x7f7918075aa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79380d3630 0x7f79381f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79380d12c0 0x7f793815cbf0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 --2- 172.21.15.170:0/3775422676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79380b6330 0x7f7938164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 >> 172.21.15.170:0/3775422676 conn(0x7f793809d830 msgr2=0x7f7938075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 shutdown_connections 2024-09-21T02:17:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:48.588+0000 7f793e4b3640 1 -- 172.21.15.170:0/3775422676 wait complete. 2024-09-21T02:17:48.607 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:17:48.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:48.608 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:17:48.618 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[41838]: fuse finished with error 0 and tester_r 0 2024-09-21T02:17:48.619 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:17:54.295 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:17:54.298 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:17:54.721 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:17:54.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:54.721 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:17:54.728 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:54.728 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:54.735 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:17:54.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.936+0000 7f08f47db640 1 -- 172.21.15.170:0/434696285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d8004d50 msgr2=0x7f08d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:54.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.936+0000 7f08f47db640 1 --2- 172.21.15.170:0/434696285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d8004d50 0x7f08d80051c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0b9720 tx=0x7f08e401fa40 comp rx=0 tx=0).stop 2024-09-21T02:17:54.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.936+0000 7f08f2d78640 1 -- 172.21.15.170:0/434696285 reap_dead start 2024-09-21T02:17:54.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.936+0000 7f08f47db640 1 -- 172.21.15.170:0/434696285 shutdown_connections 2024-09-21T02:17:54.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.936+0000 7f08f47db640 1 -- 172.21.15.170:0/434696285 >> 172.21.15.170:0/434696285 conn(0x7f08ec09d830 msgr2=0x7f08ec08bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:54.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 -- 172.21.15.170:0/434696285 shutdown_connections 2024-09-21T02:17:54.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 -- 172.21.15.170:0/434696285 wait complete. 2024-09-21T02:17:54.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 Processor -- start 2024-09-21T02:17:54.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 -- start start 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08d8004d50 0x7f08ec0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 0x7f08ec1e5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec1f4ce0 0x7f08ec1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec0b46a0 con 0x7f08ec1d7230 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec0b4520 con 0x7f08ec1f4ce0 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f47db640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f08ec0b5df0 con 0x7f08d8004d50 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 0x7f08ec1e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 0x7f08ec1e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41826/0 (socket says 172.21.15.170:41826) 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 -- 172.21.15.170:0/1993628835 learned_addr learned my addr 172.21.15.170:0/1993628835 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f2577640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08d8004d50 0x7f08ec0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f2d78640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec1f4ce0 0x7f08ec1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 -- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08d8004d50 msgr2=0x7f08ec0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08d8004d50 0x7f08ec0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 -- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec1f4ce0 msgr2=0x7f08ec1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec1f4ce0 0x7f08ec1f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 -- 172.21.15.170:0/1993628835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec1e8440 con 0x7f08ec1d7230 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f2577640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08d8004d50 0x7f08ec0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.940+0000 7f08f1d76640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 0x7f08ec1e5350 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f08e800f7a0 tx=0x7f08e800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e800bc50 con 0x7f08ec1d7230 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08e8005020 con 0x7f08ec1d7230 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ec1e86b0 con 0x7f08ec1d7230 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8005470 con 0x7f08ec1d7230 2024-09-21T02:17:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ec1e8c70 con 0x7f08ec1d7230 2024-09-21T02:17:54.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ec0d0d80 con 0x7f08ec1d7230 2024-09-21T02:17:54.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f08e8019030 con 0x7f08ec1d7230 2024-09-21T02:17:54.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08e2ffd640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08d0073560 0x7f08d0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:54.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f08e8094d70 con 0x7f08ec1d7230 2024-09-21T02:17:54.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08f2577640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08d0073560 0x7f08d0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:54.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.944+0000 7f08f2577640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08d0073560 0x7f08d0075a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08dc005fd0 tx=0x7f08dc00b530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:54.948+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f08e8099070 con 0x7f08ec1d7230 2024-09-21T02:17:55.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f08ec0ce5b0 con 0x7f08ec1d7230 2024-09-21T02:17:55.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08e2ffd640 1 -- 172.21.15.170:0/1993628835 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7f08e805f2b0 con 0x7f08ec1d7230 2024-09-21T02:17:55.130 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:17:55.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08d0073560 msgr2=0x7f08d0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08d0073560 0x7f08d0075a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08dc005fd0 tx=0x7f08dc00b530 comp rx=0 tx=0).stop 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 msgr2=0x7f08ec1e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 0x7f08ec1e5350 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f08e800f7a0 tx=0x7f08e800fc60 comp rx=0 tx=0).stop 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 shutdown_connections 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f08d0073560 0x7f08d0075a70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec1f4ce0 0x7f08ec1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec1d7230 0x7f08ec1e5350 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 --2- 172.21.15.170:0/1993628835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08d8004d50 0x7f08ec0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 >> 172.21.15.170:0/1993628835 conn(0x7f08ec09d830 msgr2=0x7f08ec1e6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.128+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 shutdown_connections 2024-09-21T02:17:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.132+0000 7f08f47db640 1 -- 172.21.15.170:0/1993628835 wait complete. 2024-09-21T02:17:55.152 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:55.152 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:55.158 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 -- 172.21.15.170:0/1873749571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb358004d50 msgr2=0x7fb3580051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 --2- 172.21.15.170:0/1873749571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb358004d50 0x7fb3580051a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb3700c7170 tx=0x7fb36801d150 comp rx=0 tx=0).stop 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb36f7fe640 1 -- 172.21.15.170:0/1873749571 reap_dead start 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 -- 172.21.15.170:0/1873749571 shutdown_connections 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 -- 172.21.15.170:0/1873749571 >> 172.21.15.170:0/1873749571 conn(0x7fb3700b1c40 msgr2=0x7fb3700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 -- 172.21.15.170:0/1873749571 shutdown_connections 2024-09-21T02:17:55.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 -- 172.21.15.170:0/1873749571 wait complete. 2024-09-21T02:17:55.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 Processor -- start 2024-09-21T02:17:55.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.308+0000 7fb3757bb640 1 -- start start 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358004d50 0x7fb3700c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 0x7fb37015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3701f48c0 0x7fb37015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700c3980 con 0x7fb3700c9d40 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700c3800 con 0x7fb358004d50 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb3700cf760 con 0x7fb3701f48c0 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 0x7fb37015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 0x7fb37015a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41838/0 (socket says 172.21.15.170:41838) 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 -- 172.21.15.170:0/3762836032 learned_addr learned my addr 172.21.15.170:0/3762836032 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36effd640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358004d50 0x7fb3700c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36f7fe640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3701f48c0 0x7fb37015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 -- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3701f48c0 msgr2=0x7fb37015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3701f48c0 0x7fb37015a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 -- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358004d50 msgr2=0x7fb3700c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358004d50 0x7fb3700c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 -- 172.21.15.170:0/3762836032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37009df50 con 0x7fb3700c9d40 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36effd640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358004d50 0x7fb3700c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36e7fc640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 0x7fb37015a170 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb36000d720 tx=0x7fb36000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360017070 con 0x7fb3700c9d40 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb360009cc0 con 0x7fb3700c9d40 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3701f3de0 con 0x7fb3700c9d40 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360005280 con 0x7fb3700c9d40 2024-09-21T02:17:55.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3701f43a0 con 0x7fb3700c9d40 2024-09-21T02:17:55.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3701f3bd0 con 0x7fb3700c9d40 2024-09-21T02:17:55.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fb360005420 con 0x7fb3700c9d40 2024-09-21T02:17:55.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3577fe640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb35c073560 0x7fb35c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7fb360094e80 con 0x7fb3700c9d40 2024-09-21T02:17:55.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.312+0000 7fb36effd640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb35c073560 0x7fb35c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.316+0000 7fb36effd640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb35c073560 0x7fb35c075a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb370070c40 tx=0x7fb364001160 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:55.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.316+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb36005f980 con 0x7fb3700c9d40 2024-09-21T02:17:55.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.512+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb37009dce0 con 0x7fb3700c9d40 2024-09-21T02:17:55.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.512+0000 7fb3577fe640 1 -- 172.21.15.170:0/3762836032 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) ==== 80+0+0 (secure 0 0 0) 0x7fb36005f280 con 0x7fb3700c9d40 2024-09-21T02:17:55.515 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:17:55.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb35c073560 msgr2=0x7fb35c075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb35c073560 0x7fb35c075a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb370070c40 tx=0x7fb364001160 comp rx=0 tx=0).stop 2024-09-21T02:17:55.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 msgr2=0x7fb37015a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 0x7fb37015a170 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb36000d720 tx=0x7fb36000dbe0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 shutdown_connections 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7fb35c073560 0x7fb35c075a70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3701f48c0 0x7fb37015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3700c9d40 0x7fb37015a170 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 --2- 172.21.15.170:0/3762836032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358004d50 0x7fb3700c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 >> 172.21.15.170:0/3762836032 conn(0x7fb3700b1c40 msgr2=0x7fb3700c8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 shutdown_connections 2024-09-21T02:17:55.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.516+0000 7fb3757bb640 1 -- 172.21.15.170:0/3762836032 wait complete. 2024-09-21T02:17:55.537 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:17:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:55.537 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:17:55.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34795]: fuse finished with error 0 and tester_r 0 2024-09-21T02:17:55.551 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:17:55.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:17:55.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:17:55.632 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:17:55.638 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:55.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 -- 172.21.15.170:0/879979911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 msgr2=0x7f67e80ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 --2- 172.21.15.170:0/879979911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e80ce4c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f67e80b2d00 tx=0x7f67d801c800 comp rx=0 tx=0).stop 2024-09-21T02:17:55.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 -- 172.21.15.170:0/879979911 shutdown_connections 2024-09-21T02:17:55.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 --2- 172.21.15.170:0/879979911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e80ce4c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 --2- 172.21.15.170:0/879979911 >> v2:172.21.15.191:3300/0 conn(0x7f67e815f990 0x7f67e8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 -- 172.21.15.170:0/879979911 >> 172.21.15.170:0/879979911 conn(0x7f67e80b1c40 msgr2=0x7f67e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:55.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 -- 172.21.15.170:0/879979911 shutdown_connections 2024-09-21T02:17:55.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.796+0000 7f67efb5b640 1 -- 172.21.15.170:0/879979911 wait complete. 2024-09-21T02:17:55.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 Processor -- start 2024-09-21T02:17:55.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- start start 2024-09-21T02:17:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e81574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cea00 0x7f67e8147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e815f990 0x7f67e80cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8169100 con 0x7f67e8070570 2024-09-21T02:17:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f67e80cb710 con 0x7f67e815f990 2024-09-21T02:17:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f67e8169280 con 0x7f67e80cea00 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e81574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e81574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54304/0 (socket says 172.21.15.170:54304) 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 -- 172.21.15.170:0/1904977788 learned_addr learned my addr 172.21.15.170:0/1904977788 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed0f6640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cea00 0x7f67e8147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ee0f8640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e815f990 0x7f67e80cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 -- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cea00 msgr2=0x7f67e8147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cea00 0x7f67e8147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 -- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e815f990 msgr2=0x7f67e80cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e815f990 0x7f67e80cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 -- 172.21.15.170:0/1904977788 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e8157ed0 con 0x7f67e8070570 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67ed8f7640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e81574f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c940 tx=0x7f67e400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4004220 con 0x7f67e8070570 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e81581b0 con 0x7f67e8070570 2024-09-21T02:17:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67e40079b0 con 0x7f67e8070570 2024-09-21T02:17:55.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40044b0 con 0x7f67e8070570 2024-09-21T02:17:55.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e8164e60 con 0x7f67e8070570 2024-09-21T02:17:55.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.800+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e80cd8a0 con 0x7f67e8070570 2024-09-21T02:17:55.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.804+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f67e4003600 con 0x7f67e8070570 2024-09-21T02:17:55.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.804+0000 7f67de7fc640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f67cc073590 0x7f67cc075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.804+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f67e40949e0 con 0x7f67e8070570 2024-09-21T02:17:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.804+0000 7f67ed0f6640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f67cc073590 0x7f67cc075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.804+0000 7f67ed0f6640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f67cc073590 0x7f67cc075aa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f67d80af000 tx=0x7f67d801b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:55.808+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f67e4014030 con 0x7f67e8070570 2024-09-21T02:17:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.000+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67e80b0540 con 0x7f67e8070570 2024-09-21T02:17:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.000+0000 7f67de7fc640 1 -- 172.21.15.170:0/1904977788 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f67e405f480 con 0x7f67e8070570 2024-09-21T02:17:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:56.004 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:17:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:17:56.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f67cc073590 msgr2=0x7f67cc075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f67cc073590 0x7f67cc075aa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f67d80af000 tx=0x7f67d801b040 comp rx=0 tx=0).stop 2024-09-21T02:17:56.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 msgr2=0x7f67e81574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e81574f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c940 tx=0x7f67e400ce00 comp rx=0 tx=0).stop 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 shutdown_connections 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f67cc073590 0x7f67cc075aa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67e80cea00 0x7f67e8147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67e815f990 0x7f67e80cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 --2- 172.21.15.170:0/1904977788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67e8070570 0x7f67e81574f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 >> 172.21.15.170:0/1904977788 conn(0x7f67e80b1c40 msgr2=0x7f67e80afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 shutdown_connections 2024-09-21T02:17:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.004+0000 7f67efb5b640 1 -- 172.21.15.170:0/1904977788 wait complete. 2024-09-21T02:17:56.024 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T02:17:56.178 INFO:teuthology.orchestra.run.smithi170.stdout:[] 2024-09-21T02:17:56.179 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T02:17:56.179 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol87450 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 -- 172.21.15.170:0/3137862689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb74004d50 msgr2=0x7feb74005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 --2- 172.21.15.170:0/3137862689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb74004d50 0x7feb74005160 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0c8320 tx=0x7feb8401d5d0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb909fa640 1 -- 172.21.15.170:0/3137862689 reap_dead start 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 -- 172.21.15.170:0/3137862689 shutdown_connections 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 -- 172.21.15.170:0/3137862689 >> 172.21.15.170:0/3137862689 conn(0x7feb8c09a9c0 msgr2=0x7feb8c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 -- 172.21.15.170:0/3137862689 shutdown_connections 2024-09-21T02:17:56.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 -- 172.21.15.170:0/3137862689 wait complete. 2024-09-21T02:17:56.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 Processor -- start 2024-09-21T02:17:56.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.336+0000 7feb9245d640 1 -- start start 2024-09-21T02:17:56.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb8c155060 0x7feb8c0cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 0x7feb8c1e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb8c1e88e0 0x7feb8c0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c0cccb0 con 0x7feb8c0c87e0 2024-09-21T02:17:56.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c0b0940 con 0x7feb8c155060 2024-09-21T02:17:56.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7feb8c0cce30 con 0x7feb8c1e88e0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 0x7feb8c1e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 0x7feb8c1e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54314/0 (socket says 172.21.15.170:54314) 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 -- 172.21.15.170:0/1699816016 learned_addr learned my addr 172.21.15.170:0/1699816016 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8bfff640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb8c155060 0x7feb8c0cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb909fa640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb8c1e88e0 0x7feb8c0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 -- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb8c1e88e0 msgr2=0x7feb8c0c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb8c1e88e0 0x7feb8c0c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 -- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb8c155060 msgr2=0x7feb8c0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb8c155060 0x7feb8c0cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 -- 172.21.15.170:0/1699816016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c1e8cf0 con 0x7feb8c0c87e0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb8b7fe640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 0x7feb8c1e83a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feb800107e0 tx=0x7feb80010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8000ece0 con 0x7feb8c0c87e0 2024-09-21T02:17:56.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb80012450 con 0x7feb8c0c87e0 2024-09-21T02:17:56.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8c0c5880 con 0x7feb8c0c87e0 2024-09-21T02:17:56.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80012880 con 0x7feb8c0c87e0 2024-09-21T02:17:56.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8c0c5b20 con 0x7feb8c0c87e0 2024-09-21T02:17:56.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8c15f880 con 0x7feb8c0c87e0 2024-09-21T02:17:56.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7feb80012a20 con 0x7feb8c0c87e0 2024-09-21T02:17:56.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.340+0000 7feb88ff9640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feb78073590 0x7feb78075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.344+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7feb80095e40 con 0x7feb8c0c87e0 2024-09-21T02:17:56.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.344+0000 7feb8bfff640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feb78073590 0x7feb78075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.344+0000 7feb8bfff640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feb78073590 0x7feb78075aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feb7c001a20 tx=0x7feb7c0018d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:56.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.344+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feb8c1e8cf0 con 0x7feb8c0c87e0 2024-09-21T02:17:56.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.544+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 --> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol87450", "target": ["mon-mgr", ""]}) -- 0x7feb8c0d19b0 con 0x7feb78073590 2024-09-21T02:17:56.559 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:56.558+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:56.560 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:56.558+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:56.560 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:56.558+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:56.560 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:56.558+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:56.560 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:56.558+0000 7f1a425f4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:56.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.624+0000 7feb88ff9640 1 -- 172.21.15.170:0/1699816016 <== mgr.4100 v2:172.21.15.191:6800/2615148678 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb8c0d19b0 con 0x7feb78073590 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feb78073590 msgr2=0x7feb78075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feb78073590 0x7feb78075aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feb7c001a20 tx=0x7feb7c0018d0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 msgr2=0x7feb8c1e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 0x7feb8c1e83a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feb800107e0 tx=0x7feb80010ca0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 shutdown_connections 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7feb78073590 0x7feb78075aa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb8c1e88e0 0x7feb8c0c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb8c0c87e0 0x7feb8c1e83a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 --2- 172.21.15.170:0/1699816016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb8c155060 0x7feb8c0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 >> 172.21.15.170:0/1699816016 conn(0x7feb8c09a9c0 msgr2=0x7feb8c0716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:56.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 shutdown_connections 2024-09-21T02:17:56.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.628+0000 7feb9245d640 1 -- 172.21.15.170:0/1699816016 wait complete. 2024-09-21T02:17:56.653 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:56.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 -- 172.21.15.170:0/1201361520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac0b22b0 msgr2=0x7f2eac0b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/1201361520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac0b22b0 0x7f2eac0b1730 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2eac073a30 tx=0x7f2ea401ccc0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 -- 172.21.15.170:0/1201361520 shutdown_connections 2024-09-21T02:17:56.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/1201361520 >> v2:172.21.15.191:3300/0 conn(0x7f2eac06d510 0x7f2eac160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/1201361520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac0b22b0 0x7f2eac0b1730 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 -- 172.21.15.170:0/1201361520 >> 172.21.15.170:0/1201361520 conn(0x7f2eac09a9c0 msgr2=0x7f2eac0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:56.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 -- 172.21.15.170:0/1201361520 shutdown_connections 2024-09-21T02:17:56.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 -- 172.21.15.170:0/1201361520 wait complete. 2024-09-21T02:17:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 Processor -- start 2024-09-21T02:17:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.804+0000 7f2eb0c20640 1 -- start start 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0b22b0 0x7f2eac1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eac0b6930 0x7f2eac1fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 0x7f2eac153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac15c620 con 0x7f2eac152e40 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac15c4a0 con 0x7f2eac0b22b0 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac15c7a0 con 0x7f2eac0b6930 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 0x7f2eac153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 0x7f2eac153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54330/0 (socket says 172.21.15.170:54330) 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 -- 172.21.15.170:0/3895192479 learned_addr learned my addr 172.21.15.170:0/3895192479 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaa59c640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0b22b0 0x7f2eac1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2ea9d9b640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eac0b6930 0x7f2eac1fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 -- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eac0b6930 msgr2=0x7f2eac1fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eac0b6930 0x7f2eac1fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 -- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0b22b0 msgr2=0x7f2eac1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0b22b0 0x7f2eac1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 -- 172.21.15.170:0/3895192479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac1fabf0 con 0x7f2eac152e40 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2ea9d9b640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eac0b6930 0x7f2eac1fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eaad9d640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 0x7f2eac153250 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c740 tx=0x7f2ea000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0004220 con 0x7f2eac152e40 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ea000ce50 con 0x7f2eac152e40 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac1faed0 con 0x7f2eac152e40 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0007c10 con 0x7f2eac152e40 2024-09-21T02:17:56.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac0c7b20 con 0x7f2eac152e40 2024-09-21T02:17:56.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac15a150 con 0x7f2eac152e40 2024-09-21T02:17:56.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.808+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2ea0003600 con 0x7f2eac152e40 2024-09-21T02:17:56.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.812+0000 7f2e9affd640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2e8c073590 0x7f2e8c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:56.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.812+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f2ea0094a10 con 0x7f2eac152e40 2024-09-21T02:17:56.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.812+0000 7f2eaa59c640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2e8c073590 0x7f2e8c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:56.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.812+0000 7f2eaa59c640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2e8c073590 0x7f2e8c075aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40af000 tx=0x7f2ea401b500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:56.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:56.816+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2ea0014030 con 0x7f2eac152e40 2024-09-21T02:17:57.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.012+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eac160390 con 0x7f2eac152e40 2024-09-21T02:17:57.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.012+0000 7f2e9affd640 1 -- 172.21.15.170:0/3895192479 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f2ea005f410 con 0x7f2eac152e40 2024-09-21T02:17:57.016 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:57.016 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:17:57.017 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2e8c073590 msgr2=0x7f2e8c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2e8c073590 0x7f2e8c075aa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40af000 tx=0x7f2ea401b500 comp rx=0 tx=0).stop 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 msgr2=0x7f2eac153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 0x7f2eac153250 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c740 tx=0x7f2ea000cc00 comp rx=0 tx=0).stop 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 shutdown_connections 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f2e8c073590 0x7f2e8c075aa0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eac0b6930 0x7f2eac1fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eac152e40 0x7f2eac153250 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 --2- 172.21.15.170:0/3895192479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0b22b0 0x7f2eac1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 >> 172.21.15.170:0/3895192479 conn(0x7f2eac09a9c0 msgr2=0x7f2eac158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 shutdown_connections 2024-09-21T02:17:57.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.016+0000 7f2eb0c20640 1 -- 172.21.15.170:0/3895192479 wait complete. 2024-09-21T02:17:57.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T02:17:57.200 INFO:teuthology.orchestra.run.smithi170.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.191:0","nonce":1074802945}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.61999363799999996,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.969624501723843,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.191:0/1074802945","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi191","pid":"28842","root":"/"}}] 2024-09-21T02:17:57.200 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi191", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28842", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.191:0", "nonce": 1074802945, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.191:0/1074802945", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.969624501723843 }, "state": "open", "uptime": 0.619993638 } ] 2024-09-21T02:17:57.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.348+0000 7f80e927d640 1 -- 172.21.15.170:0/3907254548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 msgr2=0x7f80e4073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.348+0000 7f80e927d640 1 --2- 172.21.15.170:0/3907254548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 0x7f80e4073cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009c80 tx=0x7f80d401cd10 comp rx=0 tx=0).stop 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- 172.21.15.170:0/3907254548 shutdown_connections 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 --2- 172.21.15.170:0/3907254548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 0x7f80e4073cc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 --2- 172.21.15.170:0/3907254548 >> v2:172.21.15.191:3300/0 conn(0x7f80e40b4440 0x7f80e4073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- 172.21.15.170:0/3907254548 >> 172.21.15.170:0/3907254548 conn(0x7f80e40b2630 msgr2=0x7f80e40b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- 172.21.15.170:0/3907254548 shutdown_connections 2024-09-21T02:17:57.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- 172.21.15.170:0/3907254548 wait complete. 2024-09-21T02:17:57.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 Processor -- start 2024-09-21T02:17:57.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- start start 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 0x7f80e414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80e40b4440 0x7f80e41fc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 0x7f80e41fc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f80e416bf30 con 0x7f80e40b4850 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f80e416bdb0 con 0x7f80e40b4440 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e927d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f80e416c0b0 con 0x7f80e40707f0 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 0x7f80e41fc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:57.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 0x7f80e41fc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54342/0 (socket says 172.21.15.170:54342) 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 -- 172.21.15.170:0/4036878724 learned_addr learned my addr 172.21.15.170:0/4036878724 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e2ffd640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 0x7f80e414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e27fc640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80e40b4440 0x7f80e41fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 -- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 msgr2=0x7f80e414d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 0x7f80e414d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 -- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80e40b4440 msgr2=0x7f80e41fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.352+0000 7f80e37fe640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80e40b4440 0x7f80e41fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80e37fe640 1 -- 172.21.15.170:0/4036878724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41537c0 con 0x7f80e40b4850 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80e37fe640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 0x7f80e41fc850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f80d401b3c0 tx=0x7f80d4005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40bc070 con 0x7f80e40b4850 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d40040c0 con 0x7f80e40b4850 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e4153aa0 con 0x7f80e40b4850 2024-09-21T02:17:57.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40b2040 con 0x7f80e40b4850 2024-09-21T02:17:57.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e4154060 con 0x7f80e40b4850 2024-09-21T02:17:57.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e4162550 con 0x7f80e40b4850 2024-09-21T02:17:57.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f80d4004510 con 0x7f80e40b4850 2024-09-21T02:17:57.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80db7fe640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f80c4073560 0x7f80c4075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:57.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f80d412e5f0 con 0x7f80e40b4850 2024-09-21T02:17:57.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.356+0000 7f80e2ffd640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f80c4073560 0x7f80c4075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:57.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.360+0000 7f80e2ffd640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f80c4073560 0x7f80c4075a70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009ad0 tx=0x7f80dc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:57.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.360+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f80d40f3290 con 0x7f80e40b4850 2024-09-21T02:17:57.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.576+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f80e4160510 con 0x7f80e40b4850 2024-09-21T02:17:57.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.588+0000 7f80db7fe640 1 -- 172.21.15.170:0/4036878724 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+506846 (secure 0 0 0) 0x7f80d4016990 con 0x7f80e40b4850 2024-09-21T02:17:57.598 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:57.598 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:17:57.598 INFO:teuthology.orchestra.run.smithi170.stdout: "epoch": 11, 2024-09-21T02:17:57.598 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.598 INFO:teuthology.orchestra.run.smithi170.stdout: "active_gid": 4100, 2024-09-21T02:17:57.598 INFO:teuthology.orchestra.run.smithi170.stdout: "active_name": "x", 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "active_addrs": { 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:6800", 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2615148678 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:6801", 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2615148678 2024-09-21T02:17:57.599 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "active_addr": "172.21.15.191:6801/2615148678", 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "active_change": "2024-09-21T02:13:04.850225+0000", 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "available": true, 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "standbys": [ 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "gid": 4122, 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "y", 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr_features": 4540719139924082687, 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: "available_modules": [ 2024-09-21T02:17:57.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "alerts", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.601 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.602 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.603 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.604 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.605 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_destination": { 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_destination", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_from_name": { 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_from_name", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "Ceph", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Email From: name", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_host": { 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_host", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SMTP server", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_password": { 2024-09-21T02:17:57.609 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_password", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Password to authenticate with", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_port": { 2024-09-21T02:17:57.610 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_port", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "465", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SMTP port", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_sender": { 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_sender", 2024-09-21T02:17:57.611 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_ssl": { 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_ssl", 2024-09-21T02:17:57.612 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_user": { 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_user", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.613 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "User to authenticate as", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.614 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "balancer", 2024-09-21T02:17:57.615 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "active": { 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "active", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.616 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "begin_time": { 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "begin_time", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0000", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.617 INFO:teuthology.orchestra.run.smithi170.stdout: "begin_weekday": { 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "begin_weekday", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "0", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "6", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.618 INFO:teuthology.orchestra.run.smithi170.stdout: "crush_compat_max_iterations": { 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "25", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "250", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "crush_compat_metrics": { 2024-09-21T02:17:57.619 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crush_compat_metrics", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.620 INFO:teuthology.orchestra.run.smithi170.stdout: "crush_compat_step": { 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crush_compat_step", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0.5", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "0.001", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "0.999", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.621 INFO:teuthology.orchestra.run.smithi170.stdout: "end_time": { 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "end_time", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "2359", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "end_weekday": { 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "end_weekday", 2024-09-21T02:17:57.622 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "0", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "6", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.623 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.624 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.626 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.627 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "min_score": { 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "min_score", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.628 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": { 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mode", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "upmap", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "crush-compat", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "none", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "read", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap", 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap-read" 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.629 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Balancer mode", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_ids": { 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pool_ids", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.630 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.631 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.632 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap_max_deviation": { 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "upmap_max_deviation", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.633 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap_max_optimizations": { 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephadm", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "agent_down_multiplier": { 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "agent_down_multiplier", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3.0", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.635 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "agent_refresh_rate": { 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "agent_refresh_rate", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "20", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "agent_starting_port": { 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "agent_starting_port", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "4721", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "allow_ptrace": { 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "allow_ptrace", 2024-09-21T02:17:57.637 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "autotune_interval": { 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "autotune_interval", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.638 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.639 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0.7", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "cephadm_log_destination": { 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephadm_log_destination", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.640 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "file", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "file,syslog", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "syslog" 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "cgroups_split": { 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cgroups_split", 2024-09-21T02:17:57.641 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "config_checks_enabled": { 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "config_checks_enabled", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.642 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "config_dashboard": { 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "config_dashboard", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_alertmanager": { 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_alertmanager", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.644 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_base": { 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_base", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.645 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_elasticsearch": { 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "elasticsearch container image", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.646 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_grafana": { 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_grafana", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.647 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_haproxy": { 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_haproxy", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "HAproxy container image", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.648 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_jaeger_agent": { 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_jaeger_collector": { 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:17:57.649 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_jaeger_query": { 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.650 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Jaeger query container image", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_keepalived": { 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_keepalived", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.651 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Keepalived container image", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_loki": { 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_loki", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:17:57.652 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Loki container image", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_nginx": { 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_nginx", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.653 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Nginx container image", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_node_exporter": { 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_node_exporter", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.654 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_nvmeof": { 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_nvmeof", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.655 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Nvme-of container image", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:17:57.656 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_prometheus": { 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_prometheus", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.657 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_promtail": { 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_promtail", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Promtail container image", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_samba": { 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_samba", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_samba_metrics": { 2024-09-21T02:17:57.659 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_snmp_gateway": { 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "container_init": { 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_init", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.661 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "daemon_cache_timeout": { 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.662 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "900", 2024-09-21T02:17:57.663 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "default_registry": { 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "default_registry", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.664 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "device_cache_timeout": { 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "device_cache_timeout", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "1800", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.665 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "device_enhanced_scan": { 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "device_enhanced_scan", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "facts_cache_timeout": { 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "facts_cache_timeout", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "grafana_dashboards_path": { 2024-09-21T02:17:57.667 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "host_check_interval": { 2024-09-21T02:17:57.668 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "host_check_interval", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "hw_monitoring": { 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "hw_monitoring", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.669 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "inventory_list_all": { 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "inventory_list_all", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.670 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.671 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "log_refresh_metadata": { 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_refresh_metadata", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.672 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.673 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.675 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.676 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.677 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "*", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "max_count_per_host": { 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_count_per_host", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.678 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "max_osd_draining_count": { 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_osd_draining_count", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.679 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.680 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.680 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.680 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.681 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.681 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:17:57.681 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.681 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.681 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "migration_current": { 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "migration_current", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "internal - do not modify", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": { 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mode", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "root", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "cephadm-package", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "root" 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:17:57.683 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "oob_default_addr": { 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "oob_default_addr", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "169.254.1.1", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:17:57.684 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "prometheus_alerts_path": { 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:17:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_insecure": { 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_insecure", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.686 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_password": { 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_password", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.687 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_url": { 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_url", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.688 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_username": { 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_username", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "secure_monitoring_stack": { 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:17:57.689 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "service_discovery_port": { 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "service_discovery_port", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.690 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8765", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.691 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "ssh_config_file": { 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssh_config_file", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.692 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.693 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "ssh_keepalive_interval": { 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "7", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.694 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "use_agent": { 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "use_agent", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.695 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "use_repo_digest": { 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "use_repo_digest", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:17:57.696 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_on_failed_host_check": { 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_on_stray_daemons": { 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_on_stray_hosts": { 2024-09-21T02:17:57.698 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crash", 2024-09-21T02:17:57.699 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.700 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.701 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.702 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.703 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "retain_interval": { 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "retain_interval", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "31536000", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.704 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.705 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_recent_interval": { 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_recent_interval", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "1209600", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.706 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "dashboard", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.707 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.708 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:17:57.710 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.711 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.712 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.713 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.714 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.715 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.716 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.717 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.718 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "admin", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:17:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_URL": { 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.721 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.722 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "admin", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.723 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.724 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.725 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.726 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "[]", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "{}", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.728 INFO:teuthology.orchestra.run.smithi170.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.729 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.730 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.731 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.733 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.734 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.735 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.736 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:17:57.737 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.738 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8", 2024-09-21T02:17:57.739 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "45", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "admin", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:17:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.744 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.745 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.746 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "cross_origin_url": { 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cross_origin_url", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "crt_file": { 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crt_file", 2024-09-21T02:17:57.748 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "debug": { 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "debug", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.749 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "jwt_token_ttl": { 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "jwt_token_ttl", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.750 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "28800", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "key_file": { 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "key_file", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.751 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.752 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.753 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.754 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.756 INFO:teuthology.orchestra.run.smithi170.stdout: "motd": { 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "motd", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "The message of the day", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.757 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "server_addr": { 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_addr", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.758 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "::", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "server_port": { 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_port", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8080", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.760 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "ssl": { 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssl", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.761 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "ssl_server_port": { 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssl_server_port", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8443", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.762 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "sso_oauth2": { 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sso_oauth2", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_behaviour": { 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_behaviour", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "redirect", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "redirect" 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_error_status_code": { 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_error_status_code", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "500", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "400", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "599", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "url_prefix": { 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "url_prefix", 2024-09-21T02:17:57.765 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "devicehealth", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "enable_monitoring": { 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enable_monitoring", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "monitor device health metrics", 2024-09-21T02:17:57.767 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.768 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.769 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.770 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.771 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "mark_out_threshold": { 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mark_out_threshold", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "2419200", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.772 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_name": { 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pool_name", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "device_health_metrics", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:17:57.773 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "retention_period": { 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "retention_period", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "15552000", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.774 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "scrape_frequency": { 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "scrape_frequency", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "86400", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.775 INFO:teuthology.orchestra.run.smithi170.stdout: "self_heal": { 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "self_heal", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:57.776 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.777 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_threshold": { 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_threshold", 2024-09-21T02:17:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "7257600", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.779 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "diskprediction_local", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.780 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.781 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.782 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.784 INFO:teuthology.orchestra.run.smithi170.stdout: "predict_interval": { 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "predict_interval", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "86400", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "predictor_model": { 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "predictor_model", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.785 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "prophetstor", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.787 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "influx", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": false, 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "batch_size": { 2024-09-21T02:17:57.788 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "batch_size", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5000", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "database": { 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "database", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.789 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "ceph", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": { 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "hostname", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.790 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.791 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "30", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "5", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.792 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.793 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.794 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.795 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "password": { 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "password", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "port": { 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "port", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8086", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "InfluxDB server port", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.799 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "ssl": { 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssl", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "false", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.800 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "threads": { 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "threads", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "32", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "username": { 2024-09-21T02:17:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "username", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "verify_ssl": { 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "verify_ssl", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.802 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "true", 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.803 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "insights", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.804 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.805 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.806 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.807 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.808 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "iostat", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.809 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.810 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.812 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.813 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.814 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "localpool", 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "failure_domain": { 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "failure_domain", 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.815 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "host", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.816 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.817 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.818 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.819 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "min_size": { 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "min_size", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.820 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "num_rep": { 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "num_rep", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3", 2024-09-21T02:17:57.821 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_num": { 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pg_num", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "128", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.822 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "prefix": { 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "prefix", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.823 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.824 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "subtree": { 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "subtree", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "rack", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:17:57.825 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mirroring", 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.826 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.828 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.829 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.830 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.831 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "nfs", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.832 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.833 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.834 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.835 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.836 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.837 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "orchestrator", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "fail_fs": { 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "fail_fs", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:17:57.838 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.839 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.841 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.842 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator": { 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "orchestrator", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.843 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "cephadm", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "rook", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "test_orchestrator" 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Orchestrator backend", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.844 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "osd_perf_query", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.845 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.846 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.847 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.848 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.849 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.850 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "osd_support", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.851 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.852 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.853 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.854 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.855 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.856 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pg_autoscaler", 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.857 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.858 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.859 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.860 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.861 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.862 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.863 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "threshold": { 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "threshold", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.864 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3.0", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1.0", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "scaling threshold", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "progress", 2024-09-21T02:17:57.865 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "allow_pg_recovery_event": { 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.866 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "enabled": { 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enabled", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.867 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.868 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.869 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.870 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.871 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "max_completed_events": { 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_completed_events", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.872 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "50", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.873 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.874 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "prometheus", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "cache": { 2024-09-21T02:17:57.875 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cache", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "exclude_perf_counters": { 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "exclude_perf_counters", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.876 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.877 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.878 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.879 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.880 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_stats_pools": { 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_stats_pools", 2024-09-21T02:17:57.881 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.882 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "300", 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "scrape_interval": { 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "scrape_interval", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "15.0", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.885 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "server_addr": { 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_addr", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "::", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:17:57.886 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "server_port": { 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_port", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "9283", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.887 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.888 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "stale_cache_strategy": { 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "stale_cache_strategy", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "log", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.889 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_behaviour": { 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_behaviour", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "default", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "default", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "error" 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.890 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_error_status_code": { 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_error_status_code", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "500", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "400", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "599", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.891 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_support", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.892 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.893 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.894 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "max_concurrent_snap_create": { 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.896 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.897 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.898 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "trash_purge_schedule": { 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "trash_purge_schedule", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.899 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "restful", 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "enable_auth": { 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enable_auth", 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.900 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "key_file": { 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "key_file", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.901 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.902 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.903 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.904 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.905 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.906 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "max_requests": { 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_requests", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.907 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "500", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "server_addr": { 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_addr", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.908 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "server_port": { 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_port", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.909 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.910 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rgw", 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.911 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.912 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.913 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.914 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.915 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.918 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "selftest", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.919 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.920 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.921 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.922 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.923 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "roption1": { 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "roption1", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.924 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "roption2": { 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "roption2", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "xyz", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption1": { 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption1", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.926 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption2": { 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption2", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.927 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption3": { 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption3", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.928 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption4": { 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption4", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.929 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption5": { 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption5", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.930 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption6": { 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption6", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption7": { 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption7", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "42", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "testkey": { 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "testkey", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "testlkey": { 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "testlkey", 2024-09-21T02:17:57.934 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "testnewline": { 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "testnewline", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.935 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "snap_schedule", 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.936 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "allow_m_granularity": { 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "allow_m_granularity", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.937 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "dump_on_update": { 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "dump_on_update", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.939 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.940 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.941 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.942 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.943 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "stats", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.945 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.946 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.947 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.950 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "status", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.951 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.952 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.953 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.954 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.956 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "telegraf", 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "address": { 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "address", 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.957 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.958 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "15", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.959 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.960 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.961 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.962 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.963 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.964 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "telemetry", 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_basic": { 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_basic", 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_crash": { 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_crash", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.966 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_device": { 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_device", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.967 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_ident": { 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_ident", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.968 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_perf": { 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_perf", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.969 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "contact": { 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "contact", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.970 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "description": { 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "description", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.971 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "device_url": { 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "device_url", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.972 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "enabled": { 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enabled", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.973 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "24", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "8", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.974 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "last_opt_revision": { 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "last_opt_revision", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "1", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.975 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "leaderboard": { 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "leaderboard", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.976 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "leaderboard_description": { 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "leaderboard_description", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.978 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.979 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.980 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.981 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "organization": { 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "organization", 2024-09-21T02:17:57.982 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "proxy": { 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "proxy", 2024-09-21T02:17:57.983 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.984 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "url": { 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "url", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.985 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "test_orchestrator", 2024-09-21T02:17:57.986 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.988 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.989 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.990 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.991 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.992 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:57.993 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.994 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.995 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:57.996 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.997 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "max_concurrent_clones": { 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_concurrent_clones", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "4", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.998 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "periodic_async_work": { 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "periodic_async_work", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:57.999 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Periodically check for async work", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot_clone_delay": { 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.000 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.002 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "discovery_interval": { 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "discovery_interval", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "100", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.003 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "identifier": { 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "identifier", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.004 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.005 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.006 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.007 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.008 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.009 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.010 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.011 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "zabbix_host": { 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix_host", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.012 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "zabbix_port": { 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix_port", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10051", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.013 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "zabbix_sender": { 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix_sender", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.014 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "modules": [ 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "iostat", 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "nfs", 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "restful" 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.015 INFO:teuthology.orchestra.run.smithi170.stdout: "available_modules": [ 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "alerts", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.016 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.017 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.018 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.019 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.020 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.021 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_destination": { 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_destination", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.022 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_from_name": { 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_from_name", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "Ceph", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.023 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Email From: name", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_host": { 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_host", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SMTP server", 2024-09-21T02:17:58.024 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_password": { 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_password", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Password to authenticate with", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_port": { 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_port", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "465", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SMTP port", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.026 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_sender": { 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_sender", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SMTP envelope sender", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_ssl": { 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_ssl", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "smtp_user": { 2024-09-21T02:17:58.028 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "smtp_user", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "User to authenticate as", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.029 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "balancer", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "active": { 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "active", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.031 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "begin_time": { 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "begin_time", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0000", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:17:58.032 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "begin_weekday": { 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "begin_weekday", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "0", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "6", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:17:58.033 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "crush_compat_max_iterations": { 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crush_compat_max_iterations", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "25", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "250", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.034 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "crush_compat_metrics": { 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crush_compat_metrics", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T02:17:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "crush_compat_step": { 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crush_compat_step", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0.5", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "0.001", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "0.999", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "aggressiveness of optimization", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "end_time": { 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "end_time", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "2359", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "end_weekday": { 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "end_weekday", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "0", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "6", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.038 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.039 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.040 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.041 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.042 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "min_score": { 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "min_score", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.043 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": { 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mode", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "upmap", 2024-09-21T02:17:58.044 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "crush-compat", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "none", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "read", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap-read" 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Balancer mode", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_ids": { 2024-09-21T02:17:58.045 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pool_ids", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:58.046 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.047 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap_max_deviation": { 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "upmap_max_deviation", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.048 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "upmap_max_optimizations": { 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "upmap_max_optimizations", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:58.049 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephadm", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "agent_down_multiplier": { 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "agent_down_multiplier", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3.0", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T02:17:58.051 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "agent_refresh_rate": { 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "agent_refresh_rate", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "20", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.052 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "agent_starting_port": { 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "agent_starting_port", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "4721", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.053 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "allow_ptrace": { 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "allow_ptrace", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.054 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "autotune_interval": { 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "autotune_interval", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "autotune_memory_target_ratio": { 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0.7", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.056 INFO:teuthology.orchestra.run.smithi170.stdout: "cephadm_log_destination": { 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephadm_log_destination", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "file", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "file,syslog", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "syslog" 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T02:17:58.057 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "cgroups_split": { 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cgroups_split", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T02:17:58.058 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "config_checks_enabled": { 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "config_checks_enabled", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.059 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "config_dashboard": { 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "config_dashboard", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.060 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_alertmanager": { 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_alertmanager", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_base": { 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_base", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Container image name, without the tag", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.062 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_elasticsearch": { 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_elasticsearch", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "elasticsearch container image", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.063 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_grafana": { 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_grafana", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_haproxy": { 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_haproxy", 2024-09-21T02:17:58.064 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "HAproxy container image", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_jaeger_agent": { 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_jaeger_agent", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.065 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Jaeger agent container image", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_jaeger_collector": { 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_jaeger_collector", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Jaeger collector container image", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_jaeger_query": { 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_jaeger_query", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Jaeger query container image", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_keepalived": { 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_keepalived", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.068 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Keepalived container image", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_loki": { 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_loki", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.069 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Loki container image", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_nginx": { 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_nginx", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.070 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Nginx container image", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_node_exporter": { 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_node_exporter", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.071 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_nvmeof": { 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_nvmeof", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T02:17:58.072 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Nvme-of container image", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_oauth2_proxy": { 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.073 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "oauth2-proxy container image", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_prometheus": { 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_prometheus", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.074 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Prometheus container image", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_promtail": { 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_promtail", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T02:17:58.075 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Promtail container image", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_samba": { 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_samba", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.076 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Samba/SMB container image", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_samba_metrics": { 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_samba_metrics", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.077 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "container_image_snmp_gateway": { 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_image_snmp_gateway", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.078 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "SNMP Gateway container image", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "container_init": { 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "container_init", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.079 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "daemon_cache_timeout": { 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "daemon_cache_timeout", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.080 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "default_cephadm_command_timeout": { 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "900", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T02:17:58.081 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "default_registry": { 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "default_registry", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "docker.io", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T02:17:58.082 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "device_cache_timeout": { 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "device_cache_timeout", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "1800", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "seconds to cache device inventory", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.083 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "device_enhanced_scan": { 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "device_enhanced_scan", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.084 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "facts_cache_timeout": { 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "facts_cache_timeout", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "seconds to cache host facts data", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.085 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "grafana_dashboards_path": { 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "grafana_dashboards_path", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.086 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "host_check_interval": { 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "host_check_interval", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to perform a host check", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.088 INFO:teuthology.orchestra.run.smithi170.stdout: "hw_monitoring": { 2024-09-21T02:17:58.088 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "hw_monitoring", 2024-09-21T02:17:58.088 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.089 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.089 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.089 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.089 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.089 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.089 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "inventory_list_all": { 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "inventory_list_all", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.090 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.091 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "log_refresh_metadata": { 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_refresh_metadata", 2024-09-21T02:17:58.092 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.093 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.094 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.095 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T02:17:58.097 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "*", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "max_count_per_host": { 2024-09-21T02:17:58.098 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_count_per_host", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "max number of daemons per service per host", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "max_osd_draining_count": { 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_osd_draining_count", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.099 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "migration_current": { 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "migration_current", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.100 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "internal - do not modify", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": { 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mode", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.101 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "root", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "cephadm-package", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "root" 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "oob_default_addr": { 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "oob_default_addr", 2024-09-21T02:17:58.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "169.254.1.1", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "prometheus_alerts_path": { 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "prometheus_alerts_path", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.103 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_insecure": { 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_insecure", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.104 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_password": { 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_password", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.105 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_url": { 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_url", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.106 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "registry_username": { 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "registry_username", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.107 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "secure_monitoring_stack": { 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "secure_monitoring_stack", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T02:17:58.108 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "service_discovery_port": { 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "service_discovery_port", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8765", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "cephadm service discovery port", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.109 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.110 INFO:teuthology.orchestra.run.smithi170.stdout: "ssh_config_file": { 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssh_config_file", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "ssh_keepalive_count_max": { 2024-09-21T02:17:58.111 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "ssh_keepalive_interval": { 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssh_keepalive_interval", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.112 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "7", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "use_agent": { 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "use_agent", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.113 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "use_repo_digest": { 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "use_repo_digest", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.114 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_on_failed_host_check": { 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_on_failed_host_check", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.115 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_on_stray_daemons": { 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_on_stray_daemons", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.116 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_on_stray_hosts": { 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_on_stray_hosts", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.117 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crash", 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.118 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.119 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.122 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "retain_interval": { 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "retain_interval", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "31536000", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.123 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.124 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_recent_interval": { 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_recent_interval", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "1209600", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.125 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "dashboard", 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.126 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.127 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "AUDIT_API_ENABLED": { 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T02:17:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "admin", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T02:17:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_URL": { 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_URL", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "admin", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "[]", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "{}", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T02:17:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T02:17:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "45", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T02:17:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "admin", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "cross_origin_url": { 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cross_origin_url", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "crt_file": { 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "crt_file", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "debug": { 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "debug", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Enable/disable debug options", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "jwt_token_ttl": { 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "jwt_token_ttl", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "28800", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "key_file": { 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "key_file", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "motd": { 2024-09-21T02:17:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "motd", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "The message of the day", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "redirect_resolve_ip_addr": { 2024-09-21T02:17:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "server_addr": { 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_addr", 2024-09-21T02:17:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "::", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "server_port": { 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_port", 2024-09-21T02:17:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8080", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "ssl": { 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssl", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "ssl_server_port": { 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssl_server_port", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8443", 2024-09-21T02:17:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "sso_oauth2": { 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sso_oauth2", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_behaviour": { 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_behaviour", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "redirect", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "redirect" 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_error_status_code": { 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_error_status_code", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "500", 2024-09-21T02:17:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "400", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "599", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "url_prefix": { 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "url_prefix", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "devicehealth", 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "enable_monitoring": { 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enable_monitoring", 2024-09-21T02:17:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "monitor device health metrics", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "mark_out_threshold": { 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mark_out_threshold", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "2419200", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_name": { 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pool_name", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "device_health_metrics", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "retention_period": { 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "retention_period", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "15552000", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how long to retain device health metrics", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "scrape_frequency": { 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "scrape_frequency", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "86400", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "self_heal": { 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "self_heal", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "warn_threshold": { 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "warn_threshold", 2024-09-21T02:17:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "7257600", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "diskprediction_local", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "predict_interval": { 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "predict_interval", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "86400", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "predictor_model": { 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "predictor_model", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "prophetstor", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "600", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "influx", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": false, 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "influxdb python module not found", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "batch_size": { 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "batch_size", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5000", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "database": { 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "database", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "ceph", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": { 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "hostname", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "InfluxDB server hostname", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "30", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "5", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "password": { 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "password", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "password of InfluxDB server user", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "port": { 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "port", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "8086", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "InfluxDB server port", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.222 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "ssl": { 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "ssl", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "false", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.224 INFO:teuthology.orchestra.run.smithi170.stdout: "threads": { 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "threads", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "32", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "username": { 2024-09-21T02:17:58.225 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "username", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "username of InfluxDB server user", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "verify_ssl": { 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "verify_ssl", 2024-09-21T02:17:58.226 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "true", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.227 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "insights", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.228 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.229 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.230 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.231 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.232 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "iostat", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.233 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.234 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.235 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.236 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.237 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.238 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "localpool", 2024-09-21T02:17:58.239 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "failure_domain": { 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "failure_domain", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "host", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "failure domain for any created local pool", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.241 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.242 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.243 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.244 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "min_size": { 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "min_size", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.245 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "default min_size for any created local pool", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "num_rep": { 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "num_rep", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3", 2024-09-21T02:17:58.246 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "default replica count for any created local pool", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_num": { 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pg_num", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "128", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.247 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "prefix": { 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "prefix", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "name prefix for any created local pool", 2024-09-21T02:17:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "subtree": { 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "subtree", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "rack", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.250 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mirroring", 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.251 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.252 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.253 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.254 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "nfs", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.257 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.260 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.261 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.262 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "orchestrator", 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "fail_fs": { 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "fail_fs", 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.263 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.264 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.265 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.266 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.267 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.268 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator": { 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "orchestrator", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "cephadm", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "rook", 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "test_orchestrator" 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.269 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Orchestrator backend", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.270 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "osd_perf_query", 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.271 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.272 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.274 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.275 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.276 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "osd_support", 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.277 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.278 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.279 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.280 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.281 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.282 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "pg_autoscaler", 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.283 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.284 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.285 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.286 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.287 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.288 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "threshold": { 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "threshold", 2024-09-21T02:17:58.289 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "3.0", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1.0", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "scaling threshold", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "progress", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "allow_pg_recovery_event": { 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "allow_pg_recovery_event", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.291 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "enabled": { 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enabled", 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.292 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.293 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.293 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.293 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.293 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.293 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.293 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.294 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.295 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.296 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.297 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "max_completed_events": { 2024-09-21T02:17:58.298 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_completed_events", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "50", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "number of past completed events to remember", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "sleep_interval": { 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sleep_interval", 2024-09-21T02:17:58.299 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "how long the module is going to sleep", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.300 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "prometheus", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.301 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "cache": { 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cache", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "exclude_perf_counters": { 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "exclude_perf_counters", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.303 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.305 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.306 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.307 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_stats_pools": { 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_stats_pools", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.308 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "300", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.309 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "scrape_interval": { 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "scrape_interval", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "15.0", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.310 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "server_addr": { 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_addr", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "::", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T02:17:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "server_port": { 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_port", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "9283", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.312 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.313 INFO:teuthology.orchestra.run.smithi170.stdout: "stale_cache_strategy": { 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "stale_cache_strategy", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "log", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.314 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_behaviour": { 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_behaviour", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "default", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "default", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "error" 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.315 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "standby_error_status_code": { 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "standby_error_status_code", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "500", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "400", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "599", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.316 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_support", 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.317 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.318 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.319 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.320 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.321 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "max_concurrent_snap_create": { 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_concurrent_snap_create", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.322 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "mirror_snapshot_schedule": { 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.323 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.324 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "trash_purge_schedule": { 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "trash_purge_schedule", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.325 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "restful", 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "enable_auth": { 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enable_auth", 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.326 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "key_file": { 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "key_file", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.327 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.331 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.332 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "max_requests": { 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_requests", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "500", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.333 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "server_addr": { 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_addr", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.334 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "server_port": { 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "server_port", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.335 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rgw", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.337 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.338 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.339 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.340 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "5", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "selftest", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "roption1": { 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "roption1", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.348 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "roption2": { 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "roption2", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.349 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "xyz", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption1": { 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption1", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.350 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption2": { 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption2", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.351 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption3": { 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption3", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "float", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.352 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption4": { 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption4", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.353 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption5": { 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption5", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.354 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption6": { 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption6", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.355 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "rwoption7": { 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rwoption7", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "1", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "42", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.356 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "testkey": { 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "testkey", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "testlkey": { 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "testlkey", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "testnewline": { 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "testnewline", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "snap_schedule", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "allow_m_granularity": { 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "allow_m_granularity", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "dump_on_update": { 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "dump_on_update", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "dump database to debug log on update", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "stats", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "status", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "telegraf", 2024-09-21T02:17:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "address": { 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "address", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "15", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.383 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.384 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.385 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.386 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.387 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.388 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "telemetry", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_basic": { 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_basic", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.389 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_crash": { 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_crash", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.390 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_device": { 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_device", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.391 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_ident": { 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_ident", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.392 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "channel_perf": { 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "channel_perf", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "contact": { 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "contact", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.394 INFO:teuthology.orchestra.run.smithi170.stdout: "description": { 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "description", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.395 INFO:teuthology.orchestra.run.smithi170.stdout: "device_url": { 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "device_url", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.396 INFO:teuthology.orchestra.run.smithi170.stdout: "enabled": { 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "enabled", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:58.397 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "24", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "8", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "last_opt_revision": { 2024-09-21T02:17:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "last_opt_revision", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "1", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "leaderboard": { 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "leaderboard", 2024-09-21T02:17:58.399 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "leaderboard_description": { 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "leaderboard_description", 2024-09-21T02:17:58.400 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.401 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.402 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.403 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.404 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.405 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "organization": { 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "organization", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.406 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "proxy": { 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "proxy", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.407 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "url": { 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "url", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "test_orchestrator", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.410 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.411 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.412 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.413 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.415 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.416 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.417 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.418 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.419 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.420 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "max_concurrent_clones": { 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "max_concurrent_clones", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "4", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.421 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "periodic_async_work": { 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "periodic_async_work", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Periodically check for async work", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot_clone_delay": { 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "snapshot_clone_delay", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot_clone_no_wait": { 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "True", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.424 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "can_run": true, 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "error_string": "", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "module_options": { 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "discovery_interval": { 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "discovery_interval", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "uint", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "100", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.426 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "identifier": { 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "identifier", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.427 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "interval": { 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "interval", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "secs", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "60", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.428 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "log_level": { 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_level", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.429 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster": { 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.430 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_cluster_level": { 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_cluster_level", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "info", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.431 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [ 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "critical", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "debug", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "error", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "info", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "warning" 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "log_to_file": { 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "log_to_file", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "bool", 2024-09-21T02:17:58.432 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "False", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "sqlite3_killpoint": { 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "sqlite3_killpoint", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "dev", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 1, 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "0", 2024-09-21T02:17:58.433 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "zabbix_host": { 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix_host", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.434 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "zabbix_port": { 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix_port", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "int", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "10051", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.435 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "zabbix_sender": { 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "zabbix_sender", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "str", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "level": "advanced", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "flags": 0, 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "min": "", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "max": "", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "enum_allowed": [], 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "desc": "", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "long_desc": "", 2024-09-21T02:17:58.436 INFO:teuthology.orchestra.run.smithi170.stdout: "tags": [], 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "see_also": [] 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "services": {}, 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "always_on_modules": { 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "octopus": [ 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "balancer", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "crash", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "devicehealth", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_autoscaler", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "progress", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_support", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "status", 2024-09-21T02:17:58.437 INFO:teuthology.orchestra.run.smithi170.stdout: "telemetry", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "volumes" 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "pacific": [ 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "balancer", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "crash", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "devicehealth", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_autoscaler", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "progress", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_support", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "status", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "telemetry", 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "volumes" 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.438 INFO:teuthology.orchestra.run.smithi170.stdout: "quincy": [ 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "balancer", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "crash", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "devicehealth", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_autoscaler", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "progress", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_support", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "status", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "telemetry", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "volumes" 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "reef": [ 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "balancer", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "crash", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "devicehealth", 2024-09-21T02:17:58.439 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_autoscaler", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "progress", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_support", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "status", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "telemetry", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "volumes" 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "squid": [ 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "balancer", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "crash", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "devicehealth", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "orchestrator", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "pg_autoscaler", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "progress", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "rbd_support", 2024-09-21T02:17:58.440 INFO:teuthology.orchestra.run.smithi170.stdout: "status", 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "telemetry", 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "volumes" 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "last_failure_osd_epoch": 0, 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "active_clients": [ 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "devicehealth", 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 1852913239 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "libcephsqlite", 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 543979301 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "rbd_support", 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.442 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 1634809445 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2890432817 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3689562231 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.444 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2357730026 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2838563242 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.445 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3207724562 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.446 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3826872798 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volumes", 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: "addrvec": [ 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v2", 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "172.21.15.191:0", 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3514032572 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:17:58.447 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f80c4073560 msgr2=0x7f80c4075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f80c4073560 0x7f80c4075a70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009ad0 tx=0x7f80dc0091e0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 msgr2=0x7f80e41fc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 0x7f80e41fc850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f80d401b3c0 tx=0x7f80d4005ae0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 shutdown_connections 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f80c4073560 0x7f80c4075a70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e40b4850 0x7f80e41fc850 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80e40b4440 0x7f80e41fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 --2- 172.21.15.170:0/4036878724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80e40707f0 0x7f80e414d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 >> 172.21.15.170:0/4036878724 conn(0x7f80e40b2630 msgr2=0x7f80e4160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:58.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 shutdown_connections 2024-09-21T02:17:58.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:57.596+0000 7f80e927d640 1 -- 172.21.15.170:0/4036878724 wait complete. 2024-09-21T02:17:58.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4deb892640 1 -- 172.21.15.170:0/2661126409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd0003a40 msgr2=0x7f4dd0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4deb892640 1 --2- 172.21.15.170:0/2661126409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd0003a40 0x7f4dd0003eb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400c290 tx=0x7f4dd401cf60 comp rx=0 tx=0).stop 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4de9e2f640 1 -- 172.21.15.170:0/2661126409 reap_dead start 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4deb892640 1 -- 172.21.15.170:0/2661126409 shutdown_connections 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4deb892640 1 -- 172.21.15.170:0/2661126409 >> 172.21.15.170:0/2661126409 conn(0x7f4de409a9c0 msgr2=0x7f4de40b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4deb892640 1 -- 172.21.15.170:0/2661126409 shutdown_connections 2024-09-21T02:17:58.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.620+0000 7f4deb892640 1 -- 172.21.15.170:0/2661126409 wait complete. 2024-09-21T02:17:58.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 Processor -- start 2024-09-21T02:17:58.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- start start 2024-09-21T02:17:58.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4dd0003a40 0x7f4de4154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:58.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4de41fb820 0x7f4de4163740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:58.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 0x7f4de4163c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:58.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4de4167920 con 0x7f4de41555f0 2024-09-21T02:17:58.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4de41677a0 con 0x7f4de41fb820 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4de4167aa0 con 0x7f4dd0003a40 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 0x7f4de4163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 0x7f4de4163c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54352/0 (socket says 172.21.15.170:54352) 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 -- 172.21.15.170:0/2974092023 learned_addr learned my addr 172.21.15.170:0/2974092023 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de962e640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4dd0003a40 0x7f4de4154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de8e2d640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4de41fb820 0x7f4de4163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 -- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4dd0003a40 msgr2=0x7f4de4154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4dd0003a40 0x7f4de4154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 -- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4de41fb820 msgr2=0x7f4de4163740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4de41fb820 0x7f4de4163740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 -- 172.21.15.170:0/2974092023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de41563a0 con 0x7f4de41555f0 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de8e2d640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4de41fb820 0x7f4de4163740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4de9e2f640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 0x7f4de4163c80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4de4075140 tx=0x7f4ddc00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc004b30 con 0x7f4de41555f0 2024-09-21T02:17:58.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de41553e0 con 0x7f4de41555f0 2024-09-21T02:17:58.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ddc004dc0 con 0x7f4de41555f0 2024-09-21T02:17:58.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00daf0 con 0x7f4de41555f0 2024-09-21T02:17:58.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de41fc250 con 0x7f4de41555f0 2024-09-21T02:17:58.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.624+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de40b0a80 con 0x7f4de41555f0 2024-09-21T02:17:58.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.628+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) ==== 97591+0+0 (secure 0 0 0) 0x7f4ddc01c020 con 0x7f4de41555f0 2024-09-21T02:17:58.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.628+0000 7f4dd9ffb640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4dc80735f0 0x7f4dc8075b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:58.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.628+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f4ddc094830 con 0x7f4de41555f0 2024-09-21T02:17:58.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.628+0000 7f4de962e640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4dc80735f0 0x7f4dc8075b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:58.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.628+0000 7f4de962e640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4dc80735f0 0x7f4dc8075b00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40ab000 tx=0x7f4dd4019da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:58.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.632+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4de41563a0 con 0x7f4de41555f0 2024-09-21T02:17:58.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.832+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7f4de406ef20 con 0x7f4de41555f0 2024-09-21T02:17:58.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.836+0000 7f4dd9ffb640 1 -- 172.21.15.170:0/2974092023 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7f4ddc05f0e0 con 0x7f4de41555f0 2024-09-21T02:17:58.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4dc80735f0 msgr2=0x7f4dc8075b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4dc80735f0 0x7f4dc8075b00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40ab000 tx=0x7f4dd4019da0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 msgr2=0x7f4de4163c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:58.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 0x7f4de4163c80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4de4075140 tx=0x7f4ddc00fbe0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 shutdown_connections 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:6800/2615148678,v1:172.21.15.191:6801/2615148678] conn(0x7f4dc80735f0 0x7f4dc8075b00 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40ab000 tx=0x7f4dd4019da0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4de41555f0 0x7f4de4163c80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4de41fb820 0x7f4de4163740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 --2- 172.21.15.170:0/2974092023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4dd0003a40 0x7f4de4154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.840+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 >> 172.21.15.170:0/2974092023 conn(0x7f4de409a9c0 msgr2=0x7f4de40718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.844+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 shutdown_connections 2024-09-21T02:17:58.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:58.844+0000 7f4deb892640 1 -- 172.21.15.170:0/2974092023 wait complete. 2024-09-21T02:17:58.851 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-09-21T02:17:58.850+0000 7f1abb6ec640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-21T02:17:58.880 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.949 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.950 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.950 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:58.950 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:17:58.948+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 -- 172.21.15.170:0/3948995087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 msgr2=0x7fad94070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 --2- 172.21.15.170:0/3948995087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94070340 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fad90009840 tx=0x7fad9001c820 comp rx=0 tx=0).stop 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 -- 172.21.15.170:0/3948995087 shutdown_connections 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 --2- 172.21.15.170:0/3948995087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94070340 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 -- 172.21.15.170:0/3948995087 >> 172.21.15.170:0/3948995087 conn(0x7fad940b1c40 msgr2=0x7fad940b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 -- 172.21.15.170:0/3948995087 shutdown_connections 2024-09-21T02:17:59.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 -- 172.21.15.170:0/3948995087 wait complete. 2024-09-21T02:17:59.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 Processor -- start 2024-09-21T02:17:59.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.052+0000 7fad9bdf5640 1 -- start start 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad940cb460 0x7fad94154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad940ce1c0 0x7fad941583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fad9416a330 con 0x7fad940cf7e0 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad9416a1b0 con 0x7fad940cb460 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fad9416a4b0 con 0x7fad940ce1c0 2024-09-21T02:17:59.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54444/0 (socket says 172.21.15.170:54444) 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 -- 172.21.15.170:0/30914515 learned_addr learned my addr 172.21.15.170:0/30914515 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad99b91640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad940cb460 0x7fad94154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad99390640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad940ce1c0 0x7fad941583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 -- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad940ce1c0 msgr2=0x7fad941583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad940ce1c0 0x7fad941583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 -- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad940cb460 msgr2=0x7fad94154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad940cb460 0x7fad94154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 -- 172.21.15.170:0/30914515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940af5b0 con 0x7fad940cf7e0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9a392640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94158910 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c730 tx=0x7fad8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c003b80 con 0x7fad940cf7e0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad8c003d20 con 0x7fad940cf7e0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad940b0330 con 0x7fad940cf7e0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c005330 con 0x7fad940cf7e0 2024-09-21T02:17:59.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9409db30 con 0x7fad940cf7e0 2024-09-21T02:17:59.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad94070340 con 0x7fad940cf7e0 2024-09-21T02:17:59.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 48655+0+0 (secure 0 0 0) 0x7fad8c0054d0 con 0x7fad940cf7e0 2024-09-21T02:17:59.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.056+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7fad8c04fd30 con 0x7fad940cf7e0 2024-09-21T02:17:59.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.060+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fad8c00ad50 con 0x7fad940cf7e0 2024-09-21T02:17:59.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.260+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad9415aee0 con 0x7fad940cf7e0 2024-09-21T02:17:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad8a7fc640 1 -- 172.21.15.170:0/30914515 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7fad8c010cf0 con 0x7fad940cf7e0 2024-09-21T02:17:59.265 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:17:59.266 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:17:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 msgr2=0x7fad94158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94158910 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00c730 tx=0x7fad8c00cbf0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 shutdown_connections 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad940ce1c0 0x7fad941583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad940cf7e0 0x7fad94158910 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 --2- 172.21.15.170:0/30914515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad940cb460 0x7fad94154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 >> 172.21.15.170:0/30914515 conn(0x7fad940b1c40 msgr2=0x7fad94155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 shutdown_connections 2024-09-21T02:17:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.264+0000 7fad9bdf5640 1 -- 172.21.15.170:0/30914515 wait complete. 2024-09-21T02:17:59.286 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T02:17:59.437 INFO:teuthology.orchestra.run.smithi170.stdout:[{"id":5617,"entity":{"name":{"type":"client","num":5617},"addr":{"type":"any","addr":"172.21.15.170:0","nonce":1013068182}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.471995163,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978570355891037,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5617 172.21.15.170:0/1013068182","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"y","hostname":"smithi170","pid":"28849","root":"/"}},{"id":5614,"entity":{"name":{"type":"client","num":5614},"addr":{"type":"any","addr":"172.21.15.170:0","nonce":3257667674}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.471995163,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978570355891037,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5614 172.21.15.170:0/3257667674","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"y","hostname":"smithi170","pid":"28849","root":"/"}}] 2024-09-21T02:17:59.437 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi170", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28849", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.170:0", "nonce": 1013068182, "type": "any" }, "name": { "num": 5617, "type": "client" } }, "id": 5617, "inst": "client.5617 172.21.15.170:0/1013068182", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978570355891037 }, "state": "open", "uptime": 0.471995163 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi170", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28849", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.170:0", "nonce": 3257667674, "type": "any" }, "name": { "num": 5614, "type": "client" } }, "id": 5614, "inst": "client.5614 172.21.15.170:0/3257667674", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978570355891037 }, "state": "open", "uptime": 0.471995163 } ] 2024-09-21T02:17:59.437 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T02:17:59.437 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:17:59.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:17:59.506 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:17:59.585 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:17:59.585 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:59.585 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:59.585 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:59.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:17:59.591 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:59.591 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:59.591 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:59.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:17:59.595 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:59.595 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:17:59.595 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:59.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:17:59.637 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:17:59.637 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:17:59.637 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:17:59.644 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:17:59.644 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:17:59.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddf48f640 1 -- 172.21.15.170:0/358461118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40051d0 msgr2=0x7f8dc400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/358461118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40051d0 0x7f8dc400b9d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009eb0 tx=0x7f8dd001d550 comp rx=0 tx=0).stop 2024-09-21T02:17:59.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddda2c640 1 -- 172.21.15.170:0/358461118 reap_dead start 2024-09-21T02:17:59.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddf48f640 1 -- 172.21.15.170:0/358461118 shutdown_connections 2024-09-21T02:17:59.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddf48f640 1 -- 172.21.15.170:0/358461118 >> 172.21.15.170:0/358461118 conn(0x7f8dd80b1c40 msgr2=0x7f8dd80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:17:59.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddf48f640 1 -- 172.21.15.170:0/358461118 shutdown_connections 2024-09-21T02:17:59.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.832+0000 7f8ddf48f640 1 -- 172.21.15.170:0/358461118 wait complete. 2024-09-21T02:17:59.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 Processor -- start 2024-09-21T02:17:59.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 -- start start 2024-09-21T02:17:59.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 0x7f8dd81f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 0x7f8dd8152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 0x7f8dd815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd80ceea0 con 0x7f8dd815e400 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd80ced20 con 0x7f8dd81f3e30 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd80cf020 con 0x7f8dd8156300 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddd22b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 0x7f8dd81f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddca2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 0x7f8dd8152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 0x7f8dd815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 0x7f8dd815e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54470/0 (socket says 172.21.15.170:54470) 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddca2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 0x7f8dd8152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:40622/0 (socket says 172.21.15.170:40622) 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 -- 172.21.15.170:0/10154598 learned_addr learned my addr 172.21.15.170:0/10154598 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddd22b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 0x7f8dd81f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:48828/0 (socket says 172.21.15.170:48828) 2024-09-21T02:17:59.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 -- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 msgr2=0x7f8dd81f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 0x7f8dd81f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 -- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 msgr2=0x7f8dd8152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 0x7f8dd8152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 -- 172.21.15.170:0/10154598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80b3f40 con 0x7f8dd815e400 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddd22b640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 0x7f8dd81f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddca2a640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 0x7f8dd8152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddda2c640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 0x7f8dd815e810 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009eb0 tx=0x7f8dd001bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0009ab0 con 0x7f8dd815e400 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dd00aa530 con 0x7f8dd815e400 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0009200 con 0x7f8dd815e400 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd809dca0 con 0x7f8dd815e400 2024-09-21T02:17:59.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.836+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd81f7940 con 0x7f8dd815e400 2024-09-21T02:17:59.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.840+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 48655+0+0 (secure 0 0 0) 0x7f8dd00aacf0 con 0x7f8dd815e400 2024-09-21T02:17:59.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.840+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd8152ca0 con 0x7f8dd815e400 2024-09-21T02:17:59.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.840+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7f8dd00eb600 con 0x7f8dd815e400 2024-09-21T02:17:59.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.844+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8dd00ae020 con 0x7f8dd815e400 2024-09-21T02:17:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.848+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 14) ==== 48782+0+0 (secure 0 0 0) 0x7f8dd0004030 con 0x7f8dd815e400 2024-09-21T02:17:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.848+0000 7f8dcdffb640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc0415e0 0x7f8dbc043a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:17:59.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.848+0000 7f8ddd22b640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc0415e0 0x7f8dbc043a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:17:59.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:17:59.852+0000 7f8ddd22b640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc0415e0 0x7f8dbc043a00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8dd815f130 tx=0x7f8dd4005d10 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.040+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dd8073c30 con 0x7f8dd815e400 2024-09-21T02:18:00.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8dcdffb640 1 -- 172.21.15.170:0/10154598 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f8dd0017260 con 0x7f8dd815e400 2024-09-21T02:18:00.045 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:00.046 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:18:00.046 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:18:00.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc0415e0 msgr2=0x7f8dbc043a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc0415e0 0x7f8dbc043a00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8dd815f130 tx=0x7f8dd4005d10 comp rx=0 tx=0).stop 2024-09-21T02:18:00.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 msgr2=0x7f8dd815e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 0x7f8dd815e810 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009eb0 tx=0x7f8dd001bd90 comp rx=0 tx=0).stop 2024-09-21T02:18:00.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 shutdown_connections 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc0415e0 0x7f8dbc043a00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd815e400 0x7f8dd815e810 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd81f3e30 0x7f8dd8152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 --2- 172.21.15.170:0/10154598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd8156300 0x7f8dd81f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 >> 172.21.15.170:0/10154598 conn(0x7f8dd80b1c40 msgr2=0x7f8dd809cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 shutdown_connections 2024-09-21T02:18:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.044+0000 7f8ddf48f640 1 -- 172.21.15.170:0/10154598 wait complete. 2024-09-21T02:18:00.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:00.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3558475003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160b50 msgr2=0x7fda640cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3558475003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160b50 0x7fda640cca40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fda640cc8b0 tx=0x7fda5c01c890 comp rx=0 tx=0).stop 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3558475003 shutdown_connections 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3558475003 >> v2:172.21.15.191:3300/0 conn(0x7fda64160f60 0x7fda640d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3558475003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160b50 0x7fda640cca40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3558475003 >> 172.21.15.170:0/3558475003 conn(0x7fda640711c0 msgr2=0x7fda640715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3558475003 shutdown_connections 2024-09-21T02:18:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.204+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3558475003 wait complete. 2024-09-21T02:18:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 Processor -- start 2024-09-21T02:18:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 -- start start 2024-09-21T02:18:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda640ccf80 0x7fda641479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda64160b50 0x7fda64155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 0x7fda64155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda6416a000 con 0x7fda64160f60 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fda64169e80 con 0x7fda640ccf80 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fda6416a180 con 0x7fda64160b50 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 0x7fda64155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 0x7fda64155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54500/0 (socket says 172.21.15.170:54500) 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 -- 172.21.15.170:0/3616026421 learned_addr learned my addr 172.21.15.170:0/3616026421 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda69ad6640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda640ccf80 0x7fda641479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda692d5640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda64160b50 0x7fda64155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 -- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda64160b50 msgr2=0x7fda64155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda64160b50 0x7fda64155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 -- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda640ccf80 msgr2=0x7fda641479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda640ccf80 0x7fda641479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 -- 172.21.15.170:0/3616026421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda64156890 con 0x7fda64160f60 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda69ad6640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda640ccf80 0x7fda641479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6a2d7640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 0x7fda64155f70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e970 tx=0x7fda6000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60004d30 con 0x7fda64160f60 2024-09-21T02:18:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda6000cb20 con 0x7fda64160f60 2024-09-21T02:18:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60005020 con 0x7fda64160f60 2024-09-21T02:18:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda64165470 con 0x7fda64160f60 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.208+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda64161850 con 0x7fda64160f60 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.212+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda641651e0 con 0x7fda64160f60 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.212+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 14) ==== 48782+0+0 (secure 0 0 0) 0x7fda600051c0 con 0x7fda64160f60 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.212+0000 7fda5a7fc640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda4803b9e0 0x7fda4803def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.212+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7fda60050a10 con 0x7fda64160f60 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.212+0000 7fda69ad6640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda4803b9e0 0x7fda4803def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.212+0000 7fda69ad6640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda4803b9e0 0x7fda4803def0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0af000 tx=0x7fda5c01b0d0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.216+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fda64156890 con 0x7fda64160f60 2024-09-21T02:18:00.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.408+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda640cca40 con 0x7fda64160f60 2024-09-21T02:18:00.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.408+0000 7fda5a7fc640 1 -- 172.21.15.170:0/3616026421 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7fda60016030 con 0x7fda64160f60 2024-09-21T02:18:00.411 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:00.411 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:18:00.411 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda4803b9e0 msgr2=0x7fda4803def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda4803b9e0 0x7fda4803def0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0af000 tx=0x7fda5c01b0d0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 msgr2=0x7fda64155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 0x7fda64155f70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e970 tx=0x7fda6000ee30 comp rx=0 tx=0).stop 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 shutdown_connections 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda4803b9e0 0x7fda4803def0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda64160b50 0x7fda64155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda64160f60 0x7fda64155f70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 --2- 172.21.15.170:0/3616026421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda640ccf80 0x7fda641479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 >> 172.21.15.170:0/3616026421 conn(0x7fda640711c0 msgr2=0x7fda640b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:00.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 shutdown_connections 2024-09-21T02:18:00.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.412+0000 7fda6bd3a640 1 -- 172.21.15.170:0/3616026421 wait complete. 2024-09-21T02:18:00.431 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 -- 172.21.15.170:0/3684795277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c004d50 msgr2=0x7fb04c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 --2- 172.21.15.170:0/3684795277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c004d50 0x7fb04c005160 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009eb0 tx=0x7fb05c01d5f0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb06359e640 1 -- 172.21.15.170:0/3684795277 reap_dead start 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 -- 172.21.15.170:0/3684795277 shutdown_connections 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 -- 172.21.15.170:0/3684795277 >> 172.21.15.170:0/3684795277 conn(0x7fb0640b1c40 msgr2=0x7fb0640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 -- 172.21.15.170:0/3684795277 shutdown_connections 2024-09-21T02:18:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 -- 172.21.15.170:0/3684795277 wait complete. 2024-09-21T02:18:00.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 Processor -- start 2024-09-21T02:18:00.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.568+0000 7fb0690d7640 1 -- start start 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c004d50 0x7fb0641f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 0x7fb06414ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0640c8030 0x7fb064156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0640cb9f0 con 0x7fb06414d3e0 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0640cb870 con 0x7fb0640c8030 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb0640cbb70 con 0x7fb04c004d50 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 0x7fb06414ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 0x7fb06414ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54512/0 (socket says 172.21.15.170:54512) 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 -- 172.21.15.170:0/2783695523 learned_addr learned my addr 172.21.15.170:0/2783695523 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb062d9d640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c004d50 0x7fb0641f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06359e640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0640c8030 0x7fb064156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 -- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c004d50 msgr2=0x7fb0641f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c004d50 0x7fb0641f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 -- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0640c8030 msgr2=0x7fb064156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0640c8030 0x7fb064156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 -- 172.21.15.170:0/2783695523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb064075df0 con 0x7fb06414d3e0 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb06259c640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 0x7fb06414ac50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb058011a80 tx=0x7fb058011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800fe10 con 0x7fb06414d3e0 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb058016070 con 0x7fb06414d3e0 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058012780 con 0x7fb06414d3e0 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0640c8440 con 0x7fb06414d3e0 2024-09-21T02:18:00.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06414dea0 con 0x7fb06414d3e0 2024-09-21T02:18:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06414d930 con 0x7fb06414d3e0 2024-09-21T02:18:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 14) ==== 48782+0+0 (secure 0 0 0) 0x7fb058018030 con 0x7fb06414d3e0 2024-09-21T02:18:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb04b7fe640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb05003b9b0 0x7fb05003dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7fb058050b50 con 0x7fb06414d3e0 2024-09-21T02:18:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb062d9d640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb05003b9b0 0x7fb05003dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.572+0000 7fb062d9d640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb05003b9b0 0x7fb05003dec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb054001c10 tx=0x7fb054001d80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.580+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb058021a00 con 0x7fb06414d3e0 2024-09-21T02:18:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.740+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0640ce5d0 con 0x7fb06414d3e0 2024-09-21T02:18:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb04b7fe640 1 -- 172.21.15.170:0/2783695523 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17539 (secure 0 0 0) 0x7fb058021310 con 0x7fb06414d3e0 2024-09-21T02:18:00.746 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:00.746 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":73,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:58.836533+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:17:27.202813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T02:17:28.049434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:0/2357730026":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/2838563242":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3514032572":"2024-09-22T02:17:58.836483+0000","172.21.15.191:6801/2615148678":"2024-09-22T02:17:58.836483+0000","172.21.15.191:6800/2615148678":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3689562231":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3826872798":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1634809445":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1852913239":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/2890432817":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/543979301":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3207724562":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1074802945":"2024-09-22T02:17:58.836483+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb05003b9b0 msgr2=0x7fb05003dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb05003b9b0 0x7fb05003dec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb054001c10 tx=0x7fb054001d80 comp rx=0 tx=0).stop 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 msgr2=0x7fb06414ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 0x7fb06414ac50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb058011a80 tx=0x7fb058011f40 comp rx=0 tx=0).stop 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 shutdown_connections 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb05003b9b0 0x7fb05003dec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0640c8030 0x7fb064156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb06414d3e0 0x7fb06414ac50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 --2- 172.21.15.170:0/2783695523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c004d50 0x7fb0641f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 >> 172.21.15.170:0/2783695523 conn(0x7fb0640b1c40 msgr2=0x7fb0640adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 shutdown_connections 2024-09-21T02:18:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.744+0000 7fb0690d7640 1 -- 172.21.15.170:0/2783695523 wait complete. 2024-09-21T02:18:00.767 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:18:00.768 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:00.887 INFO:tasks.ceph.mgr.x.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T02:18:00.887 INFO:tasks.ceph.mgr.x.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T02:18:00.887 INFO:tasks.ceph.mgr.x.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 -- 172.21.15.170:0/3393336952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 msgr2=0x7f18180ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3393336952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f18180ce4c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1818074c60 tx=0x7f180c01c890 comp rx=0 tx=0).stop 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 -- 172.21.15.170:0/3393336952 shutdown_connections 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3393336952 >> v2:172.21.15.191:3300/0 conn(0x7f18180cea00 0x7f18180cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3393336952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f18180ce4c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3393336952 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181815f990 0x7f1818070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 -- 172.21.15.170:0/3393336952 >> 172.21.15.170:0/3393336952 conn(0x7f18180b1c40 msgr2=0x7f18180b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 -- 172.21.15.170:0/3393336952 shutdown_connections 2024-09-21T02:18:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 -- 172.21.15.170:0/3393336952 wait complete. 2024-09-21T02:18:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 Processor -- start 2024-09-21T02:18:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.912+0000 7f181d4ef640 1 -- start start 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f1818156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18180cea00 0x7f181815ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181815f990 0x7f181815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1818073ae0 con 0x7f1818070570 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1818073960 con 0x7f18180cea00 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1818073c60 con 0x7f181815f990 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f1818156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f1818156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54536/0 (socket says 172.21.15.170:54536) 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 -- 172.21.15.170:0/3432091560 learned_addr learned my addr 172.21.15.170:0/3432091560 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:00.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f18167fc640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18180cea00 0x7f181815ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f18177fe640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181815f990 0x7f181815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 -- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181815f990 msgr2=0x7f181815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181815f990 0x7f181815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 -- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18180cea00 msgr2=0x7f181815ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18180cea00 0x7f181815ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 -- 172.21.15.170:0/3432091560 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1818156c00 con 0x7f1818070570 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f1816ffd640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f1818156100 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f180800e7a0 tx=0x7f180800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180800ccc0 con 0x7f1818070570 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1808010430 con 0x7f1818070570 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18180b4490 con 0x7f1818070570 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808010860 con 0x7f1818070570 2024-09-21T02:18:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18180b4a50 con 0x7f1818070570 2024-09-21T02:18:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18180767c0 con 0x7f1818070570 2024-09-21T02:18:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 14) ==== 48782+0+0 (secure 0 0 0) 0x7f1808010a00 con 0x7f1818070570 2024-09-21T02:18:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f17ff7fe640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f180403b9e0 0x7f180403def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7f1808050af0 con 0x7f1818070570 2024-09-21T02:18:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f18167fc640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f180403b9e0 0x7f180403def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.916+0000 7f18167fc640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f180403b9e0 0x7f180403def0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f180c0af000 tx=0x7f180c01b0d0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:00.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:00.920+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1808018d80 con 0x7f1818070570 2024-09-21T02:18:01.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.120+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 15) ==== 48990+0+0 (secure 0 0 0) 0x7f1808010cf0 con 0x7f1818070570 2024-09-21T02:18:01.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.132+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1818073e40 con 0x7f1818070570 2024-09-21T02:18:01.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.132+0000 7f17ff7fe640 1 -- 172.21.15.170:0/3432091560 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f1808053720 con 0x7f1818070570 2024-09-21T02:18:01.137 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:01.137 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:18:01.137 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f180403b9e0 msgr2=0x7f180403def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f180403b9e0 0x7f180403def0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f180c0af000 tx=0x7f180c01b0d0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 msgr2=0x7f1818156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f1818156100 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f180800e7a0 tx=0x7f180800ec60 comp rx=0 tx=0).stop 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 shutdown_connections 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f180403b9e0 0x7f180403def0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181815f990 0x7f181815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18180cea00 0x7f181815ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 --2- 172.21.15.170:0/3432091560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1818070570 0x7f1818156100 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 >> 172.21.15.170:0/3432091560 conn(0x7f18180b1c40 msgr2=0x7f18180ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:01.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 shutdown_connections 2024-09-21T02:18:01.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.136+0000 7f181d4ef640 1 -- 172.21.15.170:0/3432091560 wait complete. 2024-09-21T02:18:01.157 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:01.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 -- 172.21.15.170:0/2091956105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b6930 msgr2=0x7f88cc06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 --2- 172.21.15.170:0/2091956105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b6930 0x7f88cc06ce00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009870 tx=0x7f88c001c820 comp rx=0 tx=0).stop 2024-09-21T02:18:01.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 -- 172.21.15.170:0/2091956105 shutdown_connections 2024-09-21T02:18:01.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 --2- 172.21.15.170:0/2091956105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b6930 0x7f88cc06ce00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 --2- 172.21.15.170:0/2091956105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88cc0b22b0 0x7f88cc0b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 -- 172.21.15.170:0/2091956105 >> 172.21.15.170:0/2091956105 conn(0x7f88cc09a9c0 msgr2=0x7f88cc0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:01.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 -- 172.21.15.170:0/2091956105 shutdown_connections 2024-09-21T02:18:01.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.304+0000 7f88d1d28640 1 -- 172.21.15.170:0/2091956105 wait complete. 2024-09-21T02:18:01.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 Processor -- start 2024-09-21T02:18:01.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 -- start start 2024-09-21T02:18:01.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88cc06d340 0x7f88cc0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b22b0 0x7f88cc0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 0x7f88cc1f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc15c4d0 con 0x7f88cc0b6930 2024-09-21T02:18:01.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc15c350 con 0x7f88cc0b22b0 2024-09-21T02:18:01.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc15c650 con 0x7f88cc06d340 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 0x7f88cc1f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 0x7f88cc1f9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54548/0 (socket says 172.21.15.170:54548) 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 -- 172.21.15.170:0/3059591299 learned_addr learned my addr 172.21.15.170:0/3059591299 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88caffd640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b22b0 0x7f88cc0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cb7fe640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88cc06d340 0x7f88cc0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 -- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88cc06d340 msgr2=0x7f88cc0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88cc06d340 0x7f88cc0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 -- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b22b0 msgr2=0x7f88cc0c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b22b0 0x7f88cc0c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 -- 172.21.15.170:0/3059591299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc1faa30 con 0x7f88cc0b6930 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88cbfff640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 0x7f88cc1f9cf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f88c400ca50 tx=0x7f88c400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4016020 con 0x7f88cc0b6930 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c400a930 con 0x7f88cc0b6930 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc0ca7f0 con 0x7f88cc0b6930 2024-09-21T02:18:01.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c400ad80 con 0x7f88cc0b6930 2024-09-21T02:18:01.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.308+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc0cb2d0 con 0x7f88cc0b6930 2024-09-21T02:18:01.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.312+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc15cb20 con 0x7f88cc0b6930 2024-09-21T02:18:01.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.312+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 48990+0+0 (secure 0 0 0) 0x7f88c4019020 con 0x7f88cc0b6930 2024-09-21T02:18:01.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.312+0000 7f88b3fff640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f88b803bc20 0x7f88b803e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.312+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7f88c40506c0 con 0x7f88cc0b6930 2024-09-21T02:18:01.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.312+0000 7f88cb7fe640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f88b803bc20 0x7f88b803e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.312+0000 7f88cb7fe640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f88b803bc20 0x7f88b803e130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005fd0 tx=0x7f88bc005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:01.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.316+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f88c4020020 con 0x7f88cc0b6930 2024-09-21T02:18:01.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.524+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88cc06ef20 con 0x7f88cc0b6930 2024-09-21T02:18:01.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.524+0000 7f88b3fff640 1 -- 172.21.15.170:0/3059591299 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5237 (secure 0 0 0) 0x7f88cc06d750 con 0x7f88cc0b6930 2024-09-21T02:18:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:01.527 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"btime":"2024-09-21T02:17:47:040540+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:17:29.085709+0000","modified":"2024-09-21T02:17:32.143480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6270":{"gid":6270,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1298680671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1298680671},{"type":"v1","addr":"172.21.15.170:6835","nonce":1298680671}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6270,"qdb_cluster":[6270]},"id":5}]} 2024-09-21T02:18:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f88b803bc20 msgr2=0x7f88b803e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f88b803bc20 0x7f88b803e130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005fd0 tx=0x7f88bc005f40 comp rx=0 tx=0).stop 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 msgr2=0x7f88cc1f9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 0x7f88cc1f9cf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f88c400ca50 tx=0x7f88c400cb40 comp rx=0 tx=0).stop 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 shutdown_connections 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f88b803bc20 0x7f88b803e130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88cc0b6930 0x7f88cc1f9cf0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88cc0b22b0 0x7f88cc0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 --2- 172.21.15.170:0/3059591299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88cc06d340 0x7f88cc0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 >> 172.21.15.170:0/3059591299 conn(0x7f88cc09a9c0 msgr2=0x7f88cc0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 shutdown_connections 2024-09-21T02:18:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.528+0000 7f88d1d28640 1 -- 172.21.15.170:0/3059591299 wait complete. 2024-09-21T02:18:01.550 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 -- 172.21.15.170:0/182990357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a080b9ba0 msgr2=0x7f4a080d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/182990357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a080b9ba0 0x7f4a080d4e00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4a0809de30 tx=0x7f4a0401c890 comp rx=0 tx=0).stop 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 -- 172.21.15.170:0/182990357 shutdown_connections 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/182990357 >> v2:172.21.15.191:3300/0 conn(0x7f4a080b6600 0x7f4a080cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/182990357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a080b9ba0 0x7f4a080d4e00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 -- 172.21.15.170:0/182990357 >> 172.21.15.170:0/182990357 conn(0x7f4a0809d7d0 msgr2=0x7f4a080d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 -- 172.21.15.170:0/182990357 shutdown_connections 2024-09-21T02:18:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.696+0000 7f4a0f44f640 1 -- 172.21.15.170:0/182990357 wait complete. 2024-09-21T02:18:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 Processor -- start 2024-09-21T02:18:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 -- start start 2024-09-21T02:18:01.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 0x7f4a080cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080b9ba0 0x7f4a0814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0815a750 0x7f4a0815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0816c6e0 con 0x7f4a0806dd80 2024-09-21T02:18:01.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0816c560 con 0x7f4a080b9ba0 2024-09-21T02:18:01.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0816c860 con 0x7f4a0815a750 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 0x7f4a080cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 0x7f4a080cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54568/0 (socket says 172.21.15.170:54568) 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 -- 172.21.15.170:0/2369782429 learned_addr learned my addr 172.21.15.170:0/2369782429 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0c9ea640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080b9ba0 0x7f4a0814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d9ec640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0815a750 0x7f4a0815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 -- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0815a750 msgr2=0x7f4a0815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0815a750 0x7f4a0815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 -- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080b9ba0 msgr2=0x7f4a0814ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080b9ba0 0x7f4a0814ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 -- 172.21.15.170:0/2369782429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0815b0a0 con 0x7f4a0806dd80 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d9ec640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0815a750 0x7f4a0815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0d1eb640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 0x7f4a080cdef0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009850 tx=0x7f4a04004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040bc070 con 0x7f4a0806dd80 2024-09-21T02:18:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a04005320 con 0x7f4a0806dd80 2024-09-21T02:18:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0815eaa0 con 0x7f4a0806dd80 2024-09-21T02:18:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040b2040 con 0x7f4a0806dd80 2024-09-21T02:18:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.700+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a080cd6e0 con 0x7f4a0806dd80 2024-09-21T02:18:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.704+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a080cd920 con 0x7f4a0806dd80 2024-09-21T02:18:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.704+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 48990+0+0 (secure 0 0 0) 0x7f4a040046d0 con 0x7f4a0806dd80 2024-09-21T02:18:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.704+0000 7f49fdffb640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f49ec03bbf0 0x7f49ec03e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.704+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7f4a040ea470 con 0x7f4a0806dd80 2024-09-21T02:18:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.704+0000 7f4a0c9ea640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f49ec03bbf0 0x7f49ec03e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.704+0000 7f4a0c9ea640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f49ec03bbf0 0x7f49ec03e100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a080d1e40 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:01.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.708+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a080cd920 con 0x7f4a0806dd80 2024-09-21T02:18:01.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.872+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a080cfc60 con 0x7f4a0806dd80 2024-09-21T02:18:01.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f49fdffb640 1 -- 172.21.15.170:0/2369782429 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17539 (secure 0 0 0) 0x7f4a0815b0a0 con 0x7f4a0806dd80 2024-09-21T02:18:01.877 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:01.877 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":73,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:17:58.836533+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:17:27.202813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T02:17:28.049434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:0/2357730026":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/2838563242":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3514032572":"2024-09-22T02:17:58.836483+0000","172.21.15.191:6801/2615148678":"2024-09-22T02:17:58.836483+0000","172.21.15.191:6800/2615148678":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3689562231":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3826872798":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1634809445":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1852913239":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/2890432817":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/543979301":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3207724562":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1074802945":"2024-09-22T02:17:58.836483+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f49ec03bbf0 msgr2=0x7f49ec03e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f49ec03bbf0 0x7f49ec03e100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a080d1e40 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 msgr2=0x7f4a080cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 0x7f4a080cdef0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009850 tx=0x7f4a04004600 comp rx=0 tx=0).stop 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 shutdown_connections 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f49ec03bbf0 0x7f49ec03e100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a0815a750 0x7f4a0815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a080b9ba0 0x7f4a0814ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 --2- 172.21.15.170:0/2369782429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806dd80 0x7f4a080cdef0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 >> 172.21.15.170:0/2369782429 conn(0x7f4a0809d7d0 msgr2=0x7f4a080b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 shutdown_connections 2024-09-21T02:18:01.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:01.876+0000 7f4a0f44f640 1 -- 172.21.15.170:0/2369782429 wait complete. 2024-09-21T02:18:01.898 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:18:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 -- 172.21.15.170:0/971083907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b22b0 msgr2=0x7fd0640b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 --2- 172.21.15.170:0/971083907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b22b0 0x7fd0640b1730 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd064073a30 tx=0x7fd05c01c880 comp rx=0 tx=0).stop 2024-09-21T02:18:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 -- 172.21.15.170:0/971083907 shutdown_connections 2024-09-21T02:18:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 --2- 172.21.15.170:0/971083907 >> v2:172.21.15.191:3300/0 conn(0x7fd06406d510 0x7fd064160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 --2- 172.21.15.170:0/971083907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b22b0 0x7fd0640b1730 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 -- 172.21.15.170:0/971083907 >> 172.21.15.170:0/971083907 conn(0x7fd06409a9c0 msgr2=0x7fd0640d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:02.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 -- 172.21.15.170:0/971083907 shutdown_connections 2024-09-21T02:18:02.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.048+0000 7fd06a99e640 1 -- 172.21.15.170:0/971083907 wait complete. 2024-09-21T02:18:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 Processor -- start 2024-09-21T02:18:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 -- start start 2024-09-21T02:18:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0640b22b0 0x7fd064167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 0x7fd0641f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0641595b0 0x7fd0641599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd06415c620 con 0x7fd0640b6930 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd06415c4a0 con 0x7fd0641595b0 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd06415c7a0 con 0x7fd0640b22b0 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 0x7fd0641f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 0x7fd0641f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54580/0 (socket says 172.21.15.170:54580) 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 -- 172.21.15.170:0/1442124096 learned_addr learned my addr 172.21.15.170:0/1442124096 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd068f3b640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0641595b0 0x7fd0641599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd063fff640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0640b22b0 0x7fd064167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 -- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0640b22b0 msgr2=0x7fd064167dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0640b22b0 0x7fd064167dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 -- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0641595b0 msgr2=0x7fd0641599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0641595b0 0x7fd0641599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 -- 172.21.15.170:0/1442124096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd064159f00 con 0x7fd0640b6930 2024-09-21T02:18:02.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd0637fe640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 0x7fd0641f88e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00ba80 tx=0x7fd04c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:02.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00cc00 con 0x7fd0640b6930 2024-09-21T02:18:02.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd04c003980 con 0x7fd0640b6930 2024-09-21T02:18:02.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0640aef60 con 0x7fd0640b6930 2024-09-21T02:18:02.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.052+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c003e10 con 0x7fd0640b6930 2024-09-21T02:18:02.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0640af520 con 0x7fd0640b6930 2024-09-21T02:18:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd064160390 con 0x7fd0640b6930 2024-09-21T02:18:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 48990+0+0 (secure 0 0 0) 0x7fd04c006d10 con 0x7fd0640b6930 2024-09-21T02:18:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd060ff9640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd04403bbf0 0x7fd04403e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6268+0+0 (secure 0 0 0) 0x7fd04c050280 con 0x7fd0640b6930 2024-09-21T02:18:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd063fff640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd04403bbf0 0x7fd04403e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.056+0000 7fd063fff640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd04403bbf0 0x7fd04403e100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0af000 tx=0x7fd05c01b0c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:02.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.060+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd064159f00 con 0x7fd0640b6930 2024-09-21T02:18:02.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.272+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd06406b1d0 con 0x7fd0640b6930 2024-09-21T02:18:02.275 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:18:02.272+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:18:02.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.276+0000 7fd060ff9640 1 -- 172.21.15.170:0/1442124096 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7fd04c013db0 con 0x7fd0640b6930 2024-09-21T02:18:02.278 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:18:02.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd04403bbf0 msgr2=0x7fd04403e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd04403bbf0 0x7fd04403e100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0af000 tx=0x7fd05c01b0c0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 msgr2=0x7fd0641f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 0x7fd0641f88e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00ba80 tx=0x7fd04c00bf40 comp rx=0 tx=0).stop 2024-09-21T02:18:02.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 shutdown_connections 2024-09-21T02:18:02.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd04403bbf0 0x7fd04403e100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0641595b0 0x7fd0641599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640b6930 0x7fd0641f88e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 --2- 172.21.15.170:0/1442124096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0640b22b0 0x7fd064167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.280+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 >> 172.21.15.170:0/1442124096 conn(0x7fd06409a9c0 msgr2=0x7fd0640b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:02.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.284+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 shutdown_connections 2024-09-21T02:18:02.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.284+0000 7fd06a99e640 1 -- 172.21.15.170:0/1442124096 wait complete. 2024-09-21T02:18:02.291 INFO:tasks.ceph.mds.c.smithi170.stderr: -7568> 2024-09-21T02:17:35.636+0000 7f2d8f1aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:18:02.305 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:18:02.371 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 -- 172.21.15.170:0/12246989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6c003a60 msgr2=0x7fcb6c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 --2- 172.21.15.170:0/12246989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6c003a60 0x7fcb6c003ed0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcb700016f0 tx=0x7fcb7001c790 comp rx=0 tx=0).stop 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 -- 172.21.15.170:0/12246989 shutdown_connections 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 --2- 172.21.15.170:0/12246989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6c003a60 0x7fcb6c003ed0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 -- 172.21.15.170:0/12246989 >> 172.21.15.170:0/12246989 conn(0x7fcb800cd5d0 msgr2=0x7fcb800cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 -- 172.21.15.170:0/12246989 shutdown_connections 2024-09-21T02:18:02.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 -- 172.21.15.170:0/12246989 wait complete. 2024-09-21T02:18:02.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.456+0000 7fcb85b25640 1 Processor -- start 2024-09-21T02:18:02.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- start start 2024-09-21T02:18:02.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb6c003a60 0x7fcb801575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb80157c70 0x7fcb800ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 0x7fcb800b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb800cc7c0 con 0x7fcb800aedd0 2024-09-21T02:18:02.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb800cc640 con 0x7fcb80157c70 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcb800cc940 con 0x7fcb6c003a60 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 0x7fcb800b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 0x7fcb800b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54608/0 (socket says 172.21.15.170:54608) 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 -- 172.21.15.170:0/2772911244 learned_addr learned my addr 172.21.15.170:0/2772911244 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7f7fe640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb6c003a60 0x7fcb801575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb77fff640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb80157c70 0x7fcb800ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 -- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb6c003a60 msgr2=0x7fcb801575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb6c003a60 0x7fcb801575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 -- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb80157c70 msgr2=0x7fcb800ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb80157c70 0x7fcb800ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 -- 172.21.15.170:0/2772911244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb800af1e0 con 0x7fcb800aedd0 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb77fff640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb80157c70 0x7fcb800ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7f7fe640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb6c003a60 0x7fcb801575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:02.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7ffff640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 0x7fcb800b37b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcb80069410 tx=0x7fcb7800eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78015070 con 0x7fcb800aedd0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb7800ca60 con 0x7fcb800aedd0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb800b4860 con 0x7fcb800aedd0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78011460 con 0x7fcb800aedd0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb800b4b00 con 0x7fcb800aedd0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 48990+0+0 (secure 0 0 0) 0x7fcb78018070 con 0x7fcb800aedd0 2024-09-21T02:18:02.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7cff9640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb6003bbf0 0x7fcb6003e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:02.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb80159510 con 0x7fcb800aedd0 2024-09-21T02:18:02.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6354+0+0 (secure 0 0 0) 0x7fcb78050bf0 con 0x7fcb800aedd0 2024-09-21T02:18:02.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7f7fe640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb6003bbf0 0x7fcb6003e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:02.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.460+0000 7fcb7f7fe640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb6003bbf0 0x7fcb6003e100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb700016c0 tx=0x7fcb70019580 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.468+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcb800af1e0 con 0x7fcb800aedd0 2024-09-21T02:18:02.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:02.680+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcb800737c0 con 0x7fcb800aedd0 2024-09-21T02:18:03.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.296+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fcb78040d10 con 0x7fcb800aedd0 2024-09-21T02:18:03.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.300+0000 7fcb7cff9640 1 -- 172.21.15.170:0/2772911244 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7fcb780620e0 con 0x7fcb800aedd0 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb6003bbf0 msgr2=0x7fcb6003e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb6003bbf0 0x7fcb6003e100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb700016c0 tx=0x7fcb70019580 comp rx=0 tx=0).stop 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 msgr2=0x7fcb800b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 0x7fcb800b37b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcb80069410 tx=0x7fcb7800eae0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 shutdown_connections 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb6003bbf0 0x7fcb6003e100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb800aedd0 0x7fcb800b37b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb80157c70 0x7fcb800ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 --2- 172.21.15.170:0/2772911244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb6c003a60 0x7fcb801575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 >> 172.21.15.170:0/2772911244 conn(0x7fcb800cd5d0 msgr2=0x7fcb800cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:03.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 shutdown_connections 2024-09-21T02:18:03.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.304+0000 7fcb85b25640 1 -- 172.21.15.170:0/2772911244 wait complete. 2024-09-21T02:18:03.337 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:18:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 -- 172.21.15.170:0/3338671166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda200ca720 msgr2=0x7fda200cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 --2- 172.21.15.170:0/3338671166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda200ca720 0x7fda200cab30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda1400a950 tx=0x7fda1401e5d0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 -- 172.21.15.170:0/3338671166 shutdown_connections 2024-09-21T02:18:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 --2- 172.21.15.170:0/3338671166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda200ca720 0x7fda200cab30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 --2- 172.21.15.170:0/3338671166 >> v2:172.21.15.191:3300/0 conn(0x7fda2009dc00 0x7fda2015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 -- 172.21.15.170:0/3338671166 >> 172.21.15.170:0/3338671166 conn(0x7fda200b1c40 msgr2=0x7fda200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:03.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 -- 172.21.15.170:0/3338671166 shutdown_connections 2024-09-21T02:18:03.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.492+0000 7fda25083640 1 -- 172.21.15.170:0/3338671166 wait complete. 2024-09-21T02:18:03.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 Processor -- start 2024-09-21T02:18:03.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 -- start start 2024-09-21T02:18:03.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 0x7fda201540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:03.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda200ca720 0x7fda200b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:03.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2015e9f0 0x7fda20165140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:03.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda20168480 con 0x7fda2009dc00 2024-09-21T02:18:03.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fda20168300 con 0x7fda200ca720 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fda20168600 con 0x7fda2015e9f0 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 0x7fda201540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 0x7fda201540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54612/0 (socket says 172.21.15.170:54612) 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 -- 172.21.15.170:0/3243178673 learned_addr learned my addr 172.21.15.170:0/3243178673 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1e59c640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda200ca720 0x7fda200b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1f59e640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2015e9f0 0x7fda20165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 -- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2015e9f0 msgr2=0x7fda20165140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2015e9f0 0x7fda20165140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 -- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda200ca720 msgr2=0x7fda200b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda200ca720 0x7fda200b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 -- 172.21.15.170:0/3243178673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda200b4180 con 0x7fda2009dc00 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1e59c640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda200ca720 0x7fda200b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:03.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda1ed9d640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 0x7fda201540c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fda1000e9f0 tx=0x7fda1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:03.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1000cc60 con 0x7fda2009dc00 2024-09-21T02:18:03.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda201f8600 con 0x7fda2009dc00 2024-09-21T02:18:03.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda10010430 con 0x7fda2009dc00 2024-09-21T02:18:03.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10010880 con 0x7fda2009dc00 2024-09-21T02:18:03.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.496+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda201f8bc0 con 0x7fda2009dc00 2024-09-21T02:18:03.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.500+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2015ad30 con 0x7fda2009dc00 2024-09-21T02:18:03.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.500+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fda10010a20 con 0x7fda2009dc00 2024-09-21T02:18:03.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.500+0000 7fda077fe640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda0c073350 0x7fda0c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:03.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.500+0000 7fda1e59c640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda0c073350 0x7fda0c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:03.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.500+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6354+0+0 (secure 0 0 0) 0x7fda100952d0 con 0x7fda2009dc00 2024-09-21T02:18:03.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.500+0000 7fda1e59c640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda0c073350 0x7fda0c075860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fda140b0000 tx=0x7fda1401c3b0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:03.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.504+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fda1005fdc0 con 0x7fda2009dc00 2024-09-21T02:18:03.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:03.664+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda200cb3c0 con 0x7fda2009dc00 2024-09-21T02:18:04.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.300+0000 7fda077fe640 1 -- 172.21.15.170:0/3243178673 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) ==== 181+0+0 (secure 0 0 0) 0x7fda1005f6c0 con 0x7fda2009dc00 2024-09-21T02:18:04.304 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:18:04.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda0c073350 msgr2=0x7fda0c075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:04.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda0c073350 0x7fda0c075860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fda140b0000 tx=0x7fda1401c3b0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 msgr2=0x7fda201540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 0x7fda201540c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fda1000e9f0 tx=0x7fda1000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 shutdown_connections 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda0c073350 0x7fda0c075860 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda2015e9f0 0x7fda20165140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda200ca720 0x7fda200b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 --2- 172.21.15.170:0/3243178673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda2009dc00 0x7fda201540c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 >> 172.21.15.170:0/3243178673 conn(0x7fda200b1c40 msgr2=0x7fda2015ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:04.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 shutdown_connections 2024-09-21T02:18:04.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.308+0000 7fda25083640 1 -- 172.21.15.170:0/3243178673 wait complete. 2024-09-21T02:18:04.334 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f7cef6640 1 -- 172.21.15.170:0/1046538615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f60003a40 msgr2=0x7f4f60003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/1046538615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f60003a40 0x7f4f60003eb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4f680016f0 tx=0x7f4f6801c7b0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f76d9d640 1 -- 172.21.15.170:0/1046538615 reap_dead start 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f7cef6640 1 -- 172.21.15.170:0/1046538615 shutdown_connections 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f7cef6640 1 -- 172.21.15.170:0/1046538615 >> 172.21.15.170:0/1046538615 conn(0x7f4f780ced80 msgr2=0x7f4f780cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f7cef6640 1 -- 172.21.15.170:0/1046538615 shutdown_connections 2024-09-21T02:18:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.496+0000 7f4f7cef6640 1 -- 172.21.15.170:0/1046538615 wait complete. 2024-09-21T02:18:04.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 Processor -- start 2024-09-21T02:18:04.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- start start 2024-09-21T02:18:04.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f60003a40 0x7f4f780b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:04.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f78147c30 0x7f4f78147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:04.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 0x7f4f780caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4f780d05e0 con 0x7f4f780ca6c0 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f780d0460 con 0x7f4f60003a40 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4f780d0760 con 0x7f4f78147c30 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 0x7f4f780caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 0x7f4f780caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54632/0 (socket says 172.21.15.170:54632) 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 -- 172.21.15.170:0/2238835466 learned_addr learned my addr 172.21.15.170:0/2238835466 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7659c640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f60003a40 0x7f4f780b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f75d9b640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f78147c30 0x7f4f78147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 -- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f78147c30 msgr2=0x7f4f78147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f78147c30 0x7f4f78147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 -- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f60003a40 msgr2=0x7f4f780b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f60003a40 0x7f4f780b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 -- 172.21.15.170:0/2238835466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f780b27a0 con 0x7f4f780ca6c0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f76d9d640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 0x7f4f780caad0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f78074f60 tx=0x7f4f70010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70017070 con 0x7f4f780ca6c0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f70015070 con 0x7f4f780ca6c0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f780b2a80 con 0x7f4f780ca6c0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70004d90 con 0x7f4f780ca6c0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f780b3040 con 0x7f4f780ca6c0 2024-09-21T02:18:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.500+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f78159240 con 0x7f4f780ca6c0 2024-09-21T02:18:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.504+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f4f70003980 con 0x7f4f780ca6c0 2024-09-21T02:18:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.504+0000 7f4f66ffd640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4f58073350 0x7f4f58075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.504+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5916+0+0 (secure 0 0 0) 0x7f4f700949e0 con 0x7f4f780ca6c0 2024-09-21T02:18:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.504+0000 7f4f7659c640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4f58073350 0x7f4f58075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.504+0000 7f4f7659c640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4f58073350 0x7f4f58075860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f680016c0 tx=0x7f4f680195a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:04.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.512+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4f7005f7a0 con 0x7f4f780ca6c0 2024-09-21T02:18:04.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:04.684+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4f780b2530 con 0x7f4f780ca6c0 2024-09-21T02:18:05.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.312+0000 7f4f66ffd640 1 -- 172.21.15.170:0/2238835466 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) ==== 169+0+0 (secure 0 0 0) 0x7f4f7005f0a0 con 0x7f4f780ca6c0 2024-09-21T02:18:05.314 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:18:05.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4f58073350 msgr2=0x7f4f58075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4f58073350 0x7f4f58075860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f680016c0 tx=0x7f4f680195a0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 msgr2=0x7f4f780caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 0x7f4f780caad0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f78074f60 tx=0x7f4f70010e70 comp rx=0 tx=0).stop 2024-09-21T02:18:05.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 shutdown_connections 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4f58073350 0x7f4f58075860 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f78147c30 0x7f4f78147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f780ca6c0 0x7f4f780caad0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 --2- 172.21.15.170:0/2238835466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f60003a40 0x7f4f780b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 >> 172.21.15.170:0/2238835466 conn(0x7f4f780ced80 msgr2=0x7f4f7815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 shutdown_connections 2024-09-21T02:18:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.320+0000 7f4f7cef6640 1 -- 172.21.15.170:0/2238835466 wait complete. 2024-09-21T02:18:05.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 -- 172.21.15.170:0/3116436341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c003a10 msgr2=0x7f762c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 --2- 172.21.15.170:0/3116436341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c003a10 0x7f762c003e80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7638009ec0 tx=0x7f763801d610 comp rx=0 tx=0).stop 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f763ffff640 1 -- 172.21.15.170:0/3116436341 reap_dead start 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 -- 172.21.15.170:0/3116436341 shutdown_connections 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 -- 172.21.15.170:0/3116436341 >> 172.21.15.170:0/3116436341 conn(0x7f76400b1c40 msgr2=0x7f76400b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 -- 172.21.15.170:0/3116436341 shutdown_connections 2024-09-21T02:18:05.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 -- 172.21.15.170:0/3116436341 wait complete. 2024-09-21T02:18:05.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 Processor -- start 2024-09-21T02:18:05.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.500+0000 7f7645ddf640 1 -- start start 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f762c003a10 0x7f764015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 0x7f76400b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f764006eeb0 0x7f764006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7640166b80 con 0x7f76400b0bc0 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7640166a00 con 0x7f762c003a10 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7640166d00 con 0x7f764006eeb0 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 0x7f76400b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 0x7f76400b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40792/0 (socket says 172.21.15.170:40792) 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 -- 172.21.15.170:0/2116478751 learned_addr learned my addr 172.21.15.170:0/2116478751 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763ffff640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f764006eeb0 0x7f764006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763f7fe640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f762c003a10 0x7f764015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 -- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f764006eeb0 msgr2=0x7f764006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f764006eeb0 0x7f764006f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 -- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f762c003a10 msgr2=0x7f764015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f762c003a10 0x7f764015fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 -- 172.21.15.170:0/2116478751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764006f800 con 0x7f76400b0bc0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763ffff640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f764006eeb0 0x7f764006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763effd640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 0x7f76400b0fd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f762800e970 tx=0x7f762800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762800cc60 con 0x7f76400b0bc0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f762800ce00 con 0x7f76400b0bc0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7640159960 con 0x7f76400b0bc0 2024-09-21T02:18:05.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628005280 con 0x7f76400b0bc0 2024-09-21T02:18:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76400aeac0 con 0x7f76400b0bc0 2024-09-21T02:18:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76400cb870 con 0x7f76400b0bc0 2024-09-21T02:18:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7628005420 con 0x7f76400b0bc0 2024-09-21T02:18:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7637fff640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7620073320 0x7f7620075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f763f7fe640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7620073320 0x7f7620075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.504+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f7628094d60 con 0x7f76400b0bc0 2024-09-21T02:18:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.508+0000 7f763f7fe640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7620073320 0x7f7620075830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7638009ec0 tx=0x7f763801a3b0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:05.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.512+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f764006f800 con 0x7f76400b0bc0 2024-09-21T02:18:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.676+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f76400b2180 con 0x7f76400b0bc0 2024-09-21T02:18:05.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.680+0000 7f7637fff640 1 -- 172.21.15.170:0/2116478751 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7f762805fcc0 con 0x7f76400b0bc0 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7620073320 msgr2=0x7f7620075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7620073320 0x7f7620075830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7638009ec0 tx=0x7f763801a3b0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 msgr2=0x7f76400b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 0x7f76400b0fd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f762800e970 tx=0x7f762800ee30 comp rx=0 tx=0).stop 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 shutdown_connections 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7620073320 0x7f7620075830 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f764006eeb0 0x7f764006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76400b0bc0 0x7f76400b0fd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 --2- 172.21.15.170:0/2116478751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f762c003a10 0x7f764015fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 >> 172.21.15.170:0/2116478751 conn(0x7f76400b1c40 msgr2=0x7f76400b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 shutdown_connections 2024-09-21T02:18:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.684+0000 7f7645ddf640 1 -- 172.21.15.170:0/2116478751 wait complete. 2024-09-21T02:18:05.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- 172.21.15.170:0/3283108758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cb460 msgr2=0x7fe8440cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 --2- 172.21.15.170:0/3283108758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cb460 0x7fe8440cb870 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe8440d1160 tx=0x7fe82c01c800 comp rx=0 tx=0).stop 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- 172.21.15.170:0/3283108758 shutdown_connections 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 --2- 172.21.15.170:0/3283108758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cb460 0x7fe8440cb870 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 --2- 172.21.15.170:0/3283108758 >> v2:172.21.15.191:3300/0 conn(0x7fe8440cf7e0 0x7fe844070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- 172.21.15.170:0/3283108758 >> 172.21.15.170:0/3283108758 conn(0x7fe8440b1c40 msgr2=0x7fe8440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- 172.21.15.170:0/3283108758 shutdown_connections 2024-09-21T02:18:05.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- 172.21.15.170:0/3283108758 wait complete. 2024-09-21T02:18:05.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 Processor -- start 2024-09-21T02:18:05.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- start start 2024-09-21T02:18:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8440cb460 0x7fe84415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 0x7fe8441585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8440cf7e0 0x7fe8440b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441690d0 con 0x7fe8440cc8c0 2024-09-21T02:18:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe844168f50 con 0x7fe8440cf7e0 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe848ffa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe844169250 con 0x7fe8440cb460 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 0x7fe8441585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 0x7fe8441585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40810/0 (socket says 172.21.15.170:40810) 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 -- 172.21.15.170:0/2993565080 learned_addr learned my addr 172.21.15.170:0/2993565080 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 -- 172.21.15.170:0/2993565080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8440cb460 msgr2=0x7fe84415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8440cb460 0x7fe84415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 -- 172.21.15.170:0/2993565080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8440cf7e0 msgr2=0x7fe8440b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8440cf7e0 0x7fe8440b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.852+0000 7fe839d9b640 1 -- 172.21.15.170:0/2993565080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84415c930 con 0x7fe8440cc8c0 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe839d9b640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 0x7fe8441585e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe82c000c10 tx=0x7fe82c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0bb070 con 0x7fe8440cc8c0 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe82c005030 con 0x7fe8440cc8c0 2024-09-21T02:18:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84415cbd0 con 0x7fe8440cc8c0 2024-09-21T02:18:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0b2040 con 0x7fe8440cc8c0 2024-09-21T02:18:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8440c9e70 con 0x7fe8440cc8c0 2024-09-21T02:18:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8440cb870 con 0x7fe8440cc8c0 2024-09-21T02:18:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fe82c0be070 con 0x7fe8440cc8c0 2024-09-21T02:18:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe83b7fe640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe824073320 0x7fe824075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7fe82c12e220 con 0x7fe8440cc8c0 2024-09-21T02:18:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe84259c640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe824073320 0x7fe824075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.856+0000 7fe84259c640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe824073320 0x7fe824075830 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe83400a860 tx=0x7fe834005b60 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:05.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:05.860+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe82c0c5020 con 0x7fe8440cc8c0 2024-09-21T02:18:06.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.080+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fe8440c9c20 con 0x7fe8440cc8c0 2024-09-21T02:18:06.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.312+0000 7fe83b7fe640 1 -- 172.21.15.170:0/2993565080 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v248) ==== 131+0+0 (secure 0 0 0) 0x7fe84415c930 con 0x7fe8440cc8c0 2024-09-21T02:18:06.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe824073320 msgr2=0x7fe824075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe824073320 0x7fe824075830 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe83400a860 tx=0x7fe834005b60 comp rx=0 tx=0).stop 2024-09-21T02:18:06.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 msgr2=0x7fe8441585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 0x7fe8441585e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe82c000c10 tx=0x7fe82c004040 comp rx=0 tx=0).stop 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 shutdown_connections 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe824073320 0x7fe824075830 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8440cf7e0 0x7fe8440b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8440cc8c0 0x7fe8441585e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 --2- 172.21.15.170:0/2993565080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8440cb460 0x7fe84415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 >> 172.21.15.170:0/2993565080 conn(0x7fe8440b1c40 msgr2=0x7fe84415b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 shutdown_connections 2024-09-21T02:18:06.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.316+0000 7fe848ffa640 1 -- 172.21.15.170:0/2993565080 wait complete. 2024-09-21T02:18:06.343 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-21T02:18:06.343 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-21T02:18:06.343 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:18:06.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 -- 172.21.15.170:0/696127771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 msgr2=0x7f33cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/696127771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33cc00b9d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009ec0 tx=0x7f33dc01d610 comp rx=0 tx=0).stop 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 -- 172.21.15.170:0/696127771 shutdown_connections 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/696127771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33cc00b9d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 -- 172.21.15.170:0/696127771 >> 172.21.15.170:0/696127771 conn(0x7f33e40b1c40 msgr2=0x7f33e40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 -- 172.21.15.170:0/696127771 shutdown_connections 2024-09-21T02:18:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.500+0000 7f33e9bcf640 1 -- 172.21.15.170:0/696127771 wait complete. 2024-09-21T02:18:06.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 Processor -- start 2024-09-21T02:18:06.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- start start 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33e409ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33e40c4e70 0x7f33e4159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e41f72f0 0x7f33e41f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33e40cf760 con 0x7f33cc0052b0 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33e40c3a10 con 0x7f33e41f72f0 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f33e40cf8e0 con 0x7f33e40c4e70 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33e409ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33e409ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40830/0 (socket says 172.21.15.170:40830) 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 -- 172.21.15.170:0/299170440 learned_addr learned my addr 172.21.15.170:0/299170440 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e2ffd640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33e40c4e70 0x7f33e4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 -- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33e40c4e70 msgr2=0x7f33e4159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e3fff640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e41f72f0 0x7f33e41f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33e40c4e70 0x7f33e4159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 -- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e41f72f0 msgr2=0x7f33e41f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e41f72f0 0x7f33e41f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 -- 172.21.15.170:0/299170440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e41f6b30 con 0x7f33cc0052b0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e37fe640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33e409ac60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f33d4004d70 tx=0x7f33d4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d40105d0 con 0x7f33cc0052b0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33d4016070 con 0x7f33cc0052b0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e409b4b0 con 0x7f33cc0052b0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4010ae0 con 0x7f33cc0052b0 2024-09-21T02:18:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41f6dd0 con 0x7f33cc0052b0 2024-09-21T02:18:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.504+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e40b3510 con 0x7f33cc0052b0 2024-09-21T02:18:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.508+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f33d4010c80 con 0x7f33cc0052b0 2024-09-21T02:18:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.508+0000 7f33cbfff640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33d0073320 0x7f33d0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.508+0000 7f33e2ffd640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33d0073320 0x7f33d0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.508+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f33d4094b40 con 0x7f33cc0052b0 2024-09-21T02:18:06.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.508+0000 7f33e2ffd640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33d0073320 0x7f33d0075830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33e40ceb40 tx=0x7f33d80015c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:06.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.512+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f33d405faa0 con 0x7f33cc0052b0 2024-09-21T02:18:06.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.696+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33e40c7620 con 0x7f33cc0052b0 2024-09-21T02:18:06.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.696+0000 7f33cbfff640 1 -- 172.21.15.170:0/299170440 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13907 (secure 0 0 0) 0x7f33d405f3a0 con 0x7f33cc0052b0 2024-09-21T02:18:06.699 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:06.700 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":76,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:05.306294+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:0/1074802945":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3207724562":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/543979301":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1852913239":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/2890432817":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/1634809445":"2024-09-22T02:17:58.836483+0000","172.21.15.170:6835/1298680671":"2024-09-22T02:18:02.277287+0000","172.21.15.191:0/3826872798":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3689562231":"2024-09-22T02:17:58.836483+0000","172.21.15.191:6800/2615148678":"2024-09-22T02:17:58.836483+0000","172.21.15.191:6801/2615148678":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/3514032572":"2024-09-22T02:17:58.836483+0000","172.21.15.170:6834/1298680671":"2024-09-22T02:18:02.277287+0000","172.21.15.191:0/2838563242":"2024-09-22T02:17:58.836483+0000","172.21.15.191:0/2357730026":"2024-09-22T02:17:58.836483+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:06.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33d0073320 msgr2=0x7f33d0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33d0073320 0x7f33d0075830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f33e40ceb40 tx=0x7f33d80015c0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 msgr2=0x7f33e409ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33e409ac60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f33d4004d70 tx=0x7f33d4001640 comp rx=0 tx=0).stop 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 shutdown_connections 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33d0073320 0x7f33d0075830 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33e40c4e70 0x7f33e4159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e41f72f0 0x7f33e41f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 --2- 172.21.15.170:0/299170440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc0052b0 0x7f33e409ac60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 >> 172.21.15.170:0/299170440 conn(0x7f33e40b1c40 msgr2=0x7f33e40cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 shutdown_connections 2024-09-21T02:18:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.700+0000 7f33e9bcf640 1 -- 172.21.15.170:0/299170440 wait complete. 2024-09-21T02:18:06.720 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:18:06.720 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:18:06.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cbe224640 1 -- 172.21.15.170:0/3773825924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca0005360 msgr2=0x7f2ca000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cbe224640 1 --2- 172.21.15.170:0/3773825924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ca0005360 0x7f2ca000b9d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80d2330 tx=0x7f2cb001d280 comp rx=0 tx=0).stop 2024-09-21T02:18:06.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cb7fff640 1 -- 172.21.15.170:0/3773825924 reap_dead start 2024-09-21T02:18:06.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cbe224640 1 -- 172.21.15.170:0/3773825924 shutdown_connections 2024-09-21T02:18:06.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cbe224640 1 -- 172.21.15.170:0/3773825924 >> 172.21.15.170:0/3773825924 conn(0x7f2cb80d1280 msgr2=0x7f2cb80d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:06.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cbe224640 1 -- 172.21.15.170:0/3773825924 shutdown_connections 2024-09-21T02:18:06.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.864+0000 7f2cbe224640 1 -- 172.21.15.170:0/3773825924 wait complete. 2024-09-21T02:18:06.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 Processor -- start 2024-09-21T02:18:06.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- start start 2024-09-21T02:18:06.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca0005360 0x7f2cb815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cb81f8f10 0x7f2cb81f9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 0x7f2cb815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80c4bb0 con 0x7f2cb815b100 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80c4a30 con 0x7f2cb81f8f10 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb815bee0 con 0x7f2ca0005360 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 0x7f2cb815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 0x7f2cb815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40848/0 (socket says 172.21.15.170:40848) 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 -- 172.21.15.170:0/407093519 learned_addr learned my addr 172.21.15.170:0/407093519 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb6ffd640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cb81f8f10 0x7f2cb81f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb77fe640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca0005360 0x7f2cb815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 -- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca0005360 msgr2=0x7f2cb815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca0005360 0x7f2cb815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 -- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cb81f8f10 msgr2=0x7f2cb81f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cb81f8f10 0x7f2cb81f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 -- 172.21.15.170:0/407093519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb815ba50 con 0x7f2cb815b100 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb6ffd640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cb81f8f10 0x7f2cb81f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:06.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cb7fff640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 0x7f2cb815b510 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0098e50 tx=0x7f2cb00065d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:06.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0002050 con 0x7f2cb815b100 2024-09-21T02:18:06.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb0004e40 con 0x7f2cb815b100 2024-09-21T02:18:06.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb80c9d60 con 0x7f2cb815b100 2024-09-21T02:18:06.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0006920 con 0x7f2cb815b100 2024-09-21T02:18:06.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb80ca320 con 0x7f2cb815b100 2024-09-21T02:18:06.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.868+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb815c830 con 0x7f2cb815b100 2024-09-21T02:18:06.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.872+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f2cb00ac020 con 0x7f2cb815b100 2024-09-21T02:18:06.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.872+0000 7f2c9ffff640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ca4073320 0x7f2ca4075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:06.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.872+0000 7f2cb77fe640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ca4073320 0x7f2ca4075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:06.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.872+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f2cb012e6d0 con 0x7f2cb815b100 2024-09-21T02:18:06.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.872+0000 7f2cb77fe640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ca4073320 0x7f2ca4075830 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c10 tx=0x7f2ca8006b80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:06.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:06.876+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2cb815ba50 con 0x7f2cb815b100 2024-09-21T02:18:07.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cb80d1830 con 0x7f2cb815b100 2024-09-21T02:18:07.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2c9ffff640 1 -- 172.21.15.170:0/407093519 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f2cb0017030 con 0x7f2cb815b100 2024-09-21T02:18:07.049 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ca4073320 msgr2=0x7f2ca4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ca4073320 0x7f2ca4075830 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c10 tx=0x7f2ca8006b80 comp rx=0 tx=0).stop 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 msgr2=0x7f2cb815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 0x7f2cb815b510 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0098e50 tx=0x7f2cb00065d0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 shutdown_connections 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ca4073320 0x7f2ca4075830 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cb815b100 0x7f2cb815b510 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cb81f8f10 0x7f2cb81f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 --2- 172.21.15.170:0/407093519 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ca0005360 0x7f2cb815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 >> 172.21.15.170:0/407093519 conn(0x7f2cb80d1280 msgr2=0x7f2cb8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 shutdown_connections 2024-09-21T02:18:07.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.048+0000 7f2cbe224640 1 -- 172.21.15.170:0/407093519 wait complete. 2024-09-21T02:18:07.072 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 -- 172.21.15.170:0/1314873642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0ce9e0 msgr2=0x7facdc0cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 --2- 172.21.15.170:0/1314873642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0ce9e0 0x7facdc0cedf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7facdc070080 tx=0x7facd401cd60 comp rx=0 tx=0).stop 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 -- 172.21.15.170:0/1314873642 shutdown_connections 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 --2- 172.21.15.170:0/1314873642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0ce9e0 0x7facdc0cedf0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 --2- 172.21.15.170:0/1314873642 >> v2:172.21.15.191:3300/0 conn(0x7facdc0cfbb0 0x7facdc0cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 --2- 172.21.15.170:0/1314873642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0afb70 0x7facdc06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 -- 172.21.15.170:0/1314873642 >> 172.21.15.170:0/1314873642 conn(0x7facdc0b25c0 msgr2=0x7facdc0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 -- 172.21.15.170:0/1314873642 shutdown_connections 2024-09-21T02:18:07.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 -- 172.21.15.170:0/1314873642 wait complete. 2024-09-21T02:18:07.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 Processor -- start 2024-09-21T02:18:07.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.224+0000 7face2421640 1 -- start start 2024-09-21T02:18:07.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 0x7facdc156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:07.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ce9e0 0x7facdc0ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:07.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facdc0cfbb0 0x7facdc0ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:07.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0ce0f0 con 0x7facdc0afb70 2024-09-21T02:18:07.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0cdf70 con 0x7facdc0cfbb0 2024-09-21T02:18:07.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0ce270 con 0x7facdc0ce9e0 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 0x7facdc156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 0x7facdc156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40870/0 (socket says 172.21.15.170:40870) 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 -- 172.21.15.170:0/2005702455 learned_addr learned my addr 172.21.15.170:0/2005702455 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facd37fe640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ce9e0 0x7facdc0ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face09be640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facdc0cfbb0 0x7facdc0ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 -- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ce9e0 msgr2=0x7facdc0ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ce9e0 0x7facdc0ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 -- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facdc0cfbb0 msgr2=0x7facdc0ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facdc0cfbb0 0x7facdc0ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 -- 172.21.15.170:0/2005702455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0b1950 con 0x7facdc0afb70 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facdbfff640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 0x7facdc156e30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d750 tx=0x7faccc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc017070 con 0x7facdc0afb70 2024-09-21T02:18:07.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faccc009d20 con 0x7facdc0afb70 2024-09-21T02:18:07.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc0b52c0 con 0x7facdc0afb70 2024-09-21T02:18:07.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc005280 con 0x7facdc0afb70 2024-09-21T02:18:07.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc0b5880 con 0x7facdc0afb70 2024-09-21T02:18:07.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc0cedf0 con 0x7facdc0afb70 2024-09-21T02:18:07.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7faccc005420 con 0x7facdc0afb70 2024-09-21T02:18:07.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.228+0000 7facd97fa640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7facbc073320 0x7facbc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:07.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.232+0000 7facd37fe640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7facbc073320 0x7facbc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:07.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.232+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7faccc094460 con 0x7facdc0afb70 2024-09-21T02:18:07.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.232+0000 7facd37fe640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7facbc073320 0x7facbc075830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facdc0b18e0 tx=0x7facc400d4b0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:07.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.236+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faccc00de60 con 0x7facdc0afb70 2024-09-21T02:18:07.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:07.476+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7facdc0ca6e0 con 0x7facdc0afb70 2024-09-21T02:18:08.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.324+0000 7facd97fa640 1 -- 172.21.15.170:0/2005702455 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v250) ==== 149+0+0 (secure 0 0 0) 0x7faccc05f3c0 con 0x7facdc0afb70 2024-09-21T02:18:08.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7facbc073320 msgr2=0x7facbc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7facbc073320 0x7facbc075830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facdc0b18e0 tx=0x7facc400d4b0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 msgr2=0x7facdc156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 0x7facdc156e30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d750 tx=0x7faccc00dc10 comp rx=0 tx=0).stop 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 shutdown_connections 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7facbc073320 0x7facbc075830 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ce9e0 0x7facdc0ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facdc0cfbb0 0x7facdc0ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 --2- 172.21.15.170:0/2005702455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0afb70 0x7facdc156e30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 >> 172.21.15.170:0/2005702455 conn(0x7facdc0b25c0 msgr2=0x7facdc158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 shutdown_connections 2024-09-21T02:18:08.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.328+0000 7face2421640 1 -- 172.21.15.170:0/2005702455 wait complete. 2024-09-21T02:18:08.356 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.508+0000 7f32cabc6640 1 -- 172.21.15.170:0/3465627409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac005360 msgr2=0x7f32ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.508+0000 7f32cabc6640 1 --2- 172.21.15.170:0/3465627409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac005360 0x7f32ac00b9d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006fd0 tx=0x7f32bc01ca20 comp rx=0 tx=0).stop 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.508+0000 7f32cabc6640 1 -- 172.21.15.170:0/3465627409 shutdown_connections 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.508+0000 7f32cabc6640 1 --2- 172.21.15.170:0/3465627409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac005360 0x7f32ac00b9d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.508+0000 7f32cabc6640 1 -- 172.21.15.170:0/3465627409 >> 172.21.15.170:0/3465627409 conn(0x7f32c40b1c40 msgr2=0x7f32c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.508+0000 7f32cabc6640 1 -- 172.21.15.170:0/3465627409 shutdown_connections 2024-09-21T02:18:08.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 -- 172.21.15.170:0/3465627409 wait complete. 2024-09-21T02:18:08.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 Processor -- start 2024-09-21T02:18:08.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 -- start start 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32ac005360 0x7f32c40c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c41523e0 0x7f32c409ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 0x7f32c409cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32c40cb010 con 0x7f32c41527f0 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f32c40cae90 con 0x7f32c41523e0 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32cabc6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f32c40cdff0 con 0x7f32ac005360 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 0x7f32c409cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 0x7f32c409cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40894/0 (socket says 172.21.15.170:40894) 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 -- 172.21.15.170:0/1923496906 learned_addr learned my addr 172.21.15.170:0/1923496906 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c8962640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32ac005360 0x7f32c40c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c3fff640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c41523e0 0x7f32c409ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 -- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32ac005360 msgr2=0x7f32c40c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32ac005360 0x7f32c40c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 -- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c41523e0 msgr2=0x7f32c409ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c41523e0 0x7f32c409ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 -- 172.21.15.170:0/1923496906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c409b740 con 0x7f32c41527f0 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c9163640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 0x7f32c409cf50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006040 tx=0x7f32bc003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.512+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0a5a20 con 0x7f32c41527f0 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32bc0b3070 con 0x7f32c41527f0 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc003fc0 con 0x7f32c41527f0 2024-09-21T02:18:08.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c409ba20 con 0x7f32c41527f0 2024-09-21T02:18:08.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c409bfe0 con 0x7f32c41527f0 2024-09-21T02:18:08.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c409b4b0 con 0x7f32c41527f0 2024-09-21T02:18:08.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f32bc006070 con 0x7f32c41527f0 2024-09-21T02:18:08.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c17fa640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f32b0073320 0x7f32b0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c8962640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f32b0073320 0x7f32b0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f32bc12e8d0 con 0x7f32c41527f0 2024-09-21T02:18:08.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.516+0000 7f32c8962640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f32b0073320 0x7f32b0075830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32c4071360 tx=0x7f32b8005f30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:08.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.520+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f32bc016730 con 0x7f32c41527f0 2024-09-21T02:18:08.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.688+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f32c4070460 con 0x7f32c41527f0 2024-09-21T02:18:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.688+0000 7f32c17fa640 1 -- 172.21.15.170:0/1923496906 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7f32bc0f9990 con 0x7f32c41527f0 2024-09-21T02:18:08.691 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:18:08.691 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:18:08.691 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:18:08.691 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:18:08.692 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:18:08.693 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:18:08.694 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f32b0073320 msgr2=0x7f32b0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f32b0073320 0x7f32b0075830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f32c4071360 tx=0x7f32b8005f30 comp rx=0 tx=0).stop 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 msgr2=0x7f32c409cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 0x7f32c409cf50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006040 tx=0x7f32bc003aa0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 shutdown_connections 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f32b0073320 0x7f32b0075830 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32c41527f0 0x7f32c409cf50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32c41523e0 0x7f32c409ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 --2- 172.21.15.170:0/1923496906 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32ac005360 0x7f32c40c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 >> 172.21.15.170:0/1923496906 conn(0x7f32c40b1c40 msgr2=0x7f32c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:08.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 shutdown_connections 2024-09-21T02:18:08.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.692+0000 7f32cabc6640 1 -- 172.21.15.170:0/1923496906 wait complete. 2024-09-21T02:18:08.712 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:18:08.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 -- 172.21.15.170:0/245166667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 msgr2=0x7f45980cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 --2- 172.21.15.170:0/245166667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f45980cb7c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4590009c80 tx=0x7f459001cd40 comp rx=0 tx=0).stop 2024-09-21T02:18:08.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 -- 172.21.15.170:0/245166667 shutdown_connections 2024-09-21T02:18:08.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 --2- 172.21.15.170:0/245166667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f45980cb7c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 --2- 172.21.15.170:0/245166667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4598074270 0x7f45980cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 -- 172.21.15.170:0/245166667 >> 172.21.15.170:0/245166667 conn(0x7f45980d0170 msgr2=0x7f45980d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:08.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 -- 172.21.15.170:0/245166667 shutdown_connections 2024-09-21T02:18:08.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.852+0000 7f459db67640 1 -- 172.21.15.170:0/245166667 wait complete. 2024-09-21T02:18:08.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 Processor -- start 2024-09-21T02:18:08.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 -- start start 2024-09-21T02:18:08.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4598074270 0x7f459815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f459814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45980cf7d0 0x7f459806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4598169040 con 0x7f4598074680 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4598168ec0 con 0x7f45980cf7d0 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f45981691c0 con 0x7f4598074270 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f459814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f459814f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40920/0 (socket says 172.21.15.170:40920) 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 -- 172.21.15.170:0/3552822933 learned_addr learned my addr 172.21.15.170:0/3552822933 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f45977fe640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4598074270 0x7f459815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4597fff640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45980cf7d0 0x7f459806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 -- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4598074270 msgr2=0x7f459815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4598074270 0x7f459815c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 -- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45980cf7d0 msgr2=0x7f459806f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45980cf7d0 0x7f459806f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 -- 172.21.15.170:0/3552822933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45981fa3d0 con 0x7f4598074680 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f4596ffd640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f459814f110 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d720 tx=0x7f458c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c017070 con 0x7f4598074680 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f458c009d20 con 0x7f4598074680 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45981fa6b0 con 0x7f4598074680 2024-09-21T02:18:08.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c005280 con 0x7f4598074680 2024-09-21T02:18:08.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.856+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45981fac70 con 0x7f4598074680 2024-09-21T02:18:08.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.860+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459815ee20 con 0x7f4598074680 2024-09-21T02:18:08.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.860+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f458c005420 con 0x7f4598074680 2024-09-21T02:18:08.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.860+0000 7f457ffff640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4584073320 0x7f4584075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:08.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.860+0000 7f45977fe640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4584073320 0x7f4584075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:08.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.860+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f458c094470 con 0x7f4598074680 2024-09-21T02:18:08.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.860+0000 7f45977fe640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4584073320 0x7f4584075830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4588001650 tx=0x7f4588009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:08.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:08.864+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f458c05f3d0 con 0x7f4598074680 2024-09-21T02:18:09.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.024+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4598073b60 con 0x7f4598074680 2024-09-21T02:18:09.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.032+0000 7f457ffff640 1 -- 172.21.15.170:0/3552822933 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f458c05ecd0 con 0x7f4598074680 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4584073320 msgr2=0x7f4584075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4584073320 0x7f4584075830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4588001650 tx=0x7f4588009040 comp rx=0 tx=0).stop 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 msgr2=0x7f459814f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f459814f110 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f458c00d720 tx=0x7f458c00dbe0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 shutdown_connections 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4584073320 0x7f4584075830 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45980cf7d0 0x7f459806f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4598074680 0x7f459814f110 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 --2- 172.21.15.170:0/3552822933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4598074270 0x7f459815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 >> 172.21.15.170:0/3552822933 conn(0x7f45980d0170 msgr2=0x7f45980b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 shutdown_connections 2024-09-21T02:18:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.036+0000 7f459db67640 1 -- 172.21.15.170:0/3552822933 wait complete. 2024-09-21T02:18:09.056 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:18:09.056 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:18:09.130 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:18:09.197 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:18:09.197 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:09.197 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:09.203 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:18:09.203 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:09.203 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:09.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:18:09.209 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:09.209 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:09.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:18:09.249 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:09.249 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:09.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:18:09.257 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:18:09.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:09.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 -- 172.21.15.170:0/1931242554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980d0e50 msgr2=0x7fe4980d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 --2- 172.21.15.170:0/1931242554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980d0e50 0x7fe4980d1260 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe4980ba1d0 tx=0x7fe48001c7e0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 -- 172.21.15.170:0/1931242554 shutdown_connections 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 --2- 172.21.15.170:0/1931242554 >> v2:172.21.15.191:3300/0 conn(0x7fe4980d2440 0x7fe4980d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 --2- 172.21.15.170:0/1931242554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980d0e50 0x7fe4980d1260 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 --2- 172.21.15.170:0/1931242554 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4980b3fa0 0x7fe4980cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 -- 172.21.15.170:0/1931242554 >> 172.21.15.170:0/1931242554 conn(0x7fe49808bf50 msgr2=0x7fe49809d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 -- 172.21.15.170:0/1931242554 shutdown_connections 2024-09-21T02:18:09.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 -- 172.21.15.170:0/1931242554 wait complete. 2024-09-21T02:18:09.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 Processor -- start 2024-09-21T02:18:09.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.452+0000 7fe49cff9640 1 -- start start 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 0x7fe4981530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4980d0e50 0x7fe4981fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4980d2440 0x7fe49815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980b19e0 con 0x7fe4980b3fa0 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980b1860 con 0x7fe4980d2440 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe4980b1b60 con 0x7fe4980d0e50 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 0x7fe4981530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 0x7fe4981530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40944/0 (socket says 172.21.15.170:40944) 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 -- 172.21.15.170:0/3641865195 learned_addr learned my addr 172.21.15.170:0/3641865195 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 -- 172.21.15.170:0/3641865195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4980d0e50 msgr2=0x7fe4981fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4980d0e50 0x7fe4981fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 -- 172.21.15.170:0/3641865195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4980d2440 msgr2=0x7fe49815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4980d2440 0x7fe49815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 -- 172.21.15.170:0/3641865195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4981fd2a0 con 0x7fe4980b3fa0 2024-09-21T02:18:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49659c640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 0x7fe4981530b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c940 tx=0x7fe48800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488007bb0 con 0x7fe4980b3fa0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe488007e40 con 0x7fe4980b3fa0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4980ce170 con 0x7fe4980b3fa0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48800f690 con 0x7fe4980b3fa0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe498153730 con 0x7fe4980b3fa0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4980d1260 con 0x7fe4980b3fa0 2024-09-21T02:18:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.456+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fe48800f830 con 0x7fe4980b3fa0 2024-09-21T02:18:09.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.460+0000 7fe48effd640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe47c073320 0x7fe47c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.460+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7fe488094b60 con 0x7fe4980b3fa0 2024-09-21T02:18:09.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.460+0000 7fe48ffff640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe47c073320 0x7fe47c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:09.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.460+0000 7fe48ffff640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe47c073320 0x7fe47c075830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4800af000 tx=0x7fe480019580 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:09.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.464+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe488014030 con 0x7fe4980b3fa0 2024-09-21T02:18:09.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.672+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4980cd740 con 0x7fe4980b3fa0 2024-09-21T02:18:09.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.672+0000 7fe48effd640 1 -- 172.21.15.170:0/3641865195 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+3923 (secure 0 0 0) 0x7fe48805fac0 con 0x7fe4980b3fa0 2024-09-21T02:18:09.675 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:09.675 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":50,"btime":"2024-09-21T02:18:03:282118+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5488,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3783525776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3783525776},{"type":"v1","addr":"172.21.15.191:6835","nonce":3783525776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3717912729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3717912729},{"type":"v1","addr":"172.21.15.191:6837","nonce":3717912729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1414893525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1414893525},{"type":"v1","addr":"172.21.15.170:6833","nonce":1414893525}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/4008260163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4008260163},{"type":"v1","addr":"172.21.15.170:6835","nonce":4008260163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-09-21T02:18:09.675 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 50 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe47c073320 msgr2=0x7fe47c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe47c073320 0x7fe47c075830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4800af000 tx=0x7fe480019580 comp rx=0 tx=0).stop 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 msgr2=0x7fe4981530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 0x7fe4981530b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c940 tx=0x7fe48800ce00 comp rx=0 tx=0).stop 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 shutdown_connections 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe47c073320 0x7fe47c075830 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4980d0e50 0x7fe4981fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4980d2440 0x7fe49815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 --2- 172.21.15.170:0/3641865195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4980b3fa0 0x7fe4981530b0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 >> 172.21.15.170:0/3641865195 conn(0x7fe49808bf50 msgr2=0x7fe4980b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:09.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 shutdown_connections 2024-09-21T02:18:09.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.676+0000 7fe49cff9640 1 -- 172.21.15.170:0/3641865195 wait complete. 2024-09-21T02:18:09.696 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:18:09.696 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:18:09.697 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:18:09.697 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:09.697 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:18:09.697 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:18:09.697 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:18:09.697 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:09.697 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:18:09.697 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:18:09.698 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:18:09.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:09.698 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:18:09.698 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:18:09.698 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:18:09.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:09.709 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:18:09.710 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:18:09.711 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:18:09.711 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:18:09.716 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:18:09.716 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:18:09.718 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:18:09.718 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:18:09.753 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:18:09.753 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:09.754 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:18:09.755 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:09.764 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:18:09.764 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:09.766 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:18:09.766 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:09.766 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:18:09.838 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:18:09.844 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:18:09.864 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:18:09.864 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- 172.21.15.170:0/557473130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440b1230 msgr2=0x7f63440b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 --2- 172.21.15.170:0/557473130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440b1230 0x7f63440b1640 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f634408c5a0 tx=0x7f633c01caf0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- 172.21.15.170:0/557473130 shutdown_connections 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 --2- 172.21.15.170:0/557473130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440b1230 0x7f63440b1640 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 --2- 172.21.15.170:0/557473130 >> v2:172.21.15.191:3300/0 conn(0x7f63440d0e50 0x7f63440d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- 172.21.15.170:0/557473130 >> 172.21.15.170:0/557473130 conn(0x7f634408bf50 msgr2=0x7f634409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- 172.21.15.170:0/557473130 shutdown_connections 2024-09-21T02:18:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- 172.21.15.170:0/557473130 wait complete. 2024-09-21T02:18:09.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 Processor -- start 2024-09-21T02:18:09.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- start start 2024-09-21T02:18:09.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63440b1230 0x7f6344158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63440b3fa0 0x7f6344157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 0x7f634415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f634416bfd0 con 0x7f63440d0e50 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f634416be50 con 0x7f63440b1230 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6348f20640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f634416c150 con 0x7f63440b3fa0 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 0x7f634415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 0x7f634415ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41008/0 (socket says 172.21.15.170:41008) 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 -- 172.21.15.170:0/440259181 learned_addr learned my addr 172.21.15.170:0/440259181 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6341d9b640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63440b3fa0 0x7f6344157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f634259c640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63440b1230 0x7f6344158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 -- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63440b3fa0 msgr2=0x7f6344157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63440b3fa0 0x7f6344157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 -- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63440b1230 msgr2=0x7f6344158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63440b1230 0x7f6344158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 -- 172.21.15.170:0/440259181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63441f8f70 con 0x7f63440d0e50 2024-09-21T02:18:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.908+0000 7f6342d9d640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 0x7f634415ba00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f633c01c560 tx=0x7f633c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0b2040 con 0x7f63440d0e50 2024-09-21T02:18:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f633c0bb070 con 0x7f63440d0e50 2024-09-21T02:18:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6344158dc0 con 0x7f63440d0e50 2024-09-21T02:18:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c004530 con 0x7f63440d0e50 2024-09-21T02:18:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6344159380 con 0x7f63440d0e50 2024-09-21T02:18:09.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63440b1640 con 0x7f63440d0e50 2024-09-21T02:18:09.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f633c0046d0 con 0x7f63440d0e50 2024-09-21T02:18:09.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6332ffd640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6324073350 0x7f6324075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:09.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5502+0+0 (secure 0 0 0) 0x7f633c12ed50 con 0x7f63440d0e50 2024-09-21T02:18:09.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f634259c640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6324073350 0x7f6324075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:09.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.912+0000 7f634259c640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6324073350 0x7f6324075860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6338001180 tx=0x7f6338001110 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:09.916+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f633c132030 con 0x7f63440d0e50 2024-09-21T02:18:10.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.092+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6344158b50 con 0x7f63440d0e50 2024-09-21T02:18:10.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.360+0000 7f6332ffd640 1 -- 172.21.15.170:0/440259181 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) ==== 97+0+0 (secure 0 0 0) 0x7f633c016770 con 0x7f63440d0e50 2024-09-21T02:18:10.364 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:18:10.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6324073350 msgr2=0x7f6324075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6324073350 0x7f6324075860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6338001180 tx=0x7f6338001110 comp rx=0 tx=0).stop 2024-09-21T02:18:10.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 msgr2=0x7f634415ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 0x7f634415ba00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f633c01c560 tx=0x7f633c009200 comp rx=0 tx=0).stop 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 shutdown_connections 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6324073350 0x7f6324075860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63440b3fa0 0x7f6344157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63440d0e50 0x7f634415ba00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 --2- 172.21.15.170:0/440259181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63440b1230 0x7f6344158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 >> 172.21.15.170:0/440259181 conn(0x7f634408bf50 msgr2=0x7f63440d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 shutdown_connections 2024-09-21T02:18:10.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.364+0000 7f6348f20640 1 -- 172.21.15.170:0/440259181 wait complete. 2024-09-21T02:18:10.392 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 -- 172.21.15.170:0/596627054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 msgr2=0x7f6db800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 --2- 172.21.15.170:0/596627054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6db800b9d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00cbfa0 tx=0x7f6dc801cdc0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 -- 172.21.15.170:0/596627054 shutdown_connections 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 --2- 172.21.15.170:0/596627054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6db800b9d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 -- 172.21.15.170:0/596627054 >> 172.21.15.170:0/596627054 conn(0x7f6dd00b1c40 msgr2=0x7f6dd00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 -- 172.21.15.170:0/596627054 shutdown_connections 2024-09-21T02:18:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.544+0000 7f6dd6316640 1 -- 172.21.15.170:0/596627054 wait complete. 2024-09-21T02:18:10.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 Processor -- start 2024-09-21T02:18:10.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- start start 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6dd00ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00ce650 0x7f6dd01f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01f99c0 0x7f6dd01f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd00cb9f0 con 0x7f6db8005310 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd00cb870 con 0x7f6dd00ce650 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd00cbb70 con 0x7f6dd01f99c0 2024-09-21T02:18:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6dd00ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6dd00ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41026/0 (socket says 172.21.15.170:41026) 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 -- 172.21.15.170:0/3939154078 learned_addr learned my addr 172.21.15.170:0/3939154078 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcf7fe640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00ce650 0x7f6dd01f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd48b3640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01f99c0 0x7f6dd01f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 -- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01f99c0 msgr2=0x7f6dd01f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01f99c0 0x7f6dd01f9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 -- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00ce650 msgr2=0x7f6dd01f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00ce650 0x7f6dd01f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 -- 172.21.15.170:0/3939154078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd00cea60 con 0x7f6db8005310 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dcffff640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6dd00ce110 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00018d0 tx=0x7f6dc0001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0015070 con 0x7f6db8005310 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dc000eb50 con 0x7f6db8005310 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0011460 con 0x7f6db8005310 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd00b0e70 con 0x7f6db8005310 2024-09-21T02:18:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd0160dc0 con 0x7f6db8005310 2024-09-21T02:18:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.548+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd00704b0 con 0x7f6db8005310 2024-09-21T02:18:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.552+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f6dc0018070 con 0x7f6db8005310 2024-09-21T02:18:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.552+0000 7f6dccff9640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6dbc073350 0x7f6dbc075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.552+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7f6dc0094ec0 con 0x7f6db8005310 2024-09-21T02:18:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.552+0000 7f6dcf7fe640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6dbc073350 0x7f6dbc075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.552+0000 7f6dcf7fe640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6dbc073350 0x7f6dbc075860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40018b0 tx=0x7f6dc40011a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:10.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.556+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6dd00cea60 con 0x7f6db8005310 2024-09-21T02:18:10.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.792+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6dd0070c60 con 0x7f6db8005310 2024-09-21T02:18:10.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.792+0000 7f6dccff9640 1 -- 172.21.15.170:0/3939154078 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5304 (secure 0 0 0) 0x7f6dc00600f0 con 0x7f6db8005310 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.796 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:18:10.797 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:10.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:18:10.799 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:18:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.804 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:18:10.805 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.807 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:18:10.808 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:18:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6dbc073350 msgr2=0x7f6dbc075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6dbc073350 0x7f6dbc075860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40018b0 tx=0x7f6dc40011a0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 msgr2=0x7f6dd00ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6dd00ce110 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00018d0 tx=0x7f6dc0001d90 comp rx=0 tx=0).stop 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 shutdown_connections 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6dbc073350 0x7f6dbc075860 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01f99c0 0x7f6dd01f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00ce650 0x7f6dd01f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 --2- 172.21.15.170:0/3939154078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8005310 0x7f6dd00ce110 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 >> 172.21.15.170:0/3939154078 conn(0x7f6dd00b1c40 msgr2=0x7f6dd00ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 shutdown_connections 2024-09-21T02:18:10.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.796+0000 7f6dd6316640 1 -- 172.21.15.170:0/3939154078 wait complete. 2024-09-21T02:18:10.819 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 -- 172.21.15.170:0/3814813142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e00ce4c0 msgr2=0x7fb0e00ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/3814813142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e00ce4c0 0x7fb0e00ce8d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00cdc00 tx=0x7fb0d001c6e0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 -- 172.21.15.170:0/3814813142 shutdown_connections 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/3814813142 >> v2:172.21.15.191:3300/0 conn(0x7fb0e00cadf0 0x7fb0e00cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/3814813142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e00ce4c0 0x7fb0e00ce8d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 -- 172.21.15.170:0/3814813142 >> 172.21.15.170:0/3814813142 conn(0x7fb0e00b1c40 msgr2=0x7fb0e00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 -- 172.21.15.170:0/3814813142 shutdown_connections 2024-09-21T02:18:10.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.972+0000 7fb0e7f78640 1 -- 172.21.15.170:0/3814813142 wait complete. 2024-09-21T02:18:10.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 Processor -- start 2024-09-21T02:18:10.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 -- start start 2024-09-21T02:18:10.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e00cadf0 0x7fb0e00c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e00ce4c0 0x7fb0e00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 0x7fb0e00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0165fe0 con 0x7fb0e015f990 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0074f00 con 0x7fb0e00ce4c0 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e0166160 con 0x7fb0e00cadf0 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 0x7fb0e00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 0x7fb0e00b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41050/0 (socket says 172.21.15.170:41050) 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 -- 172.21.15.170:0/2164536983 learned_addr learned my addr 172.21.15.170:0/2164536983 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e5513640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e00ce4c0 0x7fb0e00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e5d14640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e00cadf0 0x7fb0e00c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 -- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e00cadf0 msgr2=0x7fb0e00c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e00cadf0 0x7fb0e00c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 -- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e00ce4c0 msgr2=0x7fb0e00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e00ce4c0 0x7fb0e00b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 -- 172.21.15.170:0/2164536983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e015c5f0 con 0x7fb0e015f990 2024-09-21T02:18:10.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e5d14640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e00cadf0 0x7fb0e00c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e6515640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 0x7fb0e00b4460 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800e9a0 tx=0x7fb0d800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8004d30 con 0x7fb0e015f990 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0d800cb20 con 0x7fb0e015f990 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e015c8d0 con 0x7fb0e015f990 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8005020 con 0x7fb0e015f990 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.976+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e015ce90 con 0x7fb0e015f990 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.980+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e00cb200 con 0x7fb0e015f990 2024-09-21T02:18:10.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.980+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fb0d80051c0 con 0x7fb0e015f990 2024-09-21T02:18:10.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.980+0000 7fb0d67fc640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb0c4073350 0x7fb0c4075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:10.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.980+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7fb0d8094a20 con 0x7fb0e015f990 2024-09-21T02:18:10.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.980+0000 7fb0e5d14640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb0c4073350 0x7fb0c4075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:10.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.980+0000 7fb0e5d14640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb0c4073350 0x7fb0c4075860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0098b0 tx=0x7fb0dc008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:10.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:10.984+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb0d8016030 con 0x7fb0e015f990 2024-09-21T02:18:11.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb0e00cd8a0 con 0x7fb0e015f990 2024-09-21T02:18:11.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0d67fc640 1 -- 172.21.15.170:0/2164536983 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7fb0d805f4a0 con 0x7fb0e015f990 2024-09-21T02:18:11.149 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:11.150 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:18:11.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:11.150 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:18:11.150 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:18:11.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:11.150 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb0c4073350 msgr2=0x7fb0c4075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb0c4073350 0x7fb0c4075860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0098b0 tx=0x7fb0dc008040 comp rx=0 tx=0).stop 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 msgr2=0x7fb0e00b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 0x7fb0e00b4460 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800e9a0 tx=0x7fb0d800ee60 comp rx=0 tx=0).stop 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 shutdown_connections 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb0c4073350 0x7fb0c4075860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e015f990 0x7fb0e00b4460 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0e00ce4c0 0x7fb0e00b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 --2- 172.21.15.170:0/2164536983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0e00cadf0 0x7fb0e00c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 >> 172.21.15.170:0/2164536983 conn(0x7fb0e00b1c40 msgr2=0x7fb0e00b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:11.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 shutdown_connections 2024-09-21T02:18:11.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.148+0000 7fb0e7f78640 1 -- 172.21.15.170:0/2164536983 wait complete. 2024-09-21T02:18:11.171 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:18:11.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 -- 172.21.15.170:0/374560771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35100cd1f0 msgr2=0x7f35100af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 --2- 172.21.15.170:0/374560771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35100cd1f0 0x7f35100af440 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3510072de0 tx=0x7f34f801ccf0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 -- 172.21.15.170:0/374560771 shutdown_connections 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 --2- 172.21.15.170:0/374560771 >> v2:172.21.15.191:3300/0 conn(0x7f35100b4d80 0x7f351006ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 --2- 172.21.15.170:0/374560771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35100cd1f0 0x7f35100af440 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 -- 172.21.15.170:0/374560771 >> 172.21.15.170:0/374560771 conn(0x7f35100b1c40 msgr2=0x7f35100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 -- 172.21.15.170:0/374560771 shutdown_connections 2024-09-21T02:18:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 -- 172.21.15.170:0/374560771 wait complete. 2024-09-21T02:18:11.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 Processor -- start 2024-09-21T02:18:11.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.324+0000 7f3514d2e640 1 -- start start 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35100b4d80 0x7f351015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35100b52d0 0x7f3510153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 0x7f3510153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3510169f10 con 0x7f3510152370 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3510169d90 con 0x7f35100b52d0 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f351016a090 con 0x7f35100b4d80 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 0x7f3510153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 0x7f3510153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41062/0 (socket says 172.21.15.170:41062) 2024-09-21T02:18:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 -- 172.21.15.170:0/25852968 learned_addr learned my addr 172.21.15.170:0/25852968 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350dd9b640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35100b52d0 0x7f3510153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350e59c640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35100b4d80 0x7f351015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 -- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35100b4d80 msgr2=0x7f351015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35100b4d80 0x7f351015b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 -- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35100b52d0 msgr2=0x7f3510153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35100b52d0 0x7f3510153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 -- 172.21.15.170:0/25852968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351015bf70 con 0x7f3510152370 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f350ed9d640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 0x7f3510153650 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f350000c970 tx=0x7f350000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500007bb0 con 0x7f3510152370 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f350000f450 con 0x7f3510152370 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500007e60 con 0x7f3510152370 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351015c250 con 0x7f3510152370 2024-09-21T02:18:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351015c810 con 0x7f3510152370 2024-09-21T02:18:11.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351006ef40 con 0x7f3510152370 2024-09-21T02:18:11.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f350000f8a0 con 0x7f3510152370 2024-09-21T02:18:11.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.328+0000 7f3506ffd640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34f0073320 0x7f34f0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:11.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.332+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 4857+0+0 (secure 0 0 0) 0x7f35000949f0 con 0x7f3510152370 2024-09-21T02:18:11.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.332+0000 7f350e59c640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34f0073320 0x7f34f0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:11.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.332+0000 7f350e59c640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34f0073320 0x7f34f0075830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3508006fd0 tx=0x7f3508008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:11.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.336+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3500014030 con 0x7f3510152370 2024-09-21T02:18:11.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:11.512+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f351015bd00 con 0x7f3510152370 2024-09-21T02:18:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.384+0000 7f3506ffd640 1 -- 172.21.15.170:0/25852968 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7f350005fbd0 con 0x7f3510152370 2024-09-21T02:18:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:18:12.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34f0073320 msgr2=0x7f34f0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:12.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34f0073320 0x7f34f0075830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3508006fd0 tx=0x7f3508008040 comp rx=0 tx=0).stop 2024-09-21T02:18:12.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 msgr2=0x7f3510153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:12.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 0x7f3510153650 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f350000c970 tx=0x7f350000ce30 comp rx=0 tx=0).stop 2024-09-21T02:18:12.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 shutdown_connections 2024-09-21T02:18:12.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34f0073320 0x7f34f0075830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510152370 0x7f3510153650 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35100b52d0 0x7f3510153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 --2- 172.21.15.170:0/25852968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35100b4d80 0x7f351015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 >> 172.21.15.170:0/25852968 conn(0x7f35100b1c40 msgr2=0x7f35100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:12.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 shutdown_connections 2024-09-21T02:18:12.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.388+0000 7f3514d2e640 1 -- 172.21.15.170:0/25852968 wait complete. 2024-09-21T02:18:12.422 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- 172.21.15.170:0/1657835169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e4415d920 msgr2=0x7f7e44071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1657835169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e4415d920 0x7f7e44071110 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7e440cf6a0 tx=0x7f7e3c01ccf0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- 172.21.15.170:0/1657835169 shutdown_connections 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1657835169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e4415d920 0x7f7e44071110 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1657835169 >> v2:172.21.15.191:3300/0 conn(0x7f7e44160af0 0x7f7e4415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- 172.21.15.170:0/1657835169 >> 172.21.15.170:0/1657835169 conn(0x7f7e440d2a30 msgr2=0x7f7e440cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- 172.21.15.170:0/1657835169 shutdown_connections 2024-09-21T02:18:12.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- 172.21.15.170:0/1657835169 wait complete. 2024-09-21T02:18:12.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 Processor -- start 2024-09-21T02:18:12.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- start start 2024-09-21T02:18:12.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e4415d920 0x7f7e441527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:12.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 0x7f7e44153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:12.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e44160af0 0x7f7e440c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:12.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4416a090 con 0x7f7e441606e0 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44169f10 con 0x7f7e4415d920 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e4b328640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4416a210 con 0x7f7e44160af0 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e488c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 0x7f7e44153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e488c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 0x7f7e44153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41092/0 (socket says 172.21.15.170:41092) 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.588+0000 7f7e488c3640 1 -- 172.21.15.170:0/1566084904 learned_addr learned my addr 172.21.15.170:0/1566084904 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e498c5640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e44160af0 0x7f7e440c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e490c4640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e4415d920 0x7f7e441527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e488c3640 1 -- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e44160af0 msgr2=0x7f7e440c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e488c3640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e44160af0 0x7f7e440c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e488c3640 1 -- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e4415d920 msgr2=0x7f7e441527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e488c3640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e4415d920 0x7f7e441527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e488c3640 1 -- 172.21.15.170:0/1566084904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e44152cf0 con 0x7f7e441606e0 2024-09-21T02:18:12.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e498c5640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e44160af0 0x7f7e440c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e488c3640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 0x7f7e44153cc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c970 tx=0x7f7e3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34007bb0 con 0x7f7e441606e0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e34007e40 con 0x7f7e441606e0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e440ca7c0 con 0x7f7e441606e0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400f6b0 con 0x7f7e441606e0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e440b5e00 con 0x7f7e441606e0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e440b61e0 con 0x7f7e441606e0 2024-09-21T02:18:12.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7e34004020 con 0x7f7e441606e0 2024-09-21T02:18:12.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.592+0000 7f7e39ffb640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e28073320 0x7f7e28075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:12.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.596+0000 7f7e490c4640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e28073320 0x7f7e28075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:12.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.596+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5227+0+0 (secure 0 0 0) 0x7f7e34094be0 con 0x7f7e441606e0 2024-09-21T02:18:12.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.596+0000 7f7e490c4640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e28073320 0x7f7e28075830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b810 tx=0x7f7e40008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.600+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7e440b61e0 con 0x7f7e441606e0 2024-09-21T02:18:12.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:12.772+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7e44071110 con 0x7f7e441606e0 2024-09-21T02:18:13.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.396+0000 7f7e39ffb640 1 -- 172.21.15.170:0/1566084904 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7f7e34014030 con 0x7f7e441606e0 2024-09-21T02:18:13.400 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:18:13.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.404+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e28073320 msgr2=0x7f7e28075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:13.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.404+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e28073320 0x7f7e28075830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b810 tx=0x7f7e40008040 comp rx=0 tx=0).stop 2024-09-21T02:18:13.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.404+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 msgr2=0x7f7e44153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.404+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 0x7f7e44153cc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c970 tx=0x7f7e3400ce30 comp rx=0 tx=0).stop 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.404+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 shutdown_connections 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e28073320 0x7f7e28075830 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e44160af0 0x7f7e440c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e441606e0 0x7f7e44153cc0 secure :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c970 tx=0x7f7e3400ce30 comp rx=0 tx=0).stop 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 --2- 172.21.15.170:0/1566084904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e4415d920 0x7f7e441527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 >> 172.21.15.170:0/1566084904 conn(0x7f7e440d2a30 msgr2=0x7f7e44156030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 shutdown_connections 2024-09-21T02:18:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.408+0000 7f7e4b328640 1 -- 172.21.15.170:0/1566084904 wait complete. 2024-09-21T02:18:13.450 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:18:13.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 -- 172.21.15.170:0/1669864661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 msgr2=0x7f034c0d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:13.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 --2- 172.21.15.170:0/1669864661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0d0b10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f034c074ba0 tx=0x7f033401c810 comp rx=0 tx=0).stop 2024-09-21T02:18:13.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 -- 172.21.15.170:0/1669864661 shutdown_connections 2024-09-21T02:18:13.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 --2- 172.21.15.170:0/1669864661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f034c15d3e0 0x7f034c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 --2- 172.21.15.170:0/1669864661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0d0b10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 --2- 172.21.15.170:0/1669864661 >> v2:172.21.15.191:3300/0 conn(0x7f034c1606e0 0x7f034c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 -- 172.21.15.170:0/1669864661 >> 172.21.15.170:0/1669864661 conn(0x7f034c0d2a30 msgr2=0x7f034c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:13.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 -- 172.21.15.170:0/1669864661 shutdown_connections 2024-09-21T02:18:13.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 -- 172.21.15.170:0/1669864661 wait complete. 2024-09-21T02:18:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 Processor -- start 2024-09-21T02:18:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.604+0000 7f035297a640 1 -- start start 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f034c15d3e0 0x7f034c159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f034c1606e0 0x7f034c1fa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0b4630 con 0x7f034c0d0700 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0b44b0 con 0x7f034c1606e0 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f034c0b47b0 con 0x7f034c15d3e0 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41108/0 (socket says 172.21.15.170:41108) 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 -- 172.21.15.170:0/2636345425 learned_addr learned my addr 172.21.15.170:0/2636345425 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034b7fe640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f034c15d3e0 0x7f034c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f0350f17640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f034c1606e0 0x7f034c1fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 -- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f034c15d3e0 msgr2=0x7f034c159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:13.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f034c15d3e0 0x7f034c159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 -- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f034c1606e0 msgr2=0x7f034c1fa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f034c1606e0 0x7f034c1fa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 -- 172.21.15.170:0/2636345425 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c158760 con 0x7f034c0d0700 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f034bfff640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0b53d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9f0 tx=0x7f033c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c00cd80 con 0x7f034c0d0700 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c158a40 con 0x7f034c0d0700 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f033c003a70 con 0x7f034c0d0700 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c004020 con 0x7f034c0d0700 2024-09-21T02:18:13.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c159000 con 0x7f034c0d0700 2024-09-21T02:18:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.608+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c15d7f0 con 0x7f034c0d0700 2024-09-21T02:18:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.612+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f033c0041c0 con 0x7f034c0d0700 2024-09-21T02:18:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.612+0000 7f0348ff9640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f032c073350 0x7f032c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.612+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5609+0+0 (secure 0 0 0) 0x7f033c0950f0 con 0x7f034c0d0700 2024-09-21T02:18:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.612+0000 7f034b7fe640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f032c073350 0x7f032c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.612+0000 7f034b7fe640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f032c073350 0x7f032c075860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03340af000 tx=0x7f033401b050 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:13.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.616+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f033c016030 con 0x7f034c0d0700 2024-09-21T02:18:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:13.824+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f034c0d1ac0 con 0x7f034c0d0700 2024-09-21T02:18:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.400+0000 7f0348ff9640 1 -- 172.21.15.170:0/2636345425 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v53) ==== 174+0+0 (secure 0 0 0) 0x7f033c05ffe0 con 0x7f034c0d0700 2024-09-21T02:18:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 12 and data pool 13 2024-09-21T02:18:14.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.404+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f032c073350 msgr2=0x7f032c075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f032c073350 0x7f032c075860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03340af000 tx=0x7f033401b050 comp rx=0 tx=0).stop 2024-09-21T02:18:14.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 msgr2=0x7f034c0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0b53d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9f0 tx=0x7f033c00eeb0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 shutdown_connections 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f032c073350 0x7f032c075860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f034c15d3e0 0x7f034c159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f034c1606e0 0x7f034c1fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 --2- 172.21.15.170:0/2636345425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f034c0d0700 0x7f034c0b53d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 >> 172.21.15.170:0/2636345425 conn(0x7f034c0d2a30 msgr2=0x7f034c156270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 shutdown_connections 2024-09-21T02:18:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.408+0000 7f035297a640 1 -- 172.21.15.170:0/2636345425 wait complete. 2024-09-21T02:18:14.458 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:14.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 -- 172.21.15.170:0/1009313192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 msgr2=0x7fd4b006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/1009313192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b006fa30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0071080 tx=0x7fd4a401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 -- 172.21.15.170:0/1009313192 shutdown_connections 2024-09-21T02:18:14.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/1009313192 >> v2:172.21.15.191:3300/0 conn(0x7fd4b00ccb00 0x7fd4b00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/1009313192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00b2f40 0x7fd4b00b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/1009313192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b006fa30 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 -- 172.21.15.170:0/1009313192 >> 172.21.15.170:0/1009313192 conn(0x7fd4b00b1c40 msgr2=0x7fd4b00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 -- 172.21.15.170:0/1009313192 shutdown_connections 2024-09-21T02:18:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 -- 172.21.15.170:0/1009313192 wait complete. 2024-09-21T02:18:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 Processor -- start 2024-09-21T02:18:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.608+0000 7fd4b48ea640 1 -- start start 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4b00b2f40 0x7fd4b01f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00ccb00 0x7fd4b009cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b00700f0 con 0x7fd4b00b5410 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b006ff70 con 0x7fd4b00b2f40 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b00ce800 con 0x7fd4b00ccb00 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b014b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41114/0 (socket says 172.21.15.170:41114) 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 -- 172.21.15.170:0/3383168965 learned_addr learned my addr 172.21.15.170:0/3383168965 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4ae59c640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4b00b2f40 0x7fd4b01f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4aed9d640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00ccb00 0x7fd4b009cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 -- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00ccb00 msgr2=0x7fd4b009cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00ccb00 0x7fd4b009cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 -- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4b00b2f40 msgr2=0x7fd4b01f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4b00b2f40 0x7fd4b01f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 -- 172.21.15.170:0/3383168965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01f7740 con 0x7fd4b00b5410 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4aed9d640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00ccb00 0x7fd4b009cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4ae59c640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4b00b2f40 0x7fd4b01f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4add9b640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b014b410 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000c6f0 tx=0x7fd4a000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0004020 con 0x7fd4b00b5410 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4a00042b0 con 0x7fd4b00b5410 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b01f7a20 con 0x7fd4b00b5410 2024-09-21T02:18:14.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0007c10 con 0x7fd4b00b5410 2024-09-21T02:18:14.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b00acfc0 con 0x7fd4b00b5410 2024-09-21T02:18:14.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b00ced60 con 0x7fd4b00b5410 2024-09-21T02:18:14.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.612+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fd4a0003600 con 0x7fd4b00b5410 2024-09-21T02:18:14.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.616+0000 7fd49effd640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd490073320 0x7fd490075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.616+0000 7fd4ae59c640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd490073320 0x7fd490075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.616+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fd4a0094480 con 0x7fd4b00b5410 2024-09-21T02:18:14.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.616+0000 7fd4ae59c640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd490073320 0x7fd490075830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40af000 tx=0x7fd4a401b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:14.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.620+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd4a0014030 con 0x7fd4b00b5410 2024-09-21T02:18:14.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.792+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4b01f74d0 con 0x7fd4b00b5410 2024-09-21T02:18:14.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.796+0000 7fd49effd640 1 -- 172.21.15.170:0/3383168965 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16793 (secure 0 0 0) 0x7fd4a005f310 con 0x7fd4b00b5410 2024-09-21T02:18:14.798 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:14.798 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":83,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:13.828645+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:11.529716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:12.787075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd490073320 msgr2=0x7fd490075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd490073320 0x7fd490075830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40af000 tx=0x7fd4a401b100 comp rx=0 tx=0).stop 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 msgr2=0x7fd4b014b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b014b410 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000c6f0 tx=0x7fd4a000cbb0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 shutdown_connections 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd490073320 0x7fd490075830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4b00ccb00 0x7fd4b009cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4b00b5410 0x7fd4b014b410 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 --2- 172.21.15.170:0/3383168965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4b00b2f40 0x7fd4b01f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 >> 172.21.15.170:0/3383168965 conn(0x7fd4b00b1c40 msgr2=0x7fd4b00c3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 shutdown_connections 2024-09-21T02:18:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.800+0000 7fd4b48ea640 1 -- 172.21.15.170:0/3383168965 wait complete. 2024-09-21T02:18:14.821 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 -- 172.21.15.170:0/1321499536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d1490 msgr2=0x7f79dc0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/1321499536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d1490 0x7f79dc0ced40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f79dc08cec0 tx=0x7f79cc01c740 comp rx=0 tx=0).stop 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 -- 172.21.15.170:0/1321499536 shutdown_connections 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/1321499536 >> v2:172.21.15.191:3300/0 conn(0x7f79dc0cf280 0x7f79dc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/1321499536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d1490 0x7f79dc0ced40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 -- 172.21.15.170:0/1321499536 >> 172.21.15.170:0/1321499536 conn(0x7f79dc09d830 msgr2=0x7f79dc08bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 -- 172.21.15.170:0/1321499536 shutdown_connections 2024-09-21T02:18:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.972+0000 7f79e3a9a640 1 -- 172.21.15.170:0/1321499536 wait complete. 2024-09-21T02:18:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 Processor -- start 2024-09-21T02:18:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 -- start start 2024-09-21T02:18:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79dc0cf280 0x7f79dc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79dc0d1490 0x7f79dc1f9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 0x7f79dc0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc16bd60 con 0x7f79dc0d3630 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc16bbe0 con 0x7f79dc0cf280 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e3a9a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc16bee0 con 0x7f79dc0d1490 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 0x7f79dc0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e1836640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79dc0cf280 0x7f79dc15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e1035640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79dc0d1490 0x7f79dc1f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e1836640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79dc0cf280 0x7f79dc15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38752/0 (socket says 172.21.15.170:38752) 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e1836640 1 -- 172.21.15.170:0/742572110 learned_addr learned my addr 172.21.15.170:0/742572110 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 0x7f79dc0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41128/0 (socket says 172.21.15.170:41128) 2024-09-21T02:18:14.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 -- 172.21.15.170:0/742572110 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79dc0d1490 msgr2=0x7f79dc1f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79dc0d1490 0x7f79dc1f9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 -- 172.21.15.170:0/742572110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79dc0cf280 msgr2=0x7f79dc15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:14.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79dc0cf280 0x7f79dc15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:14.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 -- 172.21.15.170:0/742572110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc15d280 con 0x7f79dc0d3630 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.976+0000 7f79e2037640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 0x7f79dc0cdc10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f79d400e990 tx=0x7f79d400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4004d30 con 0x7f79dc0d3630 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79dc15d560 con 0x7f79dc0d3630 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79d400caa0 con 0x7f79dc0d3630 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4005020 con 0x7f79dc0d3630 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79dc0b1bd0 con 0x7f79dc0d3630 2024-09-21T02:18:14.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79dc06a820 con 0x7f79dc0d3630 2024-09-21T02:18:14.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f79d40051c0 con 0x7f79dc0d3630 2024-09-21T02:18:14.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79d27fc640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f79c0073350 0x7f79c0075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:14.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f79d40950d0 con 0x7f79dc0d3630 2024-09-21T02:18:14.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79e1836640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f79c0073350 0x7f79c0075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:14.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.980+0000 7f79e1836640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f79c0073350 0x7f79c0075860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000f70 tx=0x7f79d8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:14.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:14.984+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f79d4016030 con 0x7f79dc0d3630 2024-09-21T02:18:15.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.160+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79dc0cf690 con 0x7f79dc0d3630 2024-09-21T02:18:15.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79d27fc640 1 -- 172.21.15.170:0/742572110 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16793 (secure 0 0 0) 0x7f79d405f910 con 0x7f79dc0d3630 2024-09-21T02:18:15.165 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:15.165 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":83,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:13.828645+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:11.529716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:12.787075+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:15.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f79c0073350 msgr2=0x7f79c0075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:15.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f79c0073350 0x7f79c0075860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000f70 tx=0x7f79d8009040 comp rx=0 tx=0).stop 2024-09-21T02:18:15.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 msgr2=0x7f79dc0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:15.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 0x7f79dc0cdc10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f79d400e990 tx=0x7f79d400ee50 comp rx=0 tx=0).stop 2024-09-21T02:18:15.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 shutdown_connections 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f79c0073350 0x7f79c0075860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79dc0d1490 0x7f79dc1f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79dc0d3630 0x7f79dc0cdc10 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 --2- 172.21.15.170:0/742572110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79dc0cf280 0x7f79dc15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 >> 172.21.15.170:0/742572110 conn(0x7f79dc09d830 msgr2=0x7f79dc06f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 shutdown_connections 2024-09-21T02:18:15.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.164+0000 7f79e3a9a640 1 -- 172.21.15.170:0/742572110 wait complete. 2024-09-21T02:18:15.188 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 -- 172.21.15.170:0/86940852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980cf7d0 msgr2=0x7f1598073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 --2- 172.21.15.170:0/86940852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980cf7d0 0x7f1598073b60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f15980726b0 tx=0x7f159401c8a0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 -- 172.21.15.170:0/86940852 shutdown_connections 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 --2- 172.21.15.170:0/86940852 >> v2:172.21.15.191:3300/0 conn(0x7f15980740a0 0x7f15980744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 --2- 172.21.15.170:0/86940852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980cf7d0 0x7f1598073b60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 -- 172.21.15.170:0/86940852 >> 172.21.15.170:0/86940852 conn(0x7f15980d0170 msgr2=0x7f15980d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 -- 172.21.15.170:0/86940852 shutdown_connections 2024-09-21T02:18:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.340+0000 7f159fc9e640 1 -- 172.21.15.170:0/86940852 wait complete. 2024-09-21T02:18:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 Processor -- start 2024-09-21T02:18:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 -- start start 2024-09-21T02:18:15.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15980740a0 0x7f15980c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:15.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 0x7f15981fca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:15.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15980cf7d0 0x7f159806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:15.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1598169040 con 0x7f15980ce070 2024-09-21T02:18:15.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1598168ec0 con 0x7f15980cf7d0 2024-09-21T02:18:15.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f15981691c0 con 0x7f15980740a0 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 0x7f15981fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 0x7f15981fca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41146/0 (socket says 172.21.15.170:41146) 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 -- 172.21.15.170:0/4195622054 learned_addr learned my addr 172.21.15.170:0/4195622054 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159da3a640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15980740a0 0x7f15980c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159e23b640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15980cf7d0 0x7f159806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 -- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15980740a0 msgr2=0x7f15980c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15980740a0 0x7f15980c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 -- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15980cf7d0 msgr2=0x7f159806f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15980cf7d0 0x7f159806f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 -- 172.21.15.170:0/4195622054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159815a050 con 0x7f15980ce070 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159d239640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 0x7f15981fca80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f158800c970 tx=0x7f158800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588004220 con 0x7f15980ce070 2024-09-21T02:18:15.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15880079b0 con 0x7f15980ce070 2024-09-21T02:18:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880044b0 con 0x7f15980ce070 2024-09-21T02:18:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159815a330 con 0x7f15980ce070 2024-09-21T02:18:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.344+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159815a8f0 con 0x7f15980ce070 2024-09-21T02:18:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.348+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15980744b0 con 0x7f15980ce070 2024-09-21T02:18:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.348+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f1588003600 con 0x7f15980ce070 2024-09-21T02:18:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.348+0000 7f158e7fc640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f157c073320 0x7f157c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.348+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f1588094530 con 0x7f15980ce070 2024-09-21T02:18:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.348+0000 7f159da3a640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f157c073320 0x7f157c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.348+0000 7f159da3a640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f157c073320 0x7f157c075830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15940af000 tx=0x7f159401b0e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:15.353 INFO:tasks.ceph.mds.a.smithi170.stderr:2024-09-21T02:18:15.352+0000 7f99df9df640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:18:15.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.352+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1588014030 con 0x7f15980ce070 2024-09-21T02:18:15.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:15.560+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f15980ce690 con 0x7f15980ce070 2024-09-21T02:18:16.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.440+0000 7f158e7fc640 1 -- 172.21.15.170:0/4195622054 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7f158805f3b0 con 0x7f15980ce070 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f157c073320 msgr2=0x7f157c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f157c073320 0x7f157c075830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15940af000 tx=0x7f159401b0e0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 msgr2=0x7f15981fca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 0x7f15981fca80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f158800c970 tx=0x7f158800ce30 comp rx=0 tx=0).stop 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 shutdown_connections 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f157c073320 0x7f157c075830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15980cf7d0 0x7f159806f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15980ce070 0x7f15981fca80 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 --2- 172.21.15.170:0/4195622054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15980740a0 0x7f15980c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 >> 172.21.15.170:0/4195622054 conn(0x7f15980d0170 msgr2=0x7f1598075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 shutdown_connections 2024-09-21T02:18:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.444+0000 7f159fc9e640 1 -- 172.21.15.170:0/4195622054 wait complete. 2024-09-21T02:18:16.468 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:18:16.469 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d7da1640 1 -- 172.21.15.170:0/3637130585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8003a40 msgr2=0x7fb1b8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/3637130585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1b8003a40 0x7fb1b8003eb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0001d60 tx=0x7fb1c001cb20 comp rx=0 tx=0).stop 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d633e640 1 -- 172.21.15.170:0/3637130585 reap_dead start 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d7da1640 1 -- 172.21.15.170:0/3637130585 shutdown_connections 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d7da1640 1 -- 172.21.15.170:0/3637130585 >> 172.21.15.170:0/3637130585 conn(0x7fb1d00d0170 msgr2=0x7fb1d00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d7da1640 1 -- 172.21.15.170:0/3637130585 shutdown_connections 2024-09-21T02:18:16.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.628+0000 7fb1d7da1640 1 -- 172.21.15.170:0/3637130585 wait complete. 2024-09-21T02:18:16.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 Processor -- start 2024-09-21T02:18:16.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- start start 2024-09-21T02:18:16.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1b8003a40 0x7fb1d0166020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:16.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 0x7fb1d014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:16.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d00c9650 0x7fb1d0147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0168fe0 con 0x7fb1d0164810 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d00b3610 con 0x7fb1d00c9650 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d0169160 con 0x7fb1b8003a40 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 0x7fb1d014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 0x7fb1d014deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45622/0 (socket says 172.21.15.170:45622) 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 -- 172.21.15.170:0/71133338 learned_addr learned my addr 172.21.15.170:0/71133338 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d5b3d640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1b8003a40 0x7fb1d0166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d633e640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d00c9650 0x7fb1d0147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 -- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1b8003a40 msgr2=0x7fb1d0166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1b8003a40 0x7fb1d0166020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 -- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d00c9650 msgr2=0x7fb1d0147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d00c9650 0x7fb1d0147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:16.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 -- 172.21.15.170:0/71133338 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d0164f80 con 0x7fb1d0164810 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d633e640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d00c9650 0x7fb1d0147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d533c640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 0x7fb1d014deb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d900 tx=0x7fb1c400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40064e0 con 0x7fb1d0164810 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1c400b8f0 con 0x7fb1d0164810 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d0165260 con 0x7fb1d0164810 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400bce0 con 0x7fb1d0164810 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d0165820 con 0x7fb1d0164810 2024-09-21T02:18:16.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.632+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d0164cf0 con 0x7fb1d0164810 2024-09-21T02:18:16.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.636+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fb1c4003600 con 0x7fb1d0164810 2024-09-21T02:18:16.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.636+0000 7fb1ce7fc640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb1bc073320 0x7fb1bc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:16.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.636+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fb1c4094b00 con 0x7fb1d0164810 2024-09-21T02:18:16.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.636+0000 7fb1d5b3d640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb1bc073320 0x7fb1bc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:16.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.636+0000 7fb1d5b3d640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb1bc073320 0x7fb1bc075830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00aa000 tx=0x7fb1c00a7040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:16.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.640+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb1c405f990 con 0x7fb1d0164810 2024-09-21T02:18:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:16.844+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb1d009da80 con 0x7fb1d0164810 2024-09-21T02:18:17.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.448+0000 7fb1ce7fc640 1 -- 172.21.15.170:0/71133338 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7fb1c405f290 con 0x7fb1d0164810 2024-09-21T02:18:17.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb1bc073320 msgr2=0x7fb1bc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb1bc073320 0x7fb1bc075830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00aa000 tx=0x7fb1c00a7040 comp rx=0 tx=0).stop 2024-09-21T02:18:17.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 msgr2=0x7fb1d014deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 0x7fb1d014deb0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d900 tx=0x7fb1c400ddc0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 shutdown_connections 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb1bc073320 0x7fb1bc075830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d00c9650 0x7fb1d0147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1d0164810 0x7fb1d014deb0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 --2- 172.21.15.170:0/71133338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1b8003a40 0x7fb1d0166020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 >> 172.21.15.170:0/71133338 conn(0x7fb1d00d0170 msgr2=0x7fb1d006fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 shutdown_connections 2024-09-21T02:18:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.452+0000 7fb1d7da1640 1 -- 172.21.15.170:0/71133338 wait complete. 2024-09-21T02:18:17.477 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:18:17.477 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 -- 172.21.15.170:0/1761555183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 msgr2=0x7f8ba40cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 --2- 172.21.15.170:0/1761555183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 0x7f8ba40cf970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009870 tx=0x7f8ba001c7d0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 -- 172.21.15.170:0/1761555183 shutdown_connections 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 --2- 172.21.15.170:0/1761555183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 0x7f8ba40cf970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 -- 172.21.15.170:0/1761555183 >> 172.21.15.170:0/1761555183 conn(0x7f8ba408bf50 msgr2=0x7f8ba409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 -- 172.21.15.170:0/1761555183 shutdown_connections 2024-09-21T02:18:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.628+0000 7f8bab804640 1 -- 172.21.15.170:0/1761555183 wait complete. 2024-09-21T02:18:17.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 Processor -- start 2024-09-21T02:18:17.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 -- start start 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ba40b1230 0x7f8ba40ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 0x7f8ba40cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 0x7f8ba4155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba416d0e0 con 0x7f8ba40d0e50 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba416cf60 con 0x7f8ba40b1230 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba416d260 con 0x7f8ba40b3fa0 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 0x7f8ba4155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 0x7f8ba4155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45638/0 (socket says 172.21.15.170:45638) 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 -- 172.21.15.170:0/3860523820 learned_addr learned my addr 172.21.15.170:0/3860523820 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba8d9f640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 0x7f8ba40cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba95a0640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ba40b1230 0x7f8ba40ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 -- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 msgr2=0x7f8ba40cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 0x7f8ba40cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 -- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ba40b1230 msgr2=0x7f8ba40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ba40b1230 0x7f8ba40ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 -- 172.21.15.170:0/3860523820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba40ce550 con 0x7f8ba40d0e50 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8ba9da1640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 0x7f8ba4155330 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c730 tx=0x7f8b9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c003ba0 con 0x7f8ba40d0e50 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b9c003d60 con 0x7f8ba40d0e50 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba40ccf10 con 0x7f8ba40d0e50 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c005330 con 0x7f8ba40d0e50 2024-09-21T02:18:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.632+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba40cc950 con 0x7f8ba40d0e50 2024-09-21T02:18:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.636+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba40cc6c0 con 0x7f8ba40d0e50 2024-09-21T02:18:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.636+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f8b9c0056e0 con 0x7f8ba40d0e50 2024-09-21T02:18:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.636+0000 7f8b99ffb640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b88073350 0x7f8b88075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.636+0000 7f8ba95a0640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b88073350 0x7f8b88075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.636+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f8b9c094500 con 0x7f8ba40d0e50 2024-09-21T02:18:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.636+0000 7f8ba95a0640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b88073350 0x7f8b88075860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00af000 tx=0x7f8ba001b010 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:17.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.640+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8b9c05f300 con 0x7f8ba40d0e50 2024-09-21T02:18:17.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.832+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ba40cf970 con 0x7f8ba40d0e50 2024-09-21T02:18:17.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.832+0000 7f8b99ffb640 1 -- 172.21.15.170:0/3860523820 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5237 (secure 0 0 0) 0x7f8b9c05ec00 con 0x7f8ba40d0e50 2024-09-21T02:18:17.836 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:17.836 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":56,"btime":"2024-09-21T02:18:17:443999+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2173918761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2173918761},{"type":"v1","addr":"172.21.15.191:6835","nonce":2173918761}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:13.828587+0000","modified":"2024-09-21T02:18:16.848304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1472234860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1472234860},{"type":"v1","addr":"172.21.15.170:6835","nonce":1472234860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":6}]} 2024-09-21T02:18:17.836 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 56 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b88073350 msgr2=0x7f8b88075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b88073350 0x7f8b88075860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00af000 tx=0x7f8ba001b010 comp rx=0 tx=0).stop 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 msgr2=0x7f8ba4155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 0x7f8ba4155330 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c730 tx=0x7f8b9c00cbf0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 shutdown_connections 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b88073350 0x7f8b88075860 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ba40b3fa0 0x7f8ba40cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ba40d0e50 0x7f8ba4155330 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 --2- 172.21.15.170:0/3860523820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ba40b1230 0x7f8ba40ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 >> 172.21.15.170:0/3860523820 conn(0x7f8ba408bf50 msgr2=0x7f8ba40d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:17.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 shutdown_connections 2024-09-21T02:18:17.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.836+0000 7f8bab804640 1 -- 172.21.15.170:0/3860523820 wait complete. 2024-09-21T02:18:17.857 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:17.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 -- 172.21.15.170:0/2267714878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f14160610 msgr2=0x7f3f140b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:17.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 --2- 172.21.15.170:0/2267714878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f14160610 0x7f3f140b44a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009c80 tx=0x7f3f0c01cda0 comp rx=0 tx=0).stop 2024-09-21T02:18:17.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 -- 172.21.15.170:0/2267714878 shutdown_connections 2024-09-21T02:18:17.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 --2- 172.21.15.170:0/2267714878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f14160610 0x7f3f140b44a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 --2- 172.21.15.170:0/2267714878 >> v2:172.21.15.191:3300/0 conn(0x7f3f140b25f0 0x7f3f140cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 -- 172.21.15.170:0/2267714878 >> 172.21.15.170:0/2267714878 conn(0x7f3f1409a9c0 msgr2=0x7f3f140d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:18.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 -- 172.21.15.170:0/2267714878 shutdown_connections 2024-09-21T02:18:18.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:17.996+0000 7f3f1c15d640 1 -- 172.21.15.170:0/2267714878 wait complete. 2024-09-21T02:18:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 Processor -- start 2024-09-21T02:18:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 -- start start 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f140b25f0 0x7f3f14157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1415faf0 0x7f3f141fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 0x7f3f1415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f14168f40 con 0x7f3f14160610 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3f14168dc0 con 0x7f3f1415faf0 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3f141690c0 con 0x7f3f140b25f0 2024-09-21T02:18:18.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 0x7f3f1415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 0x7f3f1415ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45652/0 (socket says 172.21.15.170:45652) 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 -- 172.21.15.170:0/3133577982 learned_addr learned my addr 172.21.15.170:0/3133577982 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f19ef9640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f140b25f0 0x7f3f14157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f196f8640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1415faf0 0x7f3f141fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 -- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f140b25f0 msgr2=0x7f3f14157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f140b25f0 0x7f3f14157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 -- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1415faf0 msgr2=0x7f3f141fcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1415faf0 0x7f3f141fcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 -- 172.21.15.170:0/3133577982 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f141fd160 con 0x7f3f14160610 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1a6fa640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 0x7f3f1415ee80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c01b450 tx=0x7f3f0c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0bc070 con 0x7f3f14160610 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 -- 172.21.15.170:0/3133577982 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1415f3c0 con 0x7f3f14160610 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f0c0ad6e0 con 0x7f3f14160610 2024-09-21T02:18:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0b2040 con 0x7f3f14160610 2024-09-21T02:18:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.000+0000 7f3f1c15d640 1 -- 172.21.15.170:0/3133577982 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f14158240 con 0x7f3f14160610 2024-09-21T02:18:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.004+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f14157fb0 con 0x7f3f14160610 2024-09-21T02:18:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.004+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f3f0c0adb30 con 0x7f3f14160610 2024-09-21T02:18:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.004+0000 7f3f0a7fc640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3ef8073320 0x7f3ef8075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.004+0000 7f3f19ef9640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3ef8073320 0x7f3ef8075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.004+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f3f0c12e510 con 0x7f3f14160610 2024-09-21T02:18:18.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.004+0000 7f3f19ef9640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3ef8073320 0x7f3ef8075830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005fd0 tx=0x7f3f10005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:18.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.008+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3f0c0c5080 con 0x7f3f14160610 2024-09-21T02:18:18.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.172+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f140cf950 con 0x7f3f14160610 2024-09-21T02:18:18.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.176+0000 7f3f0a7fc640 1 -- 172.21.15.170:0/3133577982 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16780 (secure 0 0 0) 0x7f3f141fd160 con 0x7f3f14160610 2024-09-21T02:18:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:18.179 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":84,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:15.404674+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:11.529716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:12.787075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3ef8073320 msgr2=0x7f3ef8075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3ef8073320 0x7f3ef8075830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005fd0 tx=0x7f3f10005f40 comp rx=0 tx=0).stop 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 msgr2=0x7f3f1415ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 0x7f3f1415ee80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c01b450 tx=0x7f3f0c005ae0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 shutdown_connections 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3ef8073320 0x7f3ef8075830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f14160610 0x7f3f1415ee80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f1415faf0 0x7f3f141fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 --2- 172.21.15.170:0/3133577982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f140b25f0 0x7f3f14157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 >> 172.21.15.170:0/3133577982 conn(0x7f3f1409a9c0 msgr2=0x7f3f141562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 shutdown_connections 2024-09-21T02:18:18.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.180+0000 7f3ef7fff640 1 -- 172.21.15.170:0/3133577982 wait complete. 2024-09-21T02:18:18.201 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:18:18.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- 172.21.15.170:0/1089544568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90003a40 msgr2=0x7f2a90003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/1089544568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90003a40 0x7f2a90003eb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009ee0 tx=0x7f2a9c01d610 comp rx=0 tx=0).stop 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aa9256640 1 -- 172.21.15.170:0/1089544568 reap_dead start 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- 172.21.15.170:0/1089544568 shutdown_connections 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- 172.21.15.170:0/1089544568 >> 172.21.15.170:0/1089544568 conn(0x7f2aa40cee30 msgr2=0x7f2aa40cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- 172.21.15.170:0/1089544568 shutdown_connections 2024-09-21T02:18:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- 172.21.15.170:0/1089544568 wait complete. 2024-09-21T02:18:18.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 Processor -- start 2024-09-21T02:18:18.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- start start 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90003a40 0x7f2aa40c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa40caef0 0x7f2aa41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 0x7f2aa415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4168da0 con 0x7f2aa415e8f0 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4168c20 con 0x7f2aa40caef0 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aaacb9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa4168f20 con 0x7f2a90003a40 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aa9256640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 0x7f2aa415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aa9256640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 0x7f2aa415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45680/0 (socket says 172.21.15.170:45680) 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.356+0000 7f2aa9256640 1 -- 172.21.15.170:0/2276871768 learned_addr learned my addr 172.21.15.170:0/2276871768 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa8a55640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90003a40 0x7f2aa40c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa3fff640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa40caef0 0x7f2aa41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa9256640 1 -- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90003a40 msgr2=0x7f2aa40c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa9256640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90003a40 0x7f2aa40c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa9256640 1 -- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa40caef0 msgr2=0x7f2aa41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa9256640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa40caef0 0x7f2aa41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa9256640 1 -- 172.21.15.170:0/2276871768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa415f240 con 0x7f2aa415e8f0 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa3fff640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa40caef0 0x7f2aa41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa8a55640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90003a40 0x7f2aa40c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa9256640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 0x7f2aa415ed00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2a94001020 tx=0x7f2a9400e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94011870 con 0x7f2aa415e8f0 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a94017070 con 0x7f2aa415e8f0 2024-09-21T02:18:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa40c9d60 con 0x7f2aa415e8f0 2024-09-21T02:18:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94011d80 con 0x7f2aa415e8f0 2024-09-21T02:18:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa40ca320 con 0x7f2aa415e8f0 2024-09-21T02:18:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa40c9ad0 con 0x7f2aa415e8f0 2024-09-21T02:18:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f2a9401c460 con 0x7f2aa415e8f0 2024-09-21T02:18:18.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa17fa640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2a84073320 0x7f2a84075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f2a94094820 con 0x7f2aa415e8f0 2024-09-21T02:18:18.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa8a55640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2a84073320 0x7f2a84075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.360+0000 7f2aa8a55640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2a84073320 0x7f2a84075830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0af000 tx=0x7f2a9c01a3b0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:18.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.364+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2aa415f240 con 0x7f2aa415e8f0 2024-09-21T02:18:18.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.532+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2aa406a270 con 0x7f2a84073320 2024-09-21T02:18:18.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.540+0000 7f2aa17fa640 1 -- 172.21.15.170:0/2276871768 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7f2a9c0aa4e0 con 0x7f2a84073320 2024-09-21T02:18:18.545 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:18.546 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:18:18.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2a84073320 msgr2=0x7f2a84075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2a84073320 0x7f2a84075830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0af000 tx=0x7f2a9c01a3b0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 msgr2=0x7f2aa415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 0x7f2aa415ed00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2a94001020 tx=0x7f2a9400e7f0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 shutdown_connections 2024-09-21T02:18:18.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2a84073320 0x7f2a84075830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa415e8f0 0x7f2aa415ed00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2aa40caef0 0x7f2aa41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 --2- 172.21.15.170:0/2276871768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90003a40 0x7f2aa40c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 >> 172.21.15.170:0/2276871768 conn(0x7f2aa40cee30 msgr2=0x7f2aa40730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:18.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 shutdown_connections 2024-09-21T02:18:18.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.544+0000 7f2aaacb9640 1 -- 172.21.15.170:0/2276871768 wait complete. 2024-09-21T02:18:18.565 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T02:18:16.859703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275192,"kb_used_data":9024,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699528,"statfs":{"total":773094113280,"available":772812316672,"internally_reserved":0,"allocated":9240576,"data_stored":4604713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.469243"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.444091+0000","last_change":"2024-09-21T02:18:14.442511+0000","last_active":"2024-09-21T02:18:14.444091+0000","last_peered":"2024-09-21T02:18:14.444091+0000","last_clean":"2024-09-21T02:18:14.444091+0000","last_became_active":"2024-09-21T02:18:14.442206+0000","last_became_peered":"2024-09-21T02:18:14.442206+0000","last_unstale":"2024-09-21T02:18:14.444091+0000","last_undegraded":"2024-09-21T02:18:14.444091+0000","last_fullsized":"2024-09-21T02:18:14.444091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:33.298823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.454938+0000","last_change":"2024-09-21T02:18:14.440433+0000","last_active":"2024-09-21T02:18:14.454938+0000","last_peered":"2024-09-21T02:18:14.454938+0000","last_clean":"2024-09-21T02:18:14.454938+0000","last_became_active":"2024-09-21T02:18:14.439408+0000","last_became_peered":"2024-09-21T02:18:14.439408+0000","last_unstale":"2024-09-21T02:18:14.454938+0000","last_undegraded":"2024-09-21T02:18:14.454938+0000","last_fullsized":"2024-09-21T02:18:14.454938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:49.889056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.443201+0000","last_change":"2024-09-21T02:18:14.440910+0000","last_active":"2024-09-21T02:18:14.443201+0000","last_peered":"2024-09-21T02:18:14.443201+0000","last_clean":"2024-09-21T02:18:14.443201+0000","last_became_active":"2024-09-21T02:18:14.440583+0000","last_became_peered":"2024-09-21T02:18:14.440583+0000","last_unstale":"2024-09-21T02:18:14.443201+0000","last_undegraded":"2024-09-21T02:18:14.443201+0000","last_fullsized":"2024-09-21T02:18:14.443201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.283268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.456222+0000","last_change":"2024-09-21T02:18:14.449909+0000","last_active":"2024-09-21T02:18:14.456222+0000","last_peered":"2024-09-21T02:18:14.456222+0000","last_clean":"2024-09-21T02:18:14.456222+0000","last_became_active":"2024-09-21T02:18:14.440669+0000","last_became_peered":"2024-09-21T02:18:14.440669+0000","last_unstale":"2024-09-21T02:18:14.456222+0000","last_undegraded":"2024-09-21T02:18:14.456222+0000","last_fullsized":"2024-09-21T02:18:14.456222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:41.028084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.452111+0000","last_change":"2024-09-21T02:18:14.451394+0000","last_active":"2024-09-21T02:18:14.452111+0000","last_peered":"2024-09-21T02:18:14.452111+0000","last_clean":"2024-09-21T02:18:14.452111+0000","last_became_active":"2024-09-21T02:18:14.451109+0000","last_became_peered":"2024-09-21T02:18:14.451109+0000","last_unstale":"2024-09-21T02:18:14.452111+0000","last_undegraded":"2024-09-21T02:18:14.452111+0000","last_fullsized":"2024-09-21T02:18:14.452111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:37.095580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458276+0000","last_change":"2024-09-21T02:18:14.457134+0000","last_active":"2024-09-21T02:18:14.458276+0000","last_peered":"2024-09-21T02:18:14.458276+0000","last_clean":"2024-09-21T02:18:14.458276+0000","last_became_active":"2024-09-21T02:18:14.456645+0000","last_became_peered":"2024-09-21T02:18:14.456645+0000","last_unstale":"2024-09-21T02:18:14.458276+0000","last_undegraded":"2024-09-21T02:18:14.458276+0000","last_fullsized":"2024-09-21T02:18:14.458276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:08.567073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.428765+0000","last_change":"2024-09-21T02:18:14.419796+0000","last_active":"2024-09-21T02:18:14.428765+0000","last_peered":"2024-09-21T02:18:14.428765+0000","last_clean":"2024-09-21T02:18:14.428765+0000","last_became_active":"2024-09-21T02:18:14.418913+0000","last_became_peered":"2024-09-21T02:18:14.418913+0000","last_unstale":"2024-09-21T02:18:14.428765+0000","last_undegraded":"2024-09-21T02:18:14.428765+0000","last_fullsized":"2024-09-21T02:18:14.428765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:11.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478386+0000","last_change":"2024-09-21T02:18:14.477543+0000","last_active":"2024-09-21T02:18:14.478386+0000","last_peered":"2024-09-21T02:18:14.478386+0000","last_clean":"2024-09-21T02:18:14.478386+0000","last_became_active":"2024-09-21T02:18:14.477189+0000","last_became_peered":"2024-09-21T02:18:14.477189+0000","last_unstale":"2024-09-21T02:18:14.478386+0000","last_undegraded":"2024-09-21T02:18:14.478386+0000","last_fullsized":"2024-09-21T02:18:14.478386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:16:05.955688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.435557+0000","last_change":"2024-09-21T02:18:14.433850+0000","last_active":"2024-09-21T02:18:14.435557+0000","last_peered":"2024-09-21T02:18:14.435557+0000","last_clean":"2024-09-21T02:18:14.435557+0000","last_became_active":"2024-09-21T02:18:14.433558+0000","last_became_peered":"2024-09-21T02:18:14.433558+0000","last_unstale":"2024-09-21T02:18:14.435557+0000","last_undegraded":"2024-09-21T02:18:14.435557+0000","last_fullsized":"2024-09-21T02:18:14.435557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:43.479479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458291+0000","last_change":"2024-09-21T02:18:14.457478+0000","last_active":"2024-09-21T02:18:14.458291+0000","last_peered":"2024-09-21T02:18:14.458291+0000","last_clean":"2024-09-21T02:18:14.458291+0000","last_became_active":"2024-09-21T02:18:14.457224+0000","last_became_peered":"2024-09-21T02:18:14.457224+0000","last_unstale":"2024-09-21T02:18:14.458291+0000","last_undegraded":"2024-09-21T02:18:14.458291+0000","last_fullsized":"2024-09-21T02:18:14.458291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:58.078603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.476915+0000","last_change":"2024-09-21T02:18:14.471915+0000","last_active":"2024-09-21T02:18:14.476915+0000","last_peered":"2024-09-21T02:18:14.476915+0000","last_clean":"2024-09-21T02:18:14.476915+0000","last_became_active":"2024-09-21T02:18:14.471304+0000","last_became_peered":"2024-09-21T02:18:14.471304+0000","last_unstale":"2024-09-21T02:18:14.476915+0000","last_undegraded":"2024-09-21T02:18:14.476915+0000","last_fullsized":"2024-09-21T02:18:14.476915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:32.610748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449960+0000","last_change":"2024-09-21T02:18:14.449041+0000","last_active":"2024-09-21T02:18:14.449960+0000","last_peered":"2024-09-21T02:18:14.449960+0000","last_clean":"2024-09-21T02:18:14.449960+0000","last_became_active":"2024-09-21T02:18:14.448665+0000","last_became_peered":"2024-09-21T02:18:14.448665+0000","last_unstale":"2024-09-21T02:18:14.449960+0000","last_undegraded":"2024-09-21T02:18:14.449960+0000","last_fullsized":"2024-09-21T02:18:14.449960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:26.562143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.439544+0000","last_change":"2024-09-21T02:18:14.432786+0000","last_active":"2024-09-21T02:18:14.439544+0000","last_peered":"2024-09-21T02:18:14.439544+0000","last_clean":"2024-09-21T02:18:14.439544+0000","last_became_active":"2024-09-21T02:18:14.432372+0000","last_became_peered":"2024-09-21T02:18:14.432372+0000","last_unstale":"2024-09-21T02:18:14.439544+0000","last_undegraded":"2024-09-21T02:18:14.439544+0000","last_fullsized":"2024-09-21T02:18:14.439544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:28.646671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.460701+0000","last_change":"2024-09-21T02:18:14.453043+0000","last_active":"2024-09-21T02:18:14.460701+0000","last_peered":"2024-09-21T02:18:14.460701+0000","last_clean":"2024-09-21T02:18:14.460701+0000","last_became_active":"2024-09-21T02:18:14.452219+0000","last_became_peered":"2024-09-21T02:18:14.452219+0000","last_unstale":"2024-09-21T02:18:14.460701+0000","last_undegraded":"2024-09-21T02:18:14.460701+0000","last_fullsized":"2024-09-21T02:18:14.460701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:33.999188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.459339+0000","last_change":"2024-09-21T02:18:14.456962+0000","last_active":"2024-09-21T02:18:14.459339+0000","last_peered":"2024-09-21T02:18:14.459339+0000","last_clean":"2024-09-21T02:18:14.459339+0000","last_became_active":"2024-09-21T02:18:14.455983+0000","last_became_peered":"2024-09-21T02:18:14.455983+0000","last_unstale":"2024-09-21T02:18:14.459339+0000","last_undegraded":"2024-09-21T02:18:14.459339+0000","last_fullsized":"2024-09-21T02:18:14.459339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:46.728150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.443258+0000","last_change":"2024-09-21T02:18:14.437566+0000","last_active":"2024-09-21T02:18:14.443258+0000","last_peered":"2024-09-21T02:18:14.443258+0000","last_clean":"2024-09-21T02:18:14.443258+0000","last_became_active":"2024-09-21T02:18:14.436791+0000","last_became_peered":"2024-09-21T02:18:14.436791+0000","last_unstale":"2024-09-21T02:18:14.443258+0000","last_undegraded":"2024-09-21T02:18:14.443258+0000","last_fullsized":"2024-09-21T02:18:14.443258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:05.776334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.450208+0000","last_change":"2024-09-21T02:18:14.449189+0000","last_active":"2024-09-21T02:18:14.450208+0000","last_peered":"2024-09-21T02:18:14.450208+0000","last_clean":"2024-09-21T02:18:14.450208+0000","last_became_active":"2024-09-21T02:18:14.448643+0000","last_became_peered":"2024-09-21T02:18:14.448643+0000","last_unstale":"2024-09-21T02:18:14.450208+0000","last_undegraded":"2024-09-21T02:18:14.450208+0000","last_fullsized":"2024-09-21T02:18:14.450208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:08.567073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474217+0000","last_change":"2024-09-21T02:18:14.469252+0000","last_active":"2024-09-21T02:18:14.474217+0000","last_peered":"2024-09-21T02:18:14.474217+0000","last_clean":"2024-09-21T02:18:14.474217+0000","last_became_active":"2024-09-21T02:18:14.468634+0000","last_became_peered":"2024-09-21T02:18:14.468634+0000","last_unstale":"2024-09-21T02:18:14.474217+0000","last_undegraded":"2024-09-21T02:18:14.474217+0000","last_fullsized":"2024-09-21T02:18:14.474217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:51.049187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.465179+0000","last_change":"2024-09-21T02:18:14.457778+0000","last_active":"2024-09-21T02:18:14.465179+0000","last_peered":"2024-09-21T02:18:14.465179+0000","last_clean":"2024-09-21T02:18:14.465179+0000","last_became_active":"2024-09-21T02:18:14.457064+0000","last_became_peered":"2024-09-21T02:18:14.457064+0000","last_unstale":"2024-09-21T02:18:14.465179+0000","last_undegraded":"2024-09-21T02:18:14.465179+0000","last_fullsized":"2024-09-21T02:18:14.465179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:12.145104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.465883+0000","last_change":"2024-09-21T02:18:14.459995+0000","last_active":"2024-09-21T02:18:14.465883+0000","last_peered":"2024-09-21T02:18:14.465883+0000","last_clean":"2024-09-21T02:18:14.465883+0000","last_became_active":"2024-09-21T02:18:14.459299+0000","last_became_peered":"2024-09-21T02:18:14.459299+0000","last_unstale":"2024-09-21T02:18:14.465883+0000","last_undegraded":"2024-09-21T02:18:14.465883+0000","last_fullsized":"2024-09-21T02:18:14.465883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:36.105764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.460541+0000","last_change":"2024-09-21T02:18:14.455760+0000","last_active":"2024-09-21T02:18:14.460541+0000","last_peered":"2024-09-21T02:18:14.460541+0000","last_clean":"2024-09-21T02:18:14.460541+0000","last_became_active":"2024-09-21T02:18:14.454922+0000","last_became_peered":"2024-09-21T02:18:14.454922+0000","last_unstale":"2024-09-21T02:18:14.460541+0000","last_undegraded":"2024-09-21T02:18:14.460541+0000","last_fullsized":"2024-09-21T02:18:14.460541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:27.030849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.450551+0000","last_change":"2024-09-21T02:18:14.441894+0000","last_active":"2024-09-21T02:18:14.450551+0000","last_peered":"2024-09-21T02:18:14.450551+0000","last_clean":"2024-09-21T02:18:14.450551+0000","last_became_active":"2024-09-21T02:18:14.441025+0000","last_became_peered":"2024-09-21T02:18:14.441025+0000","last_unstale":"2024-09-21T02:18:14.450551+0000","last_undegraded":"2024-09-21T02:18:14.450551+0000","last_fullsized":"2024-09-21T02:18:14.450551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:34.675299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449949+0000","last_change":"2024-09-21T02:18:14.448891+0000","last_active":"2024-09-21T02:18:14.449949+0000","last_peered":"2024-09-21T02:18:14.449949+0000","last_clean":"2024-09-21T02:18:14.449949+0000","last_became_active":"2024-09-21T02:18:14.448546+0000","last_became_peered":"2024-09-21T02:18:14.448546+0000","last_unstale":"2024-09-21T02:18:14.449949+0000","last_undegraded":"2024-09-21T02:18:14.449949+0000","last_fullsized":"2024-09-21T02:18:14.449949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.238723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.477033+0000","last_change":"2024-09-21T02:18:14.472678+0000","last_active":"2024-09-21T02:18:14.477033+0000","last_peered":"2024-09-21T02:18:14.477033+0000","last_clean":"2024-09-21T02:18:14.477033+0000","last_became_active":"2024-09-21T02:18:14.471237+0000","last_became_peered":"2024-09-21T02:18:14.471237+0000","last_unstale":"2024-09-21T02:18:14.477033+0000","last_undegraded":"2024-09-21T02:18:14.477033+0000","last_fullsized":"2024-09-21T02:18:14.477033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:28.684773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474297+0000","last_change":"2024-09-21T02:18:14.469439+0000","last_active":"2024-09-21T02:18:14.474297+0000","last_peered":"2024-09-21T02:18:14.474297+0000","last_clean":"2024-09-21T02:18:14.474297+0000","last_became_active":"2024-09-21T02:18:14.468153+0000","last_became_peered":"2024-09-21T02:18:14.468153+0000","last_unstale":"2024-09-21T02:18:14.474297+0000","last_undegraded":"2024-09-21T02:18:14.474297+0000","last_fullsized":"2024-09-21T02:18:14.474297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:00.543962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.445247+0000","last_change":"2024-09-21T02:18:14.440260+0000","last_active":"2024-09-21T02:18:14.445247+0000","last_peered":"2024-09-21T02:18:14.445247+0000","last_clean":"2024-09-21T02:18:14.445247+0000","last_became_active":"2024-09-21T02:18:14.439634+0000","last_became_peered":"2024-09-21T02:18:14.439634+0000","last_unstale":"2024-09-21T02:18:14.445247+0000","last_undegraded":"2024-09-21T02:18:14.445247+0000","last_fullsized":"2024-09-21T02:18:14.445247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:42.351533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.463396+0000","last_change":"2024-09-21T02:18:14.462988+0000","last_active":"2024-09-21T02:18:14.463396+0000","last_peered":"2024-09-21T02:18:14.463396+0000","last_clean":"2024-09-21T02:18:14.463396+0000","last_became_active":"2024-09-21T02:18:14.462668+0000","last_became_peered":"2024-09-21T02:18:14.462668+0000","last_unstale":"2024-09-21T02:18:14.463396+0000","last_undegraded":"2024-09-21T02:18:14.463396+0000","last_fullsized":"2024-09-21T02:18:14.463396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:01.188752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442566+0000","last_change":"2024-09-21T02:18:14.438106+0000","last_active":"2024-09-21T02:18:14.442566+0000","last_peered":"2024-09-21T02:18:14.442566+0000","last_clean":"2024-09-21T02:18:14.442566+0000","last_became_active":"2024-09-21T02:18:14.437747+0000","last_became_peered":"2024-09-21T02:18:14.437747+0000","last_unstale":"2024-09-21T02:18:14.442566+0000","last_undegraded":"2024-09-21T02:18:14.442566+0000","last_fullsized":"2024-09-21T02:18:14.442566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:03.234592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.448889+0000","last_change":"2024-09-21T02:18:14.423363+0000","last_active":"2024-09-21T02:18:14.448889+0000","last_peered":"2024-09-21T02:18:14.448889+0000","last_clean":"2024-09-21T02:18:14.448889+0000","last_became_active":"2024-09-21T02:18:14.420919+0000","last_became_peered":"2024-09-21T02:18:14.420919+0000","last_unstale":"2024-09-21T02:18:14.448889+0000","last_undegraded":"2024-09-21T02:18:14.448889+0000","last_fullsized":"2024-09-21T02:18:14.448889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:13.071985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.460631+0000","last_change":"2024-09-21T02:18:14.453125+0000","last_active":"2024-09-21T02:18:14.460631+0000","last_peered":"2024-09-21T02:18:14.460631+0000","last_clean":"2024-09-21T02:18:14.460631+0000","last_became_active":"2024-09-21T02:18:14.452195+0000","last_became_peered":"2024-09-21T02:18:14.452195+0000","last_unstale":"2024-09-21T02:18:14.460631+0000","last_undegraded":"2024-09-21T02:18:14.460631+0000","last_fullsized":"2024-09-21T02:18:14.460631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:36.103455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458004+0000","last_change":"2024-09-21T02:18:14.456357+0000","last_active":"2024-09-21T02:18:14.458004+0000","last_peered":"2024-09-21T02:18:14.458004+0000","last_clean":"2024-09-21T02:18:14.458004+0000","last_became_active":"2024-09-21T02:18:14.455997+0000","last_became_peered":"2024-09-21T02:18:14.455997+0000","last_unstale":"2024-09-21T02:18:14.458004+0000","last_undegraded":"2024-09-21T02:18:14.458004+0000","last_fullsized":"2024-09-21T02:18:14.458004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.238723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458161+0000","last_change":"2024-09-21T02:18:14.456648+0000","last_active":"2024-09-21T02:18:14.458161+0000","last_peered":"2024-09-21T02:18:14.458161+0000","last_clean":"2024-09-21T02:18:14.458161+0000","last_became_active":"2024-09-21T02:18:14.456296+0000","last_became_peered":"2024-09-21T02:18:14.456296+0000","last_unstale":"2024-09-21T02:18:14.458161+0000","last_undegraded":"2024-09-21T02:18:14.458161+0000","last_fullsized":"2024-09-21T02:18:14.458161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:26.562143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.433291+0000","last_change":"2024-09-21T02:18:14.431779+0000","last_active":"2024-09-21T02:18:14.433291+0000","last_peered":"2024-09-21T02:18:14.433291+0000","last_clean":"2024-09-21T02:18:14.433291+0000","last_became_active":"2024-09-21T02:18:14.431455+0000","last_became_peered":"2024-09-21T02:18:14.431455+0000","last_unstale":"2024-09-21T02:18:14.433291+0000","last_undegraded":"2024-09-21T02:18:14.433291+0000","last_fullsized":"2024-09-21T02:18:14.433291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:45.576522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.477463+0000","last_change":"2024-09-21T02:18:13.439570+0000","last_active":"2024-09-21T02:18:14.477463+0000","last_peered":"2024-09-21T02:18:14.477463+0000","last_clean":"2024-09-21T02:18:14.477463+0000","last_became_active":"2024-09-21T02:18:13.439020+0000","last_became_peered":"2024-09-21T02:18:13.439020+0000","last_unstale":"2024-09-21T02:18:14.477463+0000","last_undegraded":"2024-09-21T02:18:14.477463+0000","last_fullsized":"2024-09-21T02:18:14.477463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:22.520314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.434062+0000","last_change":"2024-09-21T02:18:14.430439+0000","last_active":"2024-09-21T02:18:14.434062+0000","last_peered":"2024-09-21T02:18:14.434062+0000","last_clean":"2024-09-21T02:18:14.434062+0000","last_became_active":"2024-09-21T02:18:14.429768+0000","last_became_peered":"2024-09-21T02:18:14.429768+0000","last_unstale":"2024-09-21T02:18:14.434062+0000","last_undegraded":"2024-09-21T02:18:14.434062+0000","last_fullsized":"2024-09-21T02:18:14.434062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:19.464328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.448223+0000","last_change":"2024-09-21T02:18:13.445278+0000","last_active":"2024-09-21T02:18:14.448223+0000","last_peered":"2024-09-21T02:18:14.448223+0000","last_clean":"2024-09-21T02:18:14.448223+0000","last_became_active":"2024-09-21T02:18:13.444819+0000","last_became_peered":"2024-09-21T02:18:13.444819+0000","last_unstale":"2024-09-21T02:18:14.448223+0000","last_undegraded":"2024-09-21T02:18:14.448223+0000","last_fullsized":"2024-09-21T02:18:14.448223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:33.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.455102+0000","last_change":"2024-09-21T02:18:14.454455+0000","last_active":"2024-09-21T02:18:14.455102+0000","last_peered":"2024-09-21T02:18:14.455102+0000","last_clean":"2024-09-21T02:18:14.455102+0000","last_became_active":"2024-09-21T02:18:14.454187+0000","last_became_peered":"2024-09-21T02:18:14.454187+0000","last_unstale":"2024-09-21T02:18:14.455102+0000","last_undegraded":"2024-09-21T02:18:14.455102+0000","last_fullsized":"2024-09-21T02:18:14.455102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:58.078603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.477098+0000","last_change":"2024-09-21T02:18:13.422096+0000","last_active":"2024-09-21T02:18:14.477098+0000","last_peered":"2024-09-21T02:18:14.477098+0000","last_clean":"2024-09-21T02:18:14.477098+0000","last_became_active":"2024-09-21T02:18:13.421119+0000","last_became_peered":"2024-09-21T02:18:13.421119+0000","last_unstale":"2024-09-21T02:18:14.477098+0000","last_undegraded":"2024-09-21T02:18:14.477098+0000","last_fullsized":"2024-09-21T02:18:14.477098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:20.428236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.475124+0000","last_change":"2024-09-21T02:18:14.474452+0000","last_active":"2024-09-21T02:18:14.475124+0000","last_peered":"2024-09-21T02:18:14.475124+0000","last_clean":"2024-09-21T02:18:14.475124+0000","last_became_active":"2024-09-21T02:18:14.474164+0000","last_became_peered":"2024-09-21T02:18:14.474164+0000","last_unstale":"2024-09-21T02:18:14.475124+0000","last_undegraded":"2024-09-21T02:18:14.475124+0000","last_fullsized":"2024-09-21T02:18:14.475124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:59.090283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.461599+0000","last_change":"2024-09-21T02:18:13.428048+0000","last_active":"2024-09-21T02:18:14.461599+0000","last_peered":"2024-09-21T02:18:14.461599+0000","last_clean":"2024-09-21T02:18:14.461599+0000","last_became_active":"2024-09-21T02:18:13.427244+0000","last_became_peered":"2024-09-21T02:18:13.427244+0000","last_unstale":"2024-09-21T02:18:14.461599+0000","last_undegraded":"2024-09-21T02:18:14.461599+0000","last_fullsized":"2024-09-21T02:18:14.461599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:40.446937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478319+0000","last_change":"2024-09-21T02:18:14.476862+0000","last_active":"2024-09-21T02:18:14.478319+0000","last_peered":"2024-09-21T02:18:14.478319+0000","last_clean":"2024-09-21T02:18:14.478319+0000","last_became_active":"2024-09-21T02:18:14.476027+0000","last_became_peered":"2024-09-21T02:18:14.476027+0000","last_unstale":"2024-09-21T02:18:14.478319+0000","last_undegraded":"2024-09-21T02:18:14.478319+0000","last_fullsized":"2024-09-21T02:18:14.478319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:46.346040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.430408+0000","last_change":"2024-09-21T02:18:13.420698+0000","last_active":"2024-09-21T02:18:14.430408+0000","last_peered":"2024-09-21T02:18:14.430408+0000","last_clean":"2024-09-21T02:18:14.430408+0000","last_became_active":"2024-09-21T02:18:13.420291+0000","last_became_peered":"2024-09-21T02:18:13.420291+0000","last_unstale":"2024-09-21T02:18:14.430408+0000","last_undegraded":"2024-09-21T02:18:14.430408+0000","last_fullsized":"2024-09-21T02:18:14.430408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:43.529177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.443776+0000","last_change":"2024-09-21T02:18:14.442893+0000","last_active":"2024-09-21T02:18:14.443776+0000","last_peered":"2024-09-21T02:18:14.443776+0000","last_clean":"2024-09-21T02:18:14.443776+0000","last_became_active":"2024-09-21T02:18:14.442456+0000","last_became_peered":"2024-09-21T02:18:14.442456+0000","last_unstale":"2024-09-21T02:18:14.443776+0000","last_undegraded":"2024-09-21T02:18:14.443776+0000","last_fullsized":"2024-09-21T02:18:14.443776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:45.576522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.462511+0000","last_change":"2024-09-21T02:18:13.420235+0000","last_active":"2024-09-21T02:18:14.462511+0000","last_peered":"2024-09-21T02:18:14.462511+0000","last_clean":"2024-09-21T02:18:14.462511+0000","last_became_active":"2024-09-21T02:18:13.419857+0000","last_became_peered":"2024-09-21T02:18:13.419857+0000","last_unstale":"2024-09-21T02:18:14.462511+0000","last_undegraded":"2024-09-21T02:18:14.462511+0000","last_fullsized":"2024-09-21T02:18:14.462511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:48.163803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.476168+0000","last_change":"2024-09-21T02:18:14.475730+0000","last_active":"2024-09-21T02:18:14.476168+0000","last_peered":"2024-09-21T02:18:14.476168+0000","last_clean":"2024-09-21T02:18:14.476168+0000","last_became_active":"2024-09-21T02:18:14.475409+0000","last_became_peered":"2024-09-21T02:18:14.475409+0000","last_unstale":"2024-09-21T02:18:14.476168+0000","last_undegraded":"2024-09-21T02:18:14.476168+0000","last_fullsized":"2024-09-21T02:18:14.476168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:56.997803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.424807+0000","last_change":"2024-09-21T02:18:13.444468+0000","last_active":"2024-09-21T02:18:14.424807+0000","last_peered":"2024-09-21T02:18:14.424807+0000","last_clean":"2024-09-21T02:18:14.424807+0000","last_became_active":"2024-09-21T02:18:13.443456+0000","last_became_peered":"2024-09-21T02:18:13.443456+0000","last_unstale":"2024-09-21T02:18:14.424807+0000","last_undegraded":"2024-09-21T02:18:14.424807+0000","last_fullsized":"2024-09-21T02:18:14.424807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:08.679962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.456087+0000","last_change":"2024-09-21T02:18:14.446856+0000","last_active":"2024-09-21T02:18:14.456087+0000","last_peered":"2024-09-21T02:18:14.456087+0000","last_clean":"2024-09-21T02:18:14.456087+0000","last_became_active":"2024-09-21T02:18:14.444846+0000","last_became_peered":"2024-09-21T02:18:14.444846+0000","last_unstale":"2024-09-21T02:18:14.456087+0000","last_undegraded":"2024-09-21T02:18:14.456087+0000","last_fullsized":"2024-09-21T02:18:14.456087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:16.108862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.425878+0000","last_change":"2024-09-21T02:18:13.444565+0000","last_active":"2024-09-21T02:18:14.425878+0000","last_peered":"2024-09-21T02:18:14.425878+0000","last_clean":"2024-09-21T02:18:14.425878+0000","last_became_active":"2024-09-21T02:18:13.443552+0000","last_became_peered":"2024-09-21T02:18:13.443552+0000","last_unstale":"2024-09-21T02:18:14.425878+0000","last_undegraded":"2024-09-21T02:18:14.425878+0000","last_fullsized":"2024-09-21T02:18:14.425878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:25.175151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449858+0000","last_change":"2024-09-21T02:18:14.443175+0000","last_active":"2024-09-21T02:18:14.449858+0000","last_peered":"2024-09-21T02:18:14.449858+0000","last_clean":"2024-09-21T02:18:14.449858+0000","last_became_active":"2024-09-21T02:18:14.442409+0000","last_became_peered":"2024-09-21T02:18:14.442409+0000","last_unstale":"2024-09-21T02:18:14.449858+0000","last_undegraded":"2024-09-21T02:18:14.449858+0000","last_fullsized":"2024-09-21T02:18:14.449858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:52.762005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.429711+0000","last_change":"2024-09-21T02:18:13.422135+0000","last_active":"2024-09-21T02:18:14.429711+0000","last_peered":"2024-09-21T02:18:14.429711+0000","last_clean":"2024-09-21T02:18:14.429711+0000","last_became_active":"2024-09-21T02:18:13.421810+0000","last_became_peered":"2024-09-21T02:18:13.421810+0000","last_unstale":"2024-09-21T02:18:14.429711+0000","last_undegraded":"2024-09-21T02:18:14.429711+0000","last_fullsized":"2024-09-21T02:18:14.429711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:49.944038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.473928+0000","last_change":"2024-09-21T02:18:14.473312+0000","last_active":"2024-09-21T02:18:14.473928+0000","last_peered":"2024-09-21T02:18:14.473928+0000","last_clean":"2024-09-21T02:18:14.473928+0000","last_became_active":"2024-09-21T02:18:14.473063+0000","last_became_peered":"2024-09-21T02:18:14.473063+0000","last_unstale":"2024-09-21T02:18:14.473928+0000","last_undegraded":"2024-09-21T02:18:14.473928+0000","last_fullsized":"2024-09-21T02:18:14.473928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:27.848671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.426668+0000","last_change":"2024-09-21T02:18:13.418626+0000","last_active":"2024-09-21T02:18:14.426668+0000","last_peered":"2024-09-21T02:18:14.426668+0000","last_clean":"2024-09-21T02:18:14.426668+0000","last_became_active":"2024-09-21T02:18:13.418109+0000","last_became_peered":"2024-09-21T02:18:13.418109+0000","last_unstale":"2024-09-21T02:18:14.426668+0000","last_undegraded":"2024-09-21T02:18:14.426668+0000","last_fullsized":"2024-09-21T02:18:14.426668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:32.421158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478800+0000","last_change":"2024-09-21T02:18:14.474526+0000","last_active":"2024-09-21T02:18:14.478800+0000","last_peered":"2024-09-21T02:18:14.478800+0000","last_clean":"2024-09-21T02:18:14.478800+0000","last_became_active":"2024-09-21T02:18:14.468379+0000","last_became_peered":"2024-09-21T02:18:14.468379+0000","last_unstale":"2024-09-21T02:18:14.478800+0000","last_undegraded":"2024-09-21T02:18:14.478800+0000","last_fullsized":"2024-09-21T02:18:14.478800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:35.046249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.479351+0000","last_change":"2024-09-21T02:18:13.434165+0000","last_active":"2024-09-21T02:18:14.479351+0000","last_peered":"2024-09-21T02:18:14.479351+0000","last_clean":"2024-09-21T02:18:14.479351+0000","last_became_active":"2024-09-21T02:18:13.433725+0000","last_became_peered":"2024-09-21T02:18:13.433725+0000","last_unstale":"2024-09-21T02:18:14.479351+0000","last_undegraded":"2024-09-21T02:18:14.479351+0000","last_fullsized":"2024-09-21T02:18:14.479351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:47.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478552+0000","last_change":"2024-09-21T02:18:14.477437+0000","last_active":"2024-09-21T02:18:14.478552+0000","last_peered":"2024-09-21T02:18:14.478552+0000","last_clean":"2024-09-21T02:18:14.478552+0000","last_became_active":"2024-09-21T02:18:14.477077+0000","last_became_peered":"2024-09-21T02:18:14.477077+0000","last_unstale":"2024-09-21T02:18:14.478552+0000","last_undegraded":"2024-09-21T02:18:14.478552+0000","last_fullsized":"2024-09-21T02:18:14.478552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:32.952815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.467656+0000","last_change":"2024-09-21T02:18:13.436262+0000","last_active":"2024-09-21T02:18:14.467656+0000","last_peered":"2024-09-21T02:18:14.467656+0000","last_clean":"2024-09-21T02:18:14.467656+0000","last_became_active":"2024-09-21T02:18:13.435918+0000","last_became_peered":"2024-09-21T02:18:13.435918+0000","last_unstale":"2024-09-21T02:18:14.467656+0000","last_undegraded":"2024-09-21T02:18:14.467656+0000","last_fullsized":"2024-09-21T02:18:14.467656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:04.772278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474774+0000","last_change":"2024-09-21T02:18:14.464283+0000","last_active":"2024-09-21T02:18:14.474774+0000","last_peered":"2024-09-21T02:18:14.474774+0000","last_clean":"2024-09-21T02:18:14.474774+0000","last_became_active":"2024-09-21T02:18:14.463247+0000","last_became_peered":"2024-09-21T02:18:14.463247+0000","last_unstale":"2024-09-21T02:18:14.474774+0000","last_undegraded":"2024-09-21T02:18:14.474774+0000","last_fullsized":"2024-09-21T02:18:14.474774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:55.643772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.466430+0000","last_change":"2024-09-21T02:18:13.436253+0000","last_active":"2024-09-21T02:18:14.466430+0000","last_peered":"2024-09-21T02:18:14.466430+0000","last_clean":"2024-09-21T02:18:14.466430+0000","last_became_active":"2024-09-21T02:18:13.435306+0000","last_became_peered":"2024-09-21T02:18:13.435306+0000","last_unstale":"2024-09-21T02:18:14.466430+0000","last_undegraded":"2024-09-21T02:18:14.466430+0000","last_fullsized":"2024-09-21T02:18:14.466430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:18.380614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.471018+0000","last_change":"2024-09-21T02:18:14.470499+0000","last_active":"2024-09-21T02:18:14.471018+0000","last_peered":"2024-09-21T02:18:14.471018+0000","last_clean":"2024-09-21T02:18:14.471018+0000","last_became_active":"2024-09-21T02:18:14.470099+0000","last_became_peered":"2024-09-21T02:18:14.470099+0000","last_unstale":"2024-09-21T02:18:14.471018+0000","last_undegraded":"2024-09-21T02:18:14.471018+0000","last_fullsized":"2024-09-21T02:18:14.471018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:33.751569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410899+0000","last_change":"2024-09-21T02:18:13.432542+0000","last_active":"2024-09-21T02:18:14.410899+0000","last_peered":"2024-09-21T02:18:14.410899+0000","last_clean":"2024-09-21T02:18:14.410899+0000","last_became_active":"2024-09-21T02:18:13.431660+0000","last_became_peered":"2024-09-21T02:18:13.431660+0000","last_unstale":"2024-09-21T02:18:14.410899+0000","last_undegraded":"2024-09-21T02:18:14.410899+0000","last_fullsized":"2024-09-21T02:18:14.410899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:39.336082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478633+0000","last_change":"2024-09-21T02:18:14.475038+0000","last_active":"2024-09-21T02:18:14.478633+0000","last_peered":"2024-09-21T02:18:14.478633+0000","last_clean":"2024-09-21T02:18:14.478633+0000","last_became_active":"2024-09-21T02:18:14.474297+0000","last_became_peered":"2024-09-21T02:18:14.474297+0000","last_unstale":"2024-09-21T02:18:14.478633+0000","last_undegraded":"2024-09-21T02:18:14.478633+0000","last_fullsized":"2024-09-21T02:18:14.478633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:11.491097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.422913+0000","last_change":"2024-09-21T02:18:13.429048+0000","last_active":"2024-09-21T02:18:14.422913+0000","last_peered":"2024-09-21T02:18:14.422913+0000","last_clean":"2024-09-21T02:18:14.422913+0000","last_became_active":"2024-09-21T02:18:13.428514+0000","last_became_peered":"2024-09-21T02:18:13.428514+0000","last_unstale":"2024-09-21T02:18:14.422913+0000","last_undegraded":"2024-09-21T02:18:14.422913+0000","last_fullsized":"2024-09-21T02:18:14.422913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:15.874001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474855+0000","last_change":"2024-09-21T02:18:14.464654+0000","last_active":"2024-09-21T02:18:14.474855+0000","last_peered":"2024-09-21T02:18:14.474855+0000","last_clean":"2024-09-21T02:18:14.474855+0000","last_became_active":"2024-09-21T02:18:14.463200+0000","last_became_peered":"2024-09-21T02:18:14.463200+0000","last_unstale":"2024-09-21T02:18:14.474855+0000","last_undegraded":"2024-09-21T02:18:14.474855+0000","last_fullsized":"2024-09-21T02:18:14.474855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:35.121991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.481573+0000","last_change":"2024-09-21T02:18:13.435771+0000","last_active":"2024-09-21T02:18:14.481573+0000","last_peered":"2024-09-21T02:18:14.481573+0000","last_clean":"2024-09-21T02:18:14.481573+0000","last_became_active":"2024-09-21T02:18:13.434918+0000","last_became_peered":"2024-09-21T02:18:13.434918+0000","last_unstale":"2024-09-21T02:18:14.481573+0000","last_undegraded":"2024-09-21T02:18:14.481573+0000","last_fullsized":"2024-09-21T02:18:14.481573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:08.084873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.441448+0000","last_change":"2024-09-21T02:18:14.439555+0000","last_active":"2024-09-21T02:18:14.441448+0000","last_peered":"2024-09-21T02:18:14.441448+0000","last_clean":"2024-09-21T02:18:14.441448+0000","last_became_active":"2024-09-21T02:18:14.439177+0000","last_became_peered":"2024-09-21T02:18:14.439177+0000","last_unstale":"2024-09-21T02:18:14.441448+0000","last_undegraded":"2024-09-21T02:18:14.441448+0000","last_fullsized":"2024-09-21T02:18:14.441448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:10.352252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.444903+0000","last_change":"2024-09-21T02:18:13.427751+0000","last_active":"2024-09-21T02:18:14.444903+0000","last_peered":"2024-09-21T02:18:14.444903+0000","last_clean":"2024-09-21T02:18:14.444903+0000","last_became_active":"2024-09-21T02:18:13.427370+0000","last_became_peered":"2024-09-21T02:18:13.427370+0000","last_unstale":"2024-09-21T02:18:14.444903+0000","last_undegraded":"2024-09-21T02:18:14.444903+0000","last_fullsized":"2024-09-21T02:18:14.444903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:43.466174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442978+0000","last_change":"2024-09-21T02:18:14.437822+0000","last_active":"2024-09-21T02:18:14.442978+0000","last_peered":"2024-09-21T02:18:14.442978+0000","last_clean":"2024-09-21T02:18:14.442978+0000","last_became_active":"2024-09-21T02:18:14.437492+0000","last_became_peered":"2024-09-21T02:18:14.437492+0000","last_unstale":"2024-09-21T02:18:14.442978+0000","last_undegraded":"2024-09-21T02:18:14.442978+0000","last_fullsized":"2024-09-21T02:18:14.442978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:27.068670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449660+0000","last_change":"2024-09-21T02:18:13.450713+0000","last_active":"2024-09-21T02:18:14.449660+0000","last_peered":"2024-09-21T02:18:14.449660+0000","last_clean":"2024-09-21T02:18:14.449660+0000","last_became_active":"2024-09-21T02:18:13.448373+0000","last_became_peered":"2024-09-21T02:18:13.448373+0000","last_unstale":"2024-09-21T02:18:14.449660+0000","last_undegraded":"2024-09-21T02:18:14.449660+0000","last_fullsized":"2024-09-21T02:18:14.449660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:18.831384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.431291+0000","last_change":"2024-09-21T02:18:14.427188+0000","last_active":"2024-09-21T02:18:14.431291+0000","last_peered":"2024-09-21T02:18:14.431291+0000","last_clean":"2024-09-21T02:18:14.431291+0000","last_became_active":"2024-09-21T02:18:14.425670+0000","last_became_peered":"2024-09-21T02:18:14.425670+0000","last_unstale":"2024-09-21T02:18:14.431291+0000","last_undegraded":"2024-09-21T02:18:14.431291+0000","last_fullsized":"2024-09-21T02:18:14.431291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.283268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.429005+0000","last_change":"2024-09-21T02:18:13.451377+0000","last_active":"2024-09-21T02:18:14.429005+0000","last_peered":"2024-09-21T02:18:14.429005+0000","last_clean":"2024-09-21T02:18:14.429005+0000","last_became_active":"2024-09-21T02:18:13.450715+0000","last_became_peered":"2024-09-21T02:18:13.450715+0000","last_unstale":"2024-09-21T02:18:14.429005+0000","last_undegraded":"2024-09-21T02:18:14.429005+0000","last_fullsized":"2024-09-21T02:18:14.429005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:45.075046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.471005+0000","last_change":"2024-09-21T02:18:14.468616+0000","last_active":"2024-09-21T02:18:14.471005+0000","last_peered":"2024-09-21T02:18:14.471005+0000","last_clean":"2024-09-21T02:18:14.471005+0000","last_became_active":"2024-09-21T02:18:14.467981+0000","last_became_peered":"2024-09-21T02:18:14.467981+0000","last_unstale":"2024-09-21T02:18:14.471005+0000","last_undegraded":"2024-09-21T02:18:14.471005+0000","last_fullsized":"2024-09-21T02:18:14.471005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.031100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.455832+0000","last_change":"2024-09-21T02:18:13.450073+0000","last_active":"2024-09-21T02:18:14.455832+0000","last_peered":"2024-09-21T02:18:14.455832+0000","last_clean":"2024-09-21T02:18:14.455832+0000","last_became_active":"2024-09-21T02:18:13.449319+0000","last_became_peered":"2024-09-21T02:18:13.449319+0000","last_unstale":"2024-09-21T02:18:14.455832+0000","last_undegraded":"2024-09-21T02:18:14.455832+0000","last_fullsized":"2024-09-21T02:18:14.455832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:52.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478752+0000","last_change":"2024-09-21T02:18:14.478084+0000","last_active":"2024-09-21T02:18:14.478752+0000","last_peered":"2024-09-21T02:18:14.478752+0000","last_clean":"2024-09-21T02:18:14.478752+0000","last_became_active":"2024-09-21T02:18:14.477852+0000","last_became_peered":"2024-09-21T02:18:14.477852+0000","last_unstale":"2024-09-21T02:18:14.478752+0000","last_undegraded":"2024-09-21T02:18:14.478752+0000","last_fullsized":"2024-09-21T02:18:14.478752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:17.648764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.411171+0000","last_change":"2024-09-21T02:18:13.439660+0000","last_active":"2024-09-21T02:18:14.411171+0000","last_peered":"2024-09-21T02:18:14.411171+0000","last_clean":"2024-09-21T02:18:14.411171+0000","last_became_active":"2024-09-21T02:18:13.438842+0000","last_became_peered":"2024-09-21T02:18:13.438842+0000","last_unstale":"2024-09-21T02:18:14.411171+0000","last_undegraded":"2024-09-21T02:18:14.411171+0000","last_fullsized":"2024-09-21T02:18:14.411171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:12.643334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.455509+0000","last_change":"2024-09-21T02:18:14.441205+0000","last_active":"2024-09-21T02:18:14.455509+0000","last_peered":"2024-09-21T02:18:14.455509+0000","last_clean":"2024-09-21T02:18:14.455509+0000","last_became_active":"2024-09-21T02:18:14.428079+0000","last_became_peered":"2024-09-21T02:18:14.428079+0000","last_unstale":"2024-09-21T02:18:14.455509+0000","last_undegraded":"2024-09-21T02:18:14.455509+0000","last_fullsized":"2024-09-21T02:18:14.455509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:49.871004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.446799+0000","last_change":"2024-09-21T02:18:13.435919+0000","last_active":"2024-09-21T02:18:14.446799+0000","last_peered":"2024-09-21T02:18:14.446799+0000","last_clean":"2024-09-21T02:18:14.446799+0000","last_became_active":"2024-09-21T02:18:13.435121+0000","last_became_peered":"2024-09-21T02:18:13.435121+0000","last_unstale":"2024-09-21T02:18:14.446799+0000","last_undegraded":"2024-09-21T02:18:14.446799+0000","last_fullsized":"2024-09-21T02:18:14.446799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:47.108644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.430515+0000","last_change":"2024-09-21T02:18:14.419974+0000","last_active":"2024-09-21T02:18:14.430515+0000","last_peered":"2024-09-21T02:18:14.430515+0000","last_clean":"2024-09-21T02:18:14.430515+0000","last_became_active":"2024-09-21T02:18:14.419113+0000","last_became_peered":"2024-09-21T02:18:14.419113+0000","last_unstale":"2024-09-21T02:18:14.430515+0000","last_undegraded":"2024-09-21T02:18:14.430515+0000","last_fullsized":"2024-09-21T02:18:14.430515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:27.068670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.426492+0000","last_change":"2024-09-21T02:18:13.452885+0000","last_active":"2024-09-21T02:18:14.426492+0000","last_peered":"2024-09-21T02:18:14.426492+0000","last_clean":"2024-09-21T02:18:14.426492+0000","last_became_active":"2024-09-21T02:18:13.452113+0000","last_became_peered":"2024-09-21T02:18:13.452113+0000","last_unstale":"2024-09-21T02:18:14.426492+0000","last_undegraded":"2024-09-21T02:18:14.426492+0000","last_fullsized":"2024-09-21T02:18:14.426492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:14.820089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.440628+0000","last_change":"2024-09-21T02:18:14.431934+0000","last_active":"2024-09-21T02:18:14.440628+0000","last_peered":"2024-09-21T02:18:14.440628+0000","last_clean":"2024-09-21T02:18:14.440628+0000","last_became_active":"2024-09-21T02:18:14.431565+0000","last_became_peered":"2024-09-21T02:18:14.431565+0000","last_unstale":"2024-09-21T02:18:14.440628+0000","last_undegraded":"2024-09-21T02:18:14.440628+0000","last_fullsized":"2024-09-21T02:18:14.440628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:39.540203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.450193+0000","last_change":"2024-09-21T02:18:13.448964+0000","last_active":"2024-09-21T02:18:14.450193+0000","last_peered":"2024-09-21T02:18:14.450193+0000","last_clean":"2024-09-21T02:18:14.450193+0000","last_became_active":"2024-09-21T02:18:13.448433+0000","last_became_peered":"2024-09-21T02:18:13.448433+0000","last_unstale":"2024-09-21T02:18:14.450193+0000","last_undegraded":"2024-09-21T02:18:14.450193+0000","last_fullsized":"2024-09-21T02:18:14.450193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:30.532640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.444952+0000","last_change":"2024-09-21T02:18:14.435784+0000","last_active":"2024-09-21T02:18:14.444952+0000","last_peered":"2024-09-21T02:18:14.444952+0000","last_clean":"2024-09-21T02:18:14.444952+0000","last_became_active":"2024-09-21T02:18:14.435264+0000","last_became_peered":"2024-09-21T02:18:14.435264+0000","last_unstale":"2024-09-21T02:18:14.444952+0000","last_undegraded":"2024-09-21T02:18:14.444952+0000","last_fullsized":"2024-09-21T02:18:14.444952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:38.729382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.463800+0000","last_change":"2024-09-21T02:18:13.440660+0000","last_active":"2024-09-21T02:18:14.463800+0000","last_peered":"2024-09-21T02:18:14.463800+0000","last_clean":"2024-09-21T02:18:14.463800+0000","last_became_active":"2024-09-21T02:18:13.437734+0000","last_became_peered":"2024-09-21T02:18:13.437734+0000","last_unstale":"2024-09-21T02:18:14.463800+0000","last_undegraded":"2024-09-21T02:18:14.463800+0000","last_fullsized":"2024-09-21T02:18:14.463800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:02.522881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.430740+0000","last_change":"2024-09-21T02:18:14.423873+0000","last_active":"2024-09-21T02:18:14.430740+0000","last_peered":"2024-09-21T02:18:14.430740+0000","last_clean":"2024-09-21T02:18:14.430740+0000","last_became_active":"2024-09-21T02:18:14.423608+0000","last_became_peered":"2024-09-21T02:18:14.423608+0000","last_unstale":"2024-09-21T02:18:14.430740+0000","last_undegraded":"2024-09-21T02:18:14.430740+0000","last_fullsized":"2024-09-21T02:18:14.430740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:27.489206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410423+0000","last_change":"2024-09-21T02:18:13.439622+0000","last_active":"2024-09-21T02:18:14.410423+0000","last_peered":"2024-09-21T02:18:14.410423+0000","last_clean":"2024-09-21T02:18:14.410423+0000","last_became_active":"2024-09-21T02:18:13.439036+0000","last_became_peered":"2024-09-21T02:18:13.439036+0000","last_unstale":"2024-09-21T02:18:14.410423+0000","last_undegraded":"2024-09-21T02:18:14.410423+0000","last_fullsized":"2024-09-21T02:18:14.410423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:32.421158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449767+0000","last_change":"2024-09-21T02:18:14.448869+0000","last_active":"2024-09-21T02:18:14.449767+0000","last_peered":"2024-09-21T02:18:14.449767+0000","last_clean":"2024-09-21T02:18:14.449767+0000","last_became_active":"2024-09-21T02:18:14.448463+0000","last_became_peered":"2024-09-21T02:18:14.448463+0000","last_unstale":"2024-09-21T02:18:14.449767+0000","last_undegraded":"2024-09-21T02:18:14.449767+0000","last_fullsized":"2024-09-21T02:18:14.449767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:10.352252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.422027+0000","last_change":"2024-09-21T02:18:13.450393+0000","last_active":"2024-09-21T02:18:14.422027+0000","last_peered":"2024-09-21T02:18:14.422027+0000","last_clean":"2024-09-21T02:18:14.422027+0000","last_became_active":"2024-09-21T02:18:13.449920+0000","last_became_peered":"2024-09-21T02:18:13.449920+0000","last_unstale":"2024-09-21T02:18:14.422027+0000","last_undegraded":"2024-09-21T02:18:14.422027+0000","last_fullsized":"2024-09-21T02:18:14.422027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.452533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.433493+0000","last_change":"2024-09-21T02:18:14.430114+0000","last_active":"2024-09-21T02:18:14.433493+0000","last_peered":"2024-09-21T02:18:14.433493+0000","last_clean":"2024-09-21T02:18:14.433493+0000","last_became_active":"2024-09-21T02:18:14.429635+0000","last_became_peered":"2024-09-21T02:18:14.429635+0000","last_unstale":"2024-09-21T02:18:14.433493+0000","last_undegraded":"2024-09-21T02:18:14.433493+0000","last_fullsized":"2024-09-21T02:18:14.433493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:22.301527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410492+0000","last_change":"2024-09-21T02:18:13.435115+0000","last_active":"2024-09-21T02:18:14.410492+0000","last_peered":"2024-09-21T02:18:14.410492+0000","last_clean":"2024-09-21T02:18:14.410492+0000","last_became_active":"2024-09-21T02:18:13.434258+0000","last_became_peered":"2024-09-21T02:18:13.434258+0000","last_unstale":"2024-09-21T02:18:14.410492+0000","last_undegraded":"2024-09-21T02:18:14.410492+0000","last_fullsized":"2024-09-21T02:18:14.410492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:59.696379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474499+0000","last_change":"2024-09-21T02:18:14.463988+0000","last_active":"2024-09-21T02:18:14.474499+0000","last_peered":"2024-09-21T02:18:14.474499+0000","last_clean":"2024-09-21T02:18:14.474499+0000","last_became_active":"2024-09-21T02:18:14.463114+0000","last_became_peered":"2024-09-21T02:18:14.463114+0000","last_unstale":"2024-09-21T02:18:14.474499+0000","last_undegraded":"2024-09-21T02:18:14.474499+0000","last_fullsized":"2024-09-21T02:18:14.474499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:00.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.424918+0000","last_change":"2024-09-21T02:18:13.420393+0000","last_active":"2024-09-21T02:18:14.424918+0000","last_peered":"2024-09-21T02:18:14.424918+0000","last_clean":"2024-09-21T02:18:14.424918+0000","last_became_active":"2024-09-21T02:18:13.419895+0000","last_became_peered":"2024-09-21T02:18:13.419895+0000","last_unstale":"2024-09-21T02:18:14.424918+0000","last_undegraded":"2024-09-21T02:18:14.424918+0000","last_fullsized":"2024-09-21T02:18:14.424918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:43.529177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442600+0000","last_change":"2024-09-21T02:18:14.440519+0000","last_active":"2024-09-21T02:18:14.442600+0000","last_peered":"2024-09-21T02:18:14.442600+0000","last_clean":"2024-09-21T02:18:14.442600+0000","last_became_active":"2024-09-21T02:18:14.440097+0000","last_became_peered":"2024-09-21T02:18:14.440097+0000","last_unstale":"2024-09-21T02:18:14.442600+0000","last_undegraded":"2024-09-21T02:18:14.442600+0000","last_fullsized":"2024-09-21T02:18:14.442600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:24.036222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.421810+0000","last_change":"2024-09-21T02:18:13.434226+0000","last_active":"2024-09-21T02:18:14.421810+0000","last_peered":"2024-09-21T02:18:14.421810+0000","last_clean":"2024-09-21T02:18:14.421810+0000","last_became_active":"2024-09-21T02:18:13.433825+0000","last_became_peered":"2024-09-21T02:18:13.433825+0000","last_unstale":"2024-09-21T02:18:14.421810+0000","last_undegraded":"2024-09-21T02:18:14.421810+0000","last_fullsized":"2024-09-21T02:18:14.421810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:07.044886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442192+0000","last_change":"2024-09-21T02:18:14.435029+0000","last_active":"2024-09-21T02:18:14.442192+0000","last_peered":"2024-09-21T02:18:14.442192+0000","last_clean":"2024-09-21T02:18:14.442192+0000","last_became_active":"2024-09-21T02:18:14.434500+0000","last_became_peered":"2024-09-21T02:18:14.434500+0000","last_unstale":"2024-09-21T02:18:14.442192+0000","last_undegraded":"2024-09-21T02:18:14.442192+0000","last_fullsized":"2024-09-21T02:18:14.442192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:49.347410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410179+0000","last_change":"2024-09-21T02:18:13.427041+0000","last_active":"2024-09-21T02:18:14.410179+0000","last_peered":"2024-09-21T02:18:14.410179+0000","last_clean":"2024-09-21T02:18:14.410179+0000","last_became_active":"2024-09-21T02:18:13.424255+0000","last_became_peered":"2024-09-21T02:18:13.424255+0000","last_unstale":"2024-09-21T02:18:14.410179+0000","last_undegraded":"2024-09-21T02:18:14.410179+0000","last_fullsized":"2024-09-21T02:18:14.410179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:33.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478846+0000","last_change":"2024-09-21T02:18:14.474421+0000","last_active":"2024-09-21T02:18:14.478846+0000","last_peered":"2024-09-21T02:18:14.478846+0000","last_clean":"2024-09-21T02:18:14.478846+0000","last_became_active":"2024-09-21T02:18:14.468628+0000","last_became_peered":"2024-09-21T02:18:14.468628+0000","last_unstale":"2024-09-21T02:18:14.478846+0000","last_undegraded":"2024-09-21T02:18:14.478846+0000","last_fullsized":"2024-09-21T02:18:14.478846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:58.350588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.446061+0000","last_change":"2024-09-21T02:18:13.445706+0000","last_active":"2024-09-21T02:18:14.446061+0000","last_peered":"2024-09-21T02:18:14.446061+0000","last_clean":"2024-09-21T02:18:14.446061+0000","last_became_active":"2024-09-21T02:18:13.445428+0000","last_became_peered":"2024-09-21T02:18:13.445428+0000","last_unstale":"2024-09-21T02:18:14.446061+0000","last_undegraded":"2024-09-21T02:18:14.446061+0000","last_fullsized":"2024-09-21T02:18:14.446061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:41.398507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":246,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.423028+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:18:14.423028+0000","last_peered":"2024-09-21T02:18:14.423028+0000","last_clean":"2024-09-21T02:18:14.423028+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:18:14.423028+0000","last_undegraded":"2024-09-21T02:18:14.423028+0000","last_fullsized":"2024-09-21T02:18:14.423028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":1015616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542208,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1015808,"data_stored":432057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509504,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":999424,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36340,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335500,"statfs":{"total":96636764160,"available":96599552000,"internally_reserved":0,"allocated":1011712,"data_stored":432023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1593344,"data_stored":1015616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36360,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335480,"statfs":{"total":96636764160,"available":96599531520,"internally_reserved":0,"allocated":1028096,"data_stored":433657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:18:18.568 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:18:18.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b133b0640 1 -- 172.21.15.170:0/3561755974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af8003a10 msgr2=0x7f7af8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b133b0640 1 --2- 172.21.15.170:0/3561755974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af8003a10 0x7f7af8003e80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009b60 tx=0x7f7b0801d2b0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b1194d640 1 -- 172.21.15.170:0/3561755974 reap_dead start 2024-09-21T02:18:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b133b0640 1 -- 172.21.15.170:0/3561755974 shutdown_connections 2024-09-21T02:18:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b133b0640 1 -- 172.21.15.170:0/3561755974 >> 172.21.15.170:0/3561755974 conn(0x7f7b0c0cee30 msgr2=0x7f7b0c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b133b0640 1 -- 172.21.15.170:0/3561755974 shutdown_connections 2024-09-21T02:18:18.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.756+0000 7f7b133b0640 1 -- 172.21.15.170:0/3561755974 wait complete. 2024-09-21T02:18:18.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 Processor -- start 2024-09-21T02:18:18.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 -- start start 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af8003a10 0x7f7b0c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b0c154f50 0x7f7b0c15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 0x7f7b0c0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c167d40 con 0x7f7b0c154990 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c167bc0 con 0x7f7b0c154f50 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c167ec0 con 0x7f7af8003a10 2024-09-21T02:18:18.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 0x7f7b0c0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 0x7f7b0c0c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45704/0 (socket says 172.21.15.170:45704) 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 -- 172.21.15.170:0/2105123584 learned_addr learned my addr 172.21.15.170:0/2105123584 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1094b640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b0c154f50 0x7f7b0c15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1114c640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af8003a10 0x7f7b0c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 -- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af8003a10 msgr2=0x7f7b0c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af8003a10 0x7f7b0c15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 -- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b0c154f50 msgr2=0x7f7b0c15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b0c154f50 0x7f7b0c15e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 -- 172.21.15.170:0/2105123584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c166300 con 0x7f7b0c154990 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b1194d640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 0x7f7b0c0c93f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0cfc40 tx=0x7f7b0400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0401a070 con 0x7f7b0c154990 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0400bce0 con 0x7f7b0c154990 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c164020 con 0x7f7b0c154990 2024-09-21T02:18:18.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040106c0 con 0x7f7b0c154990 2024-09-21T02:18:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.760+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c0cb880 con 0x7f7b0c154990 2024-09-21T02:18:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.764+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c166070 con 0x7f7b0c154990 2024-09-21T02:18:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.764+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7b0401d070 con 0x7f7b0c154990 2024-09-21T02:18:18.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.764+0000 7f7b01ffb640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073320 0x7f7af0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:18.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.764+0000 7f7b1114c640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073320 0x7f7af0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:18.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.764+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f7b04093c20 con 0x7f7b0c154990 2024-09-21T02:18:18.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.764+0000 7f7b1114c640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073320 0x7f7af0075830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b080ae000 tx=0x7f7b0801a050 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:18.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.768+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7b0405eab0 con 0x7f7b0c154990 2024-09-21T02:18:18.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.928+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b0c06a270 con 0x7f7af0073320 2024-09-21T02:18:18.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.940+0000 7f7b01ffb640 1 -- 172.21.15.170:0/2105123584 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252067 (secure 0 0 0) 0x7f7b08005180 con 0x7f7af0073320 2024-09-21T02:18:18.944 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:18.946 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:18:18.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073320 msgr2=0x7f7af0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073320 0x7f7af0075830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b080ae000 tx=0x7f7b0801a050 comp rx=0 tx=0).stop 2024-09-21T02:18:18.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 msgr2=0x7f7b0c0c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 0x7f7b0c0c93f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0cfc40 tx=0x7f7b0400dd10 comp rx=0 tx=0).stop 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 shutdown_connections 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073320 0x7f7af0075830 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b0c154990 0x7f7b0c0c93f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b0c154f50 0x7f7b0c15e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 --2- 172.21.15.170:0/2105123584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af8003a10 0x7f7b0c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 >> 172.21.15.170:0/2105123584 conn(0x7f7b0c0cee30 msgr2=0x7f7b0c0b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 shutdown_connections 2024-09-21T02:18:18.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:18.944+0000 7f7b133b0640 1 -- 172.21.15.170:0/2105123584 wait complete. 2024-09-21T02:18:18.966 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-21T02:18:18.860370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275192,"kb_used_data":9024,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754699528,"statfs":{"total":773094113280,"available":772812316672,"internally_reserved":0,"allocated":9240576,"data_stored":4604713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001858"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.444091+0000","last_change":"2024-09-21T02:18:14.442511+0000","last_active":"2024-09-21T02:18:14.444091+0000","last_peered":"2024-09-21T02:18:14.444091+0000","last_clean":"2024-09-21T02:18:14.444091+0000","last_became_active":"2024-09-21T02:18:14.442206+0000","last_became_peered":"2024-09-21T02:18:14.442206+0000","last_unstale":"2024-09-21T02:18:14.444091+0000","last_undegraded":"2024-09-21T02:18:14.444091+0000","last_fullsized":"2024-09-21T02:18:14.444091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:50:33.298823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.454938+0000","last_change":"2024-09-21T02:18:14.440433+0000","last_active":"2024-09-21T02:18:14.454938+0000","last_peered":"2024-09-21T02:18:14.454938+0000","last_clean":"2024-09-21T02:18:14.454938+0000","last_became_active":"2024-09-21T02:18:14.439408+0000","last_became_peered":"2024-09-21T02:18:14.439408+0000","last_unstale":"2024-09-21T02:18:14.454938+0000","last_undegraded":"2024-09-21T02:18:14.454938+0000","last_fullsized":"2024-09-21T02:18:14.454938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:49.889056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.443201+0000","last_change":"2024-09-21T02:18:14.440910+0000","last_active":"2024-09-21T02:18:14.443201+0000","last_peered":"2024-09-21T02:18:14.443201+0000","last_clean":"2024-09-21T02:18:14.443201+0000","last_became_active":"2024-09-21T02:18:14.440583+0000","last_became_peered":"2024-09-21T02:18:14.440583+0000","last_unstale":"2024-09-21T02:18:14.443201+0000","last_undegraded":"2024-09-21T02:18:14.443201+0000","last_fullsized":"2024-09-21T02:18:14.443201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.283268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.456222+0000","last_change":"2024-09-21T02:18:14.449909+0000","last_active":"2024-09-21T02:18:14.456222+0000","last_peered":"2024-09-21T02:18:14.456222+0000","last_clean":"2024-09-21T02:18:14.456222+0000","last_became_active":"2024-09-21T02:18:14.440669+0000","last_became_peered":"2024-09-21T02:18:14.440669+0000","last_unstale":"2024-09-21T02:18:14.456222+0000","last_undegraded":"2024-09-21T02:18:14.456222+0000","last_fullsized":"2024-09-21T02:18:14.456222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:41.028084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.452111+0000","last_change":"2024-09-21T02:18:14.451394+0000","last_active":"2024-09-21T02:18:14.452111+0000","last_peered":"2024-09-21T02:18:14.452111+0000","last_clean":"2024-09-21T02:18:14.452111+0000","last_became_active":"2024-09-21T02:18:14.451109+0000","last_became_peered":"2024-09-21T02:18:14.451109+0000","last_unstale":"2024-09-21T02:18:14.452111+0000","last_undegraded":"2024-09-21T02:18:14.452111+0000","last_fullsized":"2024-09-21T02:18:14.452111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:37.095580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458276+0000","last_change":"2024-09-21T02:18:14.457134+0000","last_active":"2024-09-21T02:18:14.458276+0000","last_peered":"2024-09-21T02:18:14.458276+0000","last_clean":"2024-09-21T02:18:14.458276+0000","last_became_active":"2024-09-21T02:18:14.456645+0000","last_became_peered":"2024-09-21T02:18:14.456645+0000","last_unstale":"2024-09-21T02:18:14.458276+0000","last_undegraded":"2024-09-21T02:18:14.458276+0000","last_fullsized":"2024-09-21T02:18:14.458276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:08.567073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.428765+0000","last_change":"2024-09-21T02:18:14.419796+0000","last_active":"2024-09-21T02:18:14.428765+0000","last_peered":"2024-09-21T02:18:14.428765+0000","last_clean":"2024-09-21T02:18:14.428765+0000","last_became_active":"2024-09-21T02:18:14.418913+0000","last_became_peered":"2024-09-21T02:18:14.418913+0000","last_unstale":"2024-09-21T02:18:14.428765+0000","last_undegraded":"2024-09-21T02:18:14.428765+0000","last_fullsized":"2024-09-21T02:18:14.428765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:11.233472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478386+0000","last_change":"2024-09-21T02:18:14.477543+0000","last_active":"2024-09-21T02:18:14.478386+0000","last_peered":"2024-09-21T02:18:14.478386+0000","last_clean":"2024-09-21T02:18:14.478386+0000","last_became_active":"2024-09-21T02:18:14.477189+0000","last_became_peered":"2024-09-21T02:18:14.477189+0000","last_unstale":"2024-09-21T02:18:14.478386+0000","last_undegraded":"2024-09-21T02:18:14.478386+0000","last_fullsized":"2024-09-21T02:18:14.478386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:16:05.955688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.435557+0000","last_change":"2024-09-21T02:18:14.433850+0000","last_active":"2024-09-21T02:18:14.435557+0000","last_peered":"2024-09-21T02:18:14.435557+0000","last_clean":"2024-09-21T02:18:14.435557+0000","last_became_active":"2024-09-21T02:18:14.433558+0000","last_became_peered":"2024-09-21T02:18:14.433558+0000","last_unstale":"2024-09-21T02:18:14.435557+0000","last_undegraded":"2024-09-21T02:18:14.435557+0000","last_fullsized":"2024-09-21T02:18:14.435557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:43.479479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458291+0000","last_change":"2024-09-21T02:18:14.457478+0000","last_active":"2024-09-21T02:18:14.458291+0000","last_peered":"2024-09-21T02:18:14.458291+0000","last_clean":"2024-09-21T02:18:14.458291+0000","last_became_active":"2024-09-21T02:18:14.457224+0000","last_became_peered":"2024-09-21T02:18:14.457224+0000","last_unstale":"2024-09-21T02:18:14.458291+0000","last_undegraded":"2024-09-21T02:18:14.458291+0000","last_fullsized":"2024-09-21T02:18:14.458291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:58.078603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.476915+0000","last_change":"2024-09-21T02:18:14.471915+0000","last_active":"2024-09-21T02:18:14.476915+0000","last_peered":"2024-09-21T02:18:14.476915+0000","last_clean":"2024-09-21T02:18:14.476915+0000","last_became_active":"2024-09-21T02:18:14.471304+0000","last_became_peered":"2024-09-21T02:18:14.471304+0000","last_unstale":"2024-09-21T02:18:14.476915+0000","last_undegraded":"2024-09-21T02:18:14.476915+0000","last_fullsized":"2024-09-21T02:18:14.476915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:32.610748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449960+0000","last_change":"2024-09-21T02:18:14.449041+0000","last_active":"2024-09-21T02:18:14.449960+0000","last_peered":"2024-09-21T02:18:14.449960+0000","last_clean":"2024-09-21T02:18:14.449960+0000","last_became_active":"2024-09-21T02:18:14.448665+0000","last_became_peered":"2024-09-21T02:18:14.448665+0000","last_unstale":"2024-09-21T02:18:14.449960+0000","last_undegraded":"2024-09-21T02:18:14.449960+0000","last_fullsized":"2024-09-21T02:18:14.449960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:26.562143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.439544+0000","last_change":"2024-09-21T02:18:14.432786+0000","last_active":"2024-09-21T02:18:14.439544+0000","last_peered":"2024-09-21T02:18:14.439544+0000","last_clean":"2024-09-21T02:18:14.439544+0000","last_became_active":"2024-09-21T02:18:14.432372+0000","last_became_peered":"2024-09-21T02:18:14.432372+0000","last_unstale":"2024-09-21T02:18:14.439544+0000","last_undegraded":"2024-09-21T02:18:14.439544+0000","last_fullsized":"2024-09-21T02:18:14.439544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:28.646671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.460701+0000","last_change":"2024-09-21T02:18:14.453043+0000","last_active":"2024-09-21T02:18:14.460701+0000","last_peered":"2024-09-21T02:18:14.460701+0000","last_clean":"2024-09-21T02:18:14.460701+0000","last_became_active":"2024-09-21T02:18:14.452219+0000","last_became_peered":"2024-09-21T02:18:14.452219+0000","last_unstale":"2024-09-21T02:18:14.460701+0000","last_undegraded":"2024-09-21T02:18:14.460701+0000","last_fullsized":"2024-09-21T02:18:14.460701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:33.999188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.459339+0000","last_change":"2024-09-21T02:18:14.456962+0000","last_active":"2024-09-21T02:18:14.459339+0000","last_peered":"2024-09-21T02:18:14.459339+0000","last_clean":"2024-09-21T02:18:14.459339+0000","last_became_active":"2024-09-21T02:18:14.455983+0000","last_became_peered":"2024-09-21T02:18:14.455983+0000","last_unstale":"2024-09-21T02:18:14.459339+0000","last_undegraded":"2024-09-21T02:18:14.459339+0000","last_fullsized":"2024-09-21T02:18:14.459339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:46.728150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.443258+0000","last_change":"2024-09-21T02:18:14.437566+0000","last_active":"2024-09-21T02:18:14.443258+0000","last_peered":"2024-09-21T02:18:14.443258+0000","last_clean":"2024-09-21T02:18:14.443258+0000","last_became_active":"2024-09-21T02:18:14.436791+0000","last_became_peered":"2024-09-21T02:18:14.436791+0000","last_unstale":"2024-09-21T02:18:14.443258+0000","last_undegraded":"2024-09-21T02:18:14.443258+0000","last_fullsized":"2024-09-21T02:18:14.443258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:05.776334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.450208+0000","last_change":"2024-09-21T02:18:14.449189+0000","last_active":"2024-09-21T02:18:14.450208+0000","last_peered":"2024-09-21T02:18:14.450208+0000","last_clean":"2024-09-21T02:18:14.450208+0000","last_became_active":"2024-09-21T02:18:14.448643+0000","last_became_peered":"2024-09-21T02:18:14.448643+0000","last_unstale":"2024-09-21T02:18:14.450208+0000","last_undegraded":"2024-09-21T02:18:14.450208+0000","last_fullsized":"2024-09-21T02:18:14.450208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:08.567073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474217+0000","last_change":"2024-09-21T02:18:14.469252+0000","last_active":"2024-09-21T02:18:14.474217+0000","last_peered":"2024-09-21T02:18:14.474217+0000","last_clean":"2024-09-21T02:18:14.474217+0000","last_became_active":"2024-09-21T02:18:14.468634+0000","last_became_peered":"2024-09-21T02:18:14.468634+0000","last_unstale":"2024-09-21T02:18:14.474217+0000","last_undegraded":"2024-09-21T02:18:14.474217+0000","last_fullsized":"2024-09-21T02:18:14.474217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:51.049187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.465179+0000","last_change":"2024-09-21T02:18:14.457778+0000","last_active":"2024-09-21T02:18:14.465179+0000","last_peered":"2024-09-21T02:18:14.465179+0000","last_clean":"2024-09-21T02:18:14.465179+0000","last_became_active":"2024-09-21T02:18:14.457064+0000","last_became_peered":"2024-09-21T02:18:14.457064+0000","last_unstale":"2024-09-21T02:18:14.465179+0000","last_undegraded":"2024-09-21T02:18:14.465179+0000","last_fullsized":"2024-09-21T02:18:14.465179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:12.145104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.465883+0000","last_change":"2024-09-21T02:18:14.459995+0000","last_active":"2024-09-21T02:18:14.465883+0000","last_peered":"2024-09-21T02:18:14.465883+0000","last_clean":"2024-09-21T02:18:14.465883+0000","last_became_active":"2024-09-21T02:18:14.459299+0000","last_became_peered":"2024-09-21T02:18:14.459299+0000","last_unstale":"2024-09-21T02:18:14.465883+0000","last_undegraded":"2024-09-21T02:18:14.465883+0000","last_fullsized":"2024-09-21T02:18:14.465883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:36.105764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.460541+0000","last_change":"2024-09-21T02:18:14.455760+0000","last_active":"2024-09-21T02:18:14.460541+0000","last_peered":"2024-09-21T02:18:14.460541+0000","last_clean":"2024-09-21T02:18:14.460541+0000","last_became_active":"2024-09-21T02:18:14.454922+0000","last_became_peered":"2024-09-21T02:18:14.454922+0000","last_unstale":"2024-09-21T02:18:14.460541+0000","last_undegraded":"2024-09-21T02:18:14.460541+0000","last_fullsized":"2024-09-21T02:18:14.460541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:27.030849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.450551+0000","last_change":"2024-09-21T02:18:14.441894+0000","last_active":"2024-09-21T02:18:14.450551+0000","last_peered":"2024-09-21T02:18:14.450551+0000","last_clean":"2024-09-21T02:18:14.450551+0000","last_became_active":"2024-09-21T02:18:14.441025+0000","last_became_peered":"2024-09-21T02:18:14.441025+0000","last_unstale":"2024-09-21T02:18:14.450551+0000","last_undegraded":"2024-09-21T02:18:14.450551+0000","last_fullsized":"2024-09-21T02:18:14.450551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:34.675299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449949+0000","last_change":"2024-09-21T02:18:14.448891+0000","last_active":"2024-09-21T02:18:14.449949+0000","last_peered":"2024-09-21T02:18:14.449949+0000","last_clean":"2024-09-21T02:18:14.449949+0000","last_became_active":"2024-09-21T02:18:14.448546+0000","last_became_peered":"2024-09-21T02:18:14.448546+0000","last_unstale":"2024-09-21T02:18:14.449949+0000","last_undegraded":"2024-09-21T02:18:14.449949+0000","last_fullsized":"2024-09-21T02:18:14.449949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.238723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.477033+0000","last_change":"2024-09-21T02:18:14.472678+0000","last_active":"2024-09-21T02:18:14.477033+0000","last_peered":"2024-09-21T02:18:14.477033+0000","last_clean":"2024-09-21T02:18:14.477033+0000","last_became_active":"2024-09-21T02:18:14.471237+0000","last_became_peered":"2024-09-21T02:18:14.471237+0000","last_unstale":"2024-09-21T02:18:14.477033+0000","last_undegraded":"2024-09-21T02:18:14.477033+0000","last_fullsized":"2024-09-21T02:18:14.477033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:28.684773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474297+0000","last_change":"2024-09-21T02:18:14.469439+0000","last_active":"2024-09-21T02:18:14.474297+0000","last_peered":"2024-09-21T02:18:14.474297+0000","last_clean":"2024-09-21T02:18:14.474297+0000","last_became_active":"2024-09-21T02:18:14.468153+0000","last_became_peered":"2024-09-21T02:18:14.468153+0000","last_unstale":"2024-09-21T02:18:14.474297+0000","last_undegraded":"2024-09-21T02:18:14.474297+0000","last_fullsized":"2024-09-21T02:18:14.474297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:00.543962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.445247+0000","last_change":"2024-09-21T02:18:14.440260+0000","last_active":"2024-09-21T02:18:14.445247+0000","last_peered":"2024-09-21T02:18:14.445247+0000","last_clean":"2024-09-21T02:18:14.445247+0000","last_became_active":"2024-09-21T02:18:14.439634+0000","last_became_peered":"2024-09-21T02:18:14.439634+0000","last_unstale":"2024-09-21T02:18:14.445247+0000","last_undegraded":"2024-09-21T02:18:14.445247+0000","last_fullsized":"2024-09-21T02:18:14.445247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:42.351533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.463396+0000","last_change":"2024-09-21T02:18:14.462988+0000","last_active":"2024-09-21T02:18:14.463396+0000","last_peered":"2024-09-21T02:18:14.463396+0000","last_clean":"2024-09-21T02:18:14.463396+0000","last_became_active":"2024-09-21T02:18:14.462668+0000","last_became_peered":"2024-09-21T02:18:14.462668+0000","last_unstale":"2024-09-21T02:18:14.463396+0000","last_undegraded":"2024-09-21T02:18:14.463396+0000","last_fullsized":"2024-09-21T02:18:14.463396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:01.188752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442566+0000","last_change":"2024-09-21T02:18:14.438106+0000","last_active":"2024-09-21T02:18:14.442566+0000","last_peered":"2024-09-21T02:18:14.442566+0000","last_clean":"2024-09-21T02:18:14.442566+0000","last_became_active":"2024-09-21T02:18:14.437747+0000","last_became_peered":"2024-09-21T02:18:14.437747+0000","last_unstale":"2024-09-21T02:18:14.442566+0000","last_undegraded":"2024-09-21T02:18:14.442566+0000","last_fullsized":"2024-09-21T02:18:14.442566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:03.234592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.448889+0000","last_change":"2024-09-21T02:18:14.423363+0000","last_active":"2024-09-21T02:18:14.448889+0000","last_peered":"2024-09-21T02:18:14.448889+0000","last_clean":"2024-09-21T02:18:14.448889+0000","last_became_active":"2024-09-21T02:18:14.420919+0000","last_became_peered":"2024-09-21T02:18:14.420919+0000","last_unstale":"2024-09-21T02:18:14.448889+0000","last_undegraded":"2024-09-21T02:18:14.448889+0000","last_fullsized":"2024-09-21T02:18:14.448889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:13.071985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.460631+0000","last_change":"2024-09-21T02:18:14.453125+0000","last_active":"2024-09-21T02:18:14.460631+0000","last_peered":"2024-09-21T02:18:14.460631+0000","last_clean":"2024-09-21T02:18:14.460631+0000","last_became_active":"2024-09-21T02:18:14.452195+0000","last_became_peered":"2024-09-21T02:18:14.452195+0000","last_unstale":"2024-09-21T02:18:14.460631+0000","last_undegraded":"2024-09-21T02:18:14.460631+0000","last_fullsized":"2024-09-21T02:18:14.460631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:36.103455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458004+0000","last_change":"2024-09-21T02:18:14.456357+0000","last_active":"2024-09-21T02:18:14.458004+0000","last_peered":"2024-09-21T02:18:14.458004+0000","last_clean":"2024-09-21T02:18:14.458004+0000","last_became_active":"2024-09-21T02:18:14.455997+0000","last_became_peered":"2024-09-21T02:18:14.455997+0000","last_unstale":"2024-09-21T02:18:14.458004+0000","last_undegraded":"2024-09-21T02:18:14.458004+0000","last_fullsized":"2024-09-21T02:18:14.458004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:03.238723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.458161+0000","last_change":"2024-09-21T02:18:14.456648+0000","last_active":"2024-09-21T02:18:14.458161+0000","last_peered":"2024-09-21T02:18:14.458161+0000","last_clean":"2024-09-21T02:18:14.458161+0000","last_became_active":"2024-09-21T02:18:14.456296+0000","last_became_peered":"2024-09-21T02:18:14.456296+0000","last_unstale":"2024-09-21T02:18:14.458161+0000","last_undegraded":"2024-09-21T02:18:14.458161+0000","last_fullsized":"2024-09-21T02:18:14.458161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:33:26.562143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.433291+0000","last_change":"2024-09-21T02:18:14.431779+0000","last_active":"2024-09-21T02:18:14.433291+0000","last_peered":"2024-09-21T02:18:14.433291+0000","last_clean":"2024-09-21T02:18:14.433291+0000","last_became_active":"2024-09-21T02:18:14.431455+0000","last_became_peered":"2024-09-21T02:18:14.431455+0000","last_unstale":"2024-09-21T02:18:14.433291+0000","last_undegraded":"2024-09-21T02:18:14.433291+0000","last_fullsized":"2024-09-21T02:18:14.433291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:45.576522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.477463+0000","last_change":"2024-09-21T02:18:13.439570+0000","last_active":"2024-09-21T02:18:14.477463+0000","last_peered":"2024-09-21T02:18:14.477463+0000","last_clean":"2024-09-21T02:18:14.477463+0000","last_became_active":"2024-09-21T02:18:13.439020+0000","last_became_peered":"2024-09-21T02:18:13.439020+0000","last_unstale":"2024-09-21T02:18:14.477463+0000","last_undegraded":"2024-09-21T02:18:14.477463+0000","last_fullsized":"2024-09-21T02:18:14.477463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:22.520314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.434062+0000","last_change":"2024-09-21T02:18:14.430439+0000","last_active":"2024-09-21T02:18:14.434062+0000","last_peered":"2024-09-21T02:18:14.434062+0000","last_clean":"2024-09-21T02:18:14.434062+0000","last_became_active":"2024-09-21T02:18:14.429768+0000","last_became_peered":"2024-09-21T02:18:14.429768+0000","last_unstale":"2024-09-21T02:18:14.434062+0000","last_undegraded":"2024-09-21T02:18:14.434062+0000","last_fullsized":"2024-09-21T02:18:14.434062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:19.464328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.448223+0000","last_change":"2024-09-21T02:18:13.445278+0000","last_active":"2024-09-21T02:18:14.448223+0000","last_peered":"2024-09-21T02:18:14.448223+0000","last_clean":"2024-09-21T02:18:14.448223+0000","last_became_active":"2024-09-21T02:18:13.444819+0000","last_became_peered":"2024-09-21T02:18:13.444819+0000","last_unstale":"2024-09-21T02:18:14.448223+0000","last_undegraded":"2024-09-21T02:18:14.448223+0000","last_fullsized":"2024-09-21T02:18:14.448223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:33.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.455102+0000","last_change":"2024-09-21T02:18:14.454455+0000","last_active":"2024-09-21T02:18:14.455102+0000","last_peered":"2024-09-21T02:18:14.455102+0000","last_clean":"2024-09-21T02:18:14.455102+0000","last_became_active":"2024-09-21T02:18:14.454187+0000","last_became_peered":"2024-09-21T02:18:14.454187+0000","last_unstale":"2024-09-21T02:18:14.455102+0000","last_undegraded":"2024-09-21T02:18:14.455102+0000","last_fullsized":"2024-09-21T02:18:14.455102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:58.078603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.477098+0000","last_change":"2024-09-21T02:18:13.422096+0000","last_active":"2024-09-21T02:18:14.477098+0000","last_peered":"2024-09-21T02:18:14.477098+0000","last_clean":"2024-09-21T02:18:14.477098+0000","last_became_active":"2024-09-21T02:18:13.421119+0000","last_became_peered":"2024-09-21T02:18:13.421119+0000","last_unstale":"2024-09-21T02:18:14.477098+0000","last_undegraded":"2024-09-21T02:18:14.477098+0000","last_fullsized":"2024-09-21T02:18:14.477098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:20.428236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.475124+0000","last_change":"2024-09-21T02:18:14.474452+0000","last_active":"2024-09-21T02:18:14.475124+0000","last_peered":"2024-09-21T02:18:14.475124+0000","last_clean":"2024-09-21T02:18:14.475124+0000","last_became_active":"2024-09-21T02:18:14.474164+0000","last_became_peered":"2024-09-21T02:18:14.474164+0000","last_unstale":"2024-09-21T02:18:14.475124+0000","last_undegraded":"2024-09-21T02:18:14.475124+0000","last_fullsized":"2024-09-21T02:18:14.475124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:59.090283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.461599+0000","last_change":"2024-09-21T02:18:13.428048+0000","last_active":"2024-09-21T02:18:14.461599+0000","last_peered":"2024-09-21T02:18:14.461599+0000","last_clean":"2024-09-21T02:18:14.461599+0000","last_became_active":"2024-09-21T02:18:13.427244+0000","last_became_peered":"2024-09-21T02:18:13.427244+0000","last_unstale":"2024-09-21T02:18:14.461599+0000","last_undegraded":"2024-09-21T02:18:14.461599+0000","last_fullsized":"2024-09-21T02:18:14.461599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:40.446937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478319+0000","last_change":"2024-09-21T02:18:14.476862+0000","last_active":"2024-09-21T02:18:14.478319+0000","last_peered":"2024-09-21T02:18:14.478319+0000","last_clean":"2024-09-21T02:18:14.478319+0000","last_became_active":"2024-09-21T02:18:14.476027+0000","last_became_peered":"2024-09-21T02:18:14.476027+0000","last_unstale":"2024-09-21T02:18:14.478319+0000","last_undegraded":"2024-09-21T02:18:14.478319+0000","last_fullsized":"2024-09-21T02:18:14.478319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:46.346040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.430408+0000","last_change":"2024-09-21T02:18:13.420698+0000","last_active":"2024-09-21T02:18:14.430408+0000","last_peered":"2024-09-21T02:18:14.430408+0000","last_clean":"2024-09-21T02:18:14.430408+0000","last_became_active":"2024-09-21T02:18:13.420291+0000","last_became_peered":"2024-09-21T02:18:13.420291+0000","last_unstale":"2024-09-21T02:18:14.430408+0000","last_undegraded":"2024-09-21T02:18:14.430408+0000","last_fullsized":"2024-09-21T02:18:14.430408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:43.529177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.443776+0000","last_change":"2024-09-21T02:18:14.442893+0000","last_active":"2024-09-21T02:18:14.443776+0000","last_peered":"2024-09-21T02:18:14.443776+0000","last_clean":"2024-09-21T02:18:14.443776+0000","last_became_active":"2024-09-21T02:18:14.442456+0000","last_became_peered":"2024-09-21T02:18:14.442456+0000","last_unstale":"2024-09-21T02:18:14.443776+0000","last_undegraded":"2024-09-21T02:18:14.443776+0000","last_fullsized":"2024-09-21T02:18:14.443776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:45.576522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.462511+0000","last_change":"2024-09-21T02:18:13.420235+0000","last_active":"2024-09-21T02:18:14.462511+0000","last_peered":"2024-09-21T02:18:14.462511+0000","last_clean":"2024-09-21T02:18:14.462511+0000","last_became_active":"2024-09-21T02:18:13.419857+0000","last_became_peered":"2024-09-21T02:18:13.419857+0000","last_unstale":"2024-09-21T02:18:14.462511+0000","last_undegraded":"2024-09-21T02:18:14.462511+0000","last_fullsized":"2024-09-21T02:18:14.462511+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:48.163803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.476168+0000","last_change":"2024-09-21T02:18:14.475730+0000","last_active":"2024-09-21T02:18:14.476168+0000","last_peered":"2024-09-21T02:18:14.476168+0000","last_clean":"2024-09-21T02:18:14.476168+0000","last_became_active":"2024-09-21T02:18:14.475409+0000","last_became_peered":"2024-09-21T02:18:14.475409+0000","last_unstale":"2024-09-21T02:18:14.476168+0000","last_undegraded":"2024-09-21T02:18:14.476168+0000","last_fullsized":"2024-09-21T02:18:14.476168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:56.997803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.424807+0000","last_change":"2024-09-21T02:18:13.444468+0000","last_active":"2024-09-21T02:18:14.424807+0000","last_peered":"2024-09-21T02:18:14.424807+0000","last_clean":"2024-09-21T02:18:14.424807+0000","last_became_active":"2024-09-21T02:18:13.443456+0000","last_became_peered":"2024-09-21T02:18:13.443456+0000","last_unstale":"2024-09-21T02:18:14.424807+0000","last_undegraded":"2024-09-21T02:18:14.424807+0000","last_fullsized":"2024-09-21T02:18:14.424807+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:08.679962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.456087+0000","last_change":"2024-09-21T02:18:14.446856+0000","last_active":"2024-09-21T02:18:14.456087+0000","last_peered":"2024-09-21T02:18:14.456087+0000","last_clean":"2024-09-21T02:18:14.456087+0000","last_became_active":"2024-09-21T02:18:14.444846+0000","last_became_peered":"2024-09-21T02:18:14.444846+0000","last_unstale":"2024-09-21T02:18:14.456087+0000","last_undegraded":"2024-09-21T02:18:14.456087+0000","last_fullsized":"2024-09-21T02:18:14.456087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:16.108862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.425878+0000","last_change":"2024-09-21T02:18:13.444565+0000","last_active":"2024-09-21T02:18:14.425878+0000","last_peered":"2024-09-21T02:18:14.425878+0000","last_clean":"2024-09-21T02:18:14.425878+0000","last_became_active":"2024-09-21T02:18:13.443552+0000","last_became_peered":"2024-09-21T02:18:13.443552+0000","last_unstale":"2024-09-21T02:18:14.425878+0000","last_undegraded":"2024-09-21T02:18:14.425878+0000","last_fullsized":"2024-09-21T02:18:14.425878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:25.175151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449858+0000","last_change":"2024-09-21T02:18:14.443175+0000","last_active":"2024-09-21T02:18:14.449858+0000","last_peered":"2024-09-21T02:18:14.449858+0000","last_clean":"2024-09-21T02:18:14.449858+0000","last_became_active":"2024-09-21T02:18:14.442409+0000","last_became_peered":"2024-09-21T02:18:14.442409+0000","last_unstale":"2024-09-21T02:18:14.449858+0000","last_undegraded":"2024-09-21T02:18:14.449858+0000","last_fullsized":"2024-09-21T02:18:14.449858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:52.762005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.429711+0000","last_change":"2024-09-21T02:18:13.422135+0000","last_active":"2024-09-21T02:18:14.429711+0000","last_peered":"2024-09-21T02:18:14.429711+0000","last_clean":"2024-09-21T02:18:14.429711+0000","last_became_active":"2024-09-21T02:18:13.421810+0000","last_became_peered":"2024-09-21T02:18:13.421810+0000","last_unstale":"2024-09-21T02:18:14.429711+0000","last_undegraded":"2024-09-21T02:18:14.429711+0000","last_fullsized":"2024-09-21T02:18:14.429711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:49.944038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.473928+0000","last_change":"2024-09-21T02:18:14.473312+0000","last_active":"2024-09-21T02:18:14.473928+0000","last_peered":"2024-09-21T02:18:14.473928+0000","last_clean":"2024-09-21T02:18:14.473928+0000","last_became_active":"2024-09-21T02:18:14.473063+0000","last_became_peered":"2024-09-21T02:18:14.473063+0000","last_unstale":"2024-09-21T02:18:14.473928+0000","last_undegraded":"2024-09-21T02:18:14.473928+0000","last_fullsized":"2024-09-21T02:18:14.473928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:27.848671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.426668+0000","last_change":"2024-09-21T02:18:13.418626+0000","last_active":"2024-09-21T02:18:14.426668+0000","last_peered":"2024-09-21T02:18:14.426668+0000","last_clean":"2024-09-21T02:18:14.426668+0000","last_became_active":"2024-09-21T02:18:13.418109+0000","last_became_peered":"2024-09-21T02:18:13.418109+0000","last_unstale":"2024-09-21T02:18:14.426668+0000","last_undegraded":"2024-09-21T02:18:14.426668+0000","last_fullsized":"2024-09-21T02:18:14.426668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:32.421158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478800+0000","last_change":"2024-09-21T02:18:14.474526+0000","last_active":"2024-09-21T02:18:14.478800+0000","last_peered":"2024-09-21T02:18:14.478800+0000","last_clean":"2024-09-21T02:18:14.478800+0000","last_became_active":"2024-09-21T02:18:14.468379+0000","last_became_peered":"2024-09-21T02:18:14.468379+0000","last_unstale":"2024-09-21T02:18:14.478800+0000","last_undegraded":"2024-09-21T02:18:14.478800+0000","last_fullsized":"2024-09-21T02:18:14.478800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:35.046249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.479351+0000","last_change":"2024-09-21T02:18:13.434165+0000","last_active":"2024-09-21T02:18:14.479351+0000","last_peered":"2024-09-21T02:18:14.479351+0000","last_clean":"2024-09-21T02:18:14.479351+0000","last_became_active":"2024-09-21T02:18:13.433725+0000","last_became_peered":"2024-09-21T02:18:13.433725+0000","last_unstale":"2024-09-21T02:18:14.479351+0000","last_undegraded":"2024-09-21T02:18:14.479351+0000","last_fullsized":"2024-09-21T02:18:14.479351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:00:47.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478552+0000","last_change":"2024-09-21T02:18:14.477437+0000","last_active":"2024-09-21T02:18:14.478552+0000","last_peered":"2024-09-21T02:18:14.478552+0000","last_clean":"2024-09-21T02:18:14.478552+0000","last_became_active":"2024-09-21T02:18:14.477077+0000","last_became_peered":"2024-09-21T02:18:14.477077+0000","last_unstale":"2024-09-21T02:18:14.478552+0000","last_undegraded":"2024-09-21T02:18:14.478552+0000","last_fullsized":"2024-09-21T02:18:14.478552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:32.952815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.467656+0000","last_change":"2024-09-21T02:18:13.436262+0000","last_active":"2024-09-21T02:18:14.467656+0000","last_peered":"2024-09-21T02:18:14.467656+0000","last_clean":"2024-09-21T02:18:14.467656+0000","last_became_active":"2024-09-21T02:18:13.435918+0000","last_became_peered":"2024-09-21T02:18:13.435918+0000","last_unstale":"2024-09-21T02:18:14.467656+0000","last_undegraded":"2024-09-21T02:18:14.467656+0000","last_fullsized":"2024-09-21T02:18:14.467656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:04.772278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474774+0000","last_change":"2024-09-21T02:18:14.464283+0000","last_active":"2024-09-21T02:18:14.474774+0000","last_peered":"2024-09-21T02:18:14.474774+0000","last_clean":"2024-09-21T02:18:14.474774+0000","last_became_active":"2024-09-21T02:18:14.463247+0000","last_became_peered":"2024-09-21T02:18:14.463247+0000","last_unstale":"2024-09-21T02:18:14.474774+0000","last_undegraded":"2024-09-21T02:18:14.474774+0000","last_fullsized":"2024-09-21T02:18:14.474774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:55.643772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.466430+0000","last_change":"2024-09-21T02:18:13.436253+0000","last_active":"2024-09-21T02:18:14.466430+0000","last_peered":"2024-09-21T02:18:14.466430+0000","last_clean":"2024-09-21T02:18:14.466430+0000","last_became_active":"2024-09-21T02:18:13.435306+0000","last_became_peered":"2024-09-21T02:18:13.435306+0000","last_unstale":"2024-09-21T02:18:14.466430+0000","last_undegraded":"2024-09-21T02:18:14.466430+0000","last_fullsized":"2024-09-21T02:18:14.466430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:18.380614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.471018+0000","last_change":"2024-09-21T02:18:14.470499+0000","last_active":"2024-09-21T02:18:14.471018+0000","last_peered":"2024-09-21T02:18:14.471018+0000","last_clean":"2024-09-21T02:18:14.471018+0000","last_became_active":"2024-09-21T02:18:14.470099+0000","last_became_peered":"2024-09-21T02:18:14.470099+0000","last_unstale":"2024-09-21T02:18:14.471018+0000","last_undegraded":"2024-09-21T02:18:14.471018+0000","last_fullsized":"2024-09-21T02:18:14.471018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:33.751569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410899+0000","last_change":"2024-09-21T02:18:13.432542+0000","last_active":"2024-09-21T02:18:14.410899+0000","last_peered":"2024-09-21T02:18:14.410899+0000","last_clean":"2024-09-21T02:18:14.410899+0000","last_became_active":"2024-09-21T02:18:13.431660+0000","last_became_peered":"2024-09-21T02:18:13.431660+0000","last_unstale":"2024-09-21T02:18:14.410899+0000","last_undegraded":"2024-09-21T02:18:14.410899+0000","last_fullsized":"2024-09-21T02:18:14.410899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:39.336082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478633+0000","last_change":"2024-09-21T02:18:14.475038+0000","last_active":"2024-09-21T02:18:14.478633+0000","last_peered":"2024-09-21T02:18:14.478633+0000","last_clean":"2024-09-21T02:18:14.478633+0000","last_became_active":"2024-09-21T02:18:14.474297+0000","last_became_peered":"2024-09-21T02:18:14.474297+0000","last_unstale":"2024-09-21T02:18:14.478633+0000","last_undegraded":"2024-09-21T02:18:14.478633+0000","last_fullsized":"2024-09-21T02:18:14.478633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:36:11.491097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.422913+0000","last_change":"2024-09-21T02:18:13.429048+0000","last_active":"2024-09-21T02:18:14.422913+0000","last_peered":"2024-09-21T02:18:14.422913+0000","last_clean":"2024-09-21T02:18:14.422913+0000","last_became_active":"2024-09-21T02:18:13.428514+0000","last_became_peered":"2024-09-21T02:18:13.428514+0000","last_unstale":"2024-09-21T02:18:14.422913+0000","last_undegraded":"2024-09-21T02:18:14.422913+0000","last_fullsized":"2024-09-21T02:18:14.422913+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:15.874001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474855+0000","last_change":"2024-09-21T02:18:14.464654+0000","last_active":"2024-09-21T02:18:14.474855+0000","last_peered":"2024-09-21T02:18:14.474855+0000","last_clean":"2024-09-21T02:18:14.474855+0000","last_became_active":"2024-09-21T02:18:14.463200+0000","last_became_peered":"2024-09-21T02:18:14.463200+0000","last_unstale":"2024-09-21T02:18:14.474855+0000","last_undegraded":"2024-09-21T02:18:14.474855+0000","last_fullsized":"2024-09-21T02:18:14.474855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:35.121991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"83'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.481573+0000","last_change":"2024-09-21T02:18:13.435771+0000","last_active":"2024-09-21T02:18:14.481573+0000","last_peered":"2024-09-21T02:18:14.481573+0000","last_clean":"2024-09-21T02:18:14.481573+0000","last_became_active":"2024-09-21T02:18:13.434918+0000","last_became_peered":"2024-09-21T02:18:13.434918+0000","last_unstale":"2024-09-21T02:18:14.481573+0000","last_undegraded":"2024-09-21T02:18:14.481573+0000","last_fullsized":"2024-09-21T02:18:14.481573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:08.084873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.441448+0000","last_change":"2024-09-21T02:18:14.439555+0000","last_active":"2024-09-21T02:18:14.441448+0000","last_peered":"2024-09-21T02:18:14.441448+0000","last_clean":"2024-09-21T02:18:14.441448+0000","last_became_active":"2024-09-21T02:18:14.439177+0000","last_became_peered":"2024-09-21T02:18:14.439177+0000","last_unstale":"2024-09-21T02:18:14.441448+0000","last_undegraded":"2024-09-21T02:18:14.441448+0000","last_fullsized":"2024-09-21T02:18:14.441448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:10.352252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.444903+0000","last_change":"2024-09-21T02:18:13.427751+0000","last_active":"2024-09-21T02:18:14.444903+0000","last_peered":"2024-09-21T02:18:14.444903+0000","last_clean":"2024-09-21T02:18:14.444903+0000","last_became_active":"2024-09-21T02:18:13.427370+0000","last_became_peered":"2024-09-21T02:18:13.427370+0000","last_unstale":"2024-09-21T02:18:14.444903+0000","last_undegraded":"2024-09-21T02:18:14.444903+0000","last_fullsized":"2024-09-21T02:18:14.444903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:43.466174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442978+0000","last_change":"2024-09-21T02:18:14.437822+0000","last_active":"2024-09-21T02:18:14.442978+0000","last_peered":"2024-09-21T02:18:14.442978+0000","last_clean":"2024-09-21T02:18:14.442978+0000","last_became_active":"2024-09-21T02:18:14.437492+0000","last_became_peered":"2024-09-21T02:18:14.437492+0000","last_unstale":"2024-09-21T02:18:14.442978+0000","last_undegraded":"2024-09-21T02:18:14.442978+0000","last_fullsized":"2024-09-21T02:18:14.442978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:27.068670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449660+0000","last_change":"2024-09-21T02:18:13.450713+0000","last_active":"2024-09-21T02:18:14.449660+0000","last_peered":"2024-09-21T02:18:14.449660+0000","last_clean":"2024-09-21T02:18:14.449660+0000","last_became_active":"2024-09-21T02:18:13.448373+0000","last_became_peered":"2024-09-21T02:18:13.448373+0000","last_unstale":"2024-09-21T02:18:14.449660+0000","last_undegraded":"2024-09-21T02:18:14.449660+0000","last_fullsized":"2024-09-21T02:18:14.449660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:19:18.831384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.431291+0000","last_change":"2024-09-21T02:18:14.427188+0000","last_active":"2024-09-21T02:18:14.431291+0000","last_peered":"2024-09-21T02:18:14.431291+0000","last_clean":"2024-09-21T02:18:14.431291+0000","last_became_active":"2024-09-21T02:18:14.425670+0000","last_became_peered":"2024-09-21T02:18:14.425670+0000","last_unstale":"2024-09-21T02:18:14.431291+0000","last_undegraded":"2024-09-21T02:18:14.431291+0000","last_fullsized":"2024-09-21T02:18:14.431291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:33.283268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.429005+0000","last_change":"2024-09-21T02:18:13.451377+0000","last_active":"2024-09-21T02:18:14.429005+0000","last_peered":"2024-09-21T02:18:14.429005+0000","last_clean":"2024-09-21T02:18:14.429005+0000","last_became_active":"2024-09-21T02:18:13.450715+0000","last_became_peered":"2024-09-21T02:18:13.450715+0000","last_unstale":"2024-09-21T02:18:14.429005+0000","last_undegraded":"2024-09-21T02:18:14.429005+0000","last_fullsized":"2024-09-21T02:18:14.429005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:45.075046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.471005+0000","last_change":"2024-09-21T02:18:14.468616+0000","last_active":"2024-09-21T02:18:14.471005+0000","last_peered":"2024-09-21T02:18:14.471005+0000","last_clean":"2024-09-21T02:18:14.471005+0000","last_became_active":"2024-09-21T02:18:14.467981+0000","last_became_peered":"2024-09-21T02:18:14.467981+0000","last_unstale":"2024-09-21T02:18:14.471005+0000","last_undegraded":"2024-09-21T02:18:14.471005+0000","last_fullsized":"2024-09-21T02:18:14.471005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:29.031100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.455832+0000","last_change":"2024-09-21T02:18:13.450073+0000","last_active":"2024-09-21T02:18:14.455832+0000","last_peered":"2024-09-21T02:18:14.455832+0000","last_clean":"2024-09-21T02:18:14.455832+0000","last_became_active":"2024-09-21T02:18:13.449319+0000","last_became_peered":"2024-09-21T02:18:13.449319+0000","last_unstale":"2024-09-21T02:18:14.455832+0000","last_undegraded":"2024-09-21T02:18:14.455832+0000","last_fullsized":"2024-09-21T02:18:14.455832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:58:52.254943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478752+0000","last_change":"2024-09-21T02:18:14.478084+0000","last_active":"2024-09-21T02:18:14.478752+0000","last_peered":"2024-09-21T02:18:14.478752+0000","last_clean":"2024-09-21T02:18:14.478752+0000","last_became_active":"2024-09-21T02:18:14.477852+0000","last_became_peered":"2024-09-21T02:18:14.477852+0000","last_unstale":"2024-09-21T02:18:14.478752+0000","last_undegraded":"2024-09-21T02:18:14.478752+0000","last_fullsized":"2024-09-21T02:18:14.478752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:17.648764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.411171+0000","last_change":"2024-09-21T02:18:13.439660+0000","last_active":"2024-09-21T02:18:14.411171+0000","last_peered":"2024-09-21T02:18:14.411171+0000","last_clean":"2024-09-21T02:18:14.411171+0000","last_became_active":"2024-09-21T02:18:13.438842+0000","last_became_peered":"2024-09-21T02:18:13.438842+0000","last_unstale":"2024-09-21T02:18:14.411171+0000","last_undegraded":"2024-09-21T02:18:14.411171+0000","last_fullsized":"2024-09-21T02:18:14.411171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:12.643334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.455509+0000","last_change":"2024-09-21T02:18:14.441205+0000","last_active":"2024-09-21T02:18:14.455509+0000","last_peered":"2024-09-21T02:18:14.455509+0000","last_clean":"2024-09-21T02:18:14.455509+0000","last_became_active":"2024-09-21T02:18:14.428079+0000","last_became_peered":"2024-09-21T02:18:14.428079+0000","last_unstale":"2024-09-21T02:18:14.455509+0000","last_undegraded":"2024-09-21T02:18:14.455509+0000","last_fullsized":"2024-09-21T02:18:14.455509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:49.871004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.446799+0000","last_change":"2024-09-21T02:18:13.435919+0000","last_active":"2024-09-21T02:18:14.446799+0000","last_peered":"2024-09-21T02:18:14.446799+0000","last_clean":"2024-09-21T02:18:14.446799+0000","last_became_active":"2024-09-21T02:18:13.435121+0000","last_became_peered":"2024-09-21T02:18:13.435121+0000","last_unstale":"2024-09-21T02:18:14.446799+0000","last_undegraded":"2024-09-21T02:18:14.446799+0000","last_fullsized":"2024-09-21T02:18:14.446799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:47.108644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.430515+0000","last_change":"2024-09-21T02:18:14.419974+0000","last_active":"2024-09-21T02:18:14.430515+0000","last_peered":"2024-09-21T02:18:14.430515+0000","last_clean":"2024-09-21T02:18:14.430515+0000","last_became_active":"2024-09-21T02:18:14.419113+0000","last_became_peered":"2024-09-21T02:18:14.419113+0000","last_unstale":"2024-09-21T02:18:14.430515+0000","last_undegraded":"2024-09-21T02:18:14.430515+0000","last_fullsized":"2024-09-21T02:18:14.430515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:27.068670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.426492+0000","last_change":"2024-09-21T02:18:13.452885+0000","last_active":"2024-09-21T02:18:14.426492+0000","last_peered":"2024-09-21T02:18:14.426492+0000","last_clean":"2024-09-21T02:18:14.426492+0000","last_became_active":"2024-09-21T02:18:13.452113+0000","last_became_peered":"2024-09-21T02:18:13.452113+0000","last_unstale":"2024-09-21T02:18:14.426492+0000","last_undegraded":"2024-09-21T02:18:14.426492+0000","last_fullsized":"2024-09-21T02:18:14.426492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:14.820089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.440628+0000","last_change":"2024-09-21T02:18:14.431934+0000","last_active":"2024-09-21T02:18:14.440628+0000","last_peered":"2024-09-21T02:18:14.440628+0000","last_clean":"2024-09-21T02:18:14.440628+0000","last_became_active":"2024-09-21T02:18:14.431565+0000","last_became_peered":"2024-09-21T02:18:14.431565+0000","last_unstale":"2024-09-21T02:18:14.440628+0000","last_undegraded":"2024-09-21T02:18:14.440628+0000","last_fullsized":"2024-09-21T02:18:14.440628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:04:39.540203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.450193+0000","last_change":"2024-09-21T02:18:13.448964+0000","last_active":"2024-09-21T02:18:14.450193+0000","last_peered":"2024-09-21T02:18:14.450193+0000","last_clean":"2024-09-21T02:18:14.450193+0000","last_became_active":"2024-09-21T02:18:13.448433+0000","last_became_peered":"2024-09-21T02:18:13.448433+0000","last_unstale":"2024-09-21T02:18:14.450193+0000","last_undegraded":"2024-09-21T02:18:14.450193+0000","last_fullsized":"2024-09-21T02:18:14.450193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:30.532640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.444952+0000","last_change":"2024-09-21T02:18:14.435784+0000","last_active":"2024-09-21T02:18:14.444952+0000","last_peered":"2024-09-21T02:18:14.444952+0000","last_clean":"2024-09-21T02:18:14.444952+0000","last_became_active":"2024-09-21T02:18:14.435264+0000","last_became_peered":"2024-09-21T02:18:14.435264+0000","last_unstale":"2024-09-21T02:18:14.444952+0000","last_undegraded":"2024-09-21T02:18:14.444952+0000","last_fullsized":"2024-09-21T02:18:14.444952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:38.729382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.463800+0000","last_change":"2024-09-21T02:18:13.440660+0000","last_active":"2024-09-21T02:18:14.463800+0000","last_peered":"2024-09-21T02:18:14.463800+0000","last_clean":"2024-09-21T02:18:14.463800+0000","last_became_active":"2024-09-21T02:18:13.437734+0000","last_became_peered":"2024-09-21T02:18:13.437734+0000","last_unstale":"2024-09-21T02:18:14.463800+0000","last_undegraded":"2024-09-21T02:18:14.463800+0000","last_fullsized":"2024-09-21T02:18:14.463800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:02.522881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.430740+0000","last_change":"2024-09-21T02:18:14.423873+0000","last_active":"2024-09-21T02:18:14.430740+0000","last_peered":"2024-09-21T02:18:14.430740+0000","last_clean":"2024-09-21T02:18:14.430740+0000","last_became_active":"2024-09-21T02:18:14.423608+0000","last_became_peered":"2024-09-21T02:18:14.423608+0000","last_unstale":"2024-09-21T02:18:14.430740+0000","last_undegraded":"2024-09-21T02:18:14.430740+0000","last_fullsized":"2024-09-21T02:18:14.430740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:27.489206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410423+0000","last_change":"2024-09-21T02:18:13.439622+0000","last_active":"2024-09-21T02:18:14.410423+0000","last_peered":"2024-09-21T02:18:14.410423+0000","last_clean":"2024-09-21T02:18:14.410423+0000","last_became_active":"2024-09-21T02:18:13.439036+0000","last_became_peered":"2024-09-21T02:18:13.439036+0000","last_unstale":"2024-09-21T02:18:14.410423+0000","last_undegraded":"2024-09-21T02:18:14.410423+0000","last_fullsized":"2024-09-21T02:18:14.410423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:32.421158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.449767+0000","last_change":"2024-09-21T02:18:14.448869+0000","last_active":"2024-09-21T02:18:14.449767+0000","last_peered":"2024-09-21T02:18:14.449767+0000","last_clean":"2024-09-21T02:18:14.449767+0000","last_became_active":"2024-09-21T02:18:14.448463+0000","last_became_peered":"2024-09-21T02:18:14.448463+0000","last_unstale":"2024-09-21T02:18:14.449767+0000","last_undegraded":"2024-09-21T02:18:14.449767+0000","last_fullsized":"2024-09-21T02:18:14.449767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:10.352252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.422027+0000","last_change":"2024-09-21T02:18:13.450393+0000","last_active":"2024-09-21T02:18:14.422027+0000","last_peered":"2024-09-21T02:18:14.422027+0000","last_clean":"2024-09-21T02:18:14.422027+0000","last_became_active":"2024-09-21T02:18:13.449920+0000","last_became_peered":"2024-09-21T02:18:13.449920+0000","last_unstale":"2024-09-21T02:18:14.422027+0000","last_undegraded":"2024-09-21T02:18:14.422027+0000","last_fullsized":"2024-09-21T02:18:14.422027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:57.452533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.433493+0000","last_change":"2024-09-21T02:18:14.430114+0000","last_active":"2024-09-21T02:18:14.433493+0000","last_peered":"2024-09-21T02:18:14.433493+0000","last_clean":"2024-09-21T02:18:14.433493+0000","last_became_active":"2024-09-21T02:18:14.429635+0000","last_became_peered":"2024-09-21T02:18:14.429635+0000","last_unstale":"2024-09-21T02:18:14.433493+0000","last_undegraded":"2024-09-21T02:18:14.433493+0000","last_fullsized":"2024-09-21T02:18:14.433493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:22.301527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410492+0000","last_change":"2024-09-21T02:18:13.435115+0000","last_active":"2024-09-21T02:18:14.410492+0000","last_peered":"2024-09-21T02:18:14.410492+0000","last_clean":"2024-09-21T02:18:14.410492+0000","last_became_active":"2024-09-21T02:18:13.434258+0000","last_became_peered":"2024-09-21T02:18:13.434258+0000","last_unstale":"2024-09-21T02:18:14.410492+0000","last_undegraded":"2024-09-21T02:18:14.410492+0000","last_fullsized":"2024-09-21T02:18:14.410492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:59.696379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.474499+0000","last_change":"2024-09-21T02:18:14.463988+0000","last_active":"2024-09-21T02:18:14.474499+0000","last_peered":"2024-09-21T02:18:14.474499+0000","last_clean":"2024-09-21T02:18:14.474499+0000","last_became_active":"2024-09-21T02:18:14.463114+0000","last_became_peered":"2024-09-21T02:18:14.463114+0000","last_unstale":"2024-09-21T02:18:14.474499+0000","last_undegraded":"2024-09-21T02:18:14.474499+0000","last_fullsized":"2024-09-21T02:18:14.474499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:00.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.424918+0000","last_change":"2024-09-21T02:18:13.420393+0000","last_active":"2024-09-21T02:18:14.424918+0000","last_peered":"2024-09-21T02:18:14.424918+0000","last_clean":"2024-09-21T02:18:14.424918+0000","last_became_active":"2024-09-21T02:18:13.419895+0000","last_became_peered":"2024-09-21T02:18:13.419895+0000","last_unstale":"2024-09-21T02:18:14.424918+0000","last_undegraded":"2024-09-21T02:18:14.424918+0000","last_fullsized":"2024-09-21T02:18:14.424918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:43.529177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442600+0000","last_change":"2024-09-21T02:18:14.440519+0000","last_active":"2024-09-21T02:18:14.442600+0000","last_peered":"2024-09-21T02:18:14.442600+0000","last_clean":"2024-09-21T02:18:14.442600+0000","last_became_active":"2024-09-21T02:18:14.440097+0000","last_became_peered":"2024-09-21T02:18:14.440097+0000","last_unstale":"2024-09-21T02:18:14.442600+0000","last_undegraded":"2024-09-21T02:18:14.442600+0000","last_fullsized":"2024-09-21T02:18:14.442600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:24.036222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.421810+0000","last_change":"2024-09-21T02:18:13.434226+0000","last_active":"2024-09-21T02:18:14.421810+0000","last_peered":"2024-09-21T02:18:14.421810+0000","last_clean":"2024-09-21T02:18:14.421810+0000","last_became_active":"2024-09-21T02:18:13.433825+0000","last_became_peered":"2024-09-21T02:18:13.433825+0000","last_unstale":"2024-09-21T02:18:14.421810+0000","last_undegraded":"2024-09-21T02:18:14.421810+0000","last_fullsized":"2024-09-21T02:18:14.421810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:07.044886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.442192+0000","last_change":"2024-09-21T02:18:14.435029+0000","last_active":"2024-09-21T02:18:14.442192+0000","last_peered":"2024-09-21T02:18:14.442192+0000","last_clean":"2024-09-21T02:18:14.442192+0000","last_became_active":"2024-09-21T02:18:14.434500+0000","last_became_peered":"2024-09-21T02:18:14.434500+0000","last_unstale":"2024-09-21T02:18:14.442192+0000","last_undegraded":"2024-09-21T02:18:14.442192+0000","last_fullsized":"2024-09-21T02:18:14.442192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:49.347410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.410179+0000","last_change":"2024-09-21T02:18:13.427041+0000","last_active":"2024-09-21T02:18:14.410179+0000","last_peered":"2024-09-21T02:18:14.410179+0000","last_clean":"2024-09-21T02:18:14.410179+0000","last_became_active":"2024-09-21T02:18:13.424255+0000","last_became_peered":"2024-09-21T02:18:13.424255+0000","last_unstale":"2024-09-21T02:18:14.410179+0000","last_undegraded":"2024-09-21T02:18:14.410179+0000","last_fullsized":"2024-09-21T02:18:14.410179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:33.567805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.478846+0000","last_change":"2024-09-21T02:18:14.474421+0000","last_active":"2024-09-21T02:18:14.478846+0000","last_peered":"2024-09-21T02:18:14.478846+0000","last_clean":"2024-09-21T02:18:14.478846+0000","last_became_active":"2024-09-21T02:18:14.468628+0000","last_became_peered":"2024-09-21T02:18:14.468628+0000","last_unstale":"2024-09-21T02:18:14.478846+0000","last_undegraded":"2024-09-21T02:18:14.478846+0000","last_fullsized":"2024-09-21T02:18:14.478846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:13.389794+0000","last_clean_scrub_stamp":"2024-09-21T02:18:13.389794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:58.350588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.446061+0000","last_change":"2024-09-21T02:18:13.445706+0000","last_active":"2024-09-21T02:18:14.446061+0000","last_peered":"2024-09-21T02:18:14.446061+0000","last_clean":"2024-09-21T02:18:14.446061+0000","last_became_active":"2024-09-21T02:18:13.445428+0000","last_became_peered":"2024-09-21T02:18:13.445428+0000","last_unstale":"2024-09-21T02:18:14.446061+0000","last_undegraded":"2024-09-21T02:18:14.446061+0000","last_fullsized":"2024-09-21T02:18:14.446061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:12.385738+0000","last_clean_scrub_stamp":"2024-09-21T02:18:12.385738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:41.398507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":246,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T02:18:14.423028+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:18:14.423028+0000","last_peered":"2024-09-21T02:18:14.423028+0000","last_clean":"2024-09-21T02:18:14.423028+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:18:14.423028+0000","last_undegraded":"2024-09-21T02:18:14.423028+0000","last_fullsized":"2024-09-21T02:18:14.423028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":999424,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":1015616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542208,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1015808,"data_stored":432057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509504,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335516,"statfs":{"total":96636764160,"available":96599568384,"internally_reserved":0,"allocated":999424,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36340,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335500,"statfs":{"total":96636764160,"available":96599552000,"internally_reserved":0,"allocated":1011712,"data_stored":432023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36904,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334936,"statfs":{"total":96636764160,"available":96598974464,"internally_reserved":0,"allocated":1593344,"data_stored":1015616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36360,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335480,"statfs":{"total":96636764160,"available":96599531520,"internally_reserved":0,"allocated":1028096,"data_stored":433657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:18:18.970 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:18:18.970 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:18:19.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/590618419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d6c004050 msgr2=0x7f7d6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/590618419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d6c004050 0x7f7d6c004810 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009b60 tx=0x7f7d7801d290 comp rx=0 tx=0).stop 2024-09-21T02:18:19.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d83fff640 1 -- 172.21.15.170:0/590618419 reap_dead start 2024-09-21T02:18:19.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/590618419 shutdown_connections 2024-09-21T02:18:19.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/590618419 >> 172.21.15.170:0/590618419 conn(0x7f7d840d0170 msgr2=0x7f7d840d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:19.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/590618419 shutdown_connections 2024-09-21T02:18:19.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/590618419 wait complete. 2024-09-21T02:18:19.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 Processor -- start 2024-09-21T02:18:19.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.168+0000 7f7d8a0ba640 1 -- start start 2024-09-21T02:18:19.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d6c004050 0x7f7d841f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d841f1950 0x7f7d8414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 0x7f7d8415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d840ceb90 con 0x7f7d8415a550 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d840b5bb0 con 0x7f7d841f1950 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7d840ced10 con 0x7f7d6c004050 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 0x7f7d8415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 0x7f7d8415a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45720/0 (socket says 172.21.15.170:45720) 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 -- 172.21.15.170:0/2157769072 learned_addr learned my addr 172.21.15.170:0/2157769072 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d837fe640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d6c004050 0x7f7d841f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d82ffd640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d841f1950 0x7f7d8414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 -- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d6c004050 msgr2=0x7f7d841f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d6c004050 0x7f7d841f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 -- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d841f1950 msgr2=0x7f7d8414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d841f1950 0x7f7d8414a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 -- 172.21.15.170:0/2157769072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d840b1010 con 0x7f7d8415a550 2024-09-21T02:18:19.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d82ffd640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d841f1950 0x7f7d8414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d83fff640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 0x7f7d8415a960 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7d840b61d0 tx=0x7f7d7c00f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00a440 con 0x7f7d8415a550 2024-09-21T02:18:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d7c00a6f0 con 0x7f7d8415a550 2024-09-21T02:18:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c017420 con 0x7f7d8415a550 2024-09-21T02:18:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8415aea0 con 0x7f7d8415a550 2024-09-21T02:18:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841f20f0 con 0x7f7d8415a550 2024-09-21T02:18:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d840745d0 con 0x7f7d8415a550 2024-09-21T02:18:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7d7c014070 con 0x7f7d8415a550 2024-09-21T02:18:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d6bfff640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7d70073350 0x7f7d70075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f7d7c097020 con 0x7f7d8415a550 2024-09-21T02:18:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d837fe640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7d70073350 0x7f7d70075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.172+0000 7f7d837fe640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7d70073350 0x7f7d70075860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d740099b0 tx=0x7f7d740091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:19.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.176+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7d8415aea0 con 0x7f7d8415a550 2024-09-21T02:18:19.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.396+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7d8406a800 con 0x7f7d8415a550 2024-09-21T02:18:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d6bfff640 1 -- 172.21.15.170:0/2157769072 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7f7d7c01d060 con 0x7f7d8415a550 2024-09-21T02:18:19.401 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:18:19.401 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:18:19.401 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:18:19.401 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:18:19.401 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:18:19.402 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7d70073350 msgr2=0x7f7d70075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7d70073350 0x7f7d70075860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d740099b0 tx=0x7f7d740091e0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 msgr2=0x7f7d8415a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 0x7f7d8415a960 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7d840b61d0 tx=0x7f7d7c00f900 comp rx=0 tx=0).stop 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 shutdown_connections 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7d70073350 0x7f7d70075860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d8415a550 0x7f7d8415a960 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d841f1950 0x7f7d8414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 --2- 172.21.15.170:0/2157769072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d6c004050 0x7f7d841f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 >> 172.21.15.170:0/2157769072 conn(0x7f7d840d0170 msgr2=0x7f7d840af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 shutdown_connections 2024-09-21T02:18:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.400+0000 7f7d8a0ba640 1 -- 172.21.15.170:0/2157769072 wait complete. 2024-09-21T02:18:19.423 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac7173640 1 -- 172.21.15.170:0/844053242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa8003a40 msgr2=0x7f4aa8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac7173640 1 --2- 172.21.15.170:0/844053242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa8003a40 0x7f4aa8003eb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000bb10 tx=0x7f4ab001cd90 comp rx=0 tx=0).stop 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac5710640 1 -- 172.21.15.170:0/844053242 reap_dead start 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac7173640 1 -- 172.21.15.170:0/844053242 shutdown_connections 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac7173640 1 -- 172.21.15.170:0/844053242 >> 172.21.15.170:0/844053242 conn(0x7f4ac009d7d0 msgr2=0x7f4ac00d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac7173640 1 -- 172.21.15.170:0/844053242 shutdown_connections 2024-09-21T02:18:19.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.560+0000 7f4ac7173640 1 -- 172.21.15.170:0/844053242 wait complete. 2024-09-21T02:18:19.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 Processor -- start 2024-09-21T02:18:19.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- start start 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa8003a40 0x7f4ac00b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ac0158390 0x7f4ac01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 0x7f4ac00b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac016b520 con 0x7f4ac00b83e0 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac016b3a0 con 0x7f4aa8003a40 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac016b6a0 con 0x7f4ac0158390 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 0x7f4ac00b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 0x7f4ac00b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45744/0 (socket says 172.21.15.170:45744) 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 -- 172.21.15.170:0/2395661969 learned_addr learned my addr 172.21.15.170:0/2395661969 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4abffff640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ac0158390 0x7f4ac01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac4f0f640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa8003a40 0x7f4ac00b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 -- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ac0158390 msgr2=0x7f4ac01560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ac0158390 0x7f4ac01560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 -- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa8003a40 msgr2=0x7f4ac00b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa8003a40 0x7f4ac00b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 -- 172.21.15.170:0/2395661969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac00b7340 con 0x7f4ac00b83e0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4abffff640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ac0158390 0x7f4ac01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac5710640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 0x7f4ac00b87f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8000f90 tx=0x7f4ab800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800b840 con 0x7f4ac00b83e0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ab8018070 con 0x7f4ac00b83e0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac00b7620 con 0x7f4ac00b83e0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800bca0 con 0x7f4ac00b83e0 2024-09-21T02:18:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac00b7be0 con 0x7f4ac00b83e0 2024-09-21T02:18:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.564+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac00cfc60 con 0x7f4ac00b83e0 2024-09-21T02:18:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.568+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f4ab801d460 con 0x7f4ac00b83e0 2024-09-21T02:18:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.568+0000 7f4abd7fa640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4aac073320 0x7f4aac075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.568+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f4ab80941b0 con 0x7f4ac00b83e0 2024-09-21T02:18:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.568+0000 7f4ac4f0f640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4aac073320 0x7f4aac075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.568+0000 7f4ac4f0f640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4aac073320 0x7f4aac075830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00b0000 tx=0x7f4ab00a7040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:19.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.572+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4ab805f040 con 0x7f4ac00b83e0 2024-09-21T02:18:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.764+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ac0156630 con 0x7f4ac00b83e0 2024-09-21T02:18:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4abd7fa640 1 -- 172.21.15.170:0/2395661969 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5237 (secure 0 0 0) 0x7f4ab805e940 con 0x7f4ac00b83e0 2024-09-21T02:18:19.769 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:19.769 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":56,"btime":"2024-09-21T02:18:17:443999+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2173918761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2173918761},{"type":"v1","addr":"172.21.15.191:6835","nonce":2173918761}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:13.828587+0000","modified":"2024-09-21T02:18:16.848304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1472234860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1472234860},{"type":"v1","addr":"172.21.15.170:6835","nonce":1472234860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":6}]} 2024-09-21T02:18:19.769 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 56 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4aac073320 msgr2=0x7f4aac075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4aac073320 0x7f4aac075830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00b0000 tx=0x7f4ab00a7040 comp rx=0 tx=0).stop 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 msgr2=0x7f4ac00b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 0x7f4ac00b87f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8000f90 tx=0x7f4ab800dd50 comp rx=0 tx=0).stop 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 shutdown_connections 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4aac073320 0x7f4aac075830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ac0158390 0x7f4ac01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac00b83e0 0x7f4ac00b87f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 --2- 172.21.15.170:0/2395661969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4aa8003a40 0x7f4ac00b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:19.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 >> 172.21.15.170:0/2395661969 conn(0x7f4ac009d7d0 msgr2=0x7f4ac00b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:19.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 shutdown_connections 2024-09-21T02:18:19.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:19.768+0000 7f4ac7173640 1 -- 172.21.15.170:0/2395661969 wait complete. 2024-09-21T02:18:20.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:18:13.828587+0000', 'modified': '2024-09-21T02:18:16.848304+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6738': {'gid': 6738, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/1472234860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1472234860}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1472234860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6738, 'qdb_cluster': [6738]} 2024-09-21T02:18:20.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:18:20.191 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T02:18:20.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 -- 172.21.15.170:0/1695961366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 msgr2=0x7f4ca006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1695961366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca006fa30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009840 tx=0x7f4c9001c820 comp rx=0 tx=0).stop 2024-09-21T02:18:20.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 -- 172.21.15.170:0/1695961366 shutdown_connections 2024-09-21T02:18:20.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1695961366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca006fa30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 -- 172.21.15.170:0/1695961366 >> 172.21.15.170:0/1695961366 conn(0x7f4ca00b1c40 msgr2=0x7f4ca00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 -- 172.21.15.170:0/1695961366 shutdown_connections 2024-09-21T02:18:20.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.356+0000 7f4ca5834640 1 -- 172.21.15.170:0/1695961366 wait complete. 2024-09-21T02:18:20.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 Processor -- start 2024-09-21T02:18:20.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 -- start start 2024-09-21T02:18:20.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca00b2f40 0x7f4ca015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca00c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ca01fc9e0 0x7f4ca0157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca016b570 con 0x7f4ca00b5410 2024-09-21T02:18:20.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0070180 con 0x7f4ca00b2f40 2024-09-21T02:18:20.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca016b6f0 con 0x7f4ca01fc9e0 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca00c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca00c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45764/0 (socket says 172.21.15.170:45764) 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/1832075719 learned_addr learned my addr 172.21.15.170:0/1832075719 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9effd640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca00b2f40 0x7f4ca015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ca01fc9e0 0x7f4ca0157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ca01fc9e0 msgr2=0x7f4ca0157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ca01fc9e0 0x7f4ca0157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca00b2f40 msgr2=0x7f4ca015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca00b2f40 0x7f4ca015bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/1832075719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca015af00 con 0x7f4ca00b5410 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca00c4330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c790 tx=0x7f4c8800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c97fff640 1 -- 172.21.15.170:0/1832075719 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88003b80 con 0x7f4ca00b5410 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca015b1e0 con 0x7f4ca00b5410 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c97fff640 1 -- 172.21.15.170:0/1832075719 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c88003d20 con 0x7f4ca00b5410 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4c97fff640 1 -- 172.21.15.170:0/1832075719 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800aa90 con 0x7f4ca00b5410 2024-09-21T02:18:20.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.360+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca015b7a0 con 0x7f4ca00b5410 2024-09-21T02:18:20.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4c97fff640 1 -- 172.21.15.170:0/1832075719 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f4c8800ac30 con 0x7f4ca00b5410 2024-09-21T02:18:20.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4c97fff640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4c80073350 0x7f4c80075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4c9effd640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4c80073350 0x7f4c80075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4c97fff640 1 -- 172.21.15.170:0/1832075719 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f4c88094320 con 0x7f4ca00b5410 2024-09-21T02:18:20.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4c9effd640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4c80073350 0x7f4c80075860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c900af000 tx=0x7f4c9001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4cacaff000 1 Processor -- start 2024-09-21T02:18:20.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.364+0000 7f4cacaff000 1 -- start start 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c54e7460 0x5556c5a286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a28be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c58c0ca0 0x5556c5a29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5556c58fd410 con 0x5556c58d7950 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5556c583ab40 con 0x5556c54e7460 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5556c5829f70 con 0x5556c58c0ca0 2024-09-21T02:18:20.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a28be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a28be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45768/0 (socket says 172.21.15.170:45768) 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/2793923277 learned_addr learned my addr 172.21.15.170:0/2793923277 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9effd640 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c54e7460 0x5556c5a286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c58c0ca0 0x5556c5a29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c58c0ca0 msgr2=0x5556c5a29120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c58c0ca0 0x5556c5a29120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c54e7460 msgr2=0x5556c5a286a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c54e7460 0x5556c5a286a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 -- 172.21.15.170:0/2793923277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c57f15b0 con 0x5556c58d7950 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a28be0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800a430 tx=0x7f4c8805a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dee878bb6571f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c957fa640 1 -- 172.21.15.170:0/2793923277 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880841c0 con 0x5556c58d7950 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c957fa640 1 -- 172.21.15.170:0/2793923277 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c88063020 con 0x5556c58d7950 2024-09-21T02:18:20.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4c957fa640 1 -- 172.21.15.170:0/2793923277 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8805ea20 con 0x5556c58d7950 2024-09-21T02:18:20.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- 172.21.15.170:0/2793923277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 msgr2=0x5556c5a28be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a28be0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800a430 tx=0x7f4c8805a9d0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- 172.21.15.170:0/2793923277 shutdown_connections 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c58c0ca0 0x5556c5a29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a28be0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2793923277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c54e7460 0x5556c5a286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- 172.21.15.170:0/2793923277 >> 172.21.15.170:0/2793923277 conn(0x5556c59add90 msgr2=0x5556c59dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- 172.21.15.170:0/2793923277 shutdown_connections 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- 172.21.15.170:0/2793923277 wait complete. 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 -- start start 2024-09-21T02:18:20.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.368+0000 7f4cacaff000 1 Processor -- start 2024-09-21T02:18:20.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a2fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5556c57c45f0 con 0x5556c58d7950 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5556c582dd80 con 0x5556c58c0ca0 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5556c57d0520 con 0x5556c54e7460 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 10 client.0 _ll_register_callbacks cb 0x5556c57d4830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a303a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45770/0 (socket says 172.21.15.170:45770) 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/2898533523 learned_addr learned my addr 172.21.15.170:0/2898533523 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a2fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9effd640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 msgr2=0x5556c5a2f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a2f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 msgr2=0x5556c5a2fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a2fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/2898533523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c57c8190 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a2fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a303a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001d40 tx=0x7f4c9800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba3c163a94d2572 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/2898533523 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980050d0 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/2898533523 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f4c98005270 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556c57c2110 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/2898533523 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f4c98020050 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 10 client.6864 fetch_fsmap learned FSMap version 56 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5556c58aef10 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/2898533523 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5556c57c2110 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/2898533523 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f4c98023020 con 0x5556c58d7950 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 10 client.6864 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 10 client.6864 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:18:20.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 20 client.6864 populate_metadata read hostname 'smithi170' 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54cc7d0 0x5556c5a39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 4 client.6864 mds_command: new command op to 6738 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 --> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556c583ca90 con 0x5556c54cc7d0 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9effd640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54cc7d0 0x5556c5a39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 10 client.6864 ms_handle_connect on v2:172.21.15.170:6834/1472234860 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c9effd640 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54cc7d0 0x5556c5a39d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ec9531a4bed365 server_cookie=f19bb25083458d6 in_seq=0 out_seq=0 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/2898533523 <== mds.0 v2:172.21.15.170:6834/1472234860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556c583ca90 con 0x5556c54cc7d0 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 10 client.6864 handle_command_reply: tid=0 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 client.6864 shutdown 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 msgr2=0x5556c5a303a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a303a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001d40 tx=0x7f4c9800f5a0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 shutdown_connections 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54cc7d0 0x5556c5a39d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a303a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a2fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4c7e7fc640 0 client.6864 ms_handle_reset on v2:172.21.15.170:6834/1472234860 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 --2- 172.21.15.170:0/2898533523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a2f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 >> 172.21.15.170:0/2898533523 conn(0x5556c59add90 msgr2=0x5556c59e4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 shutdown_connections 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 1 -- 172.21.15.170:0/2898533523 wait complete. 2024-09-21T02:18:20.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.372+0000 7f4cacaff000 20 client.6864 trim_cache size 0 max 16384 2024-09-21T02:18:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 Processor -- start 2024-09-21T02:18:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 -- start start 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0205630 con 0x5556c58d7950 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0204700 con 0x5556c58c0ca0 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca01463b0 con 0x5556c54e7460 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45784/0 (socket says 172.21.15.170:45784) 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/441178871 learned_addr learned my addr 172.21.15.170:0/441178871 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/441178871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 msgr2=0x5556c5a9a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/441178871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 msgr2=0x5556c5a9ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.392+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/441178871 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c5885db0 con 0x5556c58d7950 2024-09-21T02:18:20.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9b0a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4c980019b0 tx=0x7f4c980019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd90658aafb64ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c94ff9640 1 -- 172.21.15.170:0/441178871 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98005270 con 0x5556c58d7950 2024-09-21T02:18:20.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c94ff9640 1 -- 172.21.15.170:0/441178871 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c98018070 con 0x5556c58d7950 2024-09-21T02:18:20.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c94ff9640 1 -- 172.21.15.170:0/441178871 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98003840 con 0x5556c58d7950 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- 172.21.15.170:0/441178871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 msgr2=0x5556c5a9b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9b0a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4c980019b0 tx=0x7f4c980019e0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- 172.21.15.170:0/441178871 shutdown_connections 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9b0a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- 172.21.15.170:0/441178871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- 172.21.15.170:0/441178871 >> 172.21.15.170:0/441178871 conn(0x5556c59add90 msgr2=0x5556c5a26620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- 172.21.15.170:0/441178871 shutdown_connections 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- 172.21.15.170:0/441178871 wait complete. 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- start start 2024-09-21T02:18:20.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 Processor -- start 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0204160 con 0x5556c58d7950 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5556c582d090 con 0x5556c58c0ca0 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5556c58323b0 con 0x5556c54e7460 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4cacaff000 10 client.0 _ll_register_callbacks cb 0x7f4c98007e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45788/0 (socket says 172.21.15.170:45788) 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/980080100 learned_addr learned my addr 172.21.15.170:0/980080100 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9e7fc640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.396+0000 7f4c9effd640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 msgr2=0x5556c5a9e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 msgr2=0x5556c5a9eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9f7fe640 1 -- 172.21.15.170:0/980080100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c57f0ab0 con 0x5556c58d7950 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9f7fe640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9f020 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4c98007fa0 tx=0x7f4c9801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e26dab2d9269a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/980080100 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98032060 con 0x5556c58d7950 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/980080100 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f4c98005310 con 0x5556c58d7950 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c98018070 con 0x5556c58d7950 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/980080100 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f4c98038050 con 0x5556c58d7950 2024-09-21T02:18:20.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 10 client.6870 fetch_fsmap learned FSMap version 56 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5556c57cf070 con 0x5556c58d7950 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/980080100 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c98018070 con 0x5556c58d7950 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/980080100 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f4c9803b020 con 0x5556c58d7950 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 10 client.6870 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 10 client.6870 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 20 client.6870 populate_metadata read hostname 'smithi170' 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54ba830 0x5556c5a5a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 4 client.6870 mds_command: new command op to 6738 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 --> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] -- command(tid 0: {"prefix": "status"}) -- 0x5556c5838540 con 0x5556c54ba830 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9effd640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54ba830 0x5556c5a5a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 10 client.6870 ms_handle_connect on v2:172.21.15.170:6834/1472234860 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c9effd640 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54ba830 0x5556c5a5a610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b47f51135352932 server_cookie=79c44c4fa5ccc446 in_seq=0 out_seq=0 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 1 -- 172.21.15.170:0/980080100 <== mds.0 v2:172.21.15.170:6834/1472234860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5556c5838540 con 0x5556c54ba830 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 10 client.6870 handle_command_reply: tid=0 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 client.6870 shutdown 2024-09-21T02:18:20.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 msgr2=0x5556c5a9f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9f020 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4c98007fa0 tx=0x7f4c9801ebb0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 shutdown_connections 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:6834/1472234860,v1:172.21.15.170:6835/1472234860] conn(0x5556c54ba830 0x5556c5a5a610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556c58d7950 0x5556c5a9f020 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5556c58c0ca0 0x5556c5a9eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 --2- 172.21.15.170:0/980080100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5556c54e7460 0x5556c5a9e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4c7e7fc640 0 client.6870 ms_handle_reset on v2:172.21.15.170:6834/1472234860 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 >> 172.21.15.170:0/980080100 conn(0x5556c59add90 msgr2=0x5556c5a396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 shutdown_connections 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 1 -- 172.21.15.170:0/980080100 wait complete. 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4cacaff000 20 client.6870 trim_cache size 0 max 16384 2024-09-21T02:18:20.406 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6738, 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.9984571750000004, 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 56, 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 84, 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 84, 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.556778924 2024-09-21T02:18:20.407 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4c80073350 msgr2=0x7f4c80075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4c80073350 0x7f4c80075860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c900af000 tx=0x7f4c9001b060 comp rx=0 tx=0).stop 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 msgr2=0x7f4ca00c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca00c4330 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c790 tx=0x7f4c8800cc50 comp rx=0 tx=0).stop 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 shutdown_connections 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4c80073350 0x7f4c80075860 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ca01fc9e0 0x7f4ca0157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca00b5410 0x7f4ca00c4330 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 --2- 172.21.15.170:0/1832075719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca00b2f40 0x7f4ca015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 >> 172.21.15.170:0/1832075719 conn(0x7f4ca00b1c40 msgr2=0x7f4ca00c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 shutdown_connections 2024-09-21T02:18:20.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.400+0000 7f4ca5834640 1 -- 172.21.15.170:0/1832075719 wait complete. 2024-09-21T02:18:20.423 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:18:20.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 -- 172.21.15.170:0/1394558203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 msgr2=0x7f7dec0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 --2- 172.21.15.170:0/1394558203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec0b3350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0b3a10 tx=0x7f7ddc01c820 comp rx=0 tx=0).stop 2024-09-21T02:18:20.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 -- 172.21.15.170:0/1394558203 shutdown_connections 2024-09-21T02:18:20.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 --2- 172.21.15.170:0/1394558203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec0b3350 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 --2- 172.21.15.170:0/1394558203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dec0b5410 0x7f7dec06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 -- 172.21.15.170:0/1394558203 >> 172.21.15.170:0/1394558203 conn(0x7f7dec0b1c40 msgr2=0x7f7dec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 -- 172.21.15.170:0/1394558203 shutdown_connections 2024-09-21T02:18:20.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 -- 172.21.15.170:0/1394558203 wait complete. 2024-09-21T02:18:20.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 Processor -- start 2024-09-21T02:18:20.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.564+0000 7f7df1e62640 1 -- start start 2024-09-21T02:18:20.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec074870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dec0b5410 0x7f7dec14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dec0ccb00 0x7f7dec15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec169fd0 con 0x7f7dec0b2f40 2024-09-21T02:18:20.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec169e50 con 0x7f7dec0ccb00 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7dec16a150 con 0x7f7dec0b5410 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec074870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec074870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45806/0 (socket says 172.21.15.170:45806) 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 -- 172.21.15.170:0/4264651298 learned_addr learned my addr 172.21.15.170:0/4264651298 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deaffd640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dec0b5410 0x7f7dec14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7debfff640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dec0ccb00 0x7f7dec15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 -- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dec0b5410 msgr2=0x7f7dec14d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dec0b5410 0x7f7dec14d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 -- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dec0ccb00 msgr2=0x7f7dec15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dec0ccb00 0x7f7dec15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 -- 172.21.15.170:0/4264651298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec15bcb0 con 0x7f7dec0b2f40 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7deb7fe640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec074870 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40058d0 tx=0x7f7dd400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4017070 con 0x7f7dec0b2f40 2024-09-21T02:18:20.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec15cc00 con 0x7f7dec0b2f40 2024-09-21T02:18:20.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dd4009d20 con 0x7f7dec0b2f40 2024-09-21T02:18:20.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40106c0 con 0x7f7dec0b2f40 2024-09-21T02:18:20.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec0c8010 con 0x7f7dec0b2f40 2024-09-21T02:18:20.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec0b3350 con 0x7f7dec0b2f40 2024-09-21T02:18:20.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.568+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7dd401a070 con 0x7f7dec0b2f40 2024-09-21T02:18:20.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.572+0000 7f7de3fff640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dcc073320 0x7f7dcc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:20.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.572+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f7dd40945b0 con 0x7f7dec0b2f40 2024-09-21T02:18:20.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.572+0000 7f7deaffd640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dcc073320 0x7f7dcc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:20.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.572+0000 7f7deaffd640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dcc073320 0x7f7dcc075830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0af000 tx=0x7f7ddc01b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:20.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.576+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7dd405ed70 con 0x7f7dec0b2f40 2024-09-21T02:18:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.772+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7dec0ce120 con 0x7f7dec0b2f40 2024-09-21T02:18:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.772+0000 7f7de3fff640 1 -- 172.21.15.170:0/4264651298 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f7dd40637a0 con 0x7f7dec0b2f40 2024-09-21T02:18:20.775 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dcc073320 msgr2=0x7f7dcc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dcc073320 0x7f7dcc075830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0af000 tx=0x7f7ddc01b060 comp rx=0 tx=0).stop 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 msgr2=0x7f7dec074870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec074870 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40058d0 tx=0x7f7dd400dde0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 shutdown_connections 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dcc073320 0x7f7dcc075830 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dec0b5410 0x7f7dec14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dec0ccb00 0x7f7dec15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 --2- 172.21.15.170:0/4264651298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dec0b2f40 0x7f7dec074870 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 >> 172.21.15.170:0/4264651298 conn(0x7f7dec0b1c40 msgr2=0x7f7dec0c5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:20.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 shutdown_connections 2024-09-21T02:18:20.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20.776+0000 7f7df1e62640 1 -- 172.21.15.170:0/4264651298 wait complete. 2024-09-21T02:18:20.796 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:18:20.796 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:18:20.797 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:18:20.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:20.797 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:18:20.803 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42215sec preferred_lft 42215sec 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:18:20.804 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:18:20.805 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:20.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:20.806 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:18:20.853 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:18:20.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:20.854 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:18:20.905 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:18:20.905 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:18:20.905 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:18:20.905 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:18:20.905 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:18:20.906 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:18:20.906 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:18:20.906 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:20.906 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:20.906 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:20.906 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:18:20.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:18:20.907 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:18:20.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:18:20.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:18:20.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:18:20.907 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:18:20.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:18:21.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:18:21.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:21.025 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:18:21.073 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:18:21.073 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:18:21.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:18:21.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:21.074 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:18:21.130 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:18:21.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:18:21.130 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:21.177 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:18:21.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:18:21.177 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:21.225 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:18:21.282 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:18:21.330 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:18:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:21.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:21.333 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:18:21.381 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:18:21.381 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:18:21.426 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:18:21.439 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:21.489 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:18:21.489 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:18:21.489 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:18:21.489 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:18:21.489 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:18:21.489 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:21.490 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:21.491 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:21.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:21.492 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:18:21.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:18:21.532+0000 7f1dfeadf580 -1 init, newargv = 0x7f1dec004730 newargc=15 2024-09-21T02:18:21.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[43481]: starting ceph client 2024-09-21T02:18:21.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[43481]: starting fuse 2024-09-21T02:18:22.640 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:18:22.657 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:18:22.712 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:18:22.713 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:18:22.714 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:18:22.714 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:22.714 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:22.714 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:22.714 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:22.714 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:22.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:22.715 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:18:22.761 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:18:22.761 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:18:22.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:18:22.762 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:18:22.763 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:18:22.886 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.43481.asok 2024-09-21T02:18:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:18:22.898 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43481.asok 2024-09-21T02:18:22.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:22.899 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43481.asok status 2024-09-21T02:18:23.080 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "43481", 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6885, 2024-09-21T02:18:23.081 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 6885 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2766381462 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:18:23.082 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 2766381462 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.6885 192.168.0.1:0/2766381462", 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/2766381462", 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 56, 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 84, 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:18:23.083 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:18:23.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.097 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:18:23.097 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:23.103 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:18:23.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:23.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.104 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:23.157 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:18:23.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc049e16640 1 -- 172.21.15.170:0/730692531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c004810 msgr2=0x7fc02c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc049e16640 1 --2- 172.21.15.170:0/730692531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc02c004810 0x7fc02c004c20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c260 tx=0x7fc03801f6c0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc043fff640 1 -- 172.21.15.170:0/730692531 reap_dead start 2024-09-21T02:18:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc049e16640 1 -- 172.21.15.170:0/730692531 shutdown_connections 2024-09-21T02:18:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc049e16640 1 -- 172.21.15.170:0/730692531 >> 172.21.15.170:0/730692531 conn(0x7fc0440cee30 msgr2=0x7fc0440cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc049e16640 1 -- 172.21.15.170:0/730692531 shutdown_connections 2024-09-21T02:18:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.356+0000 7fc049e16640 1 -- 172.21.15.170:0/730692531 wait complete. 2024-09-21T02:18:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 Processor -- start 2024-09-21T02:18:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- start start 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc02c004810 0x7fc0440b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc04415f410 0x7fc0440c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 0x7fc0441f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440d1620 con 0x7fc0440c6390 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440d14a0 con 0x7fc04415f410 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc0440d17a0 con 0x7fc02c004810 2024-09-21T02:18:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 0x7fc0441f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 0x7fc0441f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45826/0 (socket says 172.21.15.170:45826) 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 -- 172.21.15.170:0/2147700031 learned_addr learned my addr 172.21.15.170:0/2147700031 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc0437fe640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc02c004810 0x7fc0440b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc042ffd640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc04415f410 0x7fc0440c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 -- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc02c004810 msgr2=0x7fc0440b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc02c004810 0x7fc0440b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 -- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc04415f410 msgr2=0x7fc0440c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc04415f410 0x7fc0440c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 -- 172.21.15.170:0/2147700031 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04406f030 con 0x7fc0440c6390 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc043fff640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 0x7fc0441f8820 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cfc40 tx=0x7fc03c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c004d30 con 0x7fc0440c6390 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc03c00ca10 con 0x7fc0440c6390 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04415e930 con 0x7fc0440c6390 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00ce40 con 0x7fc0440c6390 2024-09-21T02:18:23.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04415eef0 con 0x7fc0440c6390 2024-09-21T02:18:23.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.360+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04406eda0 con 0x7fc0440c6390 2024-09-21T02:18:23.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.364+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fc03c010430 con 0x7fc0440c6390 2024-09-21T02:18:23.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.364+0000 7fc02bfff640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc030073320 0x7fc030075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:23.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.364+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fc03c0945a0 con 0x7fc0440c6390 2024-09-21T02:18:23.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.364+0000 7fc0437fe640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc030073320 0x7fc030075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:23.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.364+0000 7fc0437fe640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc030073320 0x7fc030075830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc034000c10 tx=0x7fc034006b60 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:23.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.368+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc03c05f4c0 con 0x7fc0440c6390 2024-09-21T02:18:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.572+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc0440d1d00 con 0x7fc0440c6390 2024-09-21T02:18:23.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc02bfff640 1 -- 172.21.15.170:0/2147700031 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7fc03c05edc0 con 0x7fc0440c6390 2024-09-21T02:18:23.576 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:18:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc030073320 msgr2=0x7fc030075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc030073320 0x7fc030075830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc034000c10 tx=0x7fc034006b60 comp rx=0 tx=0).stop 2024-09-21T02:18:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 msgr2=0x7fc0441f8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 0x7fc0441f8820 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cfc40 tx=0x7fc03c00ed80 comp rx=0 tx=0).stop 2024-09-21T02:18:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 shutdown_connections 2024-09-21T02:18:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc030073320 0x7fc030075830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0440c6390 0x7fc0441f8820 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc04415f410 0x7fc0440c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 --2- 172.21.15.170:0/2147700031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc02c004810 0x7fc0440b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:23.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 >> 172.21.15.170:0/2147700031 conn(0x7fc0440cee30 msgr2=0x7fc0440b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:23.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 shutdown_connections 2024-09-21T02:18:23.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:23.576+0000 7fc049e16640 1 -- 172.21.15.170:0/2147700031 wait complete. 2024-09-21T02:18:23.596 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:18:23.597 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:18:23.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.597 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:18:23.604 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:18:23.604 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:23.605 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.606 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:23.607 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:18:23.608 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:23.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.608 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:18:23.661 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:18:23.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.663 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:18:23.717 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:18:23.717 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:18:23.717 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:23.718 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:18:23.718 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:18:23.718 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:18:23.718 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:18:23.718 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:18:23.719 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:18:23.719 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:18:23.805 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:18:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:18:23.836 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:18:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:18:23.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.843 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:18:23.848 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:18:23.848 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:18:23.848 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:18:23.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:23.849 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:18:23.901 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:18:23.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:18:23.901 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:23.947 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:18:23.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:18:23.947 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:23.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:18:24.053 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:18:24.105 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:18:24.106 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:18:24.107 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:18:24.107 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:18:24.107 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:24.107 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:24.107 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:24.107 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:24.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:24.108 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:18:24.155 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:18:24.155 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:18:24.201 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:18:24.216 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:24.263 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:18:24.263 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:18:24.263 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:18:24.264 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:18:24.265 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:24.266 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:24.266 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:24.266 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:24.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:24.267 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:18:24.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:18:24.305+0000 7f7d5bac5580 -1 init, newargv = 0x7f7d50001600 newargc=15 2024-09-21T02:18:24.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34995]: starting ceph client 2024-09-21T02:18:24.318 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34995]: starting fuse 2024-09-21T02:18:24.335 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:18:24.337 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:18:25.414 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:18:25.430 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:18:25.477 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:18:25.478 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:25.479 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:18:25.480 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:25.480 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:18:25.480 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:25.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:25.481 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:18:25.527 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:18:25.528 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:18:25.528 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:18:25.529 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:18:25.653 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.34995.asok 2024-09-21T02:18:25.656 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:18:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:18:25.665 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34995.asok 2024-09-21T02:18:25.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:25.665 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34995.asok status 2024-09-21T02:18:25.847 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:18:25.847 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:18:25.847 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:18:25.847 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "34995", 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5857, 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5857 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:18:25.848 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 257143436 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 257143436 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5857 192.168.0.1:0/257143436", 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/257143436", 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 56, 2024-09-21T02:18:25.849 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 84, 2024-09-21T02:18:25.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:18:25.850 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:18:25.850 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:18:25.850 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:18:25.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:25.863 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:18:25.863 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:25.869 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:18:25.869 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:25.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:25.869 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:25.923 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 -- 172.21.15.170:0/1848916398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccb50 msgr2=0x7f77f00ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 --2- 172.21.15.170:0/1848916398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccb50 0x7f77f00ca720 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f77f0071d80 tx=0x7f77e401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 -- 172.21.15.170:0/1848916398 shutdown_connections 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 --2- 172.21.15.170:0/1848916398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77f00ccf60 0x7f77f00cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 --2- 172.21.15.170:0/1848916398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccb50 0x7f77f00ca720 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 --2- 172.21.15.170:0/1848916398 >> v2:172.21.15.191:3300/0 conn(0x7f77f009dc00 0x7f77f015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 -- 172.21.15.170:0/1848916398 >> 172.21.15.170:0/1848916398 conn(0x7f77f00b1c40 msgr2=0x7f77f00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 -- 172.21.15.170:0/1848916398 shutdown_connections 2024-09-21T02:18:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 -- 172.21.15.170:0/1848916398 wait complete. 2024-09-21T02:18:26.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 Processor -- start 2024-09-21T02:18:26.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.096+0000 7f77f6876640 1 -- start start 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77f009dc00 0x7f77f0152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77f00ccb50 0x7f77f015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 0x7f77f00adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f77f006f660 con 0x7f77f00ccf60 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77f006f4e0 con 0x7f77f009dc00 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f77f006f7e0 con 0x7f77f00ccb50 2024-09-21T02:18:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 0x7f77f00adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 0x7f77f00adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54544/0 (socket says 172.21.15.170:54544) 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 -- 172.21.15.170:0/619313801 learned_addr learned my addr 172.21.15.170:0/619313801 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77effff640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77f009dc00 0x7f77f0152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77ef7fe640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77f00ccb50 0x7f77f015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 -- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77f00ccb50 msgr2=0x7f77f015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77f00ccb50 0x7f77f015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 -- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77f009dc00 msgr2=0x7f77f0152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77f009dc00 0x7f77f0152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 -- 172.21.15.170:0/619313801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f0152ee0 con 0x7f77f00ccf60 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f4e13640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 0x7f77f00adf70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f77e800cb10 tx=0x7f77e8007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8007dd0 con 0x7f77f00ccf60 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77e8003ab0 con 0x7f77f00ccf60 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f0158d90 con 0x7f77f00ccf60 2024-09-21T02:18:26.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8004030 con 0x7f77f00ccf60 2024-09-21T02:18:26.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f015d130 con 0x7f77f00ccf60 2024-09-21T02:18:26.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f015d3c0 con 0x7f77f00ccf60 2024-09-21T02:18:26.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.100+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f77e80041d0 con 0x7f77f00ccf60 2024-09-21T02:18:26.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.104+0000 7f77ecff9640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77dc073320 0x7f77dc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:26.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.104+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f77e8094ea0 con 0x7f77f00ccf60 2024-09-21T02:18:26.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.104+0000 7f77effff640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77dc073320 0x7f77dc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:26.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.104+0000 7f77effff640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77dc073320 0x7f77dc075830 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77e0005d70 tx=0x7f77e0005d00 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:26.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.108+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f77e8014070 con 0x7f77f00ccf60 2024-09-21T02:18:26.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.312+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f77f006a7d0 con 0x7f77f00ccf60 2024-09-21T02:18:26.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.928+0000 7f77ecff9640 1 -- 172.21.15.170:0/619313801 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7f77e805fca0 con 0x7f77f00ccf60 2024-09-21T02:18:26.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.928+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77dc073320 msgr2=0x7f77dc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.928+0000 7f77f6876640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77dc073320 0x7f77dc075830 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77e0005d70 tx=0x7f77e0005d00 comp rx=0 tx=0).stop 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.928+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 msgr2=0x7f77f00adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.928+0000 7f77f6876640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 0x7f77f00adf70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f77e800cb10 tx=0x7f77e8007220 comp rx=0 tx=0).stop 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 shutdown_connections 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77dc073320 0x7f77dc075830 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77f00ccb50 0x7f77f015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00ccf60 0x7f77f00adf70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 --2- 172.21.15.170:0/619313801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77f009dc00 0x7f77f0152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 >> 172.21.15.170:0/619313801 conn(0x7f77f00b1c40 msgr2=0x7f77f00b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 shutdown_connections 2024-09-21T02:18:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:26.932+0000 7f77f6876640 1 -- 172.21.15.170:0/619313801 wait complete. 2024-09-21T02:18:26.957 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:18:27.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc55b911640 1 -- 172.21.15.170:0/2661748178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc540003a10 msgr2=0x7fc540003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc55b911640 1 --2- 172.21.15.170:0/2661748178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc540003a10 0x7fc540003e80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc55000bb10 tx=0x7fc55001cdb0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc559eae640 1 -- 172.21.15.170:0/2661748178 reap_dead start 2024-09-21T02:18:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc55b911640 1 -- 172.21.15.170:0/2661748178 shutdown_connections 2024-09-21T02:18:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc55b911640 1 -- 172.21.15.170:0/2661748178 >> 172.21.15.170:0/2661748178 conn(0x7fc55409d7d0 msgr2=0x7fc5540d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc55b911640 1 -- 172.21.15.170:0/2661748178 shutdown_connections 2024-09-21T02:18:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.112+0000 7fc55b911640 1 -- 172.21.15.170:0/2661748178 wait complete. 2024-09-21T02:18:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 Processor -- start 2024-09-21T02:18:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- start start 2024-09-21T02:18:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc540003a10 0x7fc5540cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 0x7fc5540cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5540cd7e0 0x7fc554151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc55416a180 con 0x7fc55415d720 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc55416a000 con 0x7fc540003a10 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc55416a300 con 0x7fc5540cd7e0 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 0x7fc5540cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 0x7fc5540cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54560/0 (socket says 172.21.15.170:54560) 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 -- 172.21.15.170:0/704533798 learned_addr learned my addr 172.21.15.170:0/704533798 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc5596ad640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc540003a10 0x7fc5540cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc559eae640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5540cd7e0 0x7fc554151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 -- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5540cd7e0 msgr2=0x7fc554151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5540cd7e0 0x7fc554151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 -- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc540003a10 msgr2=0x7fc5540cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc540003a10 0x7fc5540cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 -- 172.21.15.170:0/704533798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5541f9c00 con 0x7fc55415d720 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc558eac640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 0x7fc5540cd2a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d750 tx=0x7fc54400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544017070 con 0x7fc55415d720 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc544009ce0 con 0x7fc55415d720 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5541f9ee0 con 0x7fc55415d720 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54400de60 con 0x7fc55415d720 2024-09-21T02:18:27.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5541fa4a0 con 0x7fc55415d720 2024-09-21T02:18:27.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5540d2440 con 0x7fc55415d720 2024-09-21T02:18:27.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.116+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fc544005280 con 0x7fc55415d720 2024-09-21T02:18:27.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.120+0000 7fc549ffb640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc538073320 0x7fc538075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.120+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fc5440945c0 con 0x7fc55415d720 2024-09-21T02:18:27.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.120+0000 7fc5596ad640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc538073320 0x7fc538075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.120+0000 7fc5596ad640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc538073320 0x7fc538075830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc5500b0000 tx=0x7fc550019b50 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:27.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.124+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc54405f3c0 con 0x7fc55415d720 2024-09-21T02:18:27.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.300+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc5540ce340 con 0x7fc538073320 2024-09-21T02:18:27.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.304+0000 7fc549ffb640 1 -- 172.21.15.170:0/704533798 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc5540ce340 con 0x7fc538073320 2024-09-21T02:18:27.306 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:18:27.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:27.306 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:18:27.306 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:27.306 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc538073320 msgr2=0x7fc538075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc538073320 0x7fc538075830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc5500b0000 tx=0x7fc550019b50 comp rx=0 tx=0).stop 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 msgr2=0x7fc5540cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 0x7fc5540cd2a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d750 tx=0x7fc54400dc10 comp rx=0 tx=0).stop 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 shutdown_connections 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc538073320 0x7fc538075830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5540cd7e0 0x7fc554151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55415d720 0x7fc5540cd2a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 --2- 172.21.15.170:0/704533798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc540003a10 0x7fc5540cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 >> 172.21.15.170:0/704533798 conn(0x7fc55409d7d0 msgr2=0x7fc5540b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 shutdown_connections 2024-09-21T02:18:27.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.308+0000 7fc55b911640 1 -- 172.21.15.170:0/704533798 wait complete. 2024-09-21T02:18:27.326 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1767055543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e34005920 msgr2=0x7f6e34005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1767055543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e34005920 0x7f6e34005d30 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6e48070d50 tx=0x7f6e3001cf20 comp rx=0 tx=0).stop 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1767055543 shutdown_connections 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1767055543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e34005920 0x7f6e34005d30 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1767055543 >> 172.21.15.170:0/1767055543 conn(0x7f6e480d0170 msgr2=0x7f6e480d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1767055543 shutdown_connections 2024-09-21T02:18:27.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.476+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1767055543 wait complete. 2024-09-21T02:18:27.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 Processor -- start 2024-09-21T02:18:27.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- start start 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e34005920 0x7f6e4815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 0x7f6e480adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e481eb9d0 0x7f6e480ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480ce970 con 0x7f6e481eb5c0 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480ce7f0 con 0x7f6e481eb9d0 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4815f200 con 0x7f6e34005920 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 0x7f6e480adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 0x7f6e480adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54590/0 (socket says 172.21.15.170:54590) 2024-09-21T02:18:27.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 -- 172.21.15.170:0/1720922912 learned_addr learned my addr 172.21.15.170:0/1720922912 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e477fe640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e34005920 0x7f6e4815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e47fff640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e481eb9d0 0x7f6e480ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 -- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e34005920 msgr2=0x7f6e4815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e34005920 0x7f6e4815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 -- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e481eb9d0 msgr2=0x7f6e480ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e481eb9d0 0x7f6e480ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 -- 172.21.15.170:0/1720922912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e481f8f30 con 0x7f6e481eb5c0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e3effd640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 0x7f6e480adcd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6e30000c10 tx=0x7f6e30006cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e300a5ae0 con 0x7f6e481eb5c0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e300a5ca0 con 0x7f6e481eb5c0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e481ec1a0 con 0x7f6e481eb5c0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e300063a0 con 0x7f6e481eb5c0 2024-09-21T02:18:27.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e481ec760 con 0x7f6e481eb5c0 2024-09-21T02:18:27.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.480+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e481ebf10 con 0x7f6e481eb5c0 2024-09-21T02:18:27.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.484+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f6e300b6030 con 0x7f6e481eb5c0 2024-09-21T02:18:27.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.484+0000 7f6e44ff9640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e28073320 0x7f6e28075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.484+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f6e3012d950 con 0x7f6e481eb5c0 2024-09-21T02:18:27.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.484+0000 7f6e477fe640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e28073320 0x7f6e28075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.484+0000 7f6e477fe640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e28073320 0x7f6e28075830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009bb0 tx=0x7f6e380091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:27.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.488+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6e481f8f30 con 0x7f6e481eb5c0 2024-09-21T02:18:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.656+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6e4815f760 con 0x7f6e481eb5c0 2024-09-21T02:18:27.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e44ff9640 1 -- 172.21.15.170:0/1720922912 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f6e30016cd0 con 0x7f6e481eb5c0 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e28073320 msgr2=0x7f6e28075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e28073320 0x7f6e28075830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009bb0 tx=0x7f6e380091e0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 msgr2=0x7f6e480adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 0x7f6e480adcd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6e30000c10 tx=0x7f6e30006cd0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 shutdown_connections 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e28073320 0x7f6e28075830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e481eb9d0 0x7f6e480ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e481eb5c0 0x7f6e480adcd0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 --2- 172.21.15.170:0/1720922912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e34005920 0x7f6e4815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 >> 172.21.15.170:0/1720922912 conn(0x7f6e480d0170 msgr2=0x7f6e4809c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 shutdown_connections 2024-09-21T02:18:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.660+0000 7f6e4e00f640 1 -- 172.21.15.170:0/1720922912 wait complete. 2024-09-21T02:18:27.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf2b16640 1 -- 172.21.15.170:0/3993450546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bd4004810 msgr2=0x7f6bd4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf2b16640 1 --2- 172.21.15.170:0/3993450546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bd4004810 0x7f6bd4004c20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc001ec0 tx=0x7f6bdc01f680 comp rx=0 tx=0).stop 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf10b3640 1 -- 172.21.15.170:0/3993450546 reap_dead start 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf2b16640 1 -- 172.21.15.170:0/3993450546 shutdown_connections 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf2b16640 1 -- 172.21.15.170:0/3993450546 >> 172.21.15.170:0/3993450546 conn(0x7f6bec08bf50 msgr2=0x7f6bec09d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf2b16640 1 -- 172.21.15.170:0/3993450546 shutdown_connections 2024-09-21T02:18:27.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.836+0000 7f6bf2b16640 1 -- 172.21.15.170:0/3993450546 wait complete. 2024-09-21T02:18:27.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 Processor -- start 2024-09-21T02:18:27.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- start start 2024-09-21T02:18:27.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bd4004810 0x7f6bec1e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec1f9770 0x7f6bec0ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bec0cab10 0x7f6bec0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec162630 con 0x7f6bec0cab10 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1624b0 con 0x7f6bec1f9770 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6bec1627b0 con 0x7f6bd4004810 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bec0cab10 0x7f6bec0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bec0cab10 0x7f6bec0b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54610/0 (socket says 172.21.15.170:54610) 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 -- 172.21.15.170:0/362071799 learned_addr learned my addr 172.21.15.170:0/362071799 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf08b2640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bd4004810 0x7f6bec1e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bebfff640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec1f9770 0x7f6bec0ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 -- 172.21.15.170:0/362071799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bd4004810 msgr2=0x7f6bec1e96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bd4004810 0x7f6bec1e96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 -- 172.21.15.170:0/362071799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec1f9770 msgr2=0x7f6bec0ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bec1f9770 0x7f6bec0ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 -- 172.21.15.170:0/362071799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec1f5ef0 con 0x7f6bec0cab10 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf10b3640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bec0cab10 0x7f6bec0b6250 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0ceb50 tx=0x7f6be4009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6be97fa640 1 -- 172.21.15.170:0/362071799 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4011910 con 0x7f6bec0cab10 2024-09-21T02:18:27.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6be97fa640 1 -- 172.21.15.170:0/362071799 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6be4019070 con 0x7f6bec0cab10 2024-09-21T02:18:27.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- 172.21.15.170:0/362071799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec1f61d0 con 0x7f6bec0cab10 2024-09-21T02:18:27.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6be97fa640 1 -- 172.21.15.170:0/362071799 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4011e60 con 0x7f6bec0cab10 2024-09-21T02:18:27.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- 172.21.15.170:0/362071799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1f6790 con 0x7f6bec0cab10 2024-09-21T02:18:27.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.840+0000 7f6bf2b16640 1 -- 172.21.15.170:0/362071799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bec0b1f80 con 0x7f6bec0cab10 2024-09-21T02:18:27.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.844+0000 7f6be97fa640 1 -- 172.21.15.170:0/362071799 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f6be401e460 con 0x7f6bec0cab10 2024-09-21T02:18:27.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.844+0000 7f6be97fa640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6bd8073320 0x7f6bd8075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:27.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.844+0000 7f6be97fa640 1 -- 172.21.15.170:0/362071799 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f6be4094ac0 con 0x7f6bec0cab10 2024-09-21T02:18:27.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.844+0000 7f6bf08b2640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6bd8073320 0x7f6bd8075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:27.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.844+0000 7f6bf08b2640 1 --2- 172.21.15.170:0/362071799 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6bd8073320 0x7f6bd8075830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6be0005d10 tx=0x7f6be0005c80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:27.848+0000 7f6be97fa640 1 -- 172.21.15.170:0/362071799 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6be405f8c0 con 0x7f6bec0cab10 2024-09-21T02:18:28.039 INFO:teuthology.orchestra.run.smithi170.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-21T02:18:28.039 INFO:teuthology.orchestra.run.smithi170.stderr:fs volume create [] : Create a CephFS volume 2024-09-21T02:18:28.039 INFO:teuthology.orchestra.run.smithi170.stderr:Error EINVAL: invalid command 2024-09-21T02:18:28.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T02:18:28.047 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 -- 172.21.15.170:0/4077882320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b5da0 msgr2=0x7fecb0165f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/4077882320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b5da0 0x7fecb0165f50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7feca8009c80 tx=0x7feca801cda0 comp rx=0 tx=0).stop 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 -- 172.21.15.170:0/4077882320 shutdown_connections 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/4077882320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b5da0 0x7fecb0165f50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/4077882320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb00cf640 0x7fecb00cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 -- 172.21.15.170:0/4077882320 >> 172.21.15.170:0/4077882320 conn(0x7fecb00d1280 msgr2=0x7fecb00d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 -- 172.21.15.170:0/4077882320 shutdown_connections 2024-09-21T02:18:28.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 -- 172.21.15.170:0/4077882320 wait complete. 2024-09-21T02:18:28.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 Processor -- start 2024-09-21T02:18:28.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.200+0000 7fecb4bf7640 1 -- start start 2024-09-21T02:18:28.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 0x7fecb0160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:28.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb00b5da0 0x7fecb0164160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:28.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb00cf640 0x7fecb015e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0169040 con 0x7fecb00b09f0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0168ec0 con 0x7fecb00cf640 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fecb01691c0 con 0x7fecb00b5da0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 0x7fecb0160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 0x7fecb0160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54624/0 (socket says 172.21.15.170:54624) 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 -- 172.21.15.170:0/21933601 learned_addr learned my addr 172.21.15.170:0/21933601 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecaed9d640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb00cf640 0x7fecb015e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecadd9b640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb00b5da0 0x7fecb0164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 -- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb00b5da0 msgr2=0x7fecb0164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb00b5da0 0x7fecb0164160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 -- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb00cf640 msgr2=0x7fecb015e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb00cf640 0x7fecb015e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 -- 172.21.15.170:0/21933601 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01603a0 con 0x7fecb00b09f0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecaed9d640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb00cf640 0x7fecb015e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecae59c640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 0x7fecb0160d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7feca000d770 tx=0x7feca000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:28.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0009c30 con 0x7fecb00b09f0 2024-09-21T02:18:28.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb0160610 con 0x7fecb00b09f0 2024-09-21T02:18:28.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feca000de80 con 0x7fecb00b09f0 2024-09-21T02:18:28.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0004280 con 0x7fecb00b09f0 2024-09-21T02:18:28.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb00c9240 con 0x7fecb00b09f0 2024-09-21T02:18:28.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb00cfa50 con 0x7fecb00b09f0 2024-09-21T02:18:28.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.204+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7feca0003600 con 0x7fecb00b09f0 2024-09-21T02:18:28.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.208+0000 7fec9effd640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec90073350 0x7fec90075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:28.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.208+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7feca00949c0 con 0x7fecb00b09f0 2024-09-21T02:18:28.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.208+0000 7fecadd9b640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec90073350 0x7fec90075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:28.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.208+0000 7fecadd9b640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec90073350 0x7fec90075860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fecb00caf30 tx=0x7feca4009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:28.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.212+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feca0096e20 con 0x7fecb00b09f0 2024-09-21T02:18:28.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:28.392+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7fecb00cf210 con 0x7fec90073350 2024-09-21T02:18:30.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.980+0000 7fec9effd640 1 -- 172.21.15.170:0/21933601 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7fecb00cf210 con 0x7fec90073350 2024-09-21T02:18:30.982 INFO:teuthology.orchestra.run.smithi170.stderr:Volume created successfully (no MDS daemons created) 2024-09-21T02:18:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec90073350 msgr2=0x7fec90075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec90073350 0x7fec90075860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fecb00caf30 tx=0x7feca4009040 comp rx=0 tx=0).stop 2024-09-21T02:18:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 msgr2=0x7fecb0160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 0x7fecb0160d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7feca000d770 tx=0x7feca000dc30 comp rx=0 tx=0).stop 2024-09-21T02:18:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 shutdown_connections 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec90073350 0x7fec90075860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fecb00b5da0 0x7fecb0164160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecb00cf640 0x7fecb015e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 --2- 172.21.15.170:0/21933601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecb00b09f0 0x7fecb0160d40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 >> 172.21.15.170:0/21933601 conn(0x7fecb00d1280 msgr2=0x7fecb00d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 shutdown_connections 2024-09-21T02:18:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:30.984+0000 7fecb4bf7640 1 -- 172.21.15.170:0/21933601 wait complete. 2024-09-21T02:18:31.014 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-21T02:18:31.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 -- 172.21.15.170:0/4136615647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e50004810 msgr2=0x7f2e50004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 --2- 172.21.15.170:0/4136615647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e50004810 0x7f2e50004c20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c001dc0 tx=0x7f2e4c01cad0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 -- 172.21.15.170:0/4136615647 shutdown_connections 2024-09-21T02:18:31.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 --2- 172.21.15.170:0/4136615647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e50004810 0x7f2e50004c20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 -- 172.21.15.170:0/4136615647 >> 172.21.15.170:0/4136615647 conn(0x7f2e640d0170 msgr2=0x7f2e640d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:31.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 -- 172.21.15.170:0/4136615647 shutdown_connections 2024-09-21T02:18:31.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.168+0000 7f2e689ce640 1 -- 172.21.15.170:0/4136615647 wait complete. 2024-09-21T02:18:31.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 Processor -- start 2024-09-21T02:18:31.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- start start 2024-09-21T02:18:31.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e50004810 0x7f2e64158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e640cc400 0x7f2e641e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e641e9830 0x7f2e640ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640b3550 con 0x7f2e640cc400 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640b33d0 con 0x7f2e641e9830 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2e640ce540 con 0x7f2e50004810 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e640cc400 0x7f2e641e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e640cc400 0x7f2e641e92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54636/0 (socket says 172.21.15.170:54636) 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 -- 172.21.15.170:0/3706620224 learned_addr learned my addr 172.21.15.170:0/3706620224 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e6259c640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e50004810 0x7f2e64158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e62d9d640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e641e9830 0x7f2e640ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 -- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e50004810 msgr2=0x7f2e64158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e50004810 0x7f2e64158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 -- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e641e9830 msgr2=0x7f2e640ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e641e9830 0x7f2e640ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 -- 172.21.15.170:0/3706620224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e641e9c40 con 0x7f2e640cc400 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e6259c640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e50004810 0x7f2e64158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:31.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e61d9b640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e640cc400 0x7f2e641e92f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ab040 tx=0x7f2e4c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:31.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e5affd640 1 -- 172.21.15.170:0/3706620224 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0a5b40 con 0x7f2e640cc400 2024-09-21T02:18:31.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e5affd640 1 -- 172.21.15.170:0/3706620224 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e4c0a5dd0 con 0x7f2e640cc400 2024-09-21T02:18:31.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- 172.21.15.170:0/3706620224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e640b2570 con 0x7f2e640cc400 2024-09-21T02:18:31.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e5affd640 1 -- 172.21.15.170:0/3706620224 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c006330 con 0x7f2e640cc400 2024-09-21T02:18:31.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- 172.21.15.170:0/3706620224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e640b08d0 con 0x7f2e640cc400 2024-09-21T02:18:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.172+0000 7f2e689ce640 1 -- 172.21.15.170:0/3706620224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e640b2740 con 0x7f2e640cc400 2024-09-21T02:18:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.176+0000 7f2e5affd640 1 -- 172.21.15.170:0/3706620224 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f2e4c0064d0 con 0x7f2e640cc400 2024-09-21T02:18:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.176+0000 7f2e5affd640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e44073350 0x7f2e44075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.176+0000 7f2e5affd640 1 -- 172.21.15.170:0/3706620224 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7f2e4c12ec10 con 0x7f2e640cc400 2024-09-21T02:18:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.176+0000 7f2e6259c640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e44073350 0x7f2e44075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.176+0000 7f2e6259c640 1 --2- 172.21.15.170:0/3706620224 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e44073350 0x7f2e44075860 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005d10 tx=0x7f2e54005ee0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:31.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.180+0000 7f2e5affd640 1 -- 172.21.15.170:0/3706620224 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2e640b2740 con 0x7f2e640cc400 2024-09-21T02:18:31.378 INFO:teuthology.orchestra.run.smithi170.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-21T02:18:31.378 INFO:teuthology.orchestra.run.smithi170.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-21T02:18:31.378 INFO:teuthology.orchestra.run.smithi170.stderr:Error EINVAL: invalid command 2024-09-21T02:18:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T02:18:31.384 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-21T02:18:31.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 -- 172.21.15.170:0/69934246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 msgr2=0x7f06e40ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 --2- 172.21.15.170:0/69934246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 0x7f06e40ce540 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f06e409e220 tx=0x7f06d401cd60 comp rx=0 tx=0).stop 2024-09-21T02:18:31.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 -- 172.21.15.170:0/69934246 shutdown_connections 2024-09-21T02:18:31.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 --2- 172.21.15.170:0/69934246 >> v2:172.21.15.191:3300/0 conn(0x7f06e406ef40 0x7f06e406f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 --2- 172.21.15.170:0/69934246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06e40cea80 0x7f06e40cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 --2- 172.21.15.170:0/69934246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 0x7f06e40ce540 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 -- 172.21.15.170:0/69934246 >> 172.21.15.170:0/69934246 conn(0x7f06e40d0170 msgr2=0x7f06e40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 -- 172.21.15.170:0/69934246 shutdown_connections 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 -- 172.21.15.170:0/69934246 wait complete. 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 Processor -- start 2024-09-21T02:18:31.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.544+0000 7f06eb0ae640 1 -- start start 2024-09-21T02:18:31.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06e406ef40 0x7f06e40b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 0x7f06e40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06e40cea80 0x7f06e40b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4070430 con 0x7f06e40b2fc0 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f06e40702b0 con 0x7f06e406ef40 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f06e40705b0 con 0x7f06e40cea80 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 0x7f06e40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 0x7f06e40b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54654/0 (socket says 172.21.15.170:54654) 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 -- 172.21.15.170:0/4182578269 learned_addr learned my addr 172.21.15.170:0/4182578269 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e964b640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06e40cea80 0x7f06e40b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e8e4a640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06e406ef40 0x7f06e40b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 -- 172.21.15.170:0/4182578269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06e40cea80 msgr2=0x7f06e40b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06e40cea80 0x7f06e40b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 -- 172.21.15.170:0/4182578269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06e406ef40 msgr2=0x7f06e40b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06e406ef40 0x7f06e40b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 -- 172.21.15.170:0/4182578269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e40ca4f0 con 0x7f06e40b2fc0 2024-09-21T02:18:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e3fff640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06e40b2fc0 0x7f06e40b1580 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f06d800d6d0 tx=0x7f06d800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e17fa640 1 -- 172.21.15.170:0/4182578269 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8017070 con 0x7f06e40b2fc0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e17fa640 1 -- 172.21.15.170:0/4182578269 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06d8005310 con 0x7f06e40b2fc0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 -- 172.21.15.170:0/4182578269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e40cbcc0 con 0x7f06e40b2fc0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06e17fa640 1 -- 172.21.15.170:0/4182578269 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80097c0 con 0x7f06e40b2fc0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 -- 172.21.15.170:0/4182578269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e409ac10 con 0x7f06e40b2fc0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.548+0000 7f06eb0ae640 1 -- 172.21.15.170:0/4182578269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e415d6d0 con 0x7f06e40b2fc0 2024-09-21T02:18:31.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.552+0000 7f06e17fa640 1 -- 172.21.15.170:0/4182578269 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f06d8009960 con 0x7f06e40b2fc0 2024-09-21T02:18:31.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.552+0000 7f06e17fa640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06d0073320 0x7f06d0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.552+0000 7f06e17fa640 1 -- 172.21.15.170:0/4182578269 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7f06d8094c00 con 0x7f06e40b2fc0 2024-09-21T02:18:31.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.552+0000 7f06e8e4a640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06d0073320 0x7f06d0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.552+0000 7f06e8e4a640 1 --2- 172.21.15.170:0/4182578269 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06d0073320 0x7f06d0075830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f06d40af000 tx=0x7f06d401b5a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:31.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.556+0000 7f06e17fa640 1 -- 172.21.15.170:0/4182578269 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f06d805f7a0 con 0x7f06e40b2fc0 2024-09-21T02:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-21T02:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] [] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-21T02:18:31.755 INFO:teuthology.orchestra.run.smithi170.stderr:Error EINVAL: invalid command 2024-09-21T02:18:31.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T02:18:31.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-21T02:18:31.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/1163542020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 msgr2=0x7ff9b406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/1163542020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b406ff50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4072940 tx=0x7ff99c01c800 comp rx=0 tx=0).stop 2024-09-21T02:18:31.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/1163542020 shutdown_connections 2024-09-21T02:18:31.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/1163542020 >> v2:172.21.15.191:3300/0 conn(0x7ff9b40cf060 0x7ff9b4070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/1163542020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b406ff50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/1163542020 >> 172.21.15.170:0/1163542020 conn(0x7ff9b40d0170 msgr2=0x7ff9b40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:31.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/1163542020 shutdown_connections 2024-09-21T02:18:31.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.924+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/1163542020 wait complete. 2024-09-21T02:18:31.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 Processor -- start 2024-09-21T02:18:31.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 -- start start 2024-09-21T02:18:31.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9b40b2fc0 0x7ff9b4142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b40b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9b40cf060 0x7ff9b41555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4169f50 con 0x7ff9b40cec50 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4169dd0 con 0x7ff9b40b2fc0 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b416a0d0 con 0x7ff9b40cf060 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b40b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b40b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54670/0 (socket says 172.21.15.170:54670) 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 -- 172.21.15.170:0/3383317459 learned_addr learned my addr 172.21.15.170:0/3383317459 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b2d9d640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9b40cf060 0x7ff9b41555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b259c640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9b40b2fc0 0x7ff9b4142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 -- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9b40cf060 msgr2=0x7ff9b41555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9b40cf060 0x7ff9b41555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 -- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9b40b2fc0 msgr2=0x7ff9b4142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9b40b2fc0 0x7ff9b4142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 -- 172.21.15.170:0/3383317459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b41fa3d0 con 0x7ff9b40cec50 2024-09-21T02:18:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b259c640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9b40b2fc0 0x7ff9b4142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b2d9d640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9b40cf060 0x7ff9b41555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b1d9b640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b40b1970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff99c01cce0 tx=0x7ff99c0a55d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b2040 con 0x7ff9b40cec50 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff99c0041b0 con 0x7ff9b40cec50 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b41fa6b0 con 0x7ff9b40cec50 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c003600 con 0x7ff9b40cec50 2024-09-21T02:18:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.928+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b41fac70 con 0x7ff9b40cec50 2024-09-21T02:18:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.932+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b41fafe0 con 0x7ff9b40cec50 2024-09-21T02:18:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.932+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7ff99c0bd030 con 0x7ff9b40cec50 2024-09-21T02:18:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.932+0000 7ff9aaffd640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff994073320 0x7ff994075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.932+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6543+0+0 (secure 0 0 0) 0x7ff99c12e3a0 con 0x7ff9b40cec50 2024-09-21T02:18:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.932+0000 7ff9b259c640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff994073320 0x7ff994075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.932+0000 7ff9b259c640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff994073320 0x7ff994075830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000f70 tx=0x7ff9ac009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:31.936+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff99c0c4060 con 0x7ff9b40cec50 2024-09-21T02:18:32.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:32.152+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7ff9b41fa160 con 0x7ff994073320 2024-09-21T02:18:32.159 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:18:32.156+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:18:32.247 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:18:35.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.183+0000 7ff9aaffd640 1 -- 172.21.15.170:0/3383317459 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7ff9b41fa160 con 0x7ff994073320 2024-09-21T02:18:35.188 INFO:teuthology.orchestra.run.smithi170.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-21T02:18:35.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff994073320 msgr2=0x7ff994075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff994073320 0x7ff994075830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000f70 tx=0x7ff9ac009040 comp rx=0 tx=0).stop 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 msgr2=0x7ff9b40b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b40b1970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff99c01cce0 tx=0x7ff99c0a55d0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 shutdown_connections 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff994073320 0x7ff994075830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9b40cf060 0x7ff9b41555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b40cec50 0x7ff9b40b1970 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 --2- 172.21.15.170:0/3383317459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9b40b2fc0 0x7ff9b4142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 >> 172.21.15.170:0/3383317459 conn(0x7ff9b40d0170 msgr2=0x7ff9b40b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 shutdown_connections 2024-09-21T02:18:35.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.187+0000 7ff9b8ca8640 1 -- 172.21.15.170:0/3383317459 wait complete. 2024-09-21T02:18:35.213 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:18:35.214 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:18:35.306 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:18:35.381 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:18:35.381 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:18:35.381 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:35.381 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:35.388 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:35.388 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:35.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:18:35.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 -- 172.21.15.170:0/3489490798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 msgr2=0x7fc160004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 --2- 172.21.15.170:0/3489490798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc160004c20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc168005fd0 tx=0x7fc16801cd60 comp rx=0 tx=0).stop 2024-09-21T02:18:35.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 -- 172.21.15.170:0/3489490798 shutdown_connections 2024-09-21T02:18:35.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 --2- 172.21.15.170:0/3489490798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc160004c20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 -- 172.21.15.170:0/3489490798 >> 172.21.15.170:0/3489490798 conn(0x7fc1780b1c40 msgr2=0x7fc1780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:35.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 -- 172.21.15.170:0/3489490798 shutdown_connections 2024-09-21T02:18:35.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.643+0000 7fc17f061640 1 -- 172.21.15.170:0/3489490798 wait complete. 2024-09-21T02:18:35.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 Processor -- start 2024-09-21T02:18:35.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 -- start start 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc17814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1780b3fd0 0x7fc1780c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1780c4320 0x7fc1780b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780cb9f0 con 0x7fc160004810 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780cb870 con 0x7fc1780c4320 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780cbb70 con 0x7fc1780b3fd0 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc17814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc17814f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53124/0 (socket says 172.21.15.170:53124) 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 -- 172.21.15.170:0/2450105109 learned_addr learned my addr 172.21.15.170:0/2450105109 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc177fff640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1780b3fd0 0x7fc1780c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17d5fe640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1780c4320 0x7fc1780b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:35.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 -- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1780b3fd0 msgr2=0x7fc1780c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1780b3fd0 0x7fc1780c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 -- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1780c4320 msgr2=0x7fc1780b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1780c4320 0x7fc1780b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 -- 172.21.15.170:0/2450105109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1780b3250 con 0x7fc160004810 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17d5fe640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1780c4320 0x7fc1780b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17cdfd640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc17814f4e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00d350 tx=0x7fc16c00d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c014070 con 0x7fc160004810 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc16c00f990 con 0x7fc160004810 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1780c4b50 con 0x7fc160004810 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00fde0 con 0x7fc160004810 2024-09-21T02:18:35.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.647+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1780b37d0 con 0x7fc160004810 2024-09-21T02:18:35.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.651+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1780cbd50 con 0x7fc160004810 2024-09-21T02:18:35.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.651+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fc16c017070 con 0x7fc160004810 2024-09-21T02:18:35.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.651+0000 7fc1757fa640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc164073320 0x7fc164075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:35.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.651+0000 7fc177fff640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc164073320 0x7fc164075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:35.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.651+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fc16c094470 con 0x7fc160004810 2024-09-21T02:18:35.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.651+0000 7fc177fff640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc164073320 0x7fc164075830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1680b0000 tx=0x7fc168019b00 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:35.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.655+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc16c05f210 con 0x7fc160004810 2024-09-21T02:18:35.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.843+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1780d0ba0 con 0x7fc160004810 2024-09-21T02:18:35.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc1757fa640 1 -- 172.21.15.170:0/2450105109 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+124 (secure 0 0 0) 0x7fc16c05eb10 con 0x7fc160004810 2024-09-21T02:18:35.849 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.191:6835/2173918761 2024-09-22T02:18:32.161134+0000 2024-09-21T02:18:35.849 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.191:6834/2173918761 2024-09-22T02:18:32.161134+0000 2024-09-21T02:18:35.849 INFO:teuthology.orchestra.run.smithi170.stderr:listed 2 entries 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc164073320 msgr2=0x7fc164075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc164073320 0x7fc164075830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1680b0000 tx=0x7fc168019b00 comp rx=0 tx=0).stop 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 msgr2=0x7fc17814f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc17814f4e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00d350 tx=0x7fc16c00d810 comp rx=0 tx=0).stop 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 shutdown_connections 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc164073320 0x7fc164075830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1780b3fd0 0x7fc1780c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1780c4320 0x7fc1780b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 --2- 172.21.15.170:0/2450105109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc160004810 0x7fc17814f4e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:35.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 >> 172.21.15.170:0/2450105109 conn(0x7fc1780b1c40 msgr2=0x7fc1780d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:35.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 shutdown_connections 2024-09-21T02:18:35.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:35.847+0000 7fc17f061640 1 -- 172.21.15.170:0/2450105109 wait complete. 2024-09-21T02:18:35.870 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:18:35.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:35.870 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:35.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[43481]: fuse finished with error 0 and tester_r 0 2024-09-21T02:18:35.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:18:35.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:35.882 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:18:35.960 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:18:35.960 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:35.960 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:35.967 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:35.967 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:36.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- 172.21.15.170:0/252122123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 msgr2=0x7f87b406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/252122123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b406fa30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f87b4071080 tx=0x7f87b001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- 172.21.15.170:0/252122123 shutdown_connections 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/252122123 >> v2:172.21.15.191:3300/0 conn(0x7f87b40ccb00 0x7f87b40ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/252122123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87b40b2f40 0x7f87b40b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/252122123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b406fa30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- 172.21.15.170:0/252122123 >> 172.21.15.170:0/252122123 conn(0x7f87b40b1c40 msgr2=0x7f87b40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:36.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- 172.21.15.170:0/252122123 shutdown_connections 2024-09-21T02:18:36.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- 172.21.15.170:0/252122123 wait complete. 2024-09-21T02:18:36.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 Processor -- start 2024-09-21T02:18:36.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- start start 2024-09-21T02:18:36.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87b40b2f40 0x7f87b409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b415f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87b40ccb00 0x7f87b414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87b40cf950 con 0x7f87b40b5410 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4070180 con 0x7f87b40ccb00 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87bbf3e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f87b40ce800 con 0x7f87b40b2f40 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.175+0000 7f87b94d9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b415f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53142/0 (socket says 172.21.15.170:53142) 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 -- 172.21.15.170:0/962671531 learned_addr learned my addr 172.21.15.170:0/962671531 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b9cda640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87b40b2f40 0x7f87b409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87ba4db640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87b40ccb00 0x7f87b414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 -- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87b40b2f40 msgr2=0x7f87b409a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87b40b2f40 0x7f87b409a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 -- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87b40ccb00 msgr2=0x7f87b414d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87b40ccb00 0x7f87b414d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 -- 172.21.15.170:0/962671531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b40b1000 con 0x7f87b40b5410 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87ba4db640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87b40ccb00 0x7f87b414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b94d9640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b415f490 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c6f0 tx=0x7f87a400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4004020 con 0x7f87b40b5410 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87a40042b0 con 0x7f87b40b5410 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b409af00 con 0x7f87b40b5410 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4007c10 con 0x7f87b40b5410 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b409b6f0 con 0x7f87b40b5410 2024-09-21T02:18:36.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b40ced00 con 0x7f87b40b5410 2024-09-21T02:18:36.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f87a4003600 con 0x7f87b40b5410 2024-09-21T02:18:36.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87aa7fc640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8798073320 0x7f8798075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b9cda640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8798073320 0x7f8798075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f87a40946b0 con 0x7f87b40b5410 2024-09-21T02:18:36.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.179+0000 7f87b9cda640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8798073320 0x7f8798075830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87b00af000 tx=0x7f87b001b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:36.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.187+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f87a4014030 con 0x7f87b40b5410 2024-09-21T02:18:36.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87b409b1a0 con 0x7f87b40b5410 2024-09-21T02:18:36.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87aa7fc640 1 -- 172.21.15.170:0/962671531 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+124 (secure 0 0 0) 0x7f87a405f3d0 con 0x7f87b40b5410 2024-09-21T02:18:36.373 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.191:6835/2173918761 2024-09-22T02:18:32.161134+0000 2024-09-21T02:18:36.373 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.191:6834/2173918761 2024-09-22T02:18:32.161134+0000 2024-09-21T02:18:36.374 INFO:teuthology.orchestra.run.smithi170.stderr:listed 2 entries 2024-09-21T02:18:36.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8798073320 msgr2=0x7f8798075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8798073320 0x7f8798075830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87b00af000 tx=0x7f87b001b100 comp rx=0 tx=0).stop 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 msgr2=0x7f87b415f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b415f490 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c6f0 tx=0x7f87a400cbb0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 shutdown_connections 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8798073320 0x7f8798075830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87b40ccb00 0x7f87b414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87b40b5410 0x7f87b415f490 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 --2- 172.21.15.170:0/962671531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87b40b2f40 0x7f87b409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 >> 172.21.15.170:0/962671531 conn(0x7f87b40b1c40 msgr2=0x7f87b415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 shutdown_connections 2024-09-21T02:18:36.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.371+0000 7f87bbf3e640 1 -- 172.21.15.170:0/962671531 wait complete. 2024-09-21T02:18:36.394 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:18:36.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:36.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:18:36.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[34995]: fuse finished with error 0 and tester_r 0 2024-09-21T02:18:36.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:18:36.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:36.410 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:18:36.485 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:18:36.485 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:36.485 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:36.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:18:36.491 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:18:36.492 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:36.492 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:36.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:18:36.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 -- 172.21.15.170:0/4159705707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c06d340 msgr2=0x7fcd6c0adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 --2- 172.21.15.170:0/4159705707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c06d340 0x7fcd6c0adfc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c073a30 tx=0x7fcd5401c770 comp rx=0 tx=0).stop 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 -- 172.21.15.170:0/4159705707 shutdown_connections 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 --2- 172.21.15.170:0/4159705707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c06d340 0x7fcd6c0adfc0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 --2- 172.21.15.170:0/4159705707 >> v2:172.21.15.191:3300/0 conn(0x7fcd6c162e80 0x7fcd6c06ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 -- 172.21.15.170:0/4159705707 >> 172.21.15.170:0/4159705707 conn(0x7fcd6c09a9c0 msgr2=0x7fcd6c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 -- 172.21.15.170:0/4159705707 shutdown_connections 2024-09-21T02:18:36.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.687+0000 7fcd7280d640 1 -- 172.21.15.170:0/4159705707 wait complete. 2024-09-21T02:18:36.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 Processor -- start 2024-09-21T02:18:36.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- start start 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06d340 0x7fcd6c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c0ae500 0x7fcd6c1f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 0x7fcd6c14b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c15eda0 con 0x7fcd6c162e80 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c15ec20 con 0x7fcd6c06d340 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6c16c890 con 0x7fcd6c0ae500 2024-09-21T02:18:36.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 0x7fcd6c14b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 0x7fcd6c14b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53158/0 (socket says 172.21.15.170:53158) 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 -- 172.21.15.170:0/1422911809 learned_addr learned my addr 172.21.15.170:0/1422911809 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd6bfff640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06d340 0x7fcd6c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd6b7fe640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c0ae500 0x7fcd6c1f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 -- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c0ae500 msgr2=0x7fcd6c1f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c0ae500 0x7fcd6c1f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 -- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06d340 msgr2=0x7fcd6c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06d340 0x7fcd6c0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 -- 172.21.15.170:0/1422911809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6c0b1fb0 con 0x7fcd6c162e80 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd6b7fe640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c0ae500 0x7fcd6c1f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd70daa640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 0x7fcd6c14b7a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcd64002240 tx=0x7fcd6400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64017070 con 0x7fcd6c162e80 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd64003c70 con 0x7fcd6c162e80 2024-09-21T02:18:36.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6c0b19f0 con 0x7fcd6c162e80 2024-09-21T02:18:36.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd640052d0 con 0x7fcd6c162e80 2024-09-21T02:18:36.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6c0b41d0 con 0x7fcd6c162e80 2024-09-21T02:18:36.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.691+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6c0adfc0 con 0x7fcd6c162e80 2024-09-21T02:18:36.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.695+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fcd6401a070 con 0x7fcd6c162e80 2024-09-21T02:18:36.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.695+0000 7fcd68ff9640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd4c073320 0x7fcd4c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:36.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.695+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fcd64094060 con 0x7fcd6c162e80 2024-09-21T02:18:36.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.695+0000 7fcd6bfff640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd4c073320 0x7fcd4c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:36.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.695+0000 7fcd6bfff640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd4c073320 0x7fcd4c075830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c005fd0 tx=0x7fcd5c005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:36.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.699+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcd64021020 con 0x7fcd6c162e80 2024-09-21T02:18:36.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.903+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd6c0cee40 con 0x7fcd6c162e80 2024-09-21T02:18:36.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.903+0000 7fcd68ff9640 1 -- 172.21.15.170:0/1422911809 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5237 (secure 0 0 0) 0x7fcd6c06d750 con 0x7fcd6c162e80 2024-09-21T02:18:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:36.907 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":62,"btime":"2024-09-21T02:18:33:163496+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1805283918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1805283918},{"type":"v1","addr":"172.21.15.191:6835","nonce":1805283918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:13.828587+0000","modified":"2024-09-21T02:18:16.848304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1472234860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1472234860},{"type":"v1","addr":"172.21.15.170:6835","nonce":1472234860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":6}]} 2024-09-21T02:18:36.907 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 62 2024-09-21T02:18:36.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd4c073320 msgr2=0x7fcd4c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd4c073320 0x7fcd4c075830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c005fd0 tx=0x7fcd5c005f40 comp rx=0 tx=0).stop 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 msgr2=0x7fcd6c14b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 0x7fcd6c14b7a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcd64002240 tx=0x7fcd6400cb90 comp rx=0 tx=0).stop 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 shutdown_connections 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd4c073320 0x7fcd4c075830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c0ae500 0x7fcd6c1f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6c162e80 0x7fcd6c14b7a0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 --2- 172.21.15.170:0/1422911809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06d340 0x7fcd6c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 >> 172.21.15.170:0/1422911809 conn(0x7fcd6c09a9c0 msgr2=0x7fcd6c15c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 shutdown_connections 2024-09-21T02:18:36.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:36.907+0000 7fcd7280d640 1 -- 172.21.15.170:0/1422911809 wait complete. 2024-09-21T02:18:36.928 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:37.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/2540941125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc003fe0 msgr2=0x7fcbbc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/2540941125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc003fe0 0x7fcbbc004810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0073630 tx=0x7fcbb801d230 comp rx=0 tx=0).stop 2024-09-21T02:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbced9d640 1 -- 172.21.15.170:0/2540941125 reap_dead start 2024-09-21T02:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/2540941125 shutdown_connections 2024-09-21T02:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/2540941125 >> 172.21.15.170:0/2540941125 conn(0x7fcbd00cd3c0 msgr2=0x7fcbd00cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/2540941125 shutdown_connections 2024-09-21T02:18:37.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/2540941125 wait complete. 2024-09-21T02:18:37.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 Processor -- start 2024-09-21T02:18:37.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.083+0000 7fcbd4a8a640 1 -- start start 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbbc003fe0 0x7fcbd00b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 0x7fcbd0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbd01fa170 0x7fcbd0157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0167df0 con 0x7fcbd00b3770 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd00c60a0 con 0x7fcbbc003fe0 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd0167f70 con 0x7fcbd01fa170 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 0x7fcbd0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 0x7fcbd0157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53182/0 (socket says 172.21.15.170:53182) 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 -- 172.21.15.170:0/1292164089 learned_addr learned my addr 172.21.15.170:0/1292164089 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbce59c640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbbc003fe0 0x7fcbd00b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbced9d640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbd01fa170 0x7fcbd0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 -- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbd01fa170 msgr2=0x7fcbd0157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbd01fa170 0x7fcbd0157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 -- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbbc003fe0 msgr2=0x7fcbd00b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbbc003fe0 0x7fcbd00b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 -- 172.21.15.170:0/1292164089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd00b5150 con 0x7fcbd00b3770 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbced9d640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbd01fa170 0x7fcbd0157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbcdd9b640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 0x7fcbd0157330 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80af040 tx=0x7fcbb8005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80b1040 con 0x7fcbd00b3770 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbb80ac7b0 con 0x7fcbd00b3770 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd01f70f0 con 0x7fcbd00b3770 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80acbe0 con 0x7fcbd00b3770 2024-09-21T02:18:37.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd01f76b0 con 0x7fcbd00b3770 2024-09-21T02:18:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd01f6e60 con 0x7fcbd00b3770 2024-09-21T02:18:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.087+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fcbb80be020 con 0x7fcbd00b3770 2024-09-21T02:18:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.091+0000 7fcbc6ffd640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcbb0073350 0x7fcbb0075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.091+0000 7fcbce59c640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcbb0073350 0x7fcbb0075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.091+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fcbb812ebf0 con 0x7fcbd00b3770 2024-09-21T02:18:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.091+0000 7fcbce59c640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcbb0073350 0x7fcbb0075860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009f10 tx=0x7fcbc8009500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:37.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.095+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcbb80c5020 con 0x7fcbd00b3770 2024-09-21T02:18:37.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.295+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbd0073680 con 0x7fcbd00b3770 2024-09-21T02:18:37.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.295+0000 7fcbc6ffd640 1 -- 172.21.15.170:0/1292164089 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5237 (secure 0 0 0) 0x7fcbb8016f40 con 0x7fcbd00b3770 2024-09-21T02:18:37.299 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:37.299 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":62,"btime":"2024-09-21T02:18:33:163496+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1805283918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1805283918},{"type":"v1","addr":"172.21.15.191:6835","nonce":1805283918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:13.828587+0000","modified":"2024-09-21T02:18:16.848304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1472234860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1472234860},{"type":"v1","addr":"172.21.15.170:6835","nonce":1472234860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":6}]} 2024-09-21T02:18:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 62 2024-09-21T02:18:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcbb0073350 msgr2=0x7fcbb0075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcbb0073350 0x7fcbb0075860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009f10 tx=0x7fcbc8009500 comp rx=0 tx=0).stop 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 msgr2=0x7fcbd0157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 0x7fcbd0157330 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80af040 tx=0x7fcbb8005620 comp rx=0 tx=0).stop 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 shutdown_connections 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcbb0073350 0x7fcbb0075860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbd01fa170 0x7fcbd0157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd00b3770 0x7fcbd0157330 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 --2- 172.21.15.170:0/1292164089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbbc003fe0 0x7fcbd00b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 >> 172.21.15.170:0/1292164089 conn(0x7fcbd00cd3c0 msgr2=0x7fcbd014eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 shutdown_connections 2024-09-21T02:18:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.299+0000 7fcbd4a8a640 1 -- 172.21.15.170:0/1292164089 wait complete. 2024-09-21T02:18:37.321 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:37.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 -- 172.21.15.170:0/1094381570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee28003a40 msgr2=0x7fee28003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 --2- 172.21.15.170:0/1094381570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee28003a40 0x7fee28003eb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c260 tx=0x7fee2c01cf40 comp rx=0 tx=0).stop 2024-09-21T02:18:37.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee3bfff640 1 -- 172.21.15.170:0/1094381570 reap_dead start 2024-09-21T02:18:37.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 -- 172.21.15.170:0/1094381570 shutdown_connections 2024-09-21T02:18:37.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 -- 172.21.15.170:0/1094381570 >> 172.21.15.170:0/1094381570 conn(0x7fee3c0cee30 msgr2=0x7fee3c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:37.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 -- 172.21.15.170:0/1094381570 shutdown_connections 2024-09-21T02:18:37.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 -- 172.21.15.170:0/1094381570 wait complete. 2024-09-21T02:18:37.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 Processor -- start 2024-09-21T02:18:37.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.467+0000 7fee41dac640 1 -- start start 2024-09-21T02:18:37.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee28003a40 0x7fee3c150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 0x7fee3c158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee3c15f130 0x7fee3c15f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c168830 con 0x7fee3c158630 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c1686b0 con 0x7fee3c15f130 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c1689b0 con 0x7fee28003a40 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 0x7fee3c158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 0x7fee3c158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53200/0 (socket says 172.21.15.170:53200) 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 -- 172.21.15.170:0/3253263902 learned_addr learned my addr 172.21.15.170:0/3253263902 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee3b7fe640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee28003a40 0x7fee3c150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee3bfff640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee3c15f130 0x7fee3c15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 -- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee28003a40 msgr2=0x7fee3c150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee28003a40 0x7fee3c150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 -- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee3c15f130 msgr2=0x7fee3c15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee3c15f130 0x7fee3c15f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 -- 172.21.15.170:0/3253263902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c154e80 con 0x7fee3c158630 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee3bfff640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee3c15f130 0x7fee3c15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee32ffd640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 0x7fee3c158a40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fee24004f10 tx=0x7fee24001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24015020 con 0x7fee3c158630 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee24011960 con 0x7fee3c158630 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c157870 con 0x7fee3c158630 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24011d90 con 0x7fee3c158630 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c157e30 con 0x7fee3c158630 2024-09-21T02:18:37.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c15d8f0 con 0x7fee3c158630 2024-09-21T02:18:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fee24018020 con 0x7fee3c158630 2024-09-21T02:18:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee38ff9640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee1c073320 0x7fee1c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee3b7fe640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee1c073320 0x7fee1c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fee240954e0 con 0x7fee3c158630 2024-09-21T02:18:37.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.471+0000 7fee3b7fe640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee1c073320 0x7fee1c075830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0ab000 tx=0x7fee2c019d80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:37.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.475+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fee2401f020 con 0x7fee3c158630 2024-09-21T02:18:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.647+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee3c0d0aa0 con 0x7fee3c158630 2024-09-21T02:18:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.647+0000 7fee38ff9640 1 -- 172.21.15.170:0/3253263902 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16911 (secure 0 0 0) 0x7fee2407f220 con 0x7fee3c158630 2024-09-21T02:18:37.651 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:37.652 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":91,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:35.169237+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:11.529716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:12.787075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6835/2173918761":"2024-09-22T02:18:32.161134+0000","172.21.15.191:6834/2173918761":"2024-09-22T02:18:32.161134+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee1c073320 msgr2=0x7fee1c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee1c073320 0x7fee1c075830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0ab000 tx=0x7fee2c019d80 comp rx=0 tx=0).stop 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 msgr2=0x7fee3c158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 0x7fee3c158a40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fee24004f10 tx=0x7fee24001640 comp rx=0 tx=0).stop 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 shutdown_connections 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee1c073320 0x7fee1c075830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee3c15f130 0x7fee3c15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee3c158630 0x7fee3c158a40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 --2- 172.21.15.170:0/3253263902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee28003a40 0x7fee3c150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 >> 172.21.15.170:0/3253263902 conn(0x7fee3c0cee30 msgr2=0x7fee3c15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 shutdown_connections 2024-09-21T02:18:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.651+0000 7fee41dac640 1 -- 172.21.15.170:0/3253263902 wait complete. 2024-09-21T02:18:37.673 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:18:37.673 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 -- 172.21.15.170:0/2478135710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e80ae6d0 msgr2=0x7f42e8169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 --2- 172.21.15.170:0/2478135710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e80ae6d0 0x7f42e8169870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009c80 tx=0x7f42e001cd10 comp rx=0 tx=0).stop 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 -- 172.21.15.170:0/2478135710 shutdown_connections 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 --2- 172.21.15.170:0/2478135710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e80ae6d0 0x7f42e8169870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 --2- 172.21.15.170:0/2478135710 >> v2:172.21.15.191:3300/0 conn(0x7f42e8162e80 0x7f42e806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 -- 172.21.15.170:0/2478135710 >> 172.21.15.170:0/2478135710 conn(0x7f42e809a9c0 msgr2=0x7f42e80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 -- 172.21.15.170:0/2478135710 shutdown_connections 2024-09-21T02:18:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.815+0000 7f42f0515640 1 -- 172.21.15.170:0/2478135710 wait complete. 2024-09-21T02:18:37.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 Processor -- start 2024-09-21T02:18:37.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 -- start start 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 0x7f42e81617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e80ae6d0 0x7f42e814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8162e80 0x7f42e81f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f42e815eda0 con 0x7f42e806d340 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f42e815ec20 con 0x7f42e80ae6d0 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f42e816c5d0 con 0x7f42e8162e80 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 0x7f42e81617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 0x7f42e81617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53216/0 (socket says 172.21.15.170:53216) 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 -- 172.21.15.170:0/662134003 learned_addr learned my addr 172.21.15.170:0/662134003 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42eeab2640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8162e80 0x7f42e81f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42edab0640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e80ae6d0 0x7f42e814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 -- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8162e80 msgr2=0x7f42e81f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8162e80 0x7f42e81f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 -- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e80ae6d0 msgr2=0x7f42e814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e80ae6d0 0x7f42e814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 -- 172.21.15.170:0/662134003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80b5290 con 0x7f42e806d340 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42ee2b1640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 0x7f42e81617d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f42e400ea20 tx=0x7f42e400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e400cd80 con 0x7f42e806d340 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42e4010430 con 0x7f42e806d340 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e814d7b0 con 0x7f42e806d340 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4010880 con 0x7f42e806d340 2024-09-21T02:18:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.819+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e80b5810 con 0x7f42e806d340 2024-09-21T02:18:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.823+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e806ce00 con 0x7f42e806d340 2024-09-21T02:18:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.823+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f42e4010a20 con 0x7f42e806d340 2024-09-21T02:18:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.823+0000 7f42deffd640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f42cc073320 0x7f42cc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.823+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f42e4095110 con 0x7f42e806d340 2024-09-21T02:18:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.823+0000 7f42edab0640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f42cc073320 0x7f42cc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.823+0000 7f42edab0640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f42cc073320 0x7f42cc075830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42e815e6d0 tx=0x7f42d8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:37.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:37.827+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f42e405feb0 con 0x7f42e806d340 2024-09-21T02:18:38.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.019+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42e806d930 con 0x7f42e806d340 2024-09-21T02:18:38.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42deffd640 1 -- 172.21.15.170:0/662134003 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5237 (secure 0 0 0) 0x7f42e405f7b0 con 0x7f42e806d340 2024-09-21T02:18:38.025 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:38.025 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":62,"btime":"2024-09-21T02:18:33:163496+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1805283918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1805283918},{"type":"v1","addr":"172.21.15.191:6835","nonce":1805283918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:13.828587+0000","modified":"2024-09-21T02:18:16.848304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1472234860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1472234860},{"type":"v1","addr":"172.21.15.170:6835","nonce":1472234860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":6}]} 2024-09-21T02:18:38.025 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 62 2024-09-21T02:18:38.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f42cc073320 msgr2=0x7f42cc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f42cc073320 0x7f42cc075830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42e815e6d0 tx=0x7f42d8009040 comp rx=0 tx=0).stop 2024-09-21T02:18:38.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 msgr2=0x7f42e81617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 0x7f42e81617d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f42e400ea20 tx=0x7f42e400eee0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 shutdown_connections 2024-09-21T02:18:38.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f42cc073320 0x7f42cc075830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f42e8162e80 0x7f42e81f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42e80ae6d0 0x7f42e814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 --2- 172.21.15.170:0/662134003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42e806d340 0x7f42e81617d0 secure :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f42e400ea20 tx=0x7f42e400eee0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 >> 172.21.15.170:0/662134003 conn(0x7f42e809a9c0 msgr2=0x7f42e815d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:38.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 shutdown_connections 2024-09-21T02:18:38.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.023+0000 7f42f0515640 1 -- 172.21.15.170:0/662134003 wait complete. 2024-09-21T02:18:38.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 -- 172.21.15.170:0/1168818935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 msgr2=0x7ff0b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1168818935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0b8004c20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400bb10 tx=0x7ff0b401cda0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 -- 172.21.15.170:0/1168818935 shutdown_connections 2024-09-21T02:18:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1168818935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0b8004c20 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 -- 172.21.15.170:0/1168818935 >> 172.21.15.170:0/1168818935 conn(0x7ff0c80b1c40 msgr2=0x7ff0c80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 -- 172.21.15.170:0/1168818935 shutdown_connections 2024-09-21T02:18:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.183+0000 7ff0cecad640 1 -- 172.21.15.170:0/1168818935 wait complete. 2024-09-21T02:18:38.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 Processor -- start 2024-09-21T02:18:38.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 -- start start 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0c81f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c8154390 0x7ff0c815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c81547a0 0x7ff0c815f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c80cf760 con 0x7ff0b8004810 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c80c3a10 con 0x7ff0c8154390 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c80cf8e0 con 0x7ff0c81547a0 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0c81f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0c81f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53236/0 (socket says 172.21.15.170:53236) 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 -- 172.21.15.170:0/1800408119 learned_addr learned my addr 172.21.15.170:0/1800408119 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0c7fff640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c8154390 0x7ff0c815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cd24a640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c81547a0 0x7ff0c815f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 -- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c81547a0 msgr2=0x7ff0c815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c81547a0 0x7ff0c815f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 -- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c8154390 msgr2=0x7ff0c815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c8154390 0x7ff0c815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 -- 172.21.15.170:0/1800408119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c81f7fc0 con 0x7ff0b8004810 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0c7fff640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c8154390 0x7ff0c815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cca49640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0c81f7a80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00e2e0 tx=0x7ff0bc00e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc012850 con 0x7ff0b8004810 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0bc012ae0 con 0x7ff0b8004810 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc01a400 con 0x7ff0b8004810 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c80af4d0 con 0x7ff0b8004810 2024-09-21T02:18:38.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.187+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c80afa90 con 0x7ff0b8004810 2024-09-21T02:18:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.191+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c80b3640 con 0x7ff0b8004810 2024-09-21T02:18:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.191+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7ff0bc00d030 con 0x7ff0b8004810 2024-09-21T02:18:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.191+0000 7ff0c57fa640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0ac073320 0x7ff0ac075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.191+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7ff0bc097840 con 0x7ff0b8004810 2024-09-21T02:18:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.191+0000 7ff0c7fff640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0ac073320 0x7ff0ac075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.191+0000 7ff0c7fff640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0ac073320 0x7ff0ac075830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40b0000 tx=0x7ff0b4019b40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:38.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.195+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff0c81f7fc0 con 0x7ff0b8004810 2024-09-21T02:18:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.387+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0c80b21f0 con 0x7ff0b8004810 2024-09-21T02:18:38.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.387+0000 7ff0c57fa640 1 -- 172.21.15.170:0/1800408119 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5237 (secure 0 0 0) 0x7ff0bc016030 con 0x7ff0b8004810 2024-09-21T02:18:38.391 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:38.392 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":62,"btime":"2024-09-21T02:18:33:163496+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1805283918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1805283918},{"type":"v1","addr":"172.21.15.191:6835","nonce":1805283918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:13.828587+0000","modified":"2024-09-21T02:18:16.848304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1472234860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1472234860},{"type":"v1","addr":"172.21.15.170:6835","nonce":1472234860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":6}]} 2024-09-21T02:18:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 62 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0ac073320 msgr2=0x7ff0ac075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0ac073320 0x7ff0ac075830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40b0000 tx=0x7ff0b4019b40 comp rx=0 tx=0).stop 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 msgr2=0x7ff0c81f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0c81f7a80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00e2e0 tx=0x7ff0bc00e7a0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 shutdown_connections 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0ac073320 0x7ff0ac075830 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0c81547a0 0x7ff0c815f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0c8154390 0x7ff0c815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 --2- 172.21.15.170:0/1800408119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8004810 0x7ff0c81f7a80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 >> 172.21.15.170:0/1800408119 conn(0x7ff0c80b1c40 msgr2=0x7ff0c80c6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:38.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 shutdown_connections 2024-09-21T02:18:38.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.391+0000 7ff0cecad640 1 -- 172.21.15.170:0/1800408119 wait complete. 2024-09-21T02:18:38.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:38.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 -- 172.21.15.170:0/1333787886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 msgr2=0x7f721c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 --2- 172.21.15.170:0/1333787886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f721c005200 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f72300ccaa0 tx=0x7f722801caf0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f722ed9d640 1 -- 172.21.15.170:0/1333787886 reap_dead start 2024-09-21T02:18:38.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 -- 172.21.15.170:0/1333787886 shutdown_connections 2024-09-21T02:18:38.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 -- 172.21.15.170:0/1333787886 >> 172.21.15.170:0/1333787886 conn(0x7f72300b1c40 msgr2=0x7f72300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:38.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 -- 172.21.15.170:0/1333787886 shutdown_connections 2024-09-21T02:18:38.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 -- 172.21.15.170:0/1333787886 wait complete. 2024-09-21T02:18:38.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 Processor -- start 2024-09-21T02:18:38.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.555+0000 7f7234f62640 1 -- start start 2024-09-21T02:18:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f723014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f723015eda0 0x7f723015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72300ccb00 0x7f72300ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72300b34d0 con 0x7f721c004d50 2024-09-21T02:18:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72300b3350 con 0x7f72300ccb00 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f72300b3650 con 0x7f723015eda0 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f723014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f723014c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53260/0 (socket says 172.21.15.170:53260) 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 -- 172.21.15.170:0/1845324057 learned_addr learned my addr 172.21.15.170:0/1845324057 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722ed9d640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72300ccb00 0x7f72300ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7227fff640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f723015eda0 0x7f723015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 -- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f723015eda0 msgr2=0x7f723015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f723015eda0 0x7f723015d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 -- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72300ccb00 msgr2=0x7f72300ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72300ccb00 0x7f72300ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 -- 172.21.15.170:0/1845324057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72300cd450 con 0x7f721c004d50 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7227fff640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f723015eda0 0x7f723015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f722e59c640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f723014c520 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7220010a10 tx=0x7f7220010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722000ece0 con 0x7f721c004d50 2024-09-21T02:18:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f722000ee80 con 0x7f721c004d50 2024-09-21T02:18:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723014ca60 con 0x7f721c004d50 2024-09-21T02:18:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220005260 con 0x7f721c004d50 2024-09-21T02:18:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723015dfe0 con 0x7f721c004d50 2024-09-21T02:18:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723006fa80 con 0x7f721c004d50 2024-09-21T02:18:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.559+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7220005400 con 0x7f721c004d50 2024-09-21T02:18:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.563+0000 7f7226ffd640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7210073320 0x7f7210075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.563+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f7220095350 con 0x7f721c004d50 2024-09-21T02:18:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.563+0000 7f7227fff640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7210073320 0x7f7210075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.563+0000 7f7227fff640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7210073320 0x7f7210075830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72300c6d40 tx=0x7f72180067b0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.567+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f72300cd450 con 0x7f721c004d50 2024-09-21T02:18:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.727+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72300c5d70 con 0x7f721c004d50 2024-09-21T02:18:38.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.727+0000 7f7226ffd640 1 -- 172.21.15.170:0/1845324057 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16911 (secure 0 0 0) 0x7f72200600f0 con 0x7f721c004d50 2024-09-21T02:18:38.732 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:38.732 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":91,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:35.169237+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:11.529716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:12.787075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6835/2173918761":"2024-09-22T02:18:32.161134+0000","172.21.15.191:6834/2173918761":"2024-09-22T02:18:32.161134+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7210073320 msgr2=0x7f7210075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7210073320 0x7f7210075830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72300c6d40 tx=0x7f72180067b0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 msgr2=0x7f723014c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f723014c520 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7220010a10 tx=0x7f7220010ed0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 shutdown_connections 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7210073320 0x7f7210075830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f723015eda0 0x7f723015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72300ccb00 0x7f72300ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 --2- 172.21.15.170:0/1845324057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f721c004d50 0x7f723014c520 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 >> 172.21.15.170:0/1845324057 conn(0x7f72300b1c40 msgr2=0x7f72300c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 shutdown_connections 2024-09-21T02:18:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.731+0000 7f7234f62640 1 -- 172.21.15.170:0/1845324057 wait complete. 2024-09-21T02:18:38.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 -- 172.21.15.170:0/3623072849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cc8c0 msgr2=0x7ffacc0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 --2- 172.21.15.170:0/3623072849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cc8c0 0x7ffacc0cccd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ffacc072620 tx=0x7ffac401cd80 comp rx=0 tx=0).stop 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 -- 172.21.15.170:0/3623072849 shutdown_connections 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 --2- 172.21.15.170:0/3623072849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cc8c0 0x7ffacc0cccd0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 --2- 172.21.15.170:0/3623072849 >> v2:172.21.15.191:3300/0 conn(0x7ffacc0cb460 0x7ffacc0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 --2- 172.21.15.170:0/3623072849 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc0cf7e0 0x7ffacc070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 -- 172.21.15.170:0/3623072849 >> 172.21.15.170:0/3623072849 conn(0x7ffacc0b1c40 msgr2=0x7ffacc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 -- 172.21.15.170:0/3623072849 shutdown_connections 2024-09-21T02:18:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 -- 172.21.15.170:0/3623072849 wait complete. 2024-09-21T02:18:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 Processor -- start 2024-09-21T02:18:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.907+0000 7ffad305e640 1 -- start start 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 0x7ffacc1f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc0cc8c0 0x7ffacc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc0cf7e0 0x7ffacc1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc0ced60 con 0x7ffacc0cb460 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc0cebe0 con 0x7ffacc0cf7e0 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc0ceee0 con 0x7ffacc0cc8c0 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 0x7ffacc1f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 0x7ffacc1f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53276/0 (socket says 172.21.15.170:53276) 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 -- 172.21.15.170:0/199852195 learned_addr learned my addr 172.21.15.170:0/199852195 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad15fb640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc0cf7e0 0x7ffacc1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffacbfff640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc0cc8c0 0x7ffacc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 -- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc0cc8c0 msgr2=0x7ffacc15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc0cc8c0 0x7ffacc15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 -- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc0cf7e0 msgr2=0x7ffacc1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc0cf7e0 0x7ffacc1599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 -- 172.21.15.170:0/199852195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc1fb420 con 0x7ffacc0cb460 2024-09-21T02:18:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad15fb640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc0cf7e0 0x7ffacc1599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad0dfa640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 0x7ffacc1f7a00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d950 tx=0x7ffabc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc017070 con 0x7ffacc0cb460 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffabc009d20 con 0x7ffacc0cb460 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc1fab80 con 0x7ffacc0cb460 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0106c0 con 0x7ffacc0cb460 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc1fb140 con 0x7ffacc0cb460 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0cccd0 con 0x7ffacc0cb460 2024-09-21T02:18:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7ffabc01a070 con 0x7ffacc0cb460 2024-09-21T02:18:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffac97fa640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffab8073350 0x7ffab8075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7ffabc094620 con 0x7ffacc0cb460 2024-09-21T02:18:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.911+0000 7ffacbfff640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffab8073350 0x7ffab8075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.915+0000 7ffacbfff640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffab8073350 0x7ffab8075860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffac00058d0 tx=0x7ffac0005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:38.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:38.919+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffabc05f3c0 con 0x7ffacc0cb460 2024-09-21T02:18:39.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.131+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffacc155f10 con 0x7ffacc0cb460 2024-09-21T02:18:39.134 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:18:39.131+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:18:39.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.135+0000 7ffac97fa640 1 -- 172.21.15.170:0/199852195 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7ffabc05ecc0 con 0x7ffacc0cb460 2024-09-21T02:18:39.136 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:18:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.135+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffab8073350 msgr2=0x7ffab8075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.135+0000 7ffad305e640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffab8073350 0x7ffab8075860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffac00058d0 tx=0x7ffac0005860 comp rx=0 tx=0).stop 2024-09-21T02:18:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.135+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 msgr2=0x7ffacc1f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.135+0000 7ffad305e640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 0x7ffacc1f7a00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d950 tx=0x7ffabc00de10 comp rx=0 tx=0).stop 2024-09-21T02:18:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 shutdown_connections 2024-09-21T02:18:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffab8073350 0x7ffab8075860 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc0cc8c0 0x7ffacc15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc0cf7e0 0x7ffacc1599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:39.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 --2- 172.21.15.170:0/199852195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc0cb460 0x7ffacc1f7a00 secure :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d950 tx=0x7ffabc00de10 comp rx=0 tx=0).stop 2024-09-21T02:18:39.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 >> 172.21.15.170:0/199852195 conn(0x7ffacc0b1c40 msgr2=0x7ffacc0b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:39.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 shutdown_connections 2024-09-21T02:18:39.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.139+0000 7ffad305e640 1 -- 172.21.15.170:0/199852195 wait complete. 2024-09-21T02:18:39.142 INFO:tasks.ceph.mds.a.smithi170.stderr: -1670> 2024-09-21T02:18:15.352+0000 7f99df9df640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:18:39.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:18:39.199 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:18:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.315+0000 7f1993357640 1 -- 172.21.15.170:0/658016998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1974003a40 msgr2=0x7f1974003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.315+0000 7f1993357640 1 --2- 172.21.15.170:0/658016998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1974003a40 0x7f1974003eb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1988009eb0 tx=0x7f198801d5d0 comp rx=0 tx=0).stop 2024-09-21T02:18:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.315+0000 7f19918f4640 1 -- 172.21.15.170:0/658016998 reap_dead start 2024-09-21T02:18:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.315+0000 7f1993357640 1 -- 172.21.15.170:0/658016998 shutdown_connections 2024-09-21T02:18:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.315+0000 7f1993357640 1 -- 172.21.15.170:0/658016998 >> 172.21.15.170:0/658016998 conn(0x7f198c0d2a30 msgr2=0x7f198c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 -- 172.21.15.170:0/658016998 shutdown_connections 2024-09-21T02:18:39.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 -- 172.21.15.170:0/658016998 wait complete. 2024-09-21T02:18:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 Processor -- start 2024-09-21T02:18:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 -- start start 2024-09-21T02:18:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1974003a40 0x7f198c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 0x7f198c0b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198c1f9eb0 0x7f198c1fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f198c160c70 con 0x7f198c0b7150 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f198c160af0 con 0x7f1974003a40 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f1993357640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f198c160df0 con 0x7f198c1f9eb0 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 0x7f198c0b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 0x7f198c0b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53310/0 (socket says 172.21.15.170:53310) 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 -- 172.21.15.170:0/440500489 learned_addr learned my addr 172.21.15.170:0/440500489 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19910f3640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1974003a40 0x7f198c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19918f4640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198c1f9eb0 0x7f198c1fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 -- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198c1f9eb0 msgr2=0x7f198c1fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198c1f9eb0 0x7f198c1fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 -- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1974003a40 msgr2=0x7f198c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1974003a40 0x7f198c0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:39.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 -- 172.21.15.170:0/440500489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c1fa800 con 0x7f198c0b7150 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19910f3640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1974003a40 0x7f198c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.319+0000 7f19908f2640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 0x7f198c0b7560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f198400e5c0 tx=0x7f198400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984015070 con 0x7f198c0b7150 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f198400ca80 con 0x7f198c0b7150 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c1585b0 con 0x7f198c0b7150 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984011460 con 0x7f198c0b7150 2024-09-21T02:18:39.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c158850 con 0x7f198c0b7150 2024-09-21T02:18:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c15d3e0 con 0x7f198c0b7150 2024-09-21T02:18:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f1984018070 con 0x7f198c0b7150 2024-09-21T02:18:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1981ffb640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1978073320 0x7f1978075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f1984094a60 con 0x7f198c0b7150 2024-09-21T02:18:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f19910f3640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1978073320 0x7f1978075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.323+0000 7f19910f3640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1978073320 0x7f1978075830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19880af000 tx=0x7f198801a370 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:39.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.327+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f198c1fa800 con 0x7f198c0b7150 2024-09-21T02:18:39.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:39.527+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f198c0b7c50 con 0x7f198c0b7150 2024-09-21T02:18:40.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.195+0000 7f1981ffb640 1 -- 172.21.15.170:0/440500489 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7f198405f7b0 con 0x7f198c0b7150 2024-09-21T02:18:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1978073320 msgr2=0x7f1978075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1978073320 0x7f1978075830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19880af000 tx=0x7f198801a370 comp rx=0 tx=0).stop 2024-09-21T02:18:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 msgr2=0x7f198c0b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 0x7f198c0b7560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f198400e5c0 tx=0x7f198400ea80 comp rx=0 tx=0).stop 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 shutdown_connections 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1978073320 0x7f1978075830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f198c1f9eb0 0x7f198c1fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f198c0b7150 0x7f198c0b7560 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 --2- 172.21.15.170:0/440500489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1974003a40 0x7f198c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 >> 172.21.15.170:0/440500489 conn(0x7f198c0d2a30 msgr2=0x7f198c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 shutdown_connections 2024-09-21T02:18:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.199+0000 7f1993357640 1 -- 172.21.15.170:0/440500489 wait complete. 2024-09-21T02:18:40.228 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:18:40.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 -- 172.21.15.170:0/2533437374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c09dc00 msgr2=0x7f579c0b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:40.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 --2- 172.21.15.170:0/2533437374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c09dc00 0x7f579c0b2f40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f578c0098f0 tx=0x7f578c01c7b0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 -- 172.21.15.170:0/2533437374 shutdown_connections 2024-09-21T02:18:40.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 --2- 172.21.15.170:0/2533437374 >> v2:172.21.15.191:3300/0 conn(0x7f579c06ff50 0x7f579c070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 --2- 172.21.15.170:0/2533437374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c0ca720 0x7f579c0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 --2- 172.21.15.170:0/2533437374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c09dc00 0x7f579c0b2f40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 -- 172.21.15.170:0/2533437374 >> 172.21.15.170:0/2533437374 conn(0x7f579c0b1c40 msgr2=0x7f579c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:40.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 -- 172.21.15.170:0/2533437374 shutdown_connections 2024-09-21T02:18:40.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 -- 172.21.15.170:0/2533437374 wait complete. 2024-09-21T02:18:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 Processor -- start 2024-09-21T02:18:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.431+0000 7f57a2679640 1 -- start start 2024-09-21T02:18:40.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c06ff50 0x7f579c09d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:40.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f579c09dc00 0x7f579c0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 0x7f579c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0ce490 con 0x7f579c0ca720 2024-09-21T02:18:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f579c15b340 con 0x7f579c09dc00 2024-09-21T02:18:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f579c0ce610 con 0x7f579c06ff50 2024-09-21T02:18:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 0x7f579c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 0x7f579c0c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53330/0 (socket says 172.21.15.170:53330) 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 -- 172.21.15.170:0/1425358750 learned_addr learned my addr 172.21.15.170:0/1425358750 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f579bfff640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c06ff50 0x7f579c09d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f579b7fe640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f579c09dc00 0x7f579c0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 -- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c06ff50 msgr2=0x7f579c09d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c06ff50 0x7f579c09d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 -- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f579c09dc00 msgr2=0x7f579c0c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f579c09dc00 0x7f579c0c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 -- 172.21.15.170:0/1425358750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c0ca250 con 0x7f579c0ca720 2024-09-21T02:18:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f579bfff640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c06ff50 0x7f579c09d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a0c16640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 0x7f579c0c7180 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f579400cae0 tx=0x7f579400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400ae30 con 0x7f579c0ca720 2024-09-21T02:18:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f579400f560 con 0x7f579c0ca720 2024-09-21T02:18:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c0c91b0 con 0x7f579c0ca720 2024-09-21T02:18:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400f9b0 con 0x7f579c0ca720 2024-09-21T02:18:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c0c9770 con 0x7f579c0ca720 2024-09-21T02:18:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.435+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579c0b2f40 con 0x7f579c0ca720 2024-09-21T02:18:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.439+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f5794004b30 con 0x7f579c0ca720 2024-09-21T02:18:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.439+0000 7f5798ff9640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5788073320 0x7f5788075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.439+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f57940952b0 con 0x7f579c0ca720 2024-09-21T02:18:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.439+0000 7f579bfff640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5788073320 0x7f5788075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.439+0000 7f579bfff640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5788073320 0x7f5788075830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f578c0af000 tx=0x7f578c01b050 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.443+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5794014030 con 0x7f579c0ca720 2024-09-21T02:18:40.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:40.607+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f579c0b0490 con 0x7f579c0ca720 2024-09-21T02:18:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.215+0000 7f5798ff9640 1 -- 172.21.15.170:0/1425358750 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f5794004e20 con 0x7f579c0ca720 2024-09-21T02:18:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5788073320 msgr2=0x7f5788075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5788073320 0x7f5788075830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f578c0af000 tx=0x7f578c01b050 comp rx=0 tx=0).stop 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 msgr2=0x7f579c0c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 0x7f579c0c7180 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f579400cae0 tx=0x7f579400a250 comp rx=0 tx=0).stop 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 shutdown_connections 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5788073320 0x7f5788075830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f579c0ca720 0x7f579c0c7180 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f579c09dc00 0x7f579c0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 --2- 172.21.15.170:0/1425358750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f579c06ff50 0x7f579c09d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 >> 172.21.15.170:0/1425358750 conn(0x7f579c0b1c40 msgr2=0x7f579c074900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:41.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.219+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 shutdown_connections 2024-09-21T02:18:41.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.223+0000 7f57a2679640 1 -- 172.21.15.170:0/1425358750 wait complete. 2024-09-21T02:18:41.249 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:18:41.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 -- 172.21.15.170:0/1871557939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 msgr2=0x7f70740cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:41.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 --2- 172.21.15.170:0/1871557939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740cab30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f70740cf2d0 tx=0x7f706801daa0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 -- 172.21.15.170:0/1871557939 shutdown_connections 2024-09-21T02:18:41.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 --2- 172.21.15.170:0/1871557939 >> v2:172.21.15.191:3300/0 conn(0x7f707406f2d0 0x7f707406f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 --2- 172.21.15.170:0/1871557939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740cab30 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 -- 172.21.15.170:0/1871557939 >> 172.21.15.170:0/1871557939 conn(0x7f70740b1c40 msgr2=0x7f70740b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:41.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 -- 172.21.15.170:0/1871557939 shutdown_connections 2024-09-21T02:18:41.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 -- 172.21.15.170:0/1871557939 wait complete. 2024-09-21T02:18:41.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 Processor -- start 2024-09-21T02:18:41.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.403+0000 7f707b349640 1 -- start start 2024-09-21T02:18:41.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f707406f2d0 0x7f7074155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:41.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f707409dc00 0x7f70740b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:41.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:41.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7074167d70 con 0x7f70740ca720 2024-09-21T02:18:41.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f70740705b0 con 0x7f707406f2d0 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7074167ef0 con 0x7f707409dc00 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53348/0 (socket says 172.21.15.170:53348) 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 -- 172.21.15.170:0/2133646777 learned_addr learned my addr 172.21.15.170:0/2133646777 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70790e5640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f707406f2d0 0x7f7074155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70788e4640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f707409dc00 0x7f70740b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 -- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f707409dc00 msgr2=0x7f70740b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f707409dc00 0x7f70740b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 -- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f707406f2d0 msgr2=0x7f7074155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f707406f2d0 0x7f7074155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 -- 172.21.15.170:0/2133646777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7074155f00 con 0x7f70740ca720 2024-09-21T02:18:41.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f70798e6640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740b4a50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f706400e970 tx=0x7f706400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:41.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064004d30 con 0x7f70740ca720 2024-09-21T02:18:41.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f706400cb20 con 0x7f70740ca720 2024-09-21T02:18:41.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70741fb080 con 0x7f70740ca720 2024-09-21T02:18:41.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064005020 con 0x7f70740ca720 2024-09-21T02:18:41.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707409c500 con 0x7f70740ca720 2024-09-21T02:18:41.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7074156170 con 0x7f70740ca720 2024-09-21T02:18:41.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.407+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f70640051c0 con 0x7f70740ca720 2024-09-21T02:18:41.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.411+0000 7f7071ffb640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7060073320 0x7f7060075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:41.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.411+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5443+0+0 (secure 0 0 0) 0x7f7064094d20 con 0x7f70740ca720 2024-09-21T02:18:41.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.411+0000 7f70790e5640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7060073320 0x7f7060075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:41.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.411+0000 7f70790e5640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7060073320 0x7f7060075830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f706c0015c0 tx=0x7f706c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:41.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.415+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7064016030 con 0x7f70740ca720 2024-09-21T02:18:41.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:41.587+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f707409c2f0 con 0x7f70740ca720 2024-09-21T02:18:42.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.219+0000 7f7071ffb640 1 -- 172.21.15.170:0/2133646777 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f706405fc20 con 0x7f70740ca720 2024-09-21T02:18:42.223 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:18:42.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7060073320 msgr2=0x7f7060075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7060073320 0x7f7060075830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f706c0015c0 tx=0x7f706c009040 comp rx=0 tx=0).stop 2024-09-21T02:18:42.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 msgr2=0x7f70740b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740b4a50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f706400e970 tx=0x7f706400ee30 comp rx=0 tx=0).stop 2024-09-21T02:18:42.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 shutdown_connections 2024-09-21T02:18:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7060073320 0x7f7060075830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f707409dc00 0x7f70740b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70740ca720 0x7f70740b4a50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 --2- 172.21.15.170:0/2133646777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f707406f2d0 0x7f7074155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 >> 172.21.15.170:0/2133646777 conn(0x7f70740b1c40 msgr2=0x7f70740ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:42.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.227+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 shutdown_connections 2024-09-21T02:18:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.231+0000 7f707b349640 1 -- 172.21.15.170:0/2133646777 wait complete. 2024-09-21T02:18:42.259 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 -- 172.21.15.170:0/378827284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180b62e0 msgr2=0x7fea180cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 --2- 172.21.15.170:0/378827284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180b62e0 0x7fea180cb6a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea08009bc0 tx=0x7fea0801cb10 comp rx=0 tx=0).stop 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 -- 172.21.15.170:0/378827284 shutdown_connections 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 --2- 172.21.15.170:0/378827284 >> v2:172.21.15.191:3300/0 conn(0x7fea18070ed0 0x7fea180712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 --2- 172.21.15.170:0/378827284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180b62e0 0x7fea180cb6a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 -- 172.21.15.170:0/378827284 >> 172.21.15.170:0/378827284 conn(0x7fea180cd3c0 msgr2=0x7fea180cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 -- 172.21.15.170:0/378827284 shutdown_connections 2024-09-21T02:18:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.411+0000 7fea1de1d640 1 -- 172.21.15.170:0/378827284 wait complete. 2024-09-21T02:18:42.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 Processor -- start 2024-09-21T02:18:42.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 -- start start 2024-09-21T02:18:42.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18070ed0 0x7fea18153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 0x7fea181617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180cfc50 0x7fea180b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea18168df0 con 0x7fea180b62e0 2024-09-21T02:18:42.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea18168c70 con 0x7fea18070ed0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fea18168f70 con 0x7fea180cfc50 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 0x7fea181617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 0x7fea181617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53360/0 (socket says 172.21.15.170:53360) 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 -- 172.21.15.170:0/2587852544 learned_addr learned my addr 172.21.15.170:0/2587852544 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea177fe640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18070ed0 0x7fea18153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea17fff640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180cfc50 0x7fea180b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 -- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180cfc50 msgr2=0x7fea180b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180cfc50 0x7fea180b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 -- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18070ed0 msgr2=0x7fea18153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18070ed0 0x7fea18153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 -- 172.21.15.170:0/2587852544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea18157760 con 0x7fea180b62e0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea17fff640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180cfc50 0x7fea180b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea16ffd640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 0x7fea181617b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fea08009b10 tx=0x7fea08005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea080b2040 con 0x7fea180b62e0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea08004130 con 0x7fea180b62e0 2024-09-21T02:18:42.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea18155a00 con 0x7fea180b62e0 2024-09-21T02:18:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08004560 con 0x7fea180b62e0 2024-09-21T02:18:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.415+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181571f0 con 0x7fea180b62e0 2024-09-21T02:18:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.419+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea181574d0 con 0x7fea180b62e0 2024-09-21T02:18:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.419+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fea080bd060 con 0x7fea180b62e0 2024-09-21T02:18:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.419+0000 7fea0ffff640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe9fc073350 0x7fe9fc075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.419+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fea0812ddd0 con 0x7fea180b62e0 2024-09-21T02:18:42.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.419+0000 7fea177fe640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe9fc073350 0x7fe9fc075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.419+0000 7fea177fe640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe9fc073350 0x7fe9fc075860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea00001870 tx=0x7fea00001380 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:42.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.423+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fea08016790 con 0x7fea180b62e0 2024-09-21T02:18:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.587+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fea180712e0 con 0x7fea180b62e0 2024-09-21T02:18:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.595+0000 7fea0ffff640 1 -- 172.21.15.170:0/2587852544 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7fea080f8f10 con 0x7fea180b62e0 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe9fc073350 msgr2=0x7fe9fc075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe9fc073350 0x7fe9fc075860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea00001870 tx=0x7fea00001380 comp rx=0 tx=0).stop 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 msgr2=0x7fea181617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 0x7fea181617b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fea08009b10 tx=0x7fea08005a60 comp rx=0 tx=0).stop 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 shutdown_connections 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe9fc073350 0x7fe9fc075860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea180cfc50 0x7fea180b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea180b62e0 0x7fea181617b0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 --2- 172.21.15.170:0/2587852544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18070ed0 0x7fea18153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 >> 172.21.15.170:0/2587852544 conn(0x7fea180cd3c0 msgr2=0x7fea18074040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 shutdown_connections 2024-09-21T02:18:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.599+0000 7fea1de1d640 1 -- 172.21.15.170:0/2587852544 wait complete. 2024-09-21T02:18:42.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10bc90640 1 -- 172.21.15.170:0/3021560918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 msgr2=0x7ff0f0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10bc90640 1 --2- 172.21.15.170:0/3021560918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff0f0004c20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009b60 tx=0x7ff0f801d290 comp rx=0 tx=0).stop 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10a22d640 1 -- 172.21.15.170:0/3021560918 reap_dead start 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10bc90640 1 -- 172.21.15.170:0/3021560918 shutdown_connections 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10bc90640 1 -- 172.21.15.170:0/3021560918 >> 172.21.15.170:0/3021560918 conn(0x7ff1040b1c40 msgr2=0x7ff1040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10bc90640 1 -- 172.21.15.170:0/3021560918 shutdown_connections 2024-09-21T02:18:42.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.771+0000 7ff10bc90640 1 -- 172.21.15.170:0/3021560918 wait complete. 2024-09-21T02:18:42.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 Processor -- start 2024-09-21T02:18:42.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- start start 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff10409cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1041e48b0 0x7ff104159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff10415e190 0x7ff10415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040cb9f0 con 0x7ff0f0004810 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040cb870 con 0x7ff1041e48b0 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff1040cbb70 con 0x7ff10415e190 2024-09-21T02:18:42.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff10409cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff10409cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53382/0 (socket says 172.21.15.170:53382) 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 -- 172.21.15.170:0/2050608703 learned_addr learned my addr 172.21.15.170:0/2050608703 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10a22d640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff10415e190 0x7ff10415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10922b640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1041e48b0 0x7ff104159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 -- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff10415e190 msgr2=0x7ff10415e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff10415e190 0x7ff10415e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 -- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1041e48b0 msgr2=0x7ff104159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1041e48b0 0x7ff104159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 -- 172.21.15.170:0/2050608703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10415eae0 con 0x7ff0f0004810 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff109a2c640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff10409cd90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff100011a80 tx=0x7ff100011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10000fdf0 con 0x7ff0f0004810 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff100016070 con 0x7ff0f0004810 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10409d510 con 0x7ff0f0004810 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000127a0 con 0x7ff0f0004810 2024-09-21T02:18:42.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10409dad0 con 0x7ff0f0004810 2024-09-21T02:18:42.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.775+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1040cd550 con 0x7ff0f0004810 2024-09-21T02:18:42.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.779+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7ff100018030 con 0x7ff0f0004810 2024-09-21T02:18:42.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.779+0000 7ff0f67fc640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0e8073350 0x7ff0e8075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:42.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.779+0000 7ff10922b640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0e8073350 0x7ff0e8075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:42.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.779+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7ff100094ba0 con 0x7ff0f0004810 2024-09-21T02:18:42.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.779+0000 7ff10922b640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0e8073350 0x7ff0e8075860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80ae000 tx=0x7ff0f801a030 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:42.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:42.783+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff10415eae0 con 0x7ff0f0004810 2024-09-21T02:18:43.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.011+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7ff104070510 con 0x7ff0f0004810 2024-09-21T02:18:43.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.235+0000 7ff0f67fc640 1 -- 172.21.15.170:0/2050608703 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v282) ==== 146+0+0 (secure 0 0 0) 0x7ff10005fba0 con 0x7ff0f0004810 2024-09-21T02:18:43.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0e8073350 msgr2=0x7ff0e8075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0e8073350 0x7ff0e8075860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80ae000 tx=0x7ff0f801a030 comp rx=0 tx=0).stop 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 msgr2=0x7ff10409cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff10409cd90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff100011a80 tx=0x7ff100011f40 comp rx=0 tx=0).stop 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 shutdown_connections 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff0e8073350 0x7ff0e8075860 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff10415e190 0x7ff10415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1041e48b0 0x7ff104159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 --2- 172.21.15.170:0/2050608703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0f0004810 0x7ff10409cd90 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 >> 172.21.15.170:0/2050608703 conn(0x7ff1040b1c40 msgr2=0x7ff1040b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 shutdown_connections 2024-09-21T02:18:43.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.239+0000 7ff10bc90640 1 -- 172.21.15.170:0/2050608703 wait complete. 2024-09-21T02:18:43.265 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-21T02:18:43.265 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-21T02:18:43.265 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:18:43.266 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 -- 172.21.15.170:0/3612790659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed8004d50 msgr2=0x7f1ed8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 --2- 172.21.15.170:0/3612790659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed8004d50 0x7f1ed8005160 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400c610 tx=0x7f1ee401d300 comp rx=0 tx=0).stop 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef27ee640 1 -- 172.21.15.170:0/3612790659 reap_dead start 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 -- 172.21.15.170:0/3612790659 shutdown_connections 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 -- 172.21.15.170:0/3612790659 >> 172.21.15.170:0/3612790659 conn(0x7f1eec0b1c40 msgr2=0x7f1eec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 -- 172.21.15.170:0/3612790659 shutdown_connections 2024-09-21T02:18:43.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 -- 172.21.15.170:0/3612790659 wait complete. 2024-09-21T02:18:43.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 Processor -- start 2024-09-21T02:18:43.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.419+0000 7f1ef4251640 1 -- start start 2024-09-21T02:18:43.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ed8004d50 0x7f1eec0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eec0c5160 0x7f1eec0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 0x7f1eec14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec0c39f0 con 0x7f1eec0ad540 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec0cf970 con 0x7f1eec0c5160 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1eec06ff70 con 0x7f1ed8004d50 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 0x7f1eec14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 0x7f1eec14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53392/0 (socket says 172.21.15.170:53392) 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 -- 172.21.15.170:0/4068365103 learned_addr learned my addr 172.21.15.170:0/4068365103 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef1fed640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ed8004d50 0x7f1eec0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef17ec640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eec0c5160 0x7f1eec0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 -- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ed8004d50 msgr2=0x7f1eec0c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ed8004d50 0x7f1eec0c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 -- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eec0c5160 msgr2=0x7f1eec0ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eec0c5160 0x7f1eec0ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 -- 172.21.15.170:0/4068365103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec150420 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef27ee640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 0x7f1eec14d560 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1ee401cdc0 tx=0x7f1ee40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40bd070 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ee4004e40 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec150700 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4003bc0 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec150cc0 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.423+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eec0b34c0 con 0x7f1eec0ad540 2024-09-21T02:18:43.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.427+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f1ee40b5070 con 0x7f1eec0ad540 2024-09-21T02:18:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.427+0000 7f1ee27fc640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1ed0073320 0x7f1ed0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.427+0000 7f1ef1fed640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1ed0073320 0x7f1ed0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.427+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f1ee412e070 con 0x7f1eec0ad540 2024-09-21T02:18:43.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.427+0000 7f1ef1fed640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1ed0073320 0x7f1ed0075830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009ad0 tx=0x7f1ee8005750 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:43.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.431+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1ee40170b0 con 0x7f1eec0ad540 2024-09-21T02:18:43.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.603+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eec0ad950 con 0x7f1eec0ad540 2024-09-21T02:18:43.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.603+0000 7f1ee27fc640 1 -- 172.21.15.170:0/4068365103 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13215 (secure 0 0 0) 0x7f1ee40011d0 con 0x7f1eec0ad540 2024-09-21T02:18:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:43.608 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":94,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:42.221884+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6835/1472234860":"2024-09-22T02:18:39.135755+0000","172.21.15.170:6834/1472234860":"2024-09-22T02:18:39.135755+0000","172.21.15.191:6835/2173918761":"2024-09-22T02:18:32.161134+0000","172.21.15.191:6834/2173918761":"2024-09-22T02:18:32.161134+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1ed0073320 msgr2=0x7f1ed0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1ed0073320 0x7f1ed0075830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009ad0 tx=0x7f1ee8005750 comp rx=0 tx=0).stop 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 msgr2=0x7f1eec14d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 0x7f1eec14d560 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1ee401cdc0 tx=0x7f1ee40014a0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 shutdown_connections 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1ed0073320 0x7f1ed0075830 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1eec0ad540 0x7f1eec14d560 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1eec0c5160 0x7f1eec0ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 --2- 172.21.15.170:0/4068365103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ed8004d50 0x7f1eec0c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 >> 172.21.15.170:0/4068365103 conn(0x7f1eec0b1c40 msgr2=0x7f1eec0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 shutdown_connections 2024-09-21T02:18:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.607+0000 7f1ef4251640 1 -- 172.21.15.170:0/4068365103 wait complete. 2024-09-21T02:18:43.632 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:18:43.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 -- 172.21.15.170:0/2154401659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 msgr2=0x7fcb100744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 --2- 172.21.15.170:0/2154401659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100744b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009840 tx=0x7fcb0c01c7c0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 -- 172.21.15.170:0/2154401659 shutdown_connections 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 --2- 172.21.15.170:0/2154401659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100744b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 --2- 172.21.15.170:0/2154401659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100cf7d0 0x7fcb10073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 -- 172.21.15.170:0/2154401659 >> 172.21.15.170:0/2154401659 conn(0x7fcb100d0170 msgr2=0x7fcb100d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 -- 172.21.15.170:0/2154401659 shutdown_connections 2024-09-21T02:18:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.783+0000 7fcb1803a640 1 -- 172.21.15.170:0/2154401659 wait complete. 2024-09-21T02:18:43.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 Processor -- start 2024-09-21T02:18:43.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 -- start start 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100ce070 0x7fcb101fa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 0x7fcb101fac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10169040 con 0x7fcb100cf7d0 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10168ec0 con 0x7fcb100740a0 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcb101691c0 con 0x7fcb100ce070 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 0x7fcb101fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 0x7fcb101fac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53404/0 (socket says 172.21.15.170:53404) 2024-09-21T02:18:43.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 -- 172.21.15.170:0/804341210 learned_addr learned my addr 172.21.15.170:0/804341210 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb155d5640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100ce070 0x7fcb101fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb15dd6640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 -- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100ce070 msgr2=0x7fcb101fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100ce070 0x7fcb101fa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 -- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 msgr2=0x7fcb100c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 -- 172.21.15.170:0/804341210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb100c6f50 con 0x7fcb100cf7d0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb155d5640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100ce070 0x7fcb101fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb15dd6640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb165d7640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 0x7fcb101fac40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800caa0 tx=0x7fcb0800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08016020 con 0x7fcb100cf7d0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb0800a930 con 0x7fcb100cf7d0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb100c84c0 con 0x7fcb100cf7d0 2024-09-21T02:18:43.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800ad80 con 0x7fcb100cf7d0 2024-09-21T02:18:43.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.787+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb100c8a80 con 0x7fcb100cf7d0 2024-09-21T02:18:43.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.791+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb100744b0 con 0x7fcb100cf7d0 2024-09-21T02:18:43.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.791+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fcb08019020 con 0x7fcb100cf7d0 2024-09-21T02:18:43.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.791+0000 7fcb067fc640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcaf4073320 0x7fcaf4075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:43.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.791+0000 7fcb15dd6640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcaf4073320 0x7fcaf4075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:43.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.791+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fcb08094280 con 0x7fcb100cf7d0 2024-09-21T02:18:43.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.791+0000 7fcb15dd6640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcaf4073320 0x7fcaf4075830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009b90 tx=0x7fcb000091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:43.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.795+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcb08020020 con 0x7fcb100cf7d0 2024-09-21T02:18:43.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.955+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb100cf140 con 0x7fcb100cf7d0 2024-09-21T02:18:43.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.955+0000 7fcb067fc640 1 -- 172.21.15.170:0/804341210 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fcb0805f440 con 0x7fcb100cf7d0 2024-09-21T02:18:43.958 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:18:43.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcaf4073320 msgr2=0x7fcaf4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcaf4073320 0x7fcaf4075830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009b90 tx=0x7fcb000091e0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 msgr2=0x7fcb101fac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 0x7fcb101fac40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800caa0 tx=0x7fcb0800cb90 comp rx=0 tx=0).stop 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 shutdown_connections 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcaf4073320 0x7fcaf4075830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb100ce070 0x7fcb101fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb100cf7d0 0x7fcb101fac40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 --2- 172.21.15.170:0/804341210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb100740a0 0x7fcb100c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 >> 172.21.15.170:0/804341210 conn(0x7fcb100d0170 msgr2=0x7fcb1015d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 shutdown_connections 2024-09-21T02:18:43.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:43.959+0000 7fcb1803a640 1 -- 172.21.15.170:0/804341210 wait complete. 2024-09-21T02:18:43.979 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 -- 172.21.15.170:0/492944339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0cb460 msgr2=0x7f0b1c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 --2- 172.21.15.170:0/492944339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0cb460 0x7f0b1c0cb870 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009900 tx=0x7f0b0c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 -- 172.21.15.170:0/492944339 shutdown_connections 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 --2- 172.21.15.170:0/492944339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1c0ce1c0 0x7f0b1c0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 --2- 172.21.15.170:0/492944339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0cb460 0x7f0b1c0cb870 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 -- 172.21.15.170:0/492944339 >> 172.21.15.170:0/492944339 conn(0x7f0b1c0b1c40 msgr2=0x7f0b1c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 -- 172.21.15.170:0/492944339 shutdown_connections 2024-09-21T02:18:44.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 -- 172.21.15.170:0/492944339 wait complete. 2024-09-21T02:18:44.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 Processor -- start 2024-09-21T02:18:44.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.131+0000 7f0b24305640 1 -- start start 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1c0cb460 0x7f0b1c09d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 0x7f0b1c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1c0cf7e0 0x7f0b1c0b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1690d0 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c168f50 con 0x7f0b1c0cf7e0 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1c169250 con 0x7f0b1c0cb460 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 0x7f0b1c1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 0x7f0b1c1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53432/0 (socket says 172.21.15.170:53432) 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 -- 172.21.15.170:0/727368528 learned_addr learned my addr 172.21.15.170:0/727368528 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b228a2640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1c0cf7e0 0x7f0b1c0b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b220a1640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1c0cb460 0x7f0b1c09d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 -- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1c0cb460 msgr2=0x7f0b1c09d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1c0cb460 0x7f0b1c09d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 -- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1c0cf7e0 msgr2=0x7f0b1c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1c0cf7e0 0x7f0b1c0b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 -- 172.21.15.170:0/727368528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c166110 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b218a0640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 0x7f0b1c1591c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0b0040 tx=0x7f0b0c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0b2040 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b0c005030 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c15a020 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c005440 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c1fd2a0 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c0cb870 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f0b0c0be060 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.135+0000 7f0b12ffd640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b00073320 0x7f0b00075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.139+0000 7f0b220a1640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b00073320 0x7f0b00075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.139+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f0b0c12d740 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.139+0000 7f0b220a1640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b00073320 0x7f0b00075830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b18001650 tx=0x7f0b18009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.143+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b0c0164d0 con 0x7f0b1c0ce1c0 2024-09-21T02:18:44.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:44.367+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f0b1c0b2460 con 0x7f0b1c0ce1c0 2024-09-21T02:18:45.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.243+0000 7f0b12ffd640 1 -- 172.21.15.170:0/727368528 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v284) ==== 152+0+0 (secure 0 0 0) 0x7f0b0c0f8900 con 0x7f0b1c0ce1c0 2024-09-21T02:18:45.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b00073320 msgr2=0x7f0b00075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b00073320 0x7f0b00075830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b18001650 tx=0x7f0b18009040 comp rx=0 tx=0).stop 2024-09-21T02:18:45.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 msgr2=0x7f0b1c1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 0x7f0b1c1591c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0b0040 tx=0x7f0b0c0040c0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 shutdown_connections 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b00073320 0x7f0b00075830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b1c0cf7e0 0x7f0b1c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c0ce1c0 0x7f0b1c1591c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 --2- 172.21.15.170:0/727368528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b1c0cb460 0x7f0b1c09d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 >> 172.21.15.170:0/727368528 conn(0x7f0b1c0b1c40 msgr2=0x7f0b1c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 shutdown_connections 2024-09-21T02:18:45.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.247+0000 7f0b24305640 1 -- 172.21.15.170:0/727368528 wait complete. 2024-09-21T02:18:45.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 -- 172.21.15.170:0/3818216352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200ce510 msgr2=0x7f71200ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 --2- 172.21.15.170:0/3818216352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200ce510 0x7f71200ce920 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f71200caf40 tx=0x7f711001cfd0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 -- 172.21.15.170:0/3818216352 shutdown_connections 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 --2- 172.21.15.170:0/3818216352 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71200cee60 0x7f71200caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 --2- 172.21.15.170:0/3818216352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200ce510 0x7f71200ce920 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 --2- 172.21.15.170:0/3818216352 >> v2:172.21.15.191:3300/0 conn(0x7f7120071bd0 0x7f712015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 -- 172.21.15.170:0/3818216352 >> 172.21.15.170:0/3818216352 conn(0x7f71200cd3c0 msgr2=0x7f71200cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 -- 172.21.15.170:0/3818216352 shutdown_connections 2024-09-21T02:18:45.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 -- 172.21.15.170:0/3818216352 wait complete. 2024-09-21T02:18:45.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 Processor -- start 2024-09-21T02:18:45.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.431+0000 7f71264f1640 1 -- start start 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7120071bd0 0x7f71200c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71200ce510 0x7f71200740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 0x7f712006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71200cc900 con 0x7f71200cee60 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71200cc780 con 0x7f71200ce510 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f71200cca80 con 0x7f7120071bd0 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 0x7f712006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 0x7f712006d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39102/0 (socket says 172.21.15.170:39102) 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 -- 172.21.15.170:0/3362834278 learned_addr learned my addr 172.21.15.170:0/3362834278 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f711f7fe640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71200ce510 0x7f71200740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f711ffff640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7120071bd0 0x7f71200c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 -- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7120071bd0 msgr2=0x7f71200c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7120071bd0 0x7f71200c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 -- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71200ce510 msgr2=0x7f71200740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71200ce510 0x7f71200740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 -- 172.21.15.170:0/3362834278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712014b0c0 con 0x7f71200cee60 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f7124a8e640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 0x7f712006d1d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f711800ca30 tx=0x7f711800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004d30 con 0x7f71200cee60 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7118007b30 con 0x7f71200cee60 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118005020 con 0x7f71200cee60 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7120152260 con 0x7f71200cee60 2024-09-21T02:18:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7120152820 con 0x7f71200cee60 2024-09-21T02:18:45.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.435+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71200ccf30 con 0x7f71200cee60 2024-09-21T02:18:45.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.439+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f71180051c0 con 0x7f71200cee60 2024-09-21T02:18:45.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.439+0000 7f711cff9640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f710c073320 0x7f710c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.439+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f7118094c00 con 0x7f71200cee60 2024-09-21T02:18:45.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.439+0000 7f711ffff640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f710c073320 0x7f710c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.439+0000 7f711ffff640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f710c073320 0x7f710c075830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7114005fd0 tx=0x7f711400c460 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:45.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.443+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f711809b080 con 0x7f71200cee60 2024-09-21T02:18:45.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7120159690 con 0x7f71200cee60 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f711cff9640 1 -- 172.21.15.170:0/3362834278 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7f712014b0c0 con 0x7f71200cee60 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:18:45.617 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:18:45.618 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:18:45.619 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:18:45.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f710c073320 msgr2=0x7f710c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f710c073320 0x7f710c075830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7114005fd0 tx=0x7f711400c460 comp rx=0 tx=0).stop 2024-09-21T02:18:45.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 msgr2=0x7f712006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 0x7f712006d1d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f711800ca30 tx=0x7f711800cef0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 shutdown_connections 2024-09-21T02:18:45.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f710c073320 0x7f710c075830 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71200cee60 0x7f712006d1d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71200ce510 0x7f71200740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 --2- 172.21.15.170:0/3362834278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7120071bd0 0x7f71200c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 >> 172.21.15.170:0/3362834278 conn(0x7f71200cd3c0 msgr2=0x7f71200cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 shutdown_connections 2024-09-21T02:18:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.615+0000 7f71264f1640 1 -- 172.21.15.170:0/3362834278 wait complete. 2024-09-21T02:18:45.638 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:18:45.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 -- 172.21.15.170:0/3682035927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074270 msgr2=0x7f8b000cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 --2- 172.21.15.170:0/3682035927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074270 0x7f8b000cb280 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009840 tx=0x7f8ae801c820 comp rx=0 tx=0).stop 2024-09-21T02:18:45.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 -- 172.21.15.170:0/3682035927 shutdown_connections 2024-09-21T02:18:45.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 --2- 172.21.15.170:0/3682035927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074270 0x7f8b000cb280 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 -- 172.21.15.170:0/3682035927 >> 172.21.15.170:0/3682035927 conn(0x7f8b000d0170 msgr2=0x7f8b000d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:45.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 -- 172.21.15.170:0/3682035927 shutdown_connections 2024-09-21T02:18:45.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 -- 172.21.15.170:0/3682035927 wait complete. 2024-09-21T02:18:45.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 Processor -- start 2024-09-21T02:18:45.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.779+0000 7f8b06338640 1 -- start start 2024-09-21T02:18:45.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b00074270 0x7f8b000c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 0x7f8b0006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b000cf7d0 0x7f8b0006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0016a2b0 con 0x7f8b00074680 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0016a130 con 0x7f8b00074270 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0016a430 con 0x7f8b000cf7d0 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 0x7f8b0006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 0x7f8b0006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39132/0 (socket says 172.21.15.170:39132) 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 -- 172.21.15.170:0/1428689922 learned_addr learned my addr 172.21.15.170:0/1428689922 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8afffff640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b00074270 0x7f8b000c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b048d5640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b000cf7d0 0x7f8b0006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 -- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b000cf7d0 msgr2=0x7f8b0006fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b000cf7d0 0x7f8b0006fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 -- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b00074270 msgr2=0x7f8b000c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b00074270 0x7f8b000c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 -- 172.21.15.170:0/1428689922 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b001fc130 con 0x7f8b00074680 2024-09-21T02:18:45.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b048d5640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b000cf7d0 0x7f8b0006fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:45.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8af7fff640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 0x7f8b0006f7d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8098200 tx=0x7f8ae80a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:45.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80042c0 con 0x7f8b00074680 2024-09-21T02:18:45.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ae8005020 con 0x7f8b00074680 2024-09-21T02:18:45.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b001fc3d0 con 0x7f8b00074680 2024-09-21T02:18:45.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80053d0 con 0x7f8b00074680 2024-09-21T02:18:45.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b001fc990 con 0x7f8b00074680 2024-09-21T02:18:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b00073b60 con 0x7f8b00074680 2024-09-21T02:18:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.783+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f8ae80c1060 con 0x7f8b00074680 2024-09-21T02:18:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.787+0000 7f8afd7fa640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8ae0073320 0x7f8ae0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.787+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f8ae812eb20 con 0x7f8b00074680 2024-09-21T02:18:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.787+0000 7f8afffff640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8ae0073320 0x7f8ae0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.787+0000 7f8afffff640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8ae0073320 0x7f8ae0075830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b810 tx=0x7f8af0008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:45.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.791+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8ae8016410 con 0x7f8b00074680 2024-09-21T02:18:45.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.959+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8b000ad280 con 0x7f8b00074680 2024-09-21T02:18:45.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.963+0000 7f8afd7fa640 1 -- 172.21.15.170:0/1428689922 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f8ae80f9c50 con 0x7f8b00074680 2024-09-21T02:18:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8ae0073320 msgr2=0x7f8ae0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8ae0073320 0x7f8ae0075830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b810 tx=0x7f8af0008040 comp rx=0 tx=0).stop 2024-09-21T02:18:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 msgr2=0x7f8b0006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 0x7f8b0006f7d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8098200 tx=0x7f8ae80a66f0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 shutdown_connections 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8ae0073320 0x7f8ae0075830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b000cf7d0 0x7f8b0006fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b00074680 0x7f8b0006f7d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 --2- 172.21.15.170:0/1428689922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b00074270 0x7f8b000c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 >> 172.21.15.170:0/1428689922 conn(0x7f8b000d0170 msgr2=0x7f8b0015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 shutdown_connections 2024-09-21T02:18:45.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:45.967+0000 7f8b06338640 1 -- 172.21.15.170:0/1428689922 wait complete. 2024-09-21T02:18:45.989 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:18:45.990 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:18:46.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:18:46.156 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:18:46.156 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:46.157 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:46.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:18:46.162 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:46.162 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:46.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:18:46.168 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:18:46.168 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:46.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:18:46.209 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:18:46.209 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:18:46.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:18:46.215 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:18:46.215 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/887882305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 msgr2=0x7f0d78070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/887882305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d78070340 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f0d78070cc0 tx=0x7f0d6801c800 comp rx=0 tx=0).stop 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/887882305 shutdown_connections 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/887882305 >> v2:172.21.15.191:3300/0 conn(0x7f0d780cb460 0x7f0d780cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/887882305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d78070340 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/887882305 >> 172.21.15.170:0/887882305 conn(0x7f0d780b1c40 msgr2=0x7f0d780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:46.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/887882305 shutdown_connections 2024-09-21T02:18:46.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/887882305 wait complete. 2024-09-21T02:18:46.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 Processor -- start 2024-09-21T02:18:46.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.407+0000 7f0d7d1e5640 1 -- start start 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d780cb460 0x7f0d78152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d780cc8c0 0x7f0d780c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d7814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781690d0 con 0x7f0d780cf7e0 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78168f50 con 0x7f0d780cc8c0 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0d78169250 con 0x7f0d780cb460 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d7814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d7814ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39140/0 (socket says 172.21.15.170:39140) 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 -- 172.21.15.170:0/3879888624 learned_addr learned my addr 172.21.15.170:0/3879888624 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d6ffff640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d780cc8c0 0x7f0d780c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d76d9d640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d780cb460 0x7f0d78152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 -- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d780cb460 msgr2=0x7f0d78152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d780cb460 0x7f0d78152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 -- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d780cc8c0 msgr2=0x7f0d780c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d780cc8c0 0x7f0d780c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 -- 172.21.15.170:0/3879888624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78152e40 con 0x7f0d780cf7e0 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7759e640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d7814ee40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000cb40 tx=0x7f0d7000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70016020 con 0x7f0d780cf7e0 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d7000ab60 con 0x7f0d780cf7e0 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70012420 con 0x7f0d780cf7e0 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7809b520 con 0x7f0d780cf7e0 2024-09-21T02:18:46.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d78151e20 con 0x7f0d780cf7e0 2024-09-21T02:18:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.411+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7809b290 con 0x7f0d780cf7e0 2024-09-21T02:18:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.415+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f0d70019020 con 0x7f0d780cf7e0 2024-09-21T02:18:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.415+0000 7f0d6f7fe640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0d58073350 0x7f0d58075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.415+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f0d70094770 con 0x7f0d780cf7e0 2024-09-21T02:18:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.415+0000 7f0d76d9d640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0d58073350 0x7f0d58075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.415+0000 7f0d76d9d640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0d58073350 0x7f0d58075860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d680af000 tx=0x7f0d6801b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:46.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.419+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0d70020020 con 0x7f0d780cf7e0 2024-09-21T02:18:46.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.627+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d78070340 con 0x7f0d780cf7e0 2024-09-21T02:18:46.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.627+0000 7f0d6f7fe640 1 -- 172.21.15.170:0/3879888624 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+3923 (secure 0 0 0) 0x7f0d7005f980 con 0x7f0d780cf7e0 2024-09-21T02:18:46.630 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:46.631 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":64,"btime":"2024-09-21T02:18:40:197702+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3394401447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3394401447},{"type":"v1","addr":"172.21.15.170:6833","nonce":3394401447}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5432,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1759649021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1759649021},{"type":"v1","addr":"172.21.15.191:6801","nonce":1759649021}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":5905,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1805283918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1805283918},{"type":"v1","addr":"172.21.15.191:6835","nonce":1805283918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3240497620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3240497620},{"type":"v1","addr":"172.21.15.170:6835","nonce":3240497620}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[]} 2024-09-21T02:18:46.631 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 64 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0d58073350 msgr2=0x7f0d58075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0d58073350 0x7f0d58075860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d680af000 tx=0x7f0d6801b040 comp rx=0 tx=0).stop 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 msgr2=0x7f0d7814ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d7814ee40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000cb40 tx=0x7f0d7000cc30 comp rx=0 tx=0).stop 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 shutdown_connections 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0d58073350 0x7f0d58075860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d780cf7e0 0x7f0d7814ee40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d780cc8c0 0x7f0d780c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 --2- 172.21.15.170:0/3879888624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d780cb460 0x7f0d78152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 >> 172.21.15.170:0/3879888624 conn(0x7f0d780b1c40 msgr2=0x7f0d780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 shutdown_connections 2024-09-21T02:18:46.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.631+0000 7f0d7d1e5640 1 -- 172.21.15.170:0/3879888624 wait complete. 2024-09-21T02:18:46.652 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:18:46.652 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:18:46.652 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:18:46.653 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:46.653 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:18:46.653 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:18:46.653 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:18:46.653 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:46.653 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:18:46.653 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:18:46.653 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:18:46.654 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:46.654 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:18:46.654 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:18:46.654 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:18:46.654 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:18:46.671 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:18:46.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:18:46.672 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:18:46.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:18:46.673 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:18:46.673 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:18:46.676 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:18:46.676 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:18:46.720 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:18:46.720 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:46.721 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:18:46.721 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:46.722 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:18:46.722 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:46.722 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:18:46.723 DEBUG:teuthology.parallel:result is None 2024-09-21T02:18:46.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:18:46.809 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:18:46.810 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:18:46.818 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:18:46.821 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 -- 172.21.15.170:0/752958534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 msgr2=0x7fe44c0b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 --2- 172.21.15.170:0/752958534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c0b64f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009840 tx=0x7fe43c01c730 comp rx=0 tx=0).stop 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 -- 172.21.15.170:0/752958534 shutdown_connections 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 --2- 172.21.15.170:0/752958534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c0b64f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 --2- 172.21.15.170:0/752958534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe44c0b21a0 0x7fe44c0b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 -- 172.21.15.170:0/752958534 >> 172.21.15.170:0/752958534 conn(0x7fe44c09a9c0 msgr2=0x7fe44c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 -- 172.21.15.170:0/752958534 shutdown_connections 2024-09-21T02:18:46.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 -- 172.21.15.170:0/752958534 wait complete. 2024-09-21T02:18:46.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 Processor -- start 2024-09-21T02:18:46.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 -- start start 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe44c0b21a0 0x7fe44c1fb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.863+0000 7fe4512cb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c1f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe44c0ccce0 0x7fe44c09d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c0b18e0 con 0x7fe44c0b60e0 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c0b1760 con 0x7fe44c0ccce0 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c16a4f0 con 0x7fe44c0b21a0 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c1f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c1f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39186/0 (socket says 172.21.15.170:39186) 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 -- 172.21.15.170:0/3688141071 learned_addr learned my addr 172.21.15.170:0/3688141071 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44affd640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe44c0b21a0 0x7fe44c1fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44b7fe640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe44c0ccce0 0x7fe44c09d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 -- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe44c0b21a0 msgr2=0x7fe44c1fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe44c0b21a0 0x7fe44c1fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 -- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe44c0ccce0 msgr2=0x7fe44c09d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe44c0ccce0 0x7fe44c09d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 -- 172.21.15.170:0/3688141071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c074bb0 con 0x7fe44c0b60e0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44a7fc640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c1f6e50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c430 tx=0x7fe43400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434017070 con 0x7fe44c0b60e0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe434003600 con 0x7fe44c0b60e0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c151fc0 con 0x7fe44c0b60e0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434003a50 con 0x7fe44c0b60e0 2024-09-21T02:18:46.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1f7a70 con 0x7fe44c0b60e0 2024-09-21T02:18:46.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c1f7500 con 0x7fe44c0b60e0 2024-09-21T02:18:46.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fe43401a070 con 0x7fe44c0b60e0 2024-09-21T02:18:46.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4437fe640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe42c073350 0x7fe42c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:46.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44affd640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe42c073350 0x7fe42c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:46.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fe434093870 con 0x7fe44c0b60e0 2024-09-21T02:18:46.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.867+0000 7fe44affd640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe42c073350 0x7fe42c075860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe44c074d80 tx=0x7fe444009630 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:46.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:46.871+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe434021020 con 0x7fe44c0b60e0 2024-09-21T02:18:47.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.043+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe44c0b25b0 con 0x7fe44c0b60e0 2024-09-21T02:18:47.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.271+0000 7fe4437fe640 1 -- 172.21.15.170:0/3688141071 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) ==== 97+0+0 (secure 0 0 0) 0x7fe44c0d1cb0 con 0x7fe44c0b60e0 2024-09-21T02:18:47.273 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe42c073350 msgr2=0x7fe42c075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe42c073350 0x7fe42c075860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe44c074d80 tx=0x7fe444009630 comp rx=0 tx=0).stop 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 msgr2=0x7fe44c1f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c1f6e50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c430 tx=0x7fe43400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 shutdown_connections 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe42c073350 0x7fe42c075860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe44c0ccce0 0x7fe44c09d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe44c0b60e0 0x7fe44c1f6e50 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 --2- 172.21.15.170:0/3688141071 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe44c0b21a0 0x7fe44c1fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 >> 172.21.15.170:0/3688141071 conn(0x7fe44c09a9c0 msgr2=0x7fe44c071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 shutdown_connections 2024-09-21T02:18:47.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.275+0000 7fe4512cb640 1 -- 172.21.15.170:0/3688141071 wait complete. 2024-09-21T02:18:47.303 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:18:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.443+0000 7f2e47020640 1 -- 172.21.15.170:0/1639889733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 msgr2=0x7f2e2c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.443+0000 7f2e47020640 1 --2- 172.21.15.170:0/1639889733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e2c00b9d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2e380016f0 tx=0x7f2e3801c7b0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.443+0000 7f2e47020640 1 -- 172.21.15.170:0/1639889733 shutdown_connections 2024-09-21T02:18:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.443+0000 7f2e47020640 1 --2- 172.21.15.170:0/1639889733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e2c00b9d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- 172.21.15.170:0/1639889733 >> 172.21.15.170:0/1639889733 conn(0x7f2e4009a9c0 msgr2=0x7f2e400d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- 172.21.15.170:0/1639889733 shutdown_connections 2024-09-21T02:18:47.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- 172.21.15.170:0/1639889733 wait complete. 2024-09-21T02:18:47.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 Processor -- start 2024-09-21T02:18:47.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- start start 2024-09-21T02:18:47.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e401555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e40146cd0 0x7f2e401efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e40144230 0x7f2e40144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400cc1e0 con 0x7f2e2c005260 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400cc060 con 0x7f2e40146cd0 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e47020640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2e400cc360 con 0x7f2e40144230 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e401555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e401555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39198/0 (socket says 172.21.15.170:39198) 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 -- 172.21.15.170:0/3723118559 learned_addr learned my addr 172.21.15.170:0/3723118559 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e3ffff640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e40146cd0 0x7f2e401efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e455bd640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e40144230 0x7f2e40144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 -- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e40144230 msgr2=0x7f2e40144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e40144230 0x7f2e40144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 -- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e40146cd0 msgr2=0x7f2e401efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e40146cd0 0x7f2e401efd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 -- 172.21.15.170:0/3723118559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40154550 con 0x7f2e2c005260 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e3ffff640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e40146cd0 0x7f2e401efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:47.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.447+0000 7f2e44dbc640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e401555f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800d560 tx=0x7f2e2800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:47.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28018070 con 0x7f2e2c005260 2024-09-21T02:18:47.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e280052e0 con 0x7f2e2c005260 2024-09-21T02:18:47.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e40154830 con 0x7f2e2c005260 2024-09-21T02:18:47.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28003860 con 0x7f2e2c005260 2024-09-21T02:18:47.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e40154df0 con 0x7f2e2c005260 2024-09-21T02:18:47.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e400cfc40 con 0x7f2e2c005260 2024-09-21T02:18:47.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f2e2801b070 con 0x7f2e2c005260 2024-09-21T02:18:47.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3d7fa640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e20073320 0x7f2e20075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3ffff640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e20073320 0x7f2e20075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f2e28093e10 con 0x7f2e2c005260 2024-09-21T02:18:47.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.451+0000 7f2e3ffff640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e20073320 0x7f2e20075830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2e40071700 tx=0x7f2e30009500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:47.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.455+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2e2805e9f0 con 0x7f2e2c005260 2024-09-21T02:18:47.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.691+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2e400ce9d0 con 0x7f2e2c005260 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.691+0000 7f2e3d7fa640 1 -- 172.21.15.170:0/3723118559 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7f2e28063420 con 0x7f2e2c005260 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:18:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:18:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:18:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:18:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:18:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:18:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:18:47.707 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:18:47.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e20073320 msgr2=0x7f2e20075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e20073320 0x7f2e20075830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2e40071700 tx=0x7f2e30009500 comp rx=0 tx=0).stop 2024-09-21T02:18:47.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 msgr2=0x7f2e401555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e401555f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800d560 tx=0x7f2e2800da20 comp rx=0 tx=0).stop 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 shutdown_connections 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2e20073320 0x7f2e20075830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e40144230 0x7f2e40144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e40146cd0 0x7f2e401efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 --2- 172.21.15.170:0/3723118559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e2c005260 0x7f2e401555f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 >> 172.21.15.170:0/3723118559 conn(0x7f2e4009a9c0 msgr2=0x7f2e4015e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 shutdown_connections 2024-09-21T02:18:47.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.695+0000 7f2e47020640 1 -- 172.21.15.170:0/3723118559 wait complete. 2024-09-21T02:18:47.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:18:47.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 -- 172.21.15.170:0/4169961749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d40b5410 msgr2=0x7f57d406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 --2- 172.21.15.170:0/4169961749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d40b5410 0x7f57d406fa30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009840 tx=0x7f57c401c820 comp rx=0 tx=0).stop 2024-09-21T02:18:47.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 -- 172.21.15.170:0/4169961749 shutdown_connections 2024-09-21T02:18:47.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 --2- 172.21.15.170:0/4169961749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57d40cd210 0x7f57d40b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 --2- 172.21.15.170:0/4169961749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d40b5410 0x7f57d406fa30 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 -- 172.21.15.170:0/4169961749 >> 172.21.15.170:0/4169961749 conn(0x7f57d40b1c40 msgr2=0x7f57d40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:47.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 -- 172.21.15.170:0/4169961749 shutdown_connections 2024-09-21T02:18:47.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.867+0000 7f57da634640 1 -- 172.21.15.170:0/4169961749 wait complete. 2024-09-21T02:18:47.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 Processor -- start 2024-09-21T02:18:47.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- start start 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57d40b2f40 0x7f57d4155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57d415a780 0x7f57d415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 0x7f57d41425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f57d4169fd0 con 0x7f57d41421c0 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f57d4169e50 con 0x7f57d415a780 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f57d416a150 con 0x7f57d40b2f40 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 0x7f57d41425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 0x7f57d41425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39224/0 (socket says 172.21.15.170:39224) 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 -- 172.21.15.170:0/3923120645 learned_addr learned my addr 172.21.15.170:0/3923120645 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d3fff640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57d40b2f40 0x7f57d4155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d37fe640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57d415a780 0x7f57d415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 -- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57d40b2f40 msgr2=0x7f57d4155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57d40b2f40 0x7f57d4155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 -- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57d415a780 msgr2=0x7f57d415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57d415a780 0x7f57d415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 -- 172.21.15.170:0/3923120645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d41555c0 con 0x7f57d41421c0 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d37fe640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57d415a780 0x7f57d415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d8bd1640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 0x7f57d41425d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00cab0 tx=0x7f57cc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00ae20 con 0x7f57d41421c0 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57cc00f560 con 0x7f57d41421c0 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d4073990 con 0x7f57d41421c0 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00f9b0 con 0x7f57d41421c0 2024-09-21T02:18:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d4073f50 con 0x7f57d41421c0 2024-09-21T02:18:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.871+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d415c890 con 0x7f57d41421c0 2024-09-21T02:18:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.875+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f57cc00fb50 con 0x7f57d41421c0 2024-09-21T02:18:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.875+0000 7f57d0ff9640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f57b4073350 0x7f57b4075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.875+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f57cc094c00 con 0x7f57d41421c0 2024-09-21T02:18:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.875+0000 7f57d3fff640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f57b4073350 0x7f57b4075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.875+0000 7f57d3fff640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f57b4073350 0x7f57b4075860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f57c40af000 tx=0x7f57c401b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:47.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:47.879+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f57cc09b080 con 0x7f57d41421c0 2024-09-21T02:18:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f57d406fa30 con 0x7f57d41421c0 2024-09-21T02:18:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57d0ff9640 1 -- 172.21.15.170:0/3923120645 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) ==== 84+0+65 (secure 0 0 0) 0x7f57cc014030 con 0x7f57d41421c0 2024-09-21T02:18:48.057 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:48.057 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:18:48.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:18:48.057 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:18:48.057 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:18:48.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:18:48.058 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f57b4073350 msgr2=0x7f57b4075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f57b4073350 0x7f57b4075860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f57c40af000 tx=0x7f57c401b060 comp rx=0 tx=0).stop 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 msgr2=0x7f57d41425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 0x7f57d41425d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00cab0 tx=0x7f57cc00cf70 comp rx=0 tx=0).stop 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 shutdown_connections 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f57b4073350 0x7f57b4075860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57d41421c0 0x7f57d41425d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57d415a780 0x7f57d415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:48.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 --2- 172.21.15.170:0/3923120645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57d40b2f40 0x7f57d4155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:48.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 >> 172.21.15.170:0/3923120645 conn(0x7f57d40b1c40 msgr2=0x7f57d40c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:48.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 shutdown_connections 2024-09-21T02:18:48.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.055+0000 7f57da634640 1 -- 172.21.15.170:0/3923120645 wait complete. 2024-09-21T02:18:48.080 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:18:48.080 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:18:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 -- 172.21.15.170:0/1592755482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f30003a10 msgr2=0x7f9f30003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 --2- 172.21.15.170:0/1592755482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f30003a10 0x7f9f30003e80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000c610 tx=0x7f9f4001d240 comp rx=0 tx=0).stop 2024-09-21T02:18:48.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f49903640 1 -- 172.21.15.170:0/1592755482 reap_dead start 2024-09-21T02:18:48.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 -- 172.21.15.170:0/1592755482 shutdown_connections 2024-09-21T02:18:48.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 -- 172.21.15.170:0/1592755482 >> 172.21.15.170:0/1592755482 conn(0x7f9f440b1c40 msgr2=0x7f9f440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:48.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 -- 172.21.15.170:0/1592755482 shutdown_connections 2024-09-21T02:18:48.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 -- 172.21.15.170:0/1592755482 wait complete. 2024-09-21T02:18:48.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 Processor -- start 2024-09-21T02:18:48.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.235+0000 7f9f4b366640 1 -- start start 2024-09-21T02:18:48.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f30003a10 0x7f9f4409ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4414f290 0x7f9f440b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 0x7f9f441571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f440af7d0 con 0x7f9f440b1370 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9f440af650 con 0x7f9f4414f290 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9f44167610 con 0x7f9f30003a10 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 0x7f9f441571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 0x7f9f441571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39250/0 (socket says 172.21.15.170:39250) 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 -- 172.21.15.170:0/712204916 learned_addr learned my addr 172.21.15.170:0/712204916 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f48901640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4414f290 0x7f9f440b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49102640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f30003a10 0x7f9f4409ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 -- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f30003a10 msgr2=0x7f9f4409ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f30003a10 0x7f9f4409ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 -- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4414f290 msgr2=0x7f9f440b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:48.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4414f290 0x7f9f440b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 -- 172.21.15.170:0/712204916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f44156a70 con 0x7f9f440b1370 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49903640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 0x7f9f441571f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9f4409dbc0 tx=0x7f9f3c011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004440 con 0x7f9f440b1370 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f3c016070 con 0x7f9f440b1370 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f44156d50 con 0x7f9f440b1370 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00faf0 con 0x7f9f440b1370 2024-09-21T02:18:48.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4409b420 con 0x7f9f440b1370 2024-09-21T02:18:48.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f44158510 con 0x7f9f440b1370 2024-09-21T02:18:48.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f9f3c00fc90 con 0x7f9f440b1370 2024-09-21T02:18:48.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f39ffb640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9f28073350 0x7f9f28075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:48.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f9f3c094a50 con 0x7f9f440b1370 2024-09-21T02:18:48.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.239+0000 7f9f49102640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9f28073350 0x7f9f28075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:48.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.243+0000 7f9f49102640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9f28073350 0x7f9f28075860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f400ac000 tx=0x7f9f4001a080 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:48.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.247+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9f3c022020 con 0x7f9f440b1370 2024-09-21T02:18:48.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:48.415+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9f4409b680 con 0x7f9f440b1370 2024-09-21T02:18:49.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.283+0000 7f9f39ffb640 1 -- 172.21.15.170:0/712204916 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) ==== 138+0+0 (secure 0 0 0) 0x7f9f3c05f560 con 0x7f9f440b1370 2024-09-21T02:18:49.286 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:18:49.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9f28073350 msgr2=0x7f9f28075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9f28073350 0x7f9f28075860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f400ac000 tx=0x7f9f4001a080 comp rx=0 tx=0).stop 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 msgr2=0x7f9f441571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 0x7f9f441571f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9f4409dbc0 tx=0x7f9f3c011be0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 shutdown_connections 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9f28073350 0x7f9f28075860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f440b1370 0x7f9f441571f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4414f290 0x7f9f440b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 --2- 172.21.15.170:0/712204916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f30003a10 0x7f9f4409ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 >> 172.21.15.170:0/712204916 conn(0x7f9f440b1c40 msgr2=0x7f9f441526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 shutdown_connections 2024-09-21T02:18:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.287+0000 7f9f4b366640 1 -- 172.21.15.170:0/712204916 wait complete. 2024-09-21T02:18:49.320 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:18:49.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 -- 172.21.15.170:0/2631894006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 msgr2=0x7f1ac406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:49.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/2631894006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac406f350 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40cca30 tx=0x7f1abc01cd60 comp rx=0 tx=0).stop 2024-09-21T02:18:49.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 -- 172.21.15.170:0/2631894006 shutdown_connections 2024-09-21T02:18:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/2631894006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac406f350 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/2631894006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac40cea80 0x7f1ac40cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/2631894006 >> v2:172.21.15.191:3300/0 conn(0x7f1ac40b2fc0 0x7f1ac40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 -- 172.21.15.170:0/2631894006 >> 172.21.15.170:0/2631894006 conn(0x7f1ac40d0170 msgr2=0x7f1ac40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 -- 172.21.15.170:0/2631894006 shutdown_connections 2024-09-21T02:18:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 -- 172.21.15.170:0/2631894006 wait complete. 2024-09-21T02:18:49.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 Processor -- start 2024-09-21T02:18:49.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.487+0000 7f1ac8b10640 1 -- start start 2024-09-21T02:18:49.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac40b2fc0 0x7f1ac4156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac40cea80 0x7f1ac4157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4070430 con 0x7f1ac406ef40 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40702b0 con 0x7f1ac40cea80 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40705b0 con 0x7f1ac40b2fc0 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac414d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39268/0 (socket says 172.21.15.170:39268) 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 -- 172.21.15.170:0/3883363845 learned_addr learned my addr 172.21.15.170:0/3883363845 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac2d9d640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac40cea80 0x7f1ac4157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac1d9b640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac40b2fc0 0x7f1ac4156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 -- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac40b2fc0 msgr2=0x7f1ac4156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac40b2fc0 0x7f1ac4156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 -- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac40cea80 msgr2=0x7f1ac4157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac40cea80 0x7f1ac4157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 -- 172.21.15.170:0/3883363845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac409a9c0 con 0x7f1ac406ef40 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac2d9d640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac40cea80 0x7f1ac4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac259c640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac414d760 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400d950 tx=0x7f1ab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4017070 con 0x7f1ac406ef40 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ab4009d20 con 0x7f1ac406ef40 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41f7290 con 0x7f1ac406ef40 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40106c0 con 0x7f1ac406ef40 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41f7850 con 0x7f1ac406ef40 2024-09-21T02:18:49.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.491+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac40ade60 con 0x7f1ac406ef40 2024-09-21T02:18:49.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.495+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f1ab401a070 con 0x7f1ac406ef40 2024-09-21T02:18:49.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.495+0000 7f1ab2ffd640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa4073320 0x7f1aa4075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:49.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.495+0000 7f1ac1d9b640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa4073320 0x7f1aa4075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:49.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.495+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5227+0+0 (secure 0 0 0) 0x7f1ab4094800 con 0x7f1ac406ef40 2024-09-21T02:18:49.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.495+0000 7f1ac1d9b640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa4073320 0x7f1aa4075830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ac406a9e0 tx=0x7f1ab8005ca0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:49.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.499+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1ab405f8c0 con 0x7f1ac406ef40 2024-09-21T02:18:49.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:49.671+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1ac40adb80 con 0x7f1ac406ef40 2024-09-21T02:18:50.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.287+0000 7f1ab2ffd640 1 -- 172.21.15.170:0/3883363845 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) ==== 170+0+0 (secure 0 0 0) 0x7f1ab405f1c0 con 0x7f1ac406ef40 2024-09-21T02:18:50.291 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:18:50.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa4073320 msgr2=0x7f1aa4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:50.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa4073320 0x7f1aa4075830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ac406a9e0 tx=0x7f1ab8005ca0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 msgr2=0x7f1ac414d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:50.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac414d760 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400d950 tx=0x7f1ab400de10 comp rx=0 tx=0).stop 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 shutdown_connections 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa4073320 0x7f1aa4075830 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac40b2fc0 0x7f1ac4156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac40cea80 0x7f1ac4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 --2- 172.21.15.170:0/3883363845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac406ef40 0x7f1ac414d760 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 >> 172.21.15.170:0/3883363845 conn(0x7f1ac40d0170 msgr2=0x7f1ac415da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 shutdown_connections 2024-09-21T02:18:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.291+0000 7f1ac8b10640 1 -- 172.21.15.170:0/3883363845 wait complete. 2024-09-21T02:18:50.335 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:18:50.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 -- 172.21.15.170:0/2790273965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb790 msgr2=0x7fa1300ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:50.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2790273965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb790 0x7fa1300ce5b0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa1300c71d0 tx=0x7fa11801db60 comp rx=0 tx=0).stop 2024-09-21T02:18:50.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 -- 172.21.15.170:0/2790273965 shutdown_connections 2024-09-21T02:18:50.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2790273965 >> v2:172.21.15.191:3300/0 conn(0x7fa13006f9a0 0x7fa13006fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2790273965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb790 0x7fa1300ce5b0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2790273965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1300cb380 0x7fa1300ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 -- 172.21.15.170:0/2790273965 >> 172.21.15.170:0/2790273965 conn(0x7fa1300d0170 msgr2=0x7fa1300d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:50.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 -- 172.21.15.170:0/2790273965 shutdown_connections 2024-09-21T02:18:50.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.491+0000 7fa1356e9640 1 -- 172.21.15.170:0/2790273965 wait complete. 2024-09-21T02:18:50.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 Processor -- start 2024-09-21T02:18:50.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 -- start start 2024-09-21T02:18:50.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa13006f9a0 0x7fa13009b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 0x7fa1300b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1300cb790 0x7fa130158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa13015fc60 con 0x7fa1300cb380 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa13015fae0 con 0x7fa1300cb790 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa13015fde0 con 0x7fa13006f9a0 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 0x7fa1300b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 0x7fa1300b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39292/0 (socket says 172.21.15.170:39292) 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 -- 172.21.15.170:0/2296566048 learned_addr learned my addr 172.21.15.170:0/2296566048 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12f7fe640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1300cb790 0x7fa130158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12effd640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa13006f9a0 0x7fa13009b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 -- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa13006f9a0 msgr2=0x7fa13009b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa13006f9a0 0x7fa13009b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 -- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1300cb790 msgr2=0x7fa130158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1300cb790 0x7fa130158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:50.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 -- 172.21.15.170:0/2296566048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1300b2250 con 0x7fa1300cb380 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12f7fe640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1300cb790 0x7fa130158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12effd640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa13006f9a0 0x7fa13009b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa12e7fc640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 0x7fa1300b1210 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa1180b1040 tx=0x7fa118005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b3040 con 0x7fa1300cb380 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa118005020 con 0x7fa1300cb380 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118005430 con 0x7fa1300cb380 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13009ab80 con 0x7fa1300cb380 2024-09-21T02:18:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.495+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13009b140 con 0x7fa1300cb380 2024-09-21T02:18:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.499+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13006fdb0 con 0x7fa1300cb380 2024-09-21T02:18:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.499+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fa1180c0060 con 0x7fa1300cb380 2024-09-21T02:18:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.499+0000 7fa1277fe640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa110073320 0x7fa110075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.499+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5609+0+0 (secure 0 0 0) 0x7fa118130c90 con 0x7fa1300cb380 2024-09-21T02:18:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.499+0000 7fa12effd640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa110073320 0x7fa110075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.499+0000 7fa12effd640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa110073320 0x7fa110075830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa128005c50 tx=0x7fa12800a450 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.503+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa1180177e0 con 0x7fa1300cb380 2024-09-21T02:18:50.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:50.711+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa1300cac80 con 0x7fa1300cb380 2024-09-21T02:18:51.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.359+0000 7fa1277fe640 1 -- 172.21.15.170:0/2296566048 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) ==== 174+0+0 (secure 0 0 0) 0x7fa1180fbc10 con 0x7fa1300cb380 2024-09-21T02:18:51.365 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 16 and data pool 17 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa110073320 msgr2=0x7fa110075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa110073320 0x7fa110075830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa128005c50 tx=0x7fa12800a450 comp rx=0 tx=0).stop 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 msgr2=0x7fa1300b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 0x7fa1300b1210 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa1180b1040 tx=0x7fa118005e00 comp rx=0 tx=0).stop 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 shutdown_connections 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa110073320 0x7fa110075830 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1300cb790 0x7fa130158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1300cb380 0x7fa1300b1210 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 --2- 172.21.15.170:0/2296566048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa13006f9a0 0x7fa13009b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.379+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 >> 172.21.15.170:0/2296566048 conn(0x7fa1300d0170 msgr2=0x7fa1300acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:51.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.391+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 shutdown_connections 2024-09-21T02:18:51.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.391+0000 7fa1356e9640 1 -- 172.21.15.170:0/2296566048 wait complete. 2024-09-21T02:18:51.414 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a3656c640 1 -- 172.21.15.170:0/3066009251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a1c003a10 msgr2=0x7f3a1c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a3656c640 1 --2- 172.21.15.170:0/3066009251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a1c003a10 0x7f3a1c003e80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3a300d1960 tx=0x7f3a2001cc60 comp rx=0 tx=0).stop 2024-09-21T02:18:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a34b09640 1 -- 172.21.15.170:0/3066009251 reap_dead start 2024-09-21T02:18:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a3656c640 1 -- 172.21.15.170:0/3066009251 shutdown_connections 2024-09-21T02:18:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a3656c640 1 -- 172.21.15.170:0/3066009251 >> 172.21.15.170:0/3066009251 conn(0x7f3a300cd3c0 msgr2=0x7f3a300cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a3656c640 1 -- 172.21.15.170:0/3066009251 shutdown_connections 2024-09-21T02:18:51.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.563+0000 7f3a3656c640 1 -- 172.21.15.170:0/3066009251 wait complete. 2024-09-21T02:18:51.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 Processor -- start 2024-09-21T02:18:51.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 -- start start 2024-09-21T02:18:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 0x7f3a300b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a3014c8c0 0x7f3a300b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a30154130 0x7f3a30154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a30166c50 con 0x7f3a3014c4b0 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3a30166dd0 con 0x7f3a3014c8c0 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3a30166f50 con 0x7f3a30154130 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 0x7f3a300b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 0x7f3a300b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39314/0 (socket says 172.21.15.170:39314) 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 -- 172.21.15.170:0/856318304 learned_addr learned my addr 172.21.15.170:0/856318304 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a277fe640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a3014c8c0 0x7f3a300b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 -- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a30154130 msgr2=0x7f3a30154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a34b09640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a30154130 0x7f3a30154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a30154130 0x7f3a30154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 -- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a3014c8c0 msgr2=0x7f3a300b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a3014c8c0 0x7f3a300b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 -- 172.21.15.170:0/856318304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a30155470 con 0x7f3a3014c4b0 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a277fe640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a3014c8c0 0x7f3a300b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a34b09640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a30154130 0x7f3a30154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2ffff640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 0x7f3a300b42e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3a20001ec0 tx=0x7f3a200056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a200ab040 con 0x7f3a3014c4b0 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a200be070 con 0x7f3a3014c4b0 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a301f9770 con 0x7f3a3014c4b0 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20004a20 con 0x7f3a3014c4b0 2024-09-21T02:18:51.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.567+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a301f9cf0 con 0x7f3a3014c4b0 2024-09-21T02:18:51.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.571+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a300cc5c0 con 0x7f3a3014c4b0 2024-09-21T02:18:51.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.571+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f3a200b6070 con 0x7f3a3014c4b0 2024-09-21T02:18:51.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.571+0000 7f3a2d7fa640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a10073320 0x7f3a10075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.571+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f3a2012fc80 con 0x7f3a3014c4b0 2024-09-21T02:18:51.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.571+0000 7f3a277fe640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a10073320 0x7f3a10075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.571+0000 7f3a277fe640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a10073320 0x7f3a10075830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a300725b0 tx=0x7f3a180014f0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:51.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.575+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3a20016970 con 0x7f3a3014c4b0 2024-09-21T02:18:51.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.751+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a30165490 con 0x7f3a3014c4b0 2024-09-21T02:18:51.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.751+0000 7f3a2d7fa640 1 -- 172.21.15.170:0/856318304 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16804 (secure 0 0 0) 0x7f3a200fabf0 con 0x7f3a3014c4b0 2024-09-21T02:18:51.756 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:51.756 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:50.719637+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:48.430867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:49.686964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:51.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a10073320 msgr2=0x7f3a10075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a10073320 0x7f3a10075830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a300725b0 tx=0x7f3a180014f0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 msgr2=0x7f3a300b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 0x7f3a300b42e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3a20001ec0 tx=0x7f3a200056e0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 shutdown_connections 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a10073320 0x7f3a10075830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3a30154130 0x7f3a30154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a3014c8c0 0x7f3a300b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 --2- 172.21.15.170:0/856318304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a3014c4b0 0x7f3a300b42e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 >> 172.21.15.170:0/856318304 conn(0x7f3a300cd3c0 msgr2=0x7f3a300cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 shutdown_connections 2024-09-21T02:18:51.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.755+0000 7f3a3656c640 1 -- 172.21.15.170:0/856318304 wait complete. 2024-09-21T02:18:51.779 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f5402b53640 1 -- 172.21.15.170:0/3986538244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 msgr2=0x7f53e8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f5402b53640 1 --2- 172.21.15.170:0/3986538244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53e8003e80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c2e0 tx=0x7f53f401cf90 comp rx=0 tx=0).stop 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f54010f0640 1 -- 172.21.15.170:0/3986538244 reap_dead start 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f5402b53640 1 -- 172.21.15.170:0/3986538244 shutdown_connections 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f5402b53640 1 -- 172.21.15.170:0/3986538244 >> 172.21.15.170:0/3986538244 conn(0x7f53fc0cd3c0 msgr2=0x7f53fc0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f5402b53640 1 -- 172.21.15.170:0/3986538244 shutdown_connections 2024-09-21T02:18:51.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.927+0000 7f5402b53640 1 -- 172.21.15.170:0/3986538244 wait complete. 2024-09-21T02:18:51.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 Processor -- start 2024-09-21T02:18:51.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 -- start start 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53fc0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53fc156a80 0x7f53fc154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53fc154190 0x7f53fc1fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc15d0c0 con 0x7f53e8003a10 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc15cf40 con 0x7f53fc156a80 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f53fc15d240 con 0x7f53fc154190 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53fc0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53fc0c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39328/0 (socket says 172.21.15.170:39328) 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 -- 172.21.15.170:0/2219694979 learned_addr learned my addr 172.21.15.170:0/2219694979 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f53fbfff640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53fc156a80 0x7f53fc154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54010f0640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53fc154190 0x7f53fc1fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 -- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53fc154190 msgr2=0x7f53fc1fa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53fc154190 0x7f53fc1fa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 -- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53fc156a80 msgr2=0x7f53fc154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53fc156a80 0x7f53fc154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 -- 172.21.15.170:0/2219694979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc0c6d00 con 0x7f53e8003a10 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f53fbfff640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53fc156a80 0x7f53fc154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f54008ef640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53fc0c67c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006040 tx=0x7f53f4006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40065e0 con 0x7f53e8003a10 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f40036f0 con 0x7f53e8003a10 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc1564d0 con 0x7f53e8003a10 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4003b00 con 0x7f53e8003a10 2024-09-21T02:18:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.931+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc154fc0 con 0x7f53e8003a10 2024-09-21T02:18:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.935+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc0af490 con 0x7f53e8003a10 2024-09-21T02:18:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.935+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f53f4003ca0 con 0x7f53e8003a10 2024-09-21T02:18:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.935+0000 7f53f97fa640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f53dc073320 0x7f53dc075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.935+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f53f412eff0 con 0x7f53e8003a10 2024-09-21T02:18:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.935+0000 7f53fbfff640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f53dc073320 0x7f53dc075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:51.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.935+0000 7f53fbfff640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f53dc073320 0x7f53dc075830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009420 tx=0x7f53e4009500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:51.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:51.939+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f53f40c8080 con 0x7f53e8003a10 2024-09-21T02:18:52.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.107+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53fc0b0710 con 0x7f53e8003a10 2024-09-21T02:18:52.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f53f97fa640 1 -- 172.21.15.170:0/2219694979 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16804 (secure 0 0 0) 0x7f53f4016d70 con 0x7f53e8003a10 2024-09-21T02:18:52.113 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:52.113 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:50.719637+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:48.430867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:49.686964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f53dc073320 msgr2=0x7f53dc075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f53dc073320 0x7f53dc075830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009420 tx=0x7f53e4009500 comp rx=0 tx=0).stop 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 msgr2=0x7f53fc0c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53fc0c67c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f53f4006040 tx=0x7f53f4006d00 comp rx=0 tx=0).stop 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 shutdown_connections 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f53dc073320 0x7f53dc075830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53fc154190 0x7f53fc1fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53fc156a80 0x7f53fc154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 --2- 172.21.15.170:0/2219694979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e8003a10 0x7f53fc0c67c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 >> 172.21.15.170:0/2219694979 conn(0x7f53fc0cd3c0 msgr2=0x7f53fc0c7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:52.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 shutdown_connections 2024-09-21T02:18:52.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.111+0000 7f5402b53640 1 -- 172.21.15.170:0/2219694979 wait complete. 2024-09-21T02:18:52.135 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 -- 172.21.15.170:0/2632476428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff408074270 msgr2=0x7ff4080cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 --2- 172.21.15.170:0/2632476428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff408074270 0x7ff4080cb280 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009840 tx=0x7ff3fc01c7c0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 -- 172.21.15.170:0/2632476428 shutdown_connections 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 --2- 172.21.15.170:0/2632476428 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 0x7ff4080cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 --2- 172.21.15.170:0/2632476428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff408074270 0x7ff4080cb280 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 -- 172.21.15.170:0/2632476428 >> 172.21.15.170:0/2632476428 conn(0x7ff4080d0170 msgr2=0x7ff4080d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 -- 172.21.15.170:0/2632476428 shutdown_connections 2024-09-21T02:18:52.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 -- 172.21.15.170:0/2632476428 wait complete. 2024-09-21T02:18:52.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 Processor -- start 2024-09-21T02:18:52.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.295+0000 7ff40d374640 1 -- start start 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff408074270 0x7ff4081f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 0x7ff4080afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 0x7ff4080b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff408167030 con 0x7ff4080cf7d0 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff408166eb0 con 0x7ff408074270 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff4081671b0 con 0x7ff408074680 2024-09-21T02:18:52.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 0x7ff4080b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 0x7ff4080b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39350/0 (socket says 172.21.15.170:39350) 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 -- 172.21.15.170:0/2658824526 learned_addr learned my addr 172.21.15.170:0/2658824526 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff406ffd640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff408074270 0x7ff4081f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4067fc640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 0x7ff4080afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 -- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 msgr2=0x7ff4080afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 0x7ff4080afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 -- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff408074270 msgr2=0x7ff4081f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff408074270 0x7ff4081f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 -- 172.21.15.170:0/2658824526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4081f6fe0 con 0x7ff4080cf7d0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4067fc640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 0x7ff4080afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff4077fe640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 0x7ff4080b1640 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff40000e9a0 tx=0x7ff40000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400004d30 con 0x7ff4080cf7d0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff40000cb20 con 0x7ff4080cf7d0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4081f7ce0 con 0x7ff4080cf7d0 2024-09-21T02:18:52.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400005020 con 0x7ff4080cf7d0 2024-09-21T02:18:52.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4080b27a0 con 0x7ff4080cf7d0 2024-09-21T02:18:52.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff408073b60 con 0x7ff4080cf7d0 2024-09-21T02:18:52.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.299+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7ff4000051c0 con 0x7ff4080cf7d0 2024-09-21T02:18:52.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.303+0000 7ff3ef7fe640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff3f4073350 0x7ff3f4075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:52.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.303+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff4000951e0 con 0x7ff4080cf7d0 2024-09-21T02:18:52.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.303+0000 7ff406ffd640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff3f4073350 0x7ff3f4075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:52.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.303+0000 7ff406ffd640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff3f4073350 0x7ff3f4075860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80046c0 tx=0x7ff3f8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:52.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.307+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff400016030 con 0x7ff4080cf7d0 2024-09-21T02:18:52.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:52.515+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff4080ada40 con 0x7ff4080cf7d0 2024-09-21T02:18:53.281 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-09-21T02:18:53.281+0000 7f1754b4d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:18:53.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.375+0000 7ff3ef7fe640 1 -- 172.21.15.170:0/2658824526 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7ff4000600c0 con 0x7ff4080cf7d0 2024-09-21T02:18:53.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff3f4073350 msgr2=0x7ff3f4075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:53.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff3f4073350 0x7ff3f4075860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80046c0 tx=0x7ff3f8009040 comp rx=0 tx=0).stop 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 msgr2=0x7ff4080b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 0x7ff4080b1640 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff40000e9a0 tx=0x7ff40000ee60 comp rx=0 tx=0).stop 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 shutdown_connections 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff3f4073350 0x7ff3f4075860 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff408074680 0x7ff4080afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4080cf7d0 0x7ff4080b1640 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 --2- 172.21.15.170:0/2658824526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff408074270 0x7ff4081f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 >> 172.21.15.170:0/2658824526 conn(0x7ff4080d0170 msgr2=0x7ff4080ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 shutdown_connections 2024-09-21T02:18:53.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.379+0000 7ff40d374640 1 -- 172.21.15.170:0/2658824526 wait complete. 2024-09-21T02:18:53.404 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:18:53.404 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 -- 172.21.15.170:0/793741408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 msgr2=0x7f453c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 --2- 172.21.15.170:0/793741408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c0cb280 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f452c009840 tx=0x7f452c01c820 comp rx=0 tx=0).stop 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 -- 172.21.15.170:0/793741408 shutdown_connections 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 --2- 172.21.15.170:0/793741408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c0cb280 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 -- 172.21.15.170:0/793741408 >> 172.21.15.170:0/793741408 conn(0x7f453c0d0170 msgr2=0x7f453c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 -- 172.21.15.170:0/793741408 shutdown_connections 2024-09-21T02:18:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.555+0000 7f4543312640 1 -- 172.21.15.170:0/793741408 wait complete. 2024-09-21T02:18:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 Processor -- start 2024-09-21T02:18:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 -- start start 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c1f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f453c074680 0x7f453c166d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f453c0cf7d0 0x7f453c157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f453c16a2b0 con 0x7f453c074270 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f453c16a130 con 0x7f453c0cf7d0 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f453c16a430 con 0x7f453c074680 2024-09-21T02:18:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c1f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c1f9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39368/0 (socket says 172.21.15.170:39368) 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 -- 172.21.15.170:0/1140166566 learned_addr learned my addr 172.21.15.170:0/1140166566 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45418af640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f453c0cf7d0 0x7f453c157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45408ad640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f453c074680 0x7f453c166d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 -- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f453c074680 msgr2=0x7f453c166d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f453c074680 0x7f453c166d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 -- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f453c0cf7d0 msgr2=0x7f453c157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f453c0cf7d0 0x7f453c157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 -- 172.21.15.170:0/1140166566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c156960 con 0x7f453c074270 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45408ad640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f453c074680 0x7f453c166d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f45410ae640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c1f9070 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f453800c3f0 tx=0x7f453800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538017070 con 0x7f453c074270 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45380053c0 con 0x7f453c074270 2024-09-21T02:18:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538003860 con 0x7f453c074270 2024-09-21T02:18:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c156c40 con 0x7f453c074270 2024-09-21T02:18:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.559+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c157200 con 0x7f453c074270 2024-09-21T02:18:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.563+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c073b60 con 0x7f453c074270 2024-09-21T02:18:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.563+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f453801a070 con 0x7f453c074270 2024-09-21T02:18:53.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.563+0000 7f4531ffb640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4520073320 0x7f4520075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:53.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.563+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4538093ab0 con 0x7f453c074270 2024-09-21T02:18:53.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.563+0000 7f45408ad640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4520073320 0x7f4520075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:53.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.563+0000 7f45408ad640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4520073320 0x7f4520075830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f452c0af000 tx=0x7f452c01b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.567+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4538021020 con 0x7f453c074270 2024-09-21T02:18:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:53.775+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f453c167260 con 0x7f453c074270 2024-09-21T02:18:54.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.375+0000 7f4531ffb640 1 -- 172.21.15.170:0/1140166566 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7f453805ea20 con 0x7f453c074270 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4520073320 msgr2=0x7f4520075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4520073320 0x7f4520075830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f452c0af000 tx=0x7f452c01b060 comp rx=0 tx=0).stop 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 msgr2=0x7f453c1f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c1f9070 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f453800c3f0 tx=0x7f453800c8b0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 shutdown_connections 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4520073320 0x7f4520075830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f453c074680 0x7f453c166d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f453c0cf7d0 0x7f453c157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 --2- 172.21.15.170:0/1140166566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f453c074270 0x7f453c1f9070 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 >> 172.21.15.170:0/1140166566 conn(0x7f453c0d0170 msgr2=0x7f453c158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:54.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 shutdown_connections 2024-09-21T02:18:54.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.379+0000 7f4543312640 1 -- 172.21.15.170:0/1140166566 wait complete. 2024-09-21T02:18:54.409 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:18:54.409 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:54.436 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:18:54.457 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 -- 172.21.15.170:0/1489311303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 msgr2=0x7f7c40070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/1489311303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c40070340 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009840 tx=0x7f7c3001c820 comp rx=0 tx=0).stop 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 -- 172.21.15.170:0/1489311303 shutdown_connections 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/1489311303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c40070340 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 -- 172.21.15.170:0/1489311303 >> 172.21.15.170:0/1489311303 conn(0x7f7c400b1c40 msgr2=0x7f7c400b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 -- 172.21.15.170:0/1489311303 shutdown_connections 2024-09-21T02:18:54.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.563+0000 7f7c46d2f640 1 -- 172.21.15.170:0/1489311303 wait complete. 2024-09-21T02:18:54.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 Processor -- start 2024-09-21T02:18:54.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- start start 2024-09-21T02:18:54.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c400cb460 0x7f7c400c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c400ce1c0 0x7f7c4016aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c401fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4006fcf0 con 0x7f7c400cf7e0 2024-09-21T02:18:54.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4006fb70 con 0x7f7c400cb460 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4016b760 con 0x7f7c400ce1c0 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c401fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c401fcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39382/0 (socket says 172.21.15.170:39382) 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 -- 172.21.15.170:0/750383766 learned_addr learned my addr 172.21.15.170:0/750383766 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c44acb640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c400cb460 0x7f7c400c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 -- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c400ce1c0 msgr2=0x7f7c4016aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c400ce1c0 0x7f7c4016aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 -- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c400cb460 msgr2=0x7f7c400c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c400cb460 0x7f7c400c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 -- 172.21.15.170:0/750383766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c401545b0 con 0x7f7c400cf7e0 2024-09-21T02:18:54.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c44acb640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c400cb460 0x7f7c400c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c452cc640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c401fcac0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800cad0 tx=0x7f7c3800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38016020 con 0x7f7c400cf7e0 2024-09-21T02:18:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c3800aae0 con 0x7f7c400cf7e0 2024-09-21T02:18:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c40154890 con 0x7f7c400cf7e0 2024-09-21T02:18:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800ce10 con 0x7f7c400cf7e0 2024-09-21T02:18:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c400c9d00 con 0x7f7c400cf7e0 2024-09-21T02:18:54.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.567+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c40154320 con 0x7f7c400cf7e0 2024-09-21T02:18:54.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.571+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7c38019020 con 0x7f7c400cf7e0 2024-09-21T02:18:54.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.571+0000 7f7c3d7fa640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c2c073350 0x7f7c2c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.571+0000 7f7c44acb640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c2c073350 0x7f7c2c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.571+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f7c380943b0 con 0x7f7c400cf7e0 2024-09-21T02:18:54.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.571+0000 7f7c44acb640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c2c073350 0x7f7c2c075860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7c300af000 tx=0x7f7c3001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:54.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.575+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7c38020030 con 0x7f7c400cf7e0 2024-09-21T02:18:54.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.783+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c40070340 con 0x7f7c400cf7e0 2024-09-21T02:18:54.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.783+0000 7f7c3d7fa640 1 -- 172.21.15.170:0/750383766 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5237 (secure 0 0 0) 0x7f7c3805f1b0 con 0x7f7c400cf7e0 2024-09-21T02:18:54.786 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:54.786 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":71,"btime":"2024-09-21T02:18:54:379135+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:50.719554+0000","modified":"2024-09-21T02:18:53.780500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6801/1778665511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1778665511},{"type":"v1","addr":"172.21.15.191:6801","nonce":1778665511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7107,"qdb_cluster":[7107]},"id":8}]} 2024-09-21T02:18:54.787 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 71 2024-09-21T02:18:54.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c2c073350 msgr2=0x7f7c2c075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c2c073350 0x7f7c2c075860 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7c300af000 tx=0x7f7c3001b060 comp rx=0 tx=0).stop 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 msgr2=0x7f7c401fcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c401fcac0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800cad0 tx=0x7f7c3800cbc0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 shutdown_connections 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c2c073350 0x7f7c2c075860 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c400ce1c0 0x7f7c4016aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c400cf7e0 0x7f7c401fcac0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 --2- 172.21.15.170:0/750383766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c400cb460 0x7f7c400c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 >> 172.21.15.170:0/750383766 conn(0x7f7c400b1c40 msgr2=0x7f7c40075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 shutdown_connections 2024-09-21T02:18:54.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.787+0000 7f7c46d2f640 1 -- 172.21.15.170:0/750383766 wait complete. 2024-09-21T02:18:54.807 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:18:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 -- 172.21.15.170:0/3351467134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb015f990 msgr2=0x7f5bb0070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/3351467134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb015f990 0x7f5bb0070030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009900 tx=0x7f5bac01c8b0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 -- 172.21.15.170:0/3351467134 shutdown_connections 2024-09-21T02:18:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/3351467134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bb00cadf0 0x7f5bb00cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/3351467134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb015f990 0x7f5bb0070030 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 -- 172.21.15.170:0/3351467134 >> 172.21.15.170:0/3351467134 conn(0x7f5bb00b1c40 msgr2=0x7f5bb00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 -- 172.21.15.170:0/3351467134 shutdown_connections 2024-09-21T02:18:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.947+0000 7f5bb86cd640 1 -- 172.21.15.170:0/3351467134 wait complete. 2024-09-21T02:18:54.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 Processor -- start 2024-09-21T02:18:54.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 -- start start 2024-09-21T02:18:54.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 0x7f5bb0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb00cadf0 0x7f5bb00ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bb015f990 0x7f5bb00ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0167ff0 con 0x7f5bb0070570 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb00cede0 con 0x7f5bb00cadf0 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb86cd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb0168170 con 0x7f5bb015f990 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 0x7f5bb0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 0x7f5bb0152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39394/0 (socket says 172.21.15.170:39394) 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 -- 172.21.15.170:0/693416778 learned_addr learned my addr 172.21.15.170:0/693416778 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb5c68640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb00cadf0 0x7f5bb00ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6c6a640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bb015f990 0x7f5bb00ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 -- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bb015f990 msgr2=0x7f5bb00ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bb015f990 0x7f5bb00ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 -- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb00cadf0 msgr2=0x7f5bb00ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:54.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb00cadf0 0x7f5bb00ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb6469640 1 -- 172.21.15.170:0/693416778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb01574b0 con 0x7f5bb0070570 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.951+0000 7f5bb5c68640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb00cadf0 0x7f5bb00ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5bb6469640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 0x7f5bb0152510 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009850 tx=0x7f5bac0a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0b2040 con 0x7f5bb0070570 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bac0043b0 con 0x7f5bb0070570 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac005280 con 0x7f5bb0070570 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb0157790 con 0x7f5bb0070570 2024-09-21T02:18:54.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb015a2b0 con 0x7f5bb0070570 2024-09-21T02:18:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb00cb200 con 0x7f5bb0070570 2024-09-21T02:18:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f5bac005420 con 0x7f5bb0070570 2024-09-21T02:18:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5ba6ffd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5b94073350 0x7f5b94075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5bb5c68640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5b94073350 0x7f5b94075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5bac0be070 con 0x7f5bb0070570 2024-09-21T02:18:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.955+0000 7f5bb5c68640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5b94073350 0x7f5b94075860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bb009d350 tx=0x7f5ba0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:54.959+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5bac0164d0 con 0x7f5bb0070570 2024-09-21T02:18:55.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.119+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bb00ce2a0 con 0x7f5bb0070570 2024-09-21T02:18:55.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5ba6ffd640 1 -- 172.21.15.170:0/693416778 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16791 (secure 0 0 0) 0x7f5bac0f91c0 con 0x7f5bb0070570 2024-09-21T02:18:55.125 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:55.125 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":102,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:18:52.331552+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:48.430867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:49.686964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:18:55.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5b94073350 msgr2=0x7f5b94075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5b94073350 0x7f5b94075860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bb009d350 tx=0x7f5ba0009040 comp rx=0 tx=0).stop 2024-09-21T02:18:55.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 msgr2=0x7f5bb0152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 0x7f5bb0152510 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009850 tx=0x7f5bac0a5770 comp rx=0 tx=0).stop 2024-09-21T02:18:55.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 shutdown_connections 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5b94073350 0x7f5b94075860 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5bb015f990 0x7f5bb00ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bb00cadf0 0x7f5bb00ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 --2- 172.21.15.170:0/693416778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5bb0070570 0x7f5bb0152510 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 >> 172.21.15.170:0/693416778 conn(0x7f5bb00b1c40 msgr2=0x7f5bb0159180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 shutdown_connections 2024-09-21T02:18:55.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.123+0000 7f5bb86cd640 1 -- 172.21.15.170:0/693416778 wait complete. 2024-09-21T02:18:55.146 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:18:55.147 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:18:55.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 -- 172.21.15.170:0/1214271681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cc8c0 msgr2=0x7fe4080cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 --2- 172.21.15.170:0/1214271681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cc8c0 0x7fe4080cccd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe4080d1160 tx=0x7fe40001cd80 comp rx=0 tx=0).stop 2024-09-21T02:18:55.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 -- 172.21.15.170:0/1214271681 shutdown_connections 2024-09-21T02:18:55.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 --2- 172.21.15.170:0/1214271681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cc8c0 0x7fe4080cccd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 --2- 172.21.15.170:0/1214271681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4080cb460 0x7fe4080cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 --2- 172.21.15.170:0/1214271681 >> v2:172.21.15.191:3300/0 conn(0x7fe4080cf7e0 0x7fe408070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 -- 172.21.15.170:0/1214271681 >> 172.21.15.170:0/1214271681 conn(0x7fe4080b1c40 msgr2=0x7fe4080b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 -- 172.21.15.170:0/1214271681 shutdown_connections 2024-09-21T02:18:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.295+0000 7fe40da08640 1 -- 172.21.15.170:0/1214271681 wait complete. 2024-09-21T02:18:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 Processor -- start 2024-09-21T02:18:55.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 -- start start 2024-09-21T02:18:55.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4080cb460 0x7fe4081f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4080cc8c0 0x7fe4080c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 0x7fe408158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4080ceb50 con 0x7fe4080cf7e0 2024-09-21T02:18:55.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe4080ce9d0 con 0x7fe4080cb460 2024-09-21T02:18:55.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe4080cecd0 con 0x7fe4080cc8c0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 0x7fe408158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 0x7fe408158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39418/0 (socket says 172.21.15.170:39418) 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 -- 172.21.15.170:0/4244035426 learned_addr learned my addr 172.21.15.170:0/4244035426 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe406ffd640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4080cb460 0x7fe4081f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe3fe7fc640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4080cc8c0 0x7fe4080c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 -- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4080cc8c0 msgr2=0x7fe4080c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4080cc8c0 0x7fe4080c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 -- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4080cb460 msgr2=0x7fe4081f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4080cb460 0x7fe4081f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 -- 172.21.15.170:0/4244035426 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4080c9fe0 con 0x7fe4080cf7e0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe4077fe640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 0x7fe408158440 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe400009bd0 tx=0x7fe400005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000bc070 con 0x7fe4080cf7e0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe400004330 con 0x7fe4080cf7e0 2024-09-21T02:18:55.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4080c5890 con 0x7fe4080cf7e0 2024-09-21T02:18:55.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000b2040 con 0x7fe4080cf7e0 2024-09-21T02:18:55.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.299+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4081f7190 con 0x7fe4080cf7e0 2024-09-21T02:18:55.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.303+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4080c5600 con 0x7fe4080cf7e0 2024-09-21T02:18:55.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.303+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fe4000bf070 con 0x7fe4080cf7e0 2024-09-21T02:18:55.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.303+0000 7fe404ff9640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe3e8073350 0x7fe3e8075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.303+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe40012e070 con 0x7fe4080cf7e0 2024-09-21T02:18:55.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.303+0000 7fe406ffd640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe3e8073350 0x7fe3e8075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.303+0000 7fe406ffd640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe3e8073350 0x7fe3e8075860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005fd0 tx=0x7fe3f800b390 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:55.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.307+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe400016970 con 0x7fe4080cf7e0 2024-09-21T02:18:55.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.463+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4080cccd0 con 0x7fe3e8073350 2024-09-21T02:18:55.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.471+0000 7fe404ff9640 1 -- 172.21.15.170:0/4244035426 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252409 (secure 0 0 0) 0x7fe4080cccd0 con 0x7fe3e8073350 2024-09-21T02:18:55.479 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:55.480 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:18:55.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe3e8073350 msgr2=0x7fe3e8075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe3e8073350 0x7fe3e8075860 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8005fd0 tx=0x7fe3f800b390 comp rx=0 tx=0).stop 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 msgr2=0x7fe408158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 0x7fe408158440 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe400009bd0 tx=0x7fe400005b10 comp rx=0 tx=0).stop 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 shutdown_connections 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe3e8073350 0x7fe3e8075860 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4080cc8c0 0x7fe4080c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4080cf7e0 0x7fe408158440 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 --2- 172.21.15.170:0/4244035426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4080cb460 0x7fe4081f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 >> 172.21.15.170:0/4244035426 conn(0x7fe4080b1c40 msgr2=0x7fe4080b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 shutdown_connections 2024-09-21T02:18:55.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.475+0000 7fe40da08640 1 -- 172.21.15.170:0/4244035426 wait complete. 2024-09-21T02:18:55.499 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-21T02:18:54.872596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281396,"kb_used_data":11080,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693324,"statfs":{"total":773094113280,"available":772805963776,"internally_reserved":0,"allocated":11345920,"data_stored":5592668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.583572"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753720+0000","last_change":"2024-09-21T02:18:51.378851+0000","last_active":"2024-09-21T02:18:52.753720+0000","last_peered":"2024-09-21T02:18:52.753720+0000","last_clean":"2024-09-21T02:18:52.753720+0000","last_became_active":"2024-09-21T02:18:51.378526+0000","last_became_peered":"2024-09-21T02:18:51.378526+0000","last_unstale":"2024-09-21T02:18:52.753720+0000","last_undegraded":"2024-09-21T02:18:52.753720+0000","last_fullsized":"2024-09-21T02:18:52.753720+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:53.869335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010701+0000","last_change":"2024-09-21T02:18:51.352342+0000","last_active":"2024-09-21T02:18:53.010701+0000","last_peered":"2024-09-21T02:18:53.010701+0000","last_clean":"2024-09-21T02:18:53.010701+0000","last_became_active":"2024-09-21T02:18:51.351524+0000","last_became_peered":"2024-09-21T02:18:51.351524+0000","last_unstale":"2024-09-21T02:18:53.010701+0000","last_undegraded":"2024-09-21T02:18:53.010701+0000","last_fullsized":"2024-09-21T02:18:53.010701+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.770042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943612+0000","last_change":"2024-09-21T02:18:51.369596+0000","last_active":"2024-09-21T02:18:52.943612+0000","last_peered":"2024-09-21T02:18:52.943612+0000","last_clean":"2024-09-21T02:18:52.943612+0000","last_became_active":"2024-09-21T02:18:51.368539+0000","last_became_peered":"2024-09-21T02:18:51.368539+0000","last_unstale":"2024-09-21T02:18:52.943612+0000","last_undegraded":"2024-09-21T02:18:52.943612+0000","last_fullsized":"2024-09-21T02:18:52.943612+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:47.106686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007846+0000","last_change":"2024-09-21T02:18:51.366490+0000","last_active":"2024-09-21T02:18:53.007846+0000","last_peered":"2024-09-21T02:18:53.007846+0000","last_clean":"2024-09-21T02:18:53.007846+0000","last_became_active":"2024-09-21T02:18:51.366155+0000","last_became_peered":"2024-09-21T02:18:51.366155+0000","last_unstale":"2024-09-21T02:18:53.007846+0000","last_undegraded":"2024-09-21T02:18:53.007846+0000","last_fullsized":"2024-09-21T02:18:53.007846+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:13.158703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344056+0000","last_change":"2024-09-21T02:18:51.376049+0000","last_active":"2024-09-21T02:18:52.344056+0000","last_peered":"2024-09-21T02:18:52.344056+0000","last_clean":"2024-09-21T02:18:52.344056+0000","last_became_active":"2024-09-21T02:18:51.375760+0000","last_became_peered":"2024-09-21T02:18:51.375760+0000","last_unstale":"2024-09-21T02:18:52.344056+0000","last_undegraded":"2024-09-21T02:18:52.344056+0000","last_fullsized":"2024-09-21T02:18:52.344056+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:16.438338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.340641+0000","last_change":"2024-09-21T02:18:51.366095+0000","last_active":"2024-09-21T02:18:52.340641+0000","last_peered":"2024-09-21T02:18:52.340641+0000","last_clean":"2024-09-21T02:18:52.340641+0000","last_became_active":"2024-09-21T02:18:51.351619+0000","last_became_peered":"2024-09-21T02:18:51.351619+0000","last_unstale":"2024-09-21T02:18:52.340641+0000","last_undegraded":"2024-09-21T02:18:52.340641+0000","last_fullsized":"2024-09-21T02:18:52.340641+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:28.542692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336041+0000","last_change":"2024-09-21T02:18:51.376854+0000","last_active":"2024-09-21T02:18:52.336041+0000","last_peered":"2024-09-21T02:18:52.336041+0000","last_clean":"2024-09-21T02:18:52.336041+0000","last_became_active":"2024-09-21T02:18:51.376379+0000","last_became_peered":"2024-09-21T02:18:51.376379+0000","last_unstale":"2024-09-21T02:18:52.336041+0000","last_undegraded":"2024-09-21T02:18:52.336041+0000","last_fullsized":"2024-09-21T02:18:52.336041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:17.189098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336396+0000","last_change":"2024-09-21T02:18:51.357638+0000","last_active":"2024-09-21T02:18:52.336396+0000","last_peered":"2024-09-21T02:18:52.336396+0000","last_clean":"2024-09-21T02:18:52.336396+0000","last_became_active":"2024-09-21T02:18:51.356789+0000","last_became_peered":"2024-09-21T02:18:51.356789+0000","last_unstale":"2024-09-21T02:18:52.336396+0000","last_undegraded":"2024-09-21T02:18:52.336396+0000","last_fullsized":"2024-09-21T02:18:52.336396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:40.792817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754051+0000","last_change":"2024-09-21T02:18:51.376931+0000","last_active":"2024-09-21T02:18:52.754051+0000","last_peered":"2024-09-21T02:18:52.754051+0000","last_clean":"2024-09-21T02:18:52.754051+0000","last_became_active":"2024-09-21T02:18:51.376542+0000","last_became_peered":"2024-09-21T02:18:51.376542+0000","last_unstale":"2024-09-21T02:18:52.754051+0000","last_undegraded":"2024-09-21T02:18:52.754051+0000","last_fullsized":"2024-09-21T02:18:52.754051+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:03:27.191274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343513+0000","last_change":"2024-09-21T02:18:51.369083+0000","last_active":"2024-09-21T02:18:52.343513+0000","last_peered":"2024-09-21T02:18:52.343513+0000","last_clean":"2024-09-21T02:18:52.343513+0000","last_became_active":"2024-09-21T02:18:51.368721+0000","last_became_peered":"2024-09-21T02:18:51.368721+0000","last_unstale":"2024-09-21T02:18:52.343513+0000","last_undegraded":"2024-09-21T02:18:52.343513+0000","last_fullsized":"2024-09-21T02:18:52.343513+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:00.422505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336638+0000","last_change":"2024-09-21T02:18:51.376798+0000","last_active":"2024-09-21T02:18:52.336638+0000","last_peered":"2024-09-21T02:18:52.336638+0000","last_clean":"2024-09-21T02:18:52.336638+0000","last_became_active":"2024-09-21T02:18:51.376417+0000","last_became_peered":"2024-09-21T02:18:51.376417+0000","last_unstale":"2024-09-21T02:18:52.336638+0000","last_undegraded":"2024-09-21T02:18:52.336638+0000","last_fullsized":"2024-09-21T02:18:52.336638+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.171310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343279+0000","last_change":"2024-09-21T02:18:51.369918+0000","last_active":"2024-09-21T02:18:52.343279+0000","last_peered":"2024-09-21T02:18:52.343279+0000","last_clean":"2024-09-21T02:18:52.343279+0000","last_became_active":"2024-09-21T02:18:51.368625+0000","last_became_peered":"2024-09-21T02:18:51.368625+0000","last_unstale":"2024-09-21T02:18:52.343279+0000","last_undegraded":"2024-09-21T02:18:52.343279+0000","last_fullsized":"2024-09-21T02:18:52.343279+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:04.387341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009534+0000","last_change":"2024-09-21T02:18:51.374866+0000","last_active":"2024-09-21T02:18:53.009534+0000","last_peered":"2024-09-21T02:18:53.009534+0000","last_clean":"2024-09-21T02:18:53.009534+0000","last_became_active":"2024-09-21T02:18:51.374429+0000","last_became_peered":"2024-09-21T02:18:51.374429+0000","last_unstale":"2024-09-21T02:18:53.009534+0000","last_undegraded":"2024-09-21T02:18:53.009534+0000","last_fullsized":"2024-09-21T02:18:53.009534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:38.267171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009384+0000","last_change":"2024-09-21T02:18:51.376621+0000","last_active":"2024-09-21T02:18:53.009384+0000","last_peered":"2024-09-21T02:18:53.009384+0000","last_clean":"2024-09-21T02:18:53.009384+0000","last_became_active":"2024-09-21T02:18:51.373018+0000","last_became_peered":"2024-09-21T02:18:51.373018+0000","last_unstale":"2024-09-21T02:18:53.009384+0000","last_undegraded":"2024-09-21T02:18:53.009384+0000","last_fullsized":"2024-09-21T02:18:53.009384+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:38.267171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753124+0000","last_change":"2024-09-21T02:18:51.362484+0000","last_active":"2024-09-21T02:18:52.753124+0000","last_peered":"2024-09-21T02:18:52.753124+0000","last_clean":"2024-09-21T02:18:52.753124+0000","last_became_active":"2024-09-21T02:18:51.361564+0000","last_became_peered":"2024-09-21T02:18:51.361564+0000","last_unstale":"2024-09-21T02:18:52.753124+0000","last_undegraded":"2024-09-21T02:18:52.753124+0000","last_fullsized":"2024-09-21T02:18:52.753124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.895473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.342817+0000","last_change":"2024-09-21T02:18:51.379928+0000","last_active":"2024-09-21T02:18:52.342817+0000","last_peered":"2024-09-21T02:18:52.342817+0000","last_clean":"2024-09-21T02:18:52.342817+0000","last_became_active":"2024-09-21T02:18:51.379551+0000","last_became_peered":"2024-09-21T02:18:51.379551+0000","last_unstale":"2024-09-21T02:18:52.342817+0000","last_undegraded":"2024-09-21T02:18:52.342817+0000","last_fullsized":"2024-09-21T02:18:52.342817+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:23.904068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.944410+0000","last_change":"2024-09-21T02:18:51.380772+0000","last_active":"2024-09-21T02:18:52.944410+0000","last_peered":"2024-09-21T02:18:52.944410+0000","last_clean":"2024-09-21T02:18:52.944410+0000","last_became_active":"2024-09-21T02:18:51.380251+0000","last_became_peered":"2024-09-21T02:18:51.380251+0000","last_unstale":"2024-09-21T02:18:52.944410+0000","last_undegraded":"2024-09-21T02:18:52.944410+0000","last_fullsized":"2024-09-21T02:18:52.944410+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:11.670754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336114+0000","last_change":"2024-09-21T02:18:51.375772+0000","last_active":"2024-09-21T02:18:52.336114+0000","last_peered":"2024-09-21T02:18:52.336114+0000","last_clean":"2024-09-21T02:18:52.336114+0000","last_became_active":"2024-09-21T02:18:51.374927+0000","last_became_peered":"2024-09-21T02:18:51.374927+0000","last_unstale":"2024-09-21T02:18:52.336114+0000","last_undegraded":"2024-09-21T02:18:52.336114+0000","last_fullsized":"2024-09-21T02:18:52.336114+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:26.910468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.944158+0000","last_change":"2024-09-21T02:18:51.368209+0000","last_active":"2024-09-21T02:18:52.944158+0000","last_peered":"2024-09-21T02:18:52.944158+0000","last_clean":"2024-09-21T02:18:52.944158+0000","last_became_active":"2024-09-21T02:18:51.367725+0000","last_became_peered":"2024-09-21T02:18:51.367725+0000","last_unstale":"2024-09-21T02:18:52.944158+0000","last_undegraded":"2024-09-21T02:18:52.944158+0000","last_fullsized":"2024-09-21T02:18:52.944158+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:11.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943691+0000","last_change":"2024-09-21T02:18:51.381829+0000","last_active":"2024-09-21T02:18:52.943691+0000","last_peered":"2024-09-21T02:18:52.943691+0000","last_clean":"2024-09-21T02:18:52.943691+0000","last_became_active":"2024-09-21T02:18:51.381468+0000","last_became_peered":"2024-09-21T02:18:51.381468+0000","last_unstale":"2024-09-21T02:18:52.943691+0000","last_undegraded":"2024-09-21T02:18:52.943691+0000","last_fullsized":"2024-09-21T02:18:52.943691+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:16.148023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336557+0000","last_change":"2024-09-21T02:18:51.376635+0000","last_active":"2024-09-21T02:18:52.336557+0000","last_peered":"2024-09-21T02:18:52.336557+0000","last_clean":"2024-09-21T02:18:52.336557+0000","last_became_active":"2024-09-21T02:18:51.374015+0000","last_became_peered":"2024-09-21T02:18:51.374015+0000","last_unstale":"2024-09-21T02:18:52.336557+0000","last_undegraded":"2024-09-21T02:18:52.336557+0000","last_fullsized":"2024-09-21T02:18:52.336557+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:01.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341211+0000","last_change":"2024-09-21T02:18:51.363682+0000","last_active":"2024-09-21T02:18:52.341211+0000","last_peered":"2024-09-21T02:18:52.341211+0000","last_clean":"2024-09-21T02:18:52.341211+0000","last_became_active":"2024-09-21T02:18:51.363304+0000","last_became_peered":"2024-09-21T02:18:51.363304+0000","last_unstale":"2024-09-21T02:18:52.341211+0000","last_undegraded":"2024-09-21T02:18:52.341211+0000","last_fullsized":"2024-09-21T02:18:52.341211+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:33.984312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009819+0000","last_change":"2024-09-21T02:18:51.386532+0000","last_active":"2024-09-21T02:18:53.009819+0000","last_peered":"2024-09-21T02:18:53.009819+0000","last_clean":"2024-09-21T02:18:53.009819+0000","last_became_active":"2024-09-21T02:18:51.386092+0000","last_became_peered":"2024-09-21T02:18:51.386092+0000","last_unstale":"2024-09-21T02:18:53.009819+0000","last_undegraded":"2024-09-21T02:18:53.009819+0000","last_fullsized":"2024-09-21T02:18:53.009819+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.770042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007333+0000","last_change":"2024-09-21T02:18:51.373297+0000","last_active":"2024-09-21T02:18:53.007333+0000","last_peered":"2024-09-21T02:18:53.007333+0000","last_clean":"2024-09-21T02:18:53.007333+0000","last_became_active":"2024-09-21T02:18:51.372098+0000","last_became_peered":"2024-09-21T02:18:51.372098+0000","last_unstale":"2024-09-21T02:18:53.007333+0000","last_undegraded":"2024-09-21T02:18:53.007333+0000","last_fullsized":"2024-09-21T02:18:53.007333+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:35.102595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336910+0000","last_change":"2024-09-21T02:18:51.362546+0000","last_active":"2024-09-21T02:18:52.336910+0000","last_peered":"2024-09-21T02:18:52.336910+0000","last_clean":"2024-09-21T02:18:52.336910+0000","last_became_active":"2024-09-21T02:18:51.361625+0000","last_became_peered":"2024-09-21T02:18:51.361625+0000","last_unstale":"2024-09-21T02:18:52.336910+0000","last_undegraded":"2024-09-21T02:18:52.336910+0000","last_fullsized":"2024-09-21T02:18:52.336910+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:58.263173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341455+0000","last_change":"2024-09-21T02:18:51.366119+0000","last_active":"2024-09-21T02:18:52.341455+0000","last_peered":"2024-09-21T02:18:52.341455+0000","last_clean":"2024-09-21T02:18:52.341455+0000","last_became_active":"2024-09-21T02:18:51.365698+0000","last_became_peered":"2024-09-21T02:18:51.365698+0000","last_unstale":"2024-09-21T02:18:52.341455+0000","last_undegraded":"2024-09-21T02:18:52.341455+0000","last_fullsized":"2024-09-21T02:18:52.341455+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:04.213388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008230+0000","last_change":"2024-09-21T02:18:51.378598+0000","last_active":"2024-09-21T02:18:53.008230+0000","last_peered":"2024-09-21T02:18:53.008230+0000","last_clean":"2024-09-21T02:18:53.008230+0000","last_became_active":"2024-09-21T02:18:51.378233+0000","last_became_peered":"2024-09-21T02:18:51.378233+0000","last_unstale":"2024-09-21T02:18:53.008230+0000","last_undegraded":"2024-09-21T02:18:53.008230+0000","last_fullsized":"2024-09-21T02:18:53.008230+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:05.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341385+0000","last_change":"2024-09-21T02:18:51.370244+0000","last_active":"2024-09-21T02:18:52.341385+0000","last_peered":"2024-09-21T02:18:52.341385+0000","last_clean":"2024-09-21T02:18:52.341385+0000","last_became_active":"2024-09-21T02:18:51.369831+0000","last_became_peered":"2024-09-21T02:18:51.369831+0000","last_unstale":"2024-09-21T02:18:52.341385+0000","last_undegraded":"2024-09-21T02:18:52.341385+0000","last_fullsized":"2024-09-21T02:18:52.341385+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:54.551271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011474+0000","last_change":"2024-09-21T02:18:51.378969+0000","last_active":"2024-09-21T02:18:53.011474+0000","last_peered":"2024-09-21T02:18:53.011474+0000","last_clean":"2024-09-21T02:18:53.011474+0000","last_became_active":"2024-09-21T02:18:51.374553+0000","last_became_peered":"2024-09-21T02:18:51.374553+0000","last_unstale":"2024-09-21T02:18:53.011474+0000","last_undegraded":"2024-09-21T02:18:53.011474+0000","last_fullsized":"2024-09-21T02:18:53.011474+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:05.236636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.335971+0000","last_change":"2024-09-21T02:18:51.369757+0000","last_active":"2024-09-21T02:18:52.335971+0000","last_peered":"2024-09-21T02:18:52.335971+0000","last_clean":"2024-09-21T02:18:52.335971+0000","last_became_active":"2024-09-21T02:18:51.368900+0000","last_became_peered":"2024-09-21T02:18:51.368900+0000","last_unstale":"2024-09-21T02:18:52.335971+0000","last_undegraded":"2024-09-21T02:18:52.335971+0000","last_fullsized":"2024-09-21T02:18:52.335971+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:41.980159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753301+0000","last_change":"2024-09-21T02:18:51.375360+0000","last_active":"2024-09-21T02:18:52.753301+0000","last_peered":"2024-09-21T02:18:52.753301+0000","last_clean":"2024-09-21T02:18:52.753301+0000","last_became_active":"2024-09-21T02:18:51.375042+0000","last_became_peered":"2024-09-21T02:18:51.375042+0000","last_unstale":"2024-09-21T02:18:52.753301+0000","last_undegraded":"2024-09-21T02:18:52.753301+0000","last_fullsized":"2024-09-21T02:18:52.753301+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:17.851797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.345054+0000","last_change":"2024-09-21T02:18:51.375598+0000","last_active":"2024-09-21T02:18:52.345054+0000","last_peered":"2024-09-21T02:18:52.345054+0000","last_clean":"2024-09-21T02:18:52.345054+0000","last_became_active":"2024-09-21T02:18:51.375140+0000","last_became_peered":"2024-09-21T02:18:51.375140+0000","last_unstale":"2024-09-21T02:18:52.345054+0000","last_undegraded":"2024-09-21T02:18:52.345054+0000","last_fullsized":"2024-09-21T02:18:52.345054+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:15.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754711+0000","last_change":"2024-09-21T02:18:51.380710+0000","last_active":"2024-09-21T02:18:52.754711+0000","last_peered":"2024-09-21T02:18:52.754711+0000","last_clean":"2024-09-21T02:18:52.754711+0000","last_became_active":"2024-09-21T02:18:51.380370+0000","last_became_peered":"2024-09-21T02:18:51.380370+0000","last_unstale":"2024-09-21T02:18:52.754711+0000","last_undegraded":"2024-09-21T02:18:52.754711+0000","last_fullsized":"2024-09-21T02:18:52.754711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.884612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010445+0000","last_change":"2024-09-21T02:18:50.321338+0000","last_active":"2024-09-21T02:18:53.010445+0000","last_peered":"2024-09-21T02:18:53.010445+0000","last_clean":"2024-09-21T02:18:53.010445+0000","last_became_active":"2024-09-21T02:18:50.320665+0000","last_became_peered":"2024-09-21T02:18:50.320665+0000","last_unstale":"2024-09-21T02:18:53.010445+0000","last_undegraded":"2024-09-21T02:18:53.010445+0000","last_fullsized":"2024-09-21T02:18:53.010445+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:08.839500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010803+0000","last_change":"2024-09-21T02:18:51.378181+0000","last_active":"2024-09-21T02:18:53.010803+0000","last_peered":"2024-09-21T02:18:53.010803+0000","last_clean":"2024-09-21T02:18:53.010803+0000","last_became_active":"2024-09-21T02:18:51.375256+0000","last_became_peered":"2024-09-21T02:18:51.375256+0000","last_unstale":"2024-09-21T02:18:53.010803+0000","last_undegraded":"2024-09-21T02:18:53.010803+0000","last_fullsized":"2024-09-21T02:18:53.010803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:09.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010874+0000","last_change":"2024-09-21T02:18:50.324595+0000","last_active":"2024-09-21T02:18:53.010874+0000","last_peered":"2024-09-21T02:18:53.010874+0000","last_clean":"2024-09-21T02:18:53.010874+0000","last_became_active":"2024-09-21T02:18:50.324144+0000","last_became_peered":"2024-09-21T02:18:50.324144+0000","last_unstale":"2024-09-21T02:18:53.010874+0000","last_undegraded":"2024-09-21T02:18:53.010874+0000","last_fullsized":"2024-09-21T02:18:53.010874+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:07.730672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341854+0000","last_change":"2024-09-21T02:18:51.367120+0000","last_active":"2024-09-21T02:18:52.341854+0000","last_peered":"2024-09-21T02:18:52.341854+0000","last_clean":"2024-09-21T02:18:52.341854+0000","last_became_active":"2024-09-21T02:18:51.366777+0000","last_became_peered":"2024-09-21T02:18:51.366777+0000","last_unstale":"2024-09-21T02:18:52.341854+0000","last_undegraded":"2024-09-21T02:18:52.341854+0000","last_fullsized":"2024-09-21T02:18:52.341854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.435325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008114+0000","last_change":"2024-09-21T02:18:50.343864+0000","last_active":"2024-09-21T02:18:53.008114+0000","last_peered":"2024-09-21T02:18:53.008114+0000","last_clean":"2024-09-21T02:18:53.008114+0000","last_became_active":"2024-09-21T02:18:50.342425+0000","last_became_peered":"2024-09-21T02:18:50.342425+0000","last_unstale":"2024-09-21T02:18:53.008114+0000","last_undegraded":"2024-09-21T02:18:53.008114+0000","last_fullsized":"2024-09-21T02:18:53.008114+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:33.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.944113+0000","last_change":"2024-09-21T02:18:51.378718+0000","last_active":"2024-09-21T02:18:52.944113+0000","last_peered":"2024-09-21T02:18:52.944113+0000","last_clean":"2024-09-21T02:18:52.944113+0000","last_became_active":"2024-09-21T02:18:51.370814+0000","last_became_peered":"2024-09-21T02:18:51.370814+0000","last_unstale":"2024-09-21T02:18:52.944113+0000","last_undegraded":"2024-09-21T02:18:52.944113+0000","last_fullsized":"2024-09-21T02:18:52.944113+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:31.960005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007447+0000","last_change":"2024-09-21T02:18:50.343726+0000","last_active":"2024-09-21T02:18:53.007447+0000","last_peered":"2024-09-21T02:18:53.007447+0000","last_clean":"2024-09-21T02:18:53.007447+0000","last_became_active":"2024-09-21T02:18:50.340809+0000","last_became_peered":"2024-09-21T02:18:50.340809+0000","last_unstale":"2024-09-21T02:18:53.007447+0000","last_undegraded":"2024-09-21T02:18:53.007447+0000","last_fullsized":"2024-09-21T02:18:53.007447+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:14.394768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007961+0000","last_change":"2024-09-21T02:18:51.373240+0000","last_active":"2024-09-21T02:18:53.007961+0000","last_peered":"2024-09-21T02:18:53.007961+0000","last_clean":"2024-09-21T02:18:53.007961+0000","last_became_active":"2024-09-21T02:18:51.372365+0000","last_became_peered":"2024-09-21T02:18:51.372365+0000","last_unstale":"2024-09-21T02:18:53.007961+0000","last_undegraded":"2024-09-21T02:18:53.007961+0000","last_fullsized":"2024-09-21T02:18:53.007961+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:20.711574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943224+0000","last_change":"2024-09-21T02:18:50.323170+0000","last_active":"2024-09-21T02:18:52.943224+0000","last_peered":"2024-09-21T02:18:52.943224+0000","last_clean":"2024-09-21T02:18:52.943224+0000","last_became_active":"2024-09-21T02:18:50.322482+0000","last_became_peered":"2024-09-21T02:18:50.322482+0000","last_unstale":"2024-09-21T02:18:52.943224+0000","last_undegraded":"2024-09-21T02:18:52.943224+0000","last_fullsized":"2024-09-21T02:18:52.943224+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.068268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336303+0000","last_change":"2024-09-21T02:18:51.371772+0000","last_active":"2024-09-21T02:18:52.336303+0000","last_peered":"2024-09-21T02:18:52.336303+0000","last_clean":"2024-09-21T02:18:52.336303+0000","last_became_active":"2024-09-21T02:18:51.370918+0000","last_became_peered":"2024-09-21T02:18:51.370918+0000","last_unstale":"2024-09-21T02:18:52.336303+0000","last_undegraded":"2024-09-21T02:18:52.336303+0000","last_fullsized":"2024-09-21T02:18:52.336303+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:28.157673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753157+0000","last_change":"2024-09-21T02:18:50.334892+0000","last_active":"2024-09-21T02:18:52.753157+0000","last_peered":"2024-09-21T02:18:52.753157+0000","last_clean":"2024-09-21T02:18:52.753157+0000","last_became_active":"2024-09-21T02:18:50.334191+0000","last_became_peered":"2024-09-21T02:18:50.334191+0000","last_unstale":"2024-09-21T02:18:52.753157+0000","last_undegraded":"2024-09-21T02:18:52.753157+0000","last_fullsized":"2024-09-21T02:18:52.753157+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:49.658604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343501+0000","last_change":"2024-09-21T02:18:51.380382+0000","last_active":"2024-09-21T02:18:52.343501+0000","last_peered":"2024-09-21T02:18:52.343501+0000","last_clean":"2024-09-21T02:18:52.343501+0000","last_became_active":"2024-09-21T02:18:51.380006+0000","last_became_peered":"2024-09-21T02:18:51.380006+0000","last_unstale":"2024-09-21T02:18:52.343501+0000","last_undegraded":"2024-09-21T02:18:52.343501+0000","last_fullsized":"2024-09-21T02:18:52.343501+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:05.004571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.335898+0000","last_change":"2024-09-21T02:18:50.335000+0000","last_active":"2024-09-21T02:18:52.335898+0000","last_peered":"2024-09-21T02:18:52.335898+0000","last_clean":"2024-09-21T02:18:52.335898+0000","last_became_active":"2024-09-21T02:18:50.334659+0000","last_became_peered":"2024-09-21T02:18:50.334659+0000","last_unstale":"2024-09-21T02:18:52.335898+0000","last_undegraded":"2024-09-21T02:18:52.335898+0000","last_fullsized":"2024-09-21T02:18:52.335898+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:50.621617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.335820+0000","last_change":"2024-09-21T02:18:51.366640+0000","last_active":"2024-09-21T02:18:52.335820+0000","last_peered":"2024-09-21T02:18:52.335820+0000","last_clean":"2024-09-21T02:18:52.335820+0000","last_became_active":"2024-09-21T02:18:51.365949+0000","last_became_peered":"2024-09-21T02:18:51.365949+0000","last_unstale":"2024-09-21T02:18:52.335820+0000","last_undegraded":"2024-09-21T02:18:52.335820+0000","last_fullsized":"2024-09-21T02:18:52.335820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:34.657350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343691+0000","last_change":"2024-09-21T02:18:50.349795+0000","last_active":"2024-09-21T02:18:52.343691+0000","last_peered":"2024-09-21T02:18:52.343691+0000","last_clean":"2024-09-21T02:18:52.343691+0000","last_became_active":"2024-09-21T02:18:50.348454+0000","last_became_peered":"2024-09-21T02:18:50.348454+0000","last_unstale":"2024-09-21T02:18:52.343691+0000","last_undegraded":"2024-09-21T02:18:52.343691+0000","last_fullsized":"2024-09-21T02:18:52.343691+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:55.271356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011367+0000","last_change":"2024-09-21T02:18:51.387062+0000","last_active":"2024-09-21T02:18:53.011367+0000","last_peered":"2024-09-21T02:18:53.011367+0000","last_clean":"2024-09-21T02:18:53.011367+0000","last_became_active":"2024-09-21T02:18:51.386443+0000","last_became_peered":"2024-09-21T02:18:51.386443+0000","last_unstale":"2024-09-21T02:18:53.011367+0000","last_undegraded":"2024-09-21T02:18:53.011367+0000","last_fullsized":"2024-09-21T02:18:53.011367+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:01.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008481+0000","last_change":"2024-09-21T02:18:50.346570+0000","last_active":"2024-09-21T02:18:53.008481+0000","last_peered":"2024-09-21T02:18:53.008481+0000","last_clean":"2024-09-21T02:18:53.008481+0000","last_became_active":"2024-09-21T02:18:50.345866+0000","last_became_peered":"2024-09-21T02:18:50.345866+0000","last_unstale":"2024-09-21T02:18:53.008481+0000","last_undegraded":"2024-09-21T02:18:53.008481+0000","last_fullsized":"2024-09-21T02:18:53.008481+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:35.714958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011656+0000","last_change":"2024-09-21T02:18:51.367061+0000","last_active":"2024-09-21T02:18:53.011656+0000","last_peered":"2024-09-21T02:18:53.011656+0000","last_clean":"2024-09-21T02:18:53.011656+0000","last_became_active":"2024-09-21T02:18:51.365641+0000","last_became_peered":"2024-09-21T02:18:51.365641+0000","last_unstale":"2024-09-21T02:18:53.011656+0000","last_undegraded":"2024-09-21T02:18:53.011656+0000","last_fullsized":"2024-09-21T02:18:53.011656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:49.672388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344061+0000","last_change":"2024-09-21T02:18:50.342751+0000","last_active":"2024-09-21T02:18:52.344061+0000","last_peered":"2024-09-21T02:18:52.344061+0000","last_clean":"2024-09-21T02:18:52.344061+0000","last_became_active":"2024-09-21T02:18:50.342042+0000","last_became_peered":"2024-09-21T02:18:50.342042+0000","last_unstale":"2024-09-21T02:18:52.344061+0000","last_undegraded":"2024-09-21T02:18:52.344061+0000","last_fullsized":"2024-09-21T02:18:52.344061+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:49.893730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.342025+0000","last_change":"2024-09-21T02:18:51.359503+0000","last_active":"2024-09-21T02:18:52.342025+0000","last_peered":"2024-09-21T02:18:52.342025+0000","last_clean":"2024-09-21T02:18:52.342025+0000","last_became_active":"2024-09-21T02:18:51.358536+0000","last_became_peered":"2024-09-21T02:18:51.358536+0000","last_unstale":"2024-09-21T02:18:52.342025+0000","last_undegraded":"2024-09-21T02:18:52.342025+0000","last_fullsized":"2024-09-21T02:18:52.342025+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:27.392053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343486+0000","last_change":"2024-09-21T02:18:50.349354+0000","last_active":"2024-09-21T02:18:52.343486+0000","last_peered":"2024-09-21T02:18:52.343486+0000","last_clean":"2024-09-21T02:18:52.343486+0000","last_became_active":"2024-09-21T02:18:50.348680+0000","last_became_peered":"2024-09-21T02:18:50.348680+0000","last_unstale":"2024-09-21T02:18:52.343486+0000","last_undegraded":"2024-09-21T02:18:52.343486+0000","last_fullsized":"2024-09-21T02:18:52.343486+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:51.815348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336813+0000","last_change":"2024-09-21T02:18:51.370188+0000","last_active":"2024-09-21T02:18:52.336813+0000","last_peered":"2024-09-21T02:18:52.336813+0000","last_clean":"2024-09-21T02:18:52.336813+0000","last_became_active":"2024-09-21T02:18:51.369814+0000","last_became_peered":"2024-09-21T02:18:51.369814+0000","last_unstale":"2024-09-21T02:18:52.336813+0000","last_undegraded":"2024-09-21T02:18:52.336813+0000","last_fullsized":"2024-09-21T02:18:52.336813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:13.827056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008043+0000","last_change":"2024-09-21T02:18:50.346941+0000","last_active":"2024-09-21T02:18:53.008043+0000","last_peered":"2024-09-21T02:18:53.008043+0000","last_clean":"2024-09-21T02:18:53.008043+0000","last_became_active":"2024-09-21T02:18:50.346112+0000","last_became_peered":"2024-09-21T02:18:50.346112+0000","last_unstale":"2024-09-21T02:18:53.008043+0000","last_undegraded":"2024-09-21T02:18:53.008043+0000","last_fullsized":"2024-09-21T02:18:53.008043+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:16.788465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343750+0000","last_change":"2024-09-21T02:18:51.379537+0000","last_active":"2024-09-21T02:18:52.343750+0000","last_peered":"2024-09-21T02:18:52.343750+0000","last_clean":"2024-09-21T02:18:52.343750+0000","last_became_active":"2024-09-21T02:18:51.379153+0000","last_became_peered":"2024-09-21T02:18:51.379153+0000","last_unstale":"2024-09-21T02:18:52.343750+0000","last_undegraded":"2024-09-21T02:18:52.343750+0000","last_fullsized":"2024-09-21T02:18:52.343750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:19.300698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344009+0000","last_change":"2024-09-21T02:18:50.336588+0000","last_active":"2024-09-21T02:18:52.344009+0000","last_peered":"2024-09-21T02:18:52.344009+0000","last_clean":"2024-09-21T02:18:52.344009+0000","last_became_active":"2024-09-21T02:18:50.335711+0000","last_became_peered":"2024-09-21T02:18:50.335711+0000","last_unstale":"2024-09-21T02:18:52.344009+0000","last_undegraded":"2024-09-21T02:18:52.344009+0000","last_fullsized":"2024-09-21T02:18:52.344009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:48.958670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008361+0000","last_change":"2024-09-21T02:18:51.374035+0000","last_active":"2024-09-21T02:18:53.008361+0000","last_peered":"2024-09-21T02:18:53.008361+0000","last_clean":"2024-09-21T02:18:53.008361+0000","last_became_active":"2024-09-21T02:18:51.373620+0000","last_became_peered":"2024-09-21T02:18:51.373620+0000","last_unstale":"2024-09-21T02:18:53.008361+0000","last_undegraded":"2024-09-21T02:18:53.008361+0000","last_fullsized":"2024-09-21T02:18:53.008361+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:51.972508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943768+0000","last_change":"2024-09-21T02:18:50.340882+0000","last_active":"2024-09-21T02:18:52.943768+0000","last_peered":"2024-09-21T02:18:52.943768+0000","last_clean":"2024-09-21T02:18:52.943768+0000","last_became_active":"2024-09-21T02:18:50.340470+0000","last_became_peered":"2024-09-21T02:18:50.340470+0000","last_unstale":"2024-09-21T02:18:52.943768+0000","last_undegraded":"2024-09-21T02:18:52.943768+0000","last_fullsized":"2024-09-21T02:18:52.943768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:06.718783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007693+0000","last_change":"2024-09-21T02:18:51.353718+0000","last_active":"2024-09-21T02:18:53.007693+0000","last_peered":"2024-09-21T02:18:53.007693+0000","last_clean":"2024-09-21T02:18:53.007693+0000","last_became_active":"2024-09-21T02:18:51.352917+0000","last_became_peered":"2024-09-21T02:18:51.352917+0000","last_unstale":"2024-09-21T02:18:53.007693+0000","last_undegraded":"2024-09-21T02:18:53.007693+0000","last_fullsized":"2024-09-21T02:18:53.007693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:13.566169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.752723+0000","last_change":"2024-09-21T02:18:50.333888+0000","last_active":"2024-09-21T02:18:52.752723+0000","last_peered":"2024-09-21T02:18:52.752723+0000","last_clean":"2024-09-21T02:18:52.752723+0000","last_became_active":"2024-09-21T02:18:50.333064+0000","last_became_peered":"2024-09-21T02:18:50.333064+0000","last_unstale":"2024-09-21T02:18:52.752723+0000","last_undegraded":"2024-09-21T02:18:52.752723+0000","last_fullsized":"2024-09-21T02:18:52.752723+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:10.591684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754553+0000","last_change":"2024-09-21T02:18:51.370287+0000","last_active":"2024-09-21T02:18:52.754553+0000","last_peered":"2024-09-21T02:18:52.754553+0000","last_clean":"2024-09-21T02:18:52.754553+0000","last_became_active":"2024-09-21T02:18:51.367728+0000","last_became_peered":"2024-09-21T02:18:51.367728+0000","last_unstale":"2024-09-21T02:18:52.754553+0000","last_undegraded":"2024-09-21T02:18:52.754553+0000","last_fullsized":"2024-09-21T02:18:52.754553+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:34.561754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.342484+0000","last_change":"2024-09-21T02:18:50.325571+0000","last_active":"2024-09-21T02:18:52.342484+0000","last_peered":"2024-09-21T02:18:52.342484+0000","last_clean":"2024-09-21T02:18:52.342484+0000","last_became_active":"2024-09-21T02:18:50.324409+0000","last_became_peered":"2024-09-21T02:18:50.324409+0000","last_unstale":"2024-09-21T02:18:52.342484+0000","last_undegraded":"2024-09-21T02:18:52.342484+0000","last_fullsized":"2024-09-21T02:18:52.342484+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:01.153260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754117+0000","last_change":"2024-09-21T02:18:51.379981+0000","last_active":"2024-09-21T02:18:52.754117+0000","last_peered":"2024-09-21T02:18:52.754117+0000","last_clean":"2024-09-21T02:18:52.754117+0000","last_became_active":"2024-09-21T02:18:51.379623+0000","last_became_peered":"2024-09-21T02:18:51.379623+0000","last_unstale":"2024-09-21T02:18:52.754117+0000","last_undegraded":"2024-09-21T02:18:52.754117+0000","last_fullsized":"2024-09-21T02:18:52.754117+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.151154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343768+0000","last_change":"2024-09-21T02:18:50.342086+0000","last_active":"2024-09-21T02:18:52.343768+0000","last_peered":"2024-09-21T02:18:52.343768+0000","last_clean":"2024-09-21T02:18:52.343768+0000","last_became_active":"2024-09-21T02:18:50.339265+0000","last_became_peered":"2024-09-21T02:18:50.339265+0000","last_unstale":"2024-09-21T02:18:52.343768+0000","last_undegraded":"2024-09-21T02:18:52.343768+0000","last_fullsized":"2024-09-21T02:18:52.343768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:17.337679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336573+0000","last_change":"2024-09-21T02:18:51.374582+0000","last_active":"2024-09-21T02:18:52.336573+0000","last_peered":"2024-09-21T02:18:52.336573+0000","last_clean":"2024-09-21T02:18:52.336573+0000","last_became_active":"2024-09-21T02:18:51.374221+0000","last_became_peered":"2024-09-21T02:18:51.374221+0000","last_unstale":"2024-09-21T02:18:52.336573+0000","last_undegraded":"2024-09-21T02:18:52.336573+0000","last_fullsized":"2024-09-21T02:18:52.336573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:09.039842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010410+0000","last_change":"2024-09-21T02:18:50.318191+0000","last_active":"2024-09-21T02:18:53.010410+0000","last_peered":"2024-09-21T02:18:53.010410+0000","last_clean":"2024-09-21T02:18:53.010410+0000","last_became_active":"2024-09-21T02:18:50.317786+0000","last_became_peered":"2024-09-21T02:18:50.317786+0000","last_unstale":"2024-09-21T02:18:53.010410+0000","last_undegraded":"2024-09-21T02:18:53.010410+0000","last_fullsized":"2024-09-21T02:18:53.010410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:12.982265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010196+0000","last_change":"2024-09-21T02:18:51.373783+0000","last_active":"2024-09-21T02:18:53.010196+0000","last_peered":"2024-09-21T02:18:53.010196+0000","last_clean":"2024-09-21T02:18:53.010196+0000","last_became_active":"2024-09-21T02:18:51.373307+0000","last_became_peered":"2024-09-21T02:18:51.373307+0000","last_unstale":"2024-09-21T02:18:53.010196+0000","last_undegraded":"2024-09-21T02:18:53.010196+0000","last_fullsized":"2024-09-21T02:18:53.010196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:49.672388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009047+0000","last_change":"2024-09-21T02:18:50.337791+0000","last_active":"2024-09-21T02:18:53.009047+0000","last_peered":"2024-09-21T02:18:53.009047+0000","last_clean":"2024-09-21T02:18:53.009047+0000","last_became_active":"2024-09-21T02:18:50.337083+0000","last_became_peered":"2024-09-21T02:18:50.337083+0000","last_unstale":"2024-09-21T02:18:53.009047+0000","last_undegraded":"2024-09-21T02:18:53.009047+0000","last_fullsized":"2024-09-21T02:18:53.009047+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:18.238218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943574+0000","last_change":"2024-09-21T02:18:51.366646+0000","last_active":"2024-09-21T02:18:52.943574+0000","last_peered":"2024-09-21T02:18:52.943574+0000","last_clean":"2024-09-21T02:18:52.943574+0000","last_became_active":"2024-09-21T02:18:51.366139+0000","last_became_peered":"2024-09-21T02:18:51.366139+0000","last_unstale":"2024-09-21T02:18:52.943574+0000","last_undegraded":"2024-09-21T02:18:52.943574+0000","last_fullsized":"2024-09-21T02:18:52.943574+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:39.608930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344169+0000","last_change":"2024-09-21T02:18:50.343500+0000","last_active":"2024-09-21T02:18:52.344169+0000","last_peered":"2024-09-21T02:18:52.344169+0000","last_clean":"2024-09-21T02:18:52.344169+0000","last_became_active":"2024-09-21T02:18:50.340428+0000","last_became_peered":"2024-09-21T02:18:50.340428+0000","last_unstale":"2024-09-21T02:18:52.344169+0000","last_undegraded":"2024-09-21T02:18:52.344169+0000","last_fullsized":"2024-09-21T02:18:52.344169+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:25.757690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753635+0000","last_change":"2024-09-21T02:18:51.385408+0000","last_active":"2024-09-21T02:18:52.753635+0000","last_peered":"2024-09-21T02:18:52.753635+0000","last_clean":"2024-09-21T02:18:52.753635+0000","last_became_active":"2024-09-21T02:18:51.385006+0000","last_became_peered":"2024-09-21T02:18:51.385006+0000","last_unstale":"2024-09-21T02:18:52.753635+0000","last_undegraded":"2024-09-21T02:18:52.753635+0000","last_fullsized":"2024-09-21T02:18:52.753635+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:44.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753601+0000","last_change":"2024-09-21T02:18:50.334101+0000","last_active":"2024-09-21T02:18:52.753601+0000","last_peered":"2024-09-21T02:18:52.753601+0000","last_clean":"2024-09-21T02:18:52.753601+0000","last_became_active":"2024-09-21T02:18:50.333198+0000","last_became_peered":"2024-09-21T02:18:50.333198+0000","last_unstale":"2024-09-21T02:18:52.753601+0000","last_undegraded":"2024-09-21T02:18:52.753601+0000","last_fullsized":"2024-09-21T02:18:52.753601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.562427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009249+0000","last_change":"2024-09-21T02:18:51.374046+0000","last_active":"2024-09-21T02:18:53.009249+0000","last_peered":"2024-09-21T02:18:53.009249+0000","last_clean":"2024-09-21T02:18:53.009249+0000","last_became_active":"2024-09-21T02:18:51.372327+0000","last_became_peered":"2024-09-21T02:18:51.372327+0000","last_unstale":"2024-09-21T02:18:53.009249+0000","last_undegraded":"2024-09-21T02:18:53.009249+0000","last_fullsized":"2024-09-21T02:18:53.009249+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:30.413449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.006992+0000","last_change":"2024-09-21T02:18:50.341195+0000","last_active":"2024-09-21T02:18:53.006992+0000","last_peered":"2024-09-21T02:18:53.006992+0000","last_clean":"2024-09-21T02:18:53.006992+0000","last_became_active":"2024-09-21T02:18:50.338787+0000","last_became_peered":"2024-09-21T02:18:50.338787+0000","last_unstale":"2024-09-21T02:18:53.006992+0000","last_undegraded":"2024-09-21T02:18:53.006992+0000","last_fullsized":"2024-09-21T02:18:53.006992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:20.909502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344717+0000","last_change":"2024-09-21T02:18:51.380073+0000","last_active":"2024-09-21T02:18:52.344717+0000","last_peered":"2024-09-21T02:18:52.344717+0000","last_clean":"2024-09-21T02:18:52.344717+0000","last_became_active":"2024-09-21T02:18:51.379718+0000","last_became_peered":"2024-09-21T02:18:51.379718+0000","last_unstale":"2024-09-21T02:18:52.344717+0000","last_undegraded":"2024-09-21T02:18:52.344717+0000","last_fullsized":"2024-09-21T02:18:52.344717+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:05.544806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007524+0000","last_change":"2024-09-21T02:18:50.329503+0000","last_active":"2024-09-21T02:18:53.007524+0000","last_peered":"2024-09-21T02:18:53.007524+0000","last_clean":"2024-09-21T02:18:53.007524+0000","last_became_active":"2024-09-21T02:18:50.328624+0000","last_became_peered":"2024-09-21T02:18:50.328624+0000","last_unstale":"2024-09-21T02:18:53.007524+0000","last_undegraded":"2024-09-21T02:18:53.007524+0000","last_fullsized":"2024-09-21T02:18:53.007524+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:53.977039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010569+0000","last_change":"2024-09-21T02:18:51.375992+0000","last_active":"2024-09-21T02:18:53.010569+0000","last_peered":"2024-09-21T02:18:53.010569+0000","last_clean":"2024-09-21T02:18:53.010569+0000","last_became_active":"2024-09-21T02:18:51.374305+0000","last_became_peered":"2024-09-21T02:18:51.374305+0000","last_unstale":"2024-09-21T02:18:53.010569+0000","last_undegraded":"2024-09-21T02:18:53.010569+0000","last_fullsized":"2024-09-21T02:18:53.010569+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:43.487671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008565+0000","last_change":"2024-09-21T02:18:50.340088+0000","last_active":"2024-09-21T02:18:53.008565+0000","last_peered":"2024-09-21T02:18:53.008565+0000","last_clean":"2024-09-21T02:18:53.008565+0000","last_became_active":"2024-09-21T02:18:50.339224+0000","last_became_peered":"2024-09-21T02:18:50.339224+0000","last_unstale":"2024-09-21T02:18:53.008565+0000","last_undegraded":"2024-09-21T02:18:53.008565+0000","last_fullsized":"2024-09-21T02:18:53.008565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:35.103374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.340949+0000","last_change":"2024-09-21T02:18:51.365258+0000","last_active":"2024-09-21T02:18:52.340949+0000","last_peered":"2024-09-21T02:18:52.340949+0000","last_clean":"2024-09-21T02:18:52.340949+0000","last_became_active":"2024-09-21T02:18:51.364913+0000","last_became_peered":"2024-09-21T02:18:51.364913+0000","last_unstale":"2024-09-21T02:18:52.340949+0000","last_undegraded":"2024-09-21T02:18:52.340949+0000","last_fullsized":"2024-09-21T02:18:52.340949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:00.730636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336246+0000","last_change":"2024-09-21T02:18:50.312672+0000","last_active":"2024-09-21T02:18:52.336246+0000","last_peered":"2024-09-21T02:18:52.336246+0000","last_clean":"2024-09-21T02:18:52.336246+0000","last_became_active":"2024-09-21T02:18:50.312205+0000","last_became_peered":"2024-09-21T02:18:50.312205+0000","last_unstale":"2024-09-21T02:18:52.336246+0000","last_undegraded":"2024-09-21T02:18:52.336246+0000","last_fullsized":"2024-09-21T02:18:52.336246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:26.380603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753844+0000","last_change":"2024-09-21T02:18:51.370755+0000","last_active":"2024-09-21T02:18:52.753844+0000","last_peered":"2024-09-21T02:18:52.753844+0000","last_clean":"2024-09-21T02:18:52.753844+0000","last_became_active":"2024-09-21T02:18:51.370115+0000","last_became_peered":"2024-09-21T02:18:51.370115+0000","last_unstale":"2024-09-21T02:18:52.753844+0000","last_undegraded":"2024-09-21T02:18:52.753844+0000","last_fullsized":"2024-09-21T02:18:52.753844+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:01.173763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008387+0000","last_change":"2024-09-21T02:18:50.330416+0000","last_active":"2024-09-21T02:18:53.008387+0000","last_peered":"2024-09-21T02:18:53.008387+0000","last_clean":"2024-09-21T02:18:53.008387+0000","last_became_active":"2024-09-21T02:18:50.329563+0000","last_became_peered":"2024-09-21T02:18:50.329563+0000","last_unstale":"2024-09-21T02:18:53.008387+0000","last_undegraded":"2024-09-21T02:18:53.008387+0000","last_fullsized":"2024-09-21T02:18:53.008387+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:12.381672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011332+0000","last_change":"2024-09-21T02:18:51.372338+0000","last_active":"2024-09-21T02:18:53.011332+0000","last_peered":"2024-09-21T02:18:53.011332+0000","last_clean":"2024-09-21T02:18:53.011332+0000","last_became_active":"2024-09-21T02:18:51.358808+0000","last_became_peered":"2024-09-21T02:18:51.358808+0000","last_unstale":"2024-09-21T02:18:53.011332+0000","last_undegraded":"2024-09-21T02:18:53.011332+0000","last_fullsized":"2024-09-21T02:18:53.011332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:27.819855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008158+0000","last_change":"2024-09-21T02:18:50.326586+0000","last_active":"2024-09-21T02:18:53.008158+0000","last_peered":"2024-09-21T02:18:53.008158+0000","last_clean":"2024-09-21T02:18:53.008158+0000","last_became_active":"2024-09-21T02:18:50.325718+0000","last_became_peered":"2024-09-21T02:18:50.325718+0000","last_unstale":"2024-09-21T02:18:53.008158+0000","last_undegraded":"2024-09-21T02:18:53.008158+0000","last_fullsized":"2024-09-21T02:18:53.008158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:39.655636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943549+0000","last_change":"2024-09-21T02:18:51.382674+0000","last_active":"2024-09-21T02:18:52.943549+0000","last_peered":"2024-09-21T02:18:52.943549+0000","last_clean":"2024-09-21T02:18:52.943549+0000","last_became_active":"2024-09-21T02:18:51.382258+0000","last_became_peered":"2024-09-21T02:18:51.382258+0000","last_unstale":"2024-09-21T02:18:52.943549+0000","last_undegraded":"2024-09-21T02:18:52.943549+0000","last_fullsized":"2024-09-21T02:18:52.943549+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:42.692409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336625+0000","last_change":"2024-09-21T02:18:50.313069+0000","last_active":"2024-09-21T02:18:52.336625+0000","last_peered":"2024-09-21T02:18:52.336625+0000","last_clean":"2024-09-21T02:18:52.336625+0000","last_became_active":"2024-09-21T02:18:50.312597+0000","last_became_peered":"2024-09-21T02:18:50.312597+0000","last_unstale":"2024-09-21T02:18:52.336625+0000","last_undegraded":"2024-09-21T02:18:52.336625+0000","last_fullsized":"2024-09-21T02:18:52.336625+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.423876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011027+0000","last_change":"2024-09-21T02:18:51.373621+0000","last_active":"2024-09-21T02:18:53.011027+0000","last_peered":"2024-09-21T02:18:53.011027+0000","last_clean":"2024-09-21T02:18:53.011027+0000","last_became_active":"2024-09-21T02:18:51.372053+0000","last_became_peered":"2024-09-21T02:18:51.372053+0000","last_unstale":"2024-09-21T02:18:53.011027+0000","last_undegraded":"2024-09-21T02:18:53.011027+0000","last_fullsized":"2024-09-21T02:18:53.011027+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:59.720175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009588+0000","last_change":"2024-09-21T02:18:50.337866+0000","last_active":"2024-09-21T02:18:53.009588+0000","last_peered":"2024-09-21T02:18:53.009588+0000","last_clean":"2024-09-21T02:18:53.009588+0000","last_became_active":"2024-09-21T02:18:50.337213+0000","last_became_peered":"2024-09-21T02:18:50.337213+0000","last_unstale":"2024-09-21T02:18:53.009588+0000","last_undegraded":"2024-09-21T02:18:53.009588+0000","last_fullsized":"2024-09-21T02:18:53.009588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:37.075437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009890+0000","last_change":"2024-09-21T02:18:51.377478+0000","last_active":"2024-09-21T02:18:53.009890+0000","last_peered":"2024-09-21T02:18:53.009890+0000","last_clean":"2024-09-21T02:18:53.009890+0000","last_became_active":"2024-09-21T02:18:51.376738+0000","last_became_peered":"2024-09-21T02:18:51.376738+0000","last_unstale":"2024-09-21T02:18:53.009890+0000","last_undegraded":"2024-09-21T02:18:53.009890+0000","last_fullsized":"2024-09-21T02:18:53.009890+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:01.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753502+0000","last_change":"2024-09-21T02:18:50.334715+0000","last_active":"2024-09-21T02:18:52.753502+0000","last_peered":"2024-09-21T02:18:52.753502+0000","last_clean":"2024-09-21T02:18:52.753502+0000","last_became_active":"2024-09-21T02:18:50.333717+0000","last_became_peered":"2024-09-21T02:18:50.333717+0000","last_unstale":"2024-09-21T02:18:52.753502+0000","last_undegraded":"2024-09-21T02:18:52.753502+0000","last_fullsized":"2024-09-21T02:18:52.753502+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:10.659304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.340186+0000","last_change":"2024-09-21T02:18:51.368560+0000","last_active":"2024-09-21T02:18:52.340186+0000","last_peered":"2024-09-21T02:18:52.340186+0000","last_clean":"2024-09-21T02:18:52.340186+0000","last_became_active":"2024-09-21T02:18:51.368179+0000","last_became_peered":"2024-09-21T02:18:51.368179+0000","last_unstale":"2024-09-21T02:18:52.340186+0000","last_undegraded":"2024-09-21T02:18:52.340186+0000","last_fullsized":"2024-09-21T02:18:52.340186+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:41.825495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.339628+0000","last_change":"2024-09-21T02:18:50.312150+0000","last_active":"2024-09-21T02:18:52.339628+0000","last_peered":"2024-09-21T02:18:52.339628+0000","last_clean":"2024-09-21T02:18:52.339628+0000","last_became_active":"2024-09-21T02:18:50.311220+0000","last_became_peered":"2024-09-21T02:18:50.311220+0000","last_unstale":"2024-09-21T02:18:52.339628+0000","last_undegraded":"2024-09-21T02:18:52.339628+0000","last_fullsized":"2024-09-21T02:18:52.339628+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:25.251261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007944+0000","last_change":"2024-09-21T02:18:51.364984+0000","last_active":"2024-09-21T02:18:53.007944+0000","last_peered":"2024-09-21T02:18:53.007944+0000","last_clean":"2024-09-21T02:18:53.007944+0000","last_became_active":"2024-09-21T02:18:51.364425+0000","last_became_peered":"2024-09-21T02:18:51.364425+0000","last_unstale":"2024-09-21T02:18:53.007944+0000","last_undegraded":"2024-09-21T02:18:53.007944+0000","last_fullsized":"2024-09-21T02:18:53.007944+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:40.670689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008621+0000","last_change":"2024-09-21T02:18:50.321012+0000","last_active":"2024-09-21T02:18:53.008621+0000","last_peered":"2024-09-21T02:18:53.008621+0000","last_clean":"2024-09-21T02:18:53.008621+0000","last_became_active":"2024-09-21T02:18:50.320553+0000","last_became_peered":"2024-09-21T02:18:50.320553+0000","last_unstale":"2024-09-21T02:18:53.008621+0000","last_undegraded":"2024-09-21T02:18:53.008621+0000","last_fullsized":"2024-09-21T02:18:53.008621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:30.947988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.942684+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:18:52.942684+0000","last_peered":"2024-09-21T02:18:52.942684+0000","last_clean":"2024-09-21T02:18:52.942684+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:18:52.942684+0000","last_undegraded":"2024-09-21T02:18:52.942684+0000","last_fullsized":"2024-09-21T02:18:52.942684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":552625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1261568,"data_stored":550889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1863680,"data_stored":1141349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542216,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":552591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1261568,"data_stored":550889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1273856,"data_stored":551527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1855488,"data_stored":1141257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36588,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335252,"statfs":{"total":96636764160,"available":96599298048,"internally_reserved":0,"allocated":1269760,"data_stored":551541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:18:55.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabef09e640 1 -- 172.21.15.170:0/2302968470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 msgr2=0x7fabd0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabef09e640 1 --2- 172.21.15.170:0/2302968470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabd0005160 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fabe0006fa0 tx=0x7fabe001f510 comp rx=0 tx=0).stop 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabed63b640 1 -- 172.21.15.170:0/2302968470 reap_dead start 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabef09e640 1 -- 172.21.15.170:0/2302968470 shutdown_connections 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabef09e640 1 -- 172.21.15.170:0/2302968470 >> 172.21.15.170:0/2302968470 conn(0x7fabe80b1c40 msgr2=0x7fabe80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabef09e640 1 -- 172.21.15.170:0/2302968470 shutdown_connections 2024-09-21T02:18:55.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.691+0000 7fabef09e640 1 -- 172.21.15.170:0/2302968470 wait complete. 2024-09-21T02:18:55.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 Processor -- start 2024-09-21T02:18:55.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 -- start start 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabe815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabe8150af0 0x7fabe80b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe8150260 0x7fabe80cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80cb500 con 0x7fabd0004d50 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8074f00 con 0x7fabe8150260 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fabe80cb680 con 0x7fabe8150af0 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabe815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabe815ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51354/0 (socket says 172.21.15.170:51354) 2024-09-21T02:18:55.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 -- 172.21.15.170:0/2470655865 learned_addr learned my addr 172.21.15.170:0/2470655865 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabe7fff640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabe8150af0 0x7fabe80b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabed63b640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe8150260 0x7fabe80cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 -- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabe8150af0 msgr2=0x7fabe80b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabe8150af0 0x7fabe80b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 -- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe8150260 msgr2=0x7fabe80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe8150260 0x7fabe80cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 -- 172.21.15.170:0/2470655865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe80afa50 con 0x7fabd0004d50 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabece3a640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabe815ec20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e970 tx=0x7fabd800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd800ccc0 con 0x7fabd0004d50 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabd800ce60 con 0x7fabd0004d50 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8005260 con 0x7fabd0004d50 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe80afd30 con 0x7fabd0004d50 2024-09-21T02:18:55.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.695+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe80b0d50 con 0x7fabd0004d50 2024-09-21T02:18:55.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.699+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe80ce6d0 con 0x7fabd0004d50 2024-09-21T02:18:55.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.699+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fabd8005400 con 0x7fabd0004d50 2024-09-21T02:18:55.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.699+0000 7fabe57fa640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fabd4073320 0x7fabd4075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:55.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.699+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fabd80951c0 con 0x7fabd0004d50 2024-09-21T02:18:55.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.699+0000 7fabe7fff640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fabd4073320 0x7fabd4075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:55.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.699+0000 7fabe7fff640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fabd4073320 0x7fabd4075830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fabe80d0ca0 tx=0x7fabdc0095f0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:55.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.703+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fabd805fa50 con 0x7fabd0004d50 2024-09-21T02:18:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.871+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabe80ce2a0 con 0x7fabd4073320 2024-09-21T02:18:55.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.883+0000 7fabe57fa640 1 -- 172.21.15.170:0/2470655865 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252409 (secure 0 0 0) 0x7fabe80ce2a0 con 0x7fabd4073320 2024-09-21T02:18:55.889 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:55.890 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:18:55.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fabd4073320 msgr2=0x7fabd4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fabd4073320 0x7fabd4075830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fabe80d0ca0 tx=0x7fabdc0095f0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 msgr2=0x7fabe815ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabe815ec20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e970 tx=0x7fabd800ee30 comp rx=0 tx=0).stop 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 shutdown_connections 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fabd4073320 0x7fabd4075830 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabe8150af0 0x7fabe80b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabe8150260 0x7fabe80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 --2- 172.21.15.170:0/2470655865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabd0004d50 0x7fabe815ec20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 >> 172.21.15.170:0/2470655865 conn(0x7fabe80b1c40 msgr2=0x7fabe80b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 shutdown_connections 2024-09-21T02:18:55.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:55.887+0000 7fabef09e640 1 -- 172.21.15.170:0/2470655865 wait complete. 2024-09-21T02:18:55.911 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-21T02:18:54.872596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281396,"kb_used_data":11080,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754693324,"statfs":{"total":773094113280,"available":772805963776,"internally_reserved":0,"allocated":11345920,"data_stored":5592668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.583572"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753720+0000","last_change":"2024-09-21T02:18:51.378851+0000","last_active":"2024-09-21T02:18:52.753720+0000","last_peered":"2024-09-21T02:18:52.753720+0000","last_clean":"2024-09-21T02:18:52.753720+0000","last_became_active":"2024-09-21T02:18:51.378526+0000","last_became_peered":"2024-09-21T02:18:51.378526+0000","last_unstale":"2024-09-21T02:18:52.753720+0000","last_undegraded":"2024-09-21T02:18:52.753720+0000","last_fullsized":"2024-09-21T02:18:52.753720+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:53.869335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010701+0000","last_change":"2024-09-21T02:18:51.352342+0000","last_active":"2024-09-21T02:18:53.010701+0000","last_peered":"2024-09-21T02:18:53.010701+0000","last_clean":"2024-09-21T02:18:53.010701+0000","last_became_active":"2024-09-21T02:18:51.351524+0000","last_became_peered":"2024-09-21T02:18:51.351524+0000","last_unstale":"2024-09-21T02:18:53.010701+0000","last_undegraded":"2024-09-21T02:18:53.010701+0000","last_fullsized":"2024-09-21T02:18:53.010701+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.770042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943612+0000","last_change":"2024-09-21T02:18:51.369596+0000","last_active":"2024-09-21T02:18:52.943612+0000","last_peered":"2024-09-21T02:18:52.943612+0000","last_clean":"2024-09-21T02:18:52.943612+0000","last_became_active":"2024-09-21T02:18:51.368539+0000","last_became_peered":"2024-09-21T02:18:51.368539+0000","last_unstale":"2024-09-21T02:18:52.943612+0000","last_undegraded":"2024-09-21T02:18:52.943612+0000","last_fullsized":"2024-09-21T02:18:52.943612+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:47.106686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007846+0000","last_change":"2024-09-21T02:18:51.366490+0000","last_active":"2024-09-21T02:18:53.007846+0000","last_peered":"2024-09-21T02:18:53.007846+0000","last_clean":"2024-09-21T02:18:53.007846+0000","last_became_active":"2024-09-21T02:18:51.366155+0000","last_became_peered":"2024-09-21T02:18:51.366155+0000","last_unstale":"2024-09-21T02:18:53.007846+0000","last_undegraded":"2024-09-21T02:18:53.007846+0000","last_fullsized":"2024-09-21T02:18:53.007846+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:13.158703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344056+0000","last_change":"2024-09-21T02:18:51.376049+0000","last_active":"2024-09-21T02:18:52.344056+0000","last_peered":"2024-09-21T02:18:52.344056+0000","last_clean":"2024-09-21T02:18:52.344056+0000","last_became_active":"2024-09-21T02:18:51.375760+0000","last_became_peered":"2024-09-21T02:18:51.375760+0000","last_unstale":"2024-09-21T02:18:52.344056+0000","last_undegraded":"2024-09-21T02:18:52.344056+0000","last_fullsized":"2024-09-21T02:18:52.344056+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:16.438338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.340641+0000","last_change":"2024-09-21T02:18:51.366095+0000","last_active":"2024-09-21T02:18:52.340641+0000","last_peered":"2024-09-21T02:18:52.340641+0000","last_clean":"2024-09-21T02:18:52.340641+0000","last_became_active":"2024-09-21T02:18:51.351619+0000","last_became_peered":"2024-09-21T02:18:51.351619+0000","last_unstale":"2024-09-21T02:18:52.340641+0000","last_undegraded":"2024-09-21T02:18:52.340641+0000","last_fullsized":"2024-09-21T02:18:52.340641+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:28.542692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336041+0000","last_change":"2024-09-21T02:18:51.376854+0000","last_active":"2024-09-21T02:18:52.336041+0000","last_peered":"2024-09-21T02:18:52.336041+0000","last_clean":"2024-09-21T02:18:52.336041+0000","last_became_active":"2024-09-21T02:18:51.376379+0000","last_became_peered":"2024-09-21T02:18:51.376379+0000","last_unstale":"2024-09-21T02:18:52.336041+0000","last_undegraded":"2024-09-21T02:18:52.336041+0000","last_fullsized":"2024-09-21T02:18:52.336041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:17.189098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336396+0000","last_change":"2024-09-21T02:18:51.357638+0000","last_active":"2024-09-21T02:18:52.336396+0000","last_peered":"2024-09-21T02:18:52.336396+0000","last_clean":"2024-09-21T02:18:52.336396+0000","last_became_active":"2024-09-21T02:18:51.356789+0000","last_became_peered":"2024-09-21T02:18:51.356789+0000","last_unstale":"2024-09-21T02:18:52.336396+0000","last_undegraded":"2024-09-21T02:18:52.336396+0000","last_fullsized":"2024-09-21T02:18:52.336396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:40.792817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754051+0000","last_change":"2024-09-21T02:18:51.376931+0000","last_active":"2024-09-21T02:18:52.754051+0000","last_peered":"2024-09-21T02:18:52.754051+0000","last_clean":"2024-09-21T02:18:52.754051+0000","last_became_active":"2024-09-21T02:18:51.376542+0000","last_became_peered":"2024-09-21T02:18:51.376542+0000","last_unstale":"2024-09-21T02:18:52.754051+0000","last_undegraded":"2024-09-21T02:18:52.754051+0000","last_fullsized":"2024-09-21T02:18:52.754051+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:03:27.191274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343513+0000","last_change":"2024-09-21T02:18:51.369083+0000","last_active":"2024-09-21T02:18:52.343513+0000","last_peered":"2024-09-21T02:18:52.343513+0000","last_clean":"2024-09-21T02:18:52.343513+0000","last_became_active":"2024-09-21T02:18:51.368721+0000","last_became_peered":"2024-09-21T02:18:51.368721+0000","last_unstale":"2024-09-21T02:18:52.343513+0000","last_undegraded":"2024-09-21T02:18:52.343513+0000","last_fullsized":"2024-09-21T02:18:52.343513+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:00.422505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336638+0000","last_change":"2024-09-21T02:18:51.376798+0000","last_active":"2024-09-21T02:18:52.336638+0000","last_peered":"2024-09-21T02:18:52.336638+0000","last_clean":"2024-09-21T02:18:52.336638+0000","last_became_active":"2024-09-21T02:18:51.376417+0000","last_became_peered":"2024-09-21T02:18:51.376417+0000","last_unstale":"2024-09-21T02:18:52.336638+0000","last_undegraded":"2024-09-21T02:18:52.336638+0000","last_fullsized":"2024-09-21T02:18:52.336638+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.171310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343279+0000","last_change":"2024-09-21T02:18:51.369918+0000","last_active":"2024-09-21T02:18:52.343279+0000","last_peered":"2024-09-21T02:18:52.343279+0000","last_clean":"2024-09-21T02:18:52.343279+0000","last_became_active":"2024-09-21T02:18:51.368625+0000","last_became_peered":"2024-09-21T02:18:51.368625+0000","last_unstale":"2024-09-21T02:18:52.343279+0000","last_undegraded":"2024-09-21T02:18:52.343279+0000","last_fullsized":"2024-09-21T02:18:52.343279+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:04.387341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009534+0000","last_change":"2024-09-21T02:18:51.374866+0000","last_active":"2024-09-21T02:18:53.009534+0000","last_peered":"2024-09-21T02:18:53.009534+0000","last_clean":"2024-09-21T02:18:53.009534+0000","last_became_active":"2024-09-21T02:18:51.374429+0000","last_became_peered":"2024-09-21T02:18:51.374429+0000","last_unstale":"2024-09-21T02:18:53.009534+0000","last_undegraded":"2024-09-21T02:18:53.009534+0000","last_fullsized":"2024-09-21T02:18:53.009534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:38.267171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009384+0000","last_change":"2024-09-21T02:18:51.376621+0000","last_active":"2024-09-21T02:18:53.009384+0000","last_peered":"2024-09-21T02:18:53.009384+0000","last_clean":"2024-09-21T02:18:53.009384+0000","last_became_active":"2024-09-21T02:18:51.373018+0000","last_became_peered":"2024-09-21T02:18:51.373018+0000","last_unstale":"2024-09-21T02:18:53.009384+0000","last_undegraded":"2024-09-21T02:18:53.009384+0000","last_fullsized":"2024-09-21T02:18:53.009384+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:38:38.267171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753124+0000","last_change":"2024-09-21T02:18:51.362484+0000","last_active":"2024-09-21T02:18:52.753124+0000","last_peered":"2024-09-21T02:18:52.753124+0000","last_clean":"2024-09-21T02:18:52.753124+0000","last_became_active":"2024-09-21T02:18:51.361564+0000","last_became_peered":"2024-09-21T02:18:51.361564+0000","last_unstale":"2024-09-21T02:18:52.753124+0000","last_undegraded":"2024-09-21T02:18:52.753124+0000","last_fullsized":"2024-09-21T02:18:52.753124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.895473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.342817+0000","last_change":"2024-09-21T02:18:51.379928+0000","last_active":"2024-09-21T02:18:52.342817+0000","last_peered":"2024-09-21T02:18:52.342817+0000","last_clean":"2024-09-21T02:18:52.342817+0000","last_became_active":"2024-09-21T02:18:51.379551+0000","last_became_peered":"2024-09-21T02:18:51.379551+0000","last_unstale":"2024-09-21T02:18:52.342817+0000","last_undegraded":"2024-09-21T02:18:52.342817+0000","last_fullsized":"2024-09-21T02:18:52.342817+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:41:23.904068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.944410+0000","last_change":"2024-09-21T02:18:51.380772+0000","last_active":"2024-09-21T02:18:52.944410+0000","last_peered":"2024-09-21T02:18:52.944410+0000","last_clean":"2024-09-21T02:18:52.944410+0000","last_became_active":"2024-09-21T02:18:51.380251+0000","last_became_peered":"2024-09-21T02:18:51.380251+0000","last_unstale":"2024-09-21T02:18:52.944410+0000","last_undegraded":"2024-09-21T02:18:52.944410+0000","last_fullsized":"2024-09-21T02:18:52.944410+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:11.670754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336114+0000","last_change":"2024-09-21T02:18:51.375772+0000","last_active":"2024-09-21T02:18:52.336114+0000","last_peered":"2024-09-21T02:18:52.336114+0000","last_clean":"2024-09-21T02:18:52.336114+0000","last_became_active":"2024-09-21T02:18:51.374927+0000","last_became_peered":"2024-09-21T02:18:51.374927+0000","last_unstale":"2024-09-21T02:18:52.336114+0000","last_undegraded":"2024-09-21T02:18:52.336114+0000","last_fullsized":"2024-09-21T02:18:52.336114+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:26.910468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.944158+0000","last_change":"2024-09-21T02:18:51.368209+0000","last_active":"2024-09-21T02:18:52.944158+0000","last_peered":"2024-09-21T02:18:52.944158+0000","last_clean":"2024-09-21T02:18:52.944158+0000","last_became_active":"2024-09-21T02:18:51.367725+0000","last_became_peered":"2024-09-21T02:18:51.367725+0000","last_unstale":"2024-09-21T02:18:52.944158+0000","last_undegraded":"2024-09-21T02:18:52.944158+0000","last_fullsized":"2024-09-21T02:18:52.944158+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:11.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943691+0000","last_change":"2024-09-21T02:18:51.381829+0000","last_active":"2024-09-21T02:18:52.943691+0000","last_peered":"2024-09-21T02:18:52.943691+0000","last_clean":"2024-09-21T02:18:52.943691+0000","last_became_active":"2024-09-21T02:18:51.381468+0000","last_became_peered":"2024-09-21T02:18:51.381468+0000","last_unstale":"2024-09-21T02:18:52.943691+0000","last_undegraded":"2024-09-21T02:18:52.943691+0000","last_fullsized":"2024-09-21T02:18:52.943691+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:16.148023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336557+0000","last_change":"2024-09-21T02:18:51.376635+0000","last_active":"2024-09-21T02:18:52.336557+0000","last_peered":"2024-09-21T02:18:52.336557+0000","last_clean":"2024-09-21T02:18:52.336557+0000","last_became_active":"2024-09-21T02:18:51.374015+0000","last_became_peered":"2024-09-21T02:18:51.374015+0000","last_unstale":"2024-09-21T02:18:52.336557+0000","last_undegraded":"2024-09-21T02:18:52.336557+0000","last_fullsized":"2024-09-21T02:18:52.336557+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:01.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341211+0000","last_change":"2024-09-21T02:18:51.363682+0000","last_active":"2024-09-21T02:18:52.341211+0000","last_peered":"2024-09-21T02:18:52.341211+0000","last_clean":"2024-09-21T02:18:52.341211+0000","last_became_active":"2024-09-21T02:18:51.363304+0000","last_became_peered":"2024-09-21T02:18:51.363304+0000","last_unstale":"2024-09-21T02:18:52.341211+0000","last_undegraded":"2024-09-21T02:18:52.341211+0000","last_fullsized":"2024-09-21T02:18:52.341211+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:33.984312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009819+0000","last_change":"2024-09-21T02:18:51.386532+0000","last_active":"2024-09-21T02:18:53.009819+0000","last_peered":"2024-09-21T02:18:53.009819+0000","last_clean":"2024-09-21T02:18:53.009819+0000","last_became_active":"2024-09-21T02:18:51.386092+0000","last_became_peered":"2024-09-21T02:18:51.386092+0000","last_unstale":"2024-09-21T02:18:53.009819+0000","last_undegraded":"2024-09-21T02:18:53.009819+0000","last_fullsized":"2024-09-21T02:18:53.009819+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.770042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007333+0000","last_change":"2024-09-21T02:18:51.373297+0000","last_active":"2024-09-21T02:18:53.007333+0000","last_peered":"2024-09-21T02:18:53.007333+0000","last_clean":"2024-09-21T02:18:53.007333+0000","last_became_active":"2024-09-21T02:18:51.372098+0000","last_became_peered":"2024-09-21T02:18:51.372098+0000","last_unstale":"2024-09-21T02:18:53.007333+0000","last_undegraded":"2024-09-21T02:18:53.007333+0000","last_fullsized":"2024-09-21T02:18:53.007333+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:35.102595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336910+0000","last_change":"2024-09-21T02:18:51.362546+0000","last_active":"2024-09-21T02:18:52.336910+0000","last_peered":"2024-09-21T02:18:52.336910+0000","last_clean":"2024-09-21T02:18:52.336910+0000","last_became_active":"2024-09-21T02:18:51.361625+0000","last_became_peered":"2024-09-21T02:18:51.361625+0000","last_unstale":"2024-09-21T02:18:52.336910+0000","last_undegraded":"2024-09-21T02:18:52.336910+0000","last_fullsized":"2024-09-21T02:18:52.336910+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:58.263173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341455+0000","last_change":"2024-09-21T02:18:51.366119+0000","last_active":"2024-09-21T02:18:52.341455+0000","last_peered":"2024-09-21T02:18:52.341455+0000","last_clean":"2024-09-21T02:18:52.341455+0000","last_became_active":"2024-09-21T02:18:51.365698+0000","last_became_peered":"2024-09-21T02:18:51.365698+0000","last_unstale":"2024-09-21T02:18:52.341455+0000","last_undegraded":"2024-09-21T02:18:52.341455+0000","last_fullsized":"2024-09-21T02:18:52.341455+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:04.213388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008230+0000","last_change":"2024-09-21T02:18:51.378598+0000","last_active":"2024-09-21T02:18:53.008230+0000","last_peered":"2024-09-21T02:18:53.008230+0000","last_clean":"2024-09-21T02:18:53.008230+0000","last_became_active":"2024-09-21T02:18:51.378233+0000","last_became_peered":"2024-09-21T02:18:51.378233+0000","last_unstale":"2024-09-21T02:18:53.008230+0000","last_undegraded":"2024-09-21T02:18:53.008230+0000","last_fullsized":"2024-09-21T02:18:53.008230+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:05.298429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341385+0000","last_change":"2024-09-21T02:18:51.370244+0000","last_active":"2024-09-21T02:18:52.341385+0000","last_peered":"2024-09-21T02:18:52.341385+0000","last_clean":"2024-09-21T02:18:52.341385+0000","last_became_active":"2024-09-21T02:18:51.369831+0000","last_became_peered":"2024-09-21T02:18:51.369831+0000","last_unstale":"2024-09-21T02:18:52.341385+0000","last_undegraded":"2024-09-21T02:18:52.341385+0000","last_fullsized":"2024-09-21T02:18:52.341385+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:54.551271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011474+0000","last_change":"2024-09-21T02:18:51.378969+0000","last_active":"2024-09-21T02:18:53.011474+0000","last_peered":"2024-09-21T02:18:53.011474+0000","last_clean":"2024-09-21T02:18:53.011474+0000","last_became_active":"2024-09-21T02:18:51.374553+0000","last_became_peered":"2024-09-21T02:18:51.374553+0000","last_unstale":"2024-09-21T02:18:53.011474+0000","last_undegraded":"2024-09-21T02:18:53.011474+0000","last_fullsized":"2024-09-21T02:18:53.011474+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:05.236636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.335971+0000","last_change":"2024-09-21T02:18:51.369757+0000","last_active":"2024-09-21T02:18:52.335971+0000","last_peered":"2024-09-21T02:18:52.335971+0000","last_clean":"2024-09-21T02:18:52.335971+0000","last_became_active":"2024-09-21T02:18:51.368900+0000","last_became_peered":"2024-09-21T02:18:51.368900+0000","last_unstale":"2024-09-21T02:18:52.335971+0000","last_undegraded":"2024-09-21T02:18:52.335971+0000","last_fullsized":"2024-09-21T02:18:52.335971+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:41.980159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753301+0000","last_change":"2024-09-21T02:18:51.375360+0000","last_active":"2024-09-21T02:18:52.753301+0000","last_peered":"2024-09-21T02:18:52.753301+0000","last_clean":"2024-09-21T02:18:52.753301+0000","last_became_active":"2024-09-21T02:18:51.375042+0000","last_became_peered":"2024-09-21T02:18:51.375042+0000","last_unstale":"2024-09-21T02:18:52.753301+0000","last_undegraded":"2024-09-21T02:18:52.753301+0000","last_fullsized":"2024-09-21T02:18:52.753301+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:17.851797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.345054+0000","last_change":"2024-09-21T02:18:51.375598+0000","last_active":"2024-09-21T02:18:52.345054+0000","last_peered":"2024-09-21T02:18:52.345054+0000","last_clean":"2024-09-21T02:18:52.345054+0000","last_became_active":"2024-09-21T02:18:51.375140+0000","last_became_peered":"2024-09-21T02:18:51.375140+0000","last_unstale":"2024-09-21T02:18:52.345054+0000","last_undegraded":"2024-09-21T02:18:52.345054+0000","last_fullsized":"2024-09-21T02:18:52.345054+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:15.627517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754711+0000","last_change":"2024-09-21T02:18:51.380710+0000","last_active":"2024-09-21T02:18:52.754711+0000","last_peered":"2024-09-21T02:18:52.754711+0000","last_clean":"2024-09-21T02:18:52.754711+0000","last_became_active":"2024-09-21T02:18:51.380370+0000","last_became_peered":"2024-09-21T02:18:51.380370+0000","last_unstale":"2024-09-21T02:18:52.754711+0000","last_undegraded":"2024-09-21T02:18:52.754711+0000","last_fullsized":"2024-09-21T02:18:52.754711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:15.884612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010445+0000","last_change":"2024-09-21T02:18:50.321338+0000","last_active":"2024-09-21T02:18:53.010445+0000","last_peered":"2024-09-21T02:18:53.010445+0000","last_clean":"2024-09-21T02:18:53.010445+0000","last_became_active":"2024-09-21T02:18:50.320665+0000","last_became_peered":"2024-09-21T02:18:50.320665+0000","last_unstale":"2024-09-21T02:18:53.010445+0000","last_undegraded":"2024-09-21T02:18:53.010445+0000","last_fullsized":"2024-09-21T02:18:53.010445+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:08.839500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010803+0000","last_change":"2024-09-21T02:18:51.378181+0000","last_active":"2024-09-21T02:18:53.010803+0000","last_peered":"2024-09-21T02:18:53.010803+0000","last_clean":"2024-09-21T02:18:53.010803+0000","last_became_active":"2024-09-21T02:18:51.375256+0000","last_became_peered":"2024-09-21T02:18:51.375256+0000","last_unstale":"2024-09-21T02:18:53.010803+0000","last_undegraded":"2024-09-21T02:18:53.010803+0000","last_fullsized":"2024-09-21T02:18:53.010803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:09.670726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010874+0000","last_change":"2024-09-21T02:18:50.324595+0000","last_active":"2024-09-21T02:18:53.010874+0000","last_peered":"2024-09-21T02:18:53.010874+0000","last_clean":"2024-09-21T02:18:53.010874+0000","last_became_active":"2024-09-21T02:18:50.324144+0000","last_became_peered":"2024-09-21T02:18:50.324144+0000","last_unstale":"2024-09-21T02:18:53.010874+0000","last_undegraded":"2024-09-21T02:18:53.010874+0000","last_fullsized":"2024-09-21T02:18:53.010874+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:07.730672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.341854+0000","last_change":"2024-09-21T02:18:51.367120+0000","last_active":"2024-09-21T02:18:52.341854+0000","last_peered":"2024-09-21T02:18:52.341854+0000","last_clean":"2024-09-21T02:18:52.341854+0000","last_became_active":"2024-09-21T02:18:51.366777+0000","last_became_peered":"2024-09-21T02:18:51.366777+0000","last_unstale":"2024-09-21T02:18:52.341854+0000","last_undegraded":"2024-09-21T02:18:52.341854+0000","last_fullsized":"2024-09-21T02:18:52.341854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.435325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008114+0000","last_change":"2024-09-21T02:18:50.343864+0000","last_active":"2024-09-21T02:18:53.008114+0000","last_peered":"2024-09-21T02:18:53.008114+0000","last_clean":"2024-09-21T02:18:53.008114+0000","last_became_active":"2024-09-21T02:18:50.342425+0000","last_became_peered":"2024-09-21T02:18:50.342425+0000","last_unstale":"2024-09-21T02:18:53.008114+0000","last_undegraded":"2024-09-21T02:18:53.008114+0000","last_fullsized":"2024-09-21T02:18:53.008114+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:33.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.944113+0000","last_change":"2024-09-21T02:18:51.378718+0000","last_active":"2024-09-21T02:18:52.944113+0000","last_peered":"2024-09-21T02:18:52.944113+0000","last_clean":"2024-09-21T02:18:52.944113+0000","last_became_active":"2024-09-21T02:18:51.370814+0000","last_became_peered":"2024-09-21T02:18:51.370814+0000","last_unstale":"2024-09-21T02:18:52.944113+0000","last_undegraded":"2024-09-21T02:18:52.944113+0000","last_fullsized":"2024-09-21T02:18:52.944113+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:31.960005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007447+0000","last_change":"2024-09-21T02:18:50.343726+0000","last_active":"2024-09-21T02:18:53.007447+0000","last_peered":"2024-09-21T02:18:53.007447+0000","last_clean":"2024-09-21T02:18:53.007447+0000","last_became_active":"2024-09-21T02:18:50.340809+0000","last_became_peered":"2024-09-21T02:18:50.340809+0000","last_unstale":"2024-09-21T02:18:53.007447+0000","last_undegraded":"2024-09-21T02:18:53.007447+0000","last_fullsized":"2024-09-21T02:18:53.007447+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:14.394768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007961+0000","last_change":"2024-09-21T02:18:51.373240+0000","last_active":"2024-09-21T02:18:53.007961+0000","last_peered":"2024-09-21T02:18:53.007961+0000","last_clean":"2024-09-21T02:18:53.007961+0000","last_became_active":"2024-09-21T02:18:51.372365+0000","last_became_peered":"2024-09-21T02:18:51.372365+0000","last_unstale":"2024-09-21T02:18:53.007961+0000","last_undegraded":"2024-09-21T02:18:53.007961+0000","last_fullsized":"2024-09-21T02:18:53.007961+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:20.711574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943224+0000","last_change":"2024-09-21T02:18:50.323170+0000","last_active":"2024-09-21T02:18:52.943224+0000","last_peered":"2024-09-21T02:18:52.943224+0000","last_clean":"2024-09-21T02:18:52.943224+0000","last_became_active":"2024-09-21T02:18:50.322482+0000","last_became_peered":"2024-09-21T02:18:50.322482+0000","last_unstale":"2024-09-21T02:18:52.943224+0000","last_undegraded":"2024-09-21T02:18:52.943224+0000","last_fullsized":"2024-09-21T02:18:52.943224+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:10.068268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336303+0000","last_change":"2024-09-21T02:18:51.371772+0000","last_active":"2024-09-21T02:18:52.336303+0000","last_peered":"2024-09-21T02:18:52.336303+0000","last_clean":"2024-09-21T02:18:52.336303+0000","last_became_active":"2024-09-21T02:18:51.370918+0000","last_became_peered":"2024-09-21T02:18:51.370918+0000","last_unstale":"2024-09-21T02:18:52.336303+0000","last_undegraded":"2024-09-21T02:18:52.336303+0000","last_fullsized":"2024-09-21T02:18:52.336303+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:28.157673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753157+0000","last_change":"2024-09-21T02:18:50.334892+0000","last_active":"2024-09-21T02:18:52.753157+0000","last_peered":"2024-09-21T02:18:52.753157+0000","last_clean":"2024-09-21T02:18:52.753157+0000","last_became_active":"2024-09-21T02:18:50.334191+0000","last_became_peered":"2024-09-21T02:18:50.334191+0000","last_unstale":"2024-09-21T02:18:52.753157+0000","last_undegraded":"2024-09-21T02:18:52.753157+0000","last_fullsized":"2024-09-21T02:18:52.753157+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:57:49.658604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343501+0000","last_change":"2024-09-21T02:18:51.380382+0000","last_active":"2024-09-21T02:18:52.343501+0000","last_peered":"2024-09-21T02:18:52.343501+0000","last_clean":"2024-09-21T02:18:52.343501+0000","last_became_active":"2024-09-21T02:18:51.380006+0000","last_became_peered":"2024-09-21T02:18:51.380006+0000","last_unstale":"2024-09-21T02:18:52.343501+0000","last_undegraded":"2024-09-21T02:18:52.343501+0000","last_fullsized":"2024-09-21T02:18:52.343501+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:05.004571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.335898+0000","last_change":"2024-09-21T02:18:50.335000+0000","last_active":"2024-09-21T02:18:52.335898+0000","last_peered":"2024-09-21T02:18:52.335898+0000","last_clean":"2024-09-21T02:18:52.335898+0000","last_became_active":"2024-09-21T02:18:50.334659+0000","last_became_peered":"2024-09-21T02:18:50.334659+0000","last_unstale":"2024-09-21T02:18:52.335898+0000","last_undegraded":"2024-09-21T02:18:52.335898+0000","last_fullsized":"2024-09-21T02:18:52.335898+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:50.621617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.335820+0000","last_change":"2024-09-21T02:18:51.366640+0000","last_active":"2024-09-21T02:18:52.335820+0000","last_peered":"2024-09-21T02:18:52.335820+0000","last_clean":"2024-09-21T02:18:52.335820+0000","last_became_active":"2024-09-21T02:18:51.365949+0000","last_became_peered":"2024-09-21T02:18:51.365949+0000","last_unstale":"2024-09-21T02:18:52.335820+0000","last_undegraded":"2024-09-21T02:18:52.335820+0000","last_fullsized":"2024-09-21T02:18:52.335820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:34.657350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343691+0000","last_change":"2024-09-21T02:18:50.349795+0000","last_active":"2024-09-21T02:18:52.343691+0000","last_peered":"2024-09-21T02:18:52.343691+0000","last_clean":"2024-09-21T02:18:52.343691+0000","last_became_active":"2024-09-21T02:18:50.348454+0000","last_became_peered":"2024-09-21T02:18:50.348454+0000","last_unstale":"2024-09-21T02:18:52.343691+0000","last_undegraded":"2024-09-21T02:18:52.343691+0000","last_fullsized":"2024-09-21T02:18:52.343691+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:55.271356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011367+0000","last_change":"2024-09-21T02:18:51.387062+0000","last_active":"2024-09-21T02:18:53.011367+0000","last_peered":"2024-09-21T02:18:53.011367+0000","last_clean":"2024-09-21T02:18:53.011367+0000","last_became_active":"2024-09-21T02:18:51.386443+0000","last_became_peered":"2024-09-21T02:18:51.386443+0000","last_unstale":"2024-09-21T02:18:53.011367+0000","last_undegraded":"2024-09-21T02:18:53.011367+0000","last_fullsized":"2024-09-21T02:18:53.011367+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:01.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008481+0000","last_change":"2024-09-21T02:18:50.346570+0000","last_active":"2024-09-21T02:18:53.008481+0000","last_peered":"2024-09-21T02:18:53.008481+0000","last_clean":"2024-09-21T02:18:53.008481+0000","last_became_active":"2024-09-21T02:18:50.345866+0000","last_became_peered":"2024-09-21T02:18:50.345866+0000","last_unstale":"2024-09-21T02:18:53.008481+0000","last_undegraded":"2024-09-21T02:18:53.008481+0000","last_fullsized":"2024-09-21T02:18:53.008481+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:35.714958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011656+0000","last_change":"2024-09-21T02:18:51.367061+0000","last_active":"2024-09-21T02:18:53.011656+0000","last_peered":"2024-09-21T02:18:53.011656+0000","last_clean":"2024-09-21T02:18:53.011656+0000","last_became_active":"2024-09-21T02:18:51.365641+0000","last_became_peered":"2024-09-21T02:18:51.365641+0000","last_unstale":"2024-09-21T02:18:53.011656+0000","last_undegraded":"2024-09-21T02:18:53.011656+0000","last_fullsized":"2024-09-21T02:18:53.011656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:49.672388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344061+0000","last_change":"2024-09-21T02:18:50.342751+0000","last_active":"2024-09-21T02:18:52.344061+0000","last_peered":"2024-09-21T02:18:52.344061+0000","last_clean":"2024-09-21T02:18:52.344061+0000","last_became_active":"2024-09-21T02:18:50.342042+0000","last_became_peered":"2024-09-21T02:18:50.342042+0000","last_unstale":"2024-09-21T02:18:52.344061+0000","last_undegraded":"2024-09-21T02:18:52.344061+0000","last_fullsized":"2024-09-21T02:18:52.344061+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:49.893730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.342025+0000","last_change":"2024-09-21T02:18:51.359503+0000","last_active":"2024-09-21T02:18:52.342025+0000","last_peered":"2024-09-21T02:18:52.342025+0000","last_clean":"2024-09-21T02:18:52.342025+0000","last_became_active":"2024-09-21T02:18:51.358536+0000","last_became_peered":"2024-09-21T02:18:51.358536+0000","last_unstale":"2024-09-21T02:18:52.342025+0000","last_undegraded":"2024-09-21T02:18:52.342025+0000","last_fullsized":"2024-09-21T02:18:52.342025+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:27.392053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343486+0000","last_change":"2024-09-21T02:18:50.349354+0000","last_active":"2024-09-21T02:18:52.343486+0000","last_peered":"2024-09-21T02:18:52.343486+0000","last_clean":"2024-09-21T02:18:52.343486+0000","last_became_active":"2024-09-21T02:18:50.348680+0000","last_became_peered":"2024-09-21T02:18:50.348680+0000","last_unstale":"2024-09-21T02:18:52.343486+0000","last_undegraded":"2024-09-21T02:18:52.343486+0000","last_fullsized":"2024-09-21T02:18:52.343486+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:27:51.815348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336813+0000","last_change":"2024-09-21T02:18:51.370188+0000","last_active":"2024-09-21T02:18:52.336813+0000","last_peered":"2024-09-21T02:18:52.336813+0000","last_clean":"2024-09-21T02:18:52.336813+0000","last_became_active":"2024-09-21T02:18:51.369814+0000","last_became_peered":"2024-09-21T02:18:51.369814+0000","last_unstale":"2024-09-21T02:18:52.336813+0000","last_undegraded":"2024-09-21T02:18:52.336813+0000","last_fullsized":"2024-09-21T02:18:52.336813+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:13.827056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":21,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008043+0000","last_change":"2024-09-21T02:18:50.346941+0000","last_active":"2024-09-21T02:18:53.008043+0000","last_peered":"2024-09-21T02:18:53.008043+0000","last_clean":"2024-09-21T02:18:53.008043+0000","last_became_active":"2024-09-21T02:18:50.346112+0000","last_became_peered":"2024-09-21T02:18:50.346112+0000","last_unstale":"2024-09-21T02:18:53.008043+0000","last_undegraded":"2024-09-21T02:18:53.008043+0000","last_fullsized":"2024-09-21T02:18:53.008043+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:11:16.788465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343750+0000","last_change":"2024-09-21T02:18:51.379537+0000","last_active":"2024-09-21T02:18:52.343750+0000","last_peered":"2024-09-21T02:18:52.343750+0000","last_clean":"2024-09-21T02:18:52.343750+0000","last_became_active":"2024-09-21T02:18:51.379153+0000","last_became_peered":"2024-09-21T02:18:51.379153+0000","last_unstale":"2024-09-21T02:18:52.343750+0000","last_undegraded":"2024-09-21T02:18:52.343750+0000","last_fullsized":"2024-09-21T02:18:52.343750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:19.300698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344009+0000","last_change":"2024-09-21T02:18:50.336588+0000","last_active":"2024-09-21T02:18:52.344009+0000","last_peered":"2024-09-21T02:18:52.344009+0000","last_clean":"2024-09-21T02:18:52.344009+0000","last_became_active":"2024-09-21T02:18:50.335711+0000","last_became_peered":"2024-09-21T02:18:50.335711+0000","last_unstale":"2024-09-21T02:18:52.344009+0000","last_undegraded":"2024-09-21T02:18:52.344009+0000","last_fullsized":"2024-09-21T02:18:52.344009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:48.958670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008361+0000","last_change":"2024-09-21T02:18:51.374035+0000","last_active":"2024-09-21T02:18:53.008361+0000","last_peered":"2024-09-21T02:18:53.008361+0000","last_clean":"2024-09-21T02:18:53.008361+0000","last_became_active":"2024-09-21T02:18:51.373620+0000","last_became_peered":"2024-09-21T02:18:51.373620+0000","last_unstale":"2024-09-21T02:18:53.008361+0000","last_undegraded":"2024-09-21T02:18:53.008361+0000","last_fullsized":"2024-09-21T02:18:53.008361+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:51.972508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943768+0000","last_change":"2024-09-21T02:18:50.340882+0000","last_active":"2024-09-21T02:18:52.943768+0000","last_peered":"2024-09-21T02:18:52.943768+0000","last_clean":"2024-09-21T02:18:52.943768+0000","last_became_active":"2024-09-21T02:18:50.340470+0000","last_became_peered":"2024-09-21T02:18:50.340470+0000","last_unstale":"2024-09-21T02:18:52.943768+0000","last_undegraded":"2024-09-21T02:18:52.943768+0000","last_fullsized":"2024-09-21T02:18:52.943768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:06.718783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007693+0000","last_change":"2024-09-21T02:18:51.353718+0000","last_active":"2024-09-21T02:18:53.007693+0000","last_peered":"2024-09-21T02:18:53.007693+0000","last_clean":"2024-09-21T02:18:53.007693+0000","last_became_active":"2024-09-21T02:18:51.352917+0000","last_became_peered":"2024-09-21T02:18:51.352917+0000","last_unstale":"2024-09-21T02:18:53.007693+0000","last_undegraded":"2024-09-21T02:18:53.007693+0000","last_fullsized":"2024-09-21T02:18:53.007693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:13.566169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":23,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.752723+0000","last_change":"2024-09-21T02:18:50.333888+0000","last_active":"2024-09-21T02:18:52.752723+0000","last_peered":"2024-09-21T02:18:52.752723+0000","last_clean":"2024-09-21T02:18:52.752723+0000","last_became_active":"2024-09-21T02:18:50.333064+0000","last_became_peered":"2024-09-21T02:18:50.333064+0000","last_unstale":"2024-09-21T02:18:52.752723+0000","last_undegraded":"2024-09-21T02:18:52.752723+0000","last_fullsized":"2024-09-21T02:18:52.752723+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:10.591684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754553+0000","last_change":"2024-09-21T02:18:51.370287+0000","last_active":"2024-09-21T02:18:52.754553+0000","last_peered":"2024-09-21T02:18:52.754553+0000","last_clean":"2024-09-21T02:18:52.754553+0000","last_became_active":"2024-09-21T02:18:51.367728+0000","last_became_peered":"2024-09-21T02:18:51.367728+0000","last_unstale":"2024-09-21T02:18:52.754553+0000","last_undegraded":"2024-09-21T02:18:52.754553+0000","last_fullsized":"2024-09-21T02:18:52.754553+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:34.561754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.342484+0000","last_change":"2024-09-21T02:18:50.325571+0000","last_active":"2024-09-21T02:18:52.342484+0000","last_peered":"2024-09-21T02:18:52.342484+0000","last_clean":"2024-09-21T02:18:52.342484+0000","last_became_active":"2024-09-21T02:18:50.324409+0000","last_became_peered":"2024-09-21T02:18:50.324409+0000","last_unstale":"2024-09-21T02:18:52.342484+0000","last_undegraded":"2024-09-21T02:18:52.342484+0000","last_fullsized":"2024-09-21T02:18:52.342484+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:01.153260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.754117+0000","last_change":"2024-09-21T02:18:51.379981+0000","last_active":"2024-09-21T02:18:52.754117+0000","last_peered":"2024-09-21T02:18:52.754117+0000","last_clean":"2024-09-21T02:18:52.754117+0000","last_became_active":"2024-09-21T02:18:51.379623+0000","last_became_peered":"2024-09-21T02:18:51.379623+0000","last_unstale":"2024-09-21T02:18:52.754117+0000","last_undegraded":"2024-09-21T02:18:52.754117+0000","last_fullsized":"2024-09-21T02:18:52.754117+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:30:44.151154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.343768+0000","last_change":"2024-09-21T02:18:50.342086+0000","last_active":"2024-09-21T02:18:52.343768+0000","last_peered":"2024-09-21T02:18:52.343768+0000","last_clean":"2024-09-21T02:18:52.343768+0000","last_became_active":"2024-09-21T02:18:50.339265+0000","last_became_peered":"2024-09-21T02:18:50.339265+0000","last_unstale":"2024-09-21T02:18:52.343768+0000","last_undegraded":"2024-09-21T02:18:52.343768+0000","last_fullsized":"2024-09-21T02:18:52.343768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:17.337679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336573+0000","last_change":"2024-09-21T02:18:51.374582+0000","last_active":"2024-09-21T02:18:52.336573+0000","last_peered":"2024-09-21T02:18:52.336573+0000","last_clean":"2024-09-21T02:18:52.336573+0000","last_became_active":"2024-09-21T02:18:51.374221+0000","last_became_peered":"2024-09-21T02:18:51.374221+0000","last_unstale":"2024-09-21T02:18:52.336573+0000","last_undegraded":"2024-09-21T02:18:52.336573+0000","last_fullsized":"2024-09-21T02:18:52.336573+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:09.039842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010410+0000","last_change":"2024-09-21T02:18:50.318191+0000","last_active":"2024-09-21T02:18:53.010410+0000","last_peered":"2024-09-21T02:18:53.010410+0000","last_clean":"2024-09-21T02:18:53.010410+0000","last_became_active":"2024-09-21T02:18:50.317786+0000","last_became_peered":"2024-09-21T02:18:50.317786+0000","last_unstale":"2024-09-21T02:18:53.010410+0000","last_undegraded":"2024-09-21T02:18:53.010410+0000","last_fullsized":"2024-09-21T02:18:53.010410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:12.982265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010196+0000","last_change":"2024-09-21T02:18:51.373783+0000","last_active":"2024-09-21T02:18:53.010196+0000","last_peered":"2024-09-21T02:18:53.010196+0000","last_clean":"2024-09-21T02:18:53.010196+0000","last_became_active":"2024-09-21T02:18:51.373307+0000","last_became_peered":"2024-09-21T02:18:51.373307+0000","last_unstale":"2024-09-21T02:18:53.010196+0000","last_undegraded":"2024-09-21T02:18:53.010196+0000","last_fullsized":"2024-09-21T02:18:53.010196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:49.672388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009047+0000","last_change":"2024-09-21T02:18:50.337791+0000","last_active":"2024-09-21T02:18:53.009047+0000","last_peered":"2024-09-21T02:18:53.009047+0000","last_clean":"2024-09-21T02:18:53.009047+0000","last_became_active":"2024-09-21T02:18:50.337083+0000","last_became_peered":"2024-09-21T02:18:50.337083+0000","last_unstale":"2024-09-21T02:18:53.009047+0000","last_undegraded":"2024-09-21T02:18:53.009047+0000","last_fullsized":"2024-09-21T02:18:53.009047+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:18.238218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943574+0000","last_change":"2024-09-21T02:18:51.366646+0000","last_active":"2024-09-21T02:18:52.943574+0000","last_peered":"2024-09-21T02:18:52.943574+0000","last_clean":"2024-09-21T02:18:52.943574+0000","last_became_active":"2024-09-21T02:18:51.366139+0000","last_became_peered":"2024-09-21T02:18:51.366139+0000","last_unstale":"2024-09-21T02:18:52.943574+0000","last_undegraded":"2024-09-21T02:18:52.943574+0000","last_fullsized":"2024-09-21T02:18:52.943574+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:39.608930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344169+0000","last_change":"2024-09-21T02:18:50.343500+0000","last_active":"2024-09-21T02:18:52.344169+0000","last_peered":"2024-09-21T02:18:52.344169+0000","last_clean":"2024-09-21T02:18:52.344169+0000","last_became_active":"2024-09-21T02:18:50.340428+0000","last_became_peered":"2024-09-21T02:18:50.340428+0000","last_unstale":"2024-09-21T02:18:52.344169+0000","last_undegraded":"2024-09-21T02:18:52.344169+0000","last_fullsized":"2024-09-21T02:18:52.344169+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:25.757690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753635+0000","last_change":"2024-09-21T02:18:51.385408+0000","last_active":"2024-09-21T02:18:52.753635+0000","last_peered":"2024-09-21T02:18:52.753635+0000","last_clean":"2024-09-21T02:18:52.753635+0000","last_became_active":"2024-09-21T02:18:51.385006+0000","last_became_peered":"2024-09-21T02:18:51.385006+0000","last_unstale":"2024-09-21T02:18:52.753635+0000","last_undegraded":"2024-09-21T02:18:52.753635+0000","last_fullsized":"2024-09-21T02:18:52.753635+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:44.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753601+0000","last_change":"2024-09-21T02:18:50.334101+0000","last_active":"2024-09-21T02:18:52.753601+0000","last_peered":"2024-09-21T02:18:52.753601+0000","last_clean":"2024-09-21T02:18:52.753601+0000","last_became_active":"2024-09-21T02:18:50.333198+0000","last_became_peered":"2024-09-21T02:18:50.333198+0000","last_unstale":"2024-09-21T02:18:52.753601+0000","last_undegraded":"2024-09-21T02:18:52.753601+0000","last_fullsized":"2024-09-21T02:18:52.753601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:46.562427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009249+0000","last_change":"2024-09-21T02:18:51.374046+0000","last_active":"2024-09-21T02:18:53.009249+0000","last_peered":"2024-09-21T02:18:53.009249+0000","last_clean":"2024-09-21T02:18:53.009249+0000","last_became_active":"2024-09-21T02:18:51.372327+0000","last_became_peered":"2024-09-21T02:18:51.372327+0000","last_unstale":"2024-09-21T02:18:53.009249+0000","last_undegraded":"2024-09-21T02:18:53.009249+0000","last_fullsized":"2024-09-21T02:18:53.009249+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:30.413449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.006992+0000","last_change":"2024-09-21T02:18:50.341195+0000","last_active":"2024-09-21T02:18:53.006992+0000","last_peered":"2024-09-21T02:18:53.006992+0000","last_clean":"2024-09-21T02:18:53.006992+0000","last_became_active":"2024-09-21T02:18:50.338787+0000","last_became_peered":"2024-09-21T02:18:50.338787+0000","last_unstale":"2024-09-21T02:18:53.006992+0000","last_undegraded":"2024-09-21T02:18:53.006992+0000","last_fullsized":"2024-09-21T02:18:53.006992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:20.909502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.344717+0000","last_change":"2024-09-21T02:18:51.380073+0000","last_active":"2024-09-21T02:18:52.344717+0000","last_peered":"2024-09-21T02:18:52.344717+0000","last_clean":"2024-09-21T02:18:52.344717+0000","last_became_active":"2024-09-21T02:18:51.379718+0000","last_became_peered":"2024-09-21T02:18:51.379718+0000","last_unstale":"2024-09-21T02:18:52.344717+0000","last_undegraded":"2024-09-21T02:18:52.344717+0000","last_fullsized":"2024-09-21T02:18:52.344717+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:05.544806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007524+0000","last_change":"2024-09-21T02:18:50.329503+0000","last_active":"2024-09-21T02:18:53.007524+0000","last_peered":"2024-09-21T02:18:53.007524+0000","last_clean":"2024-09-21T02:18:53.007524+0000","last_became_active":"2024-09-21T02:18:50.328624+0000","last_became_peered":"2024-09-21T02:18:50.328624+0000","last_unstale":"2024-09-21T02:18:53.007524+0000","last_undegraded":"2024-09-21T02:18:53.007524+0000","last_fullsized":"2024-09-21T02:18:53.007524+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:53.977039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.010569+0000","last_change":"2024-09-21T02:18:51.375992+0000","last_active":"2024-09-21T02:18:53.010569+0000","last_peered":"2024-09-21T02:18:53.010569+0000","last_clean":"2024-09-21T02:18:53.010569+0000","last_became_active":"2024-09-21T02:18:51.374305+0000","last_became_peered":"2024-09-21T02:18:51.374305+0000","last_unstale":"2024-09-21T02:18:53.010569+0000","last_undegraded":"2024-09-21T02:18:53.010569+0000","last_fullsized":"2024-09-21T02:18:53.010569+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:14:43.487671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008565+0000","last_change":"2024-09-21T02:18:50.340088+0000","last_active":"2024-09-21T02:18:53.008565+0000","last_peered":"2024-09-21T02:18:53.008565+0000","last_clean":"2024-09-21T02:18:53.008565+0000","last_became_active":"2024-09-21T02:18:50.339224+0000","last_became_peered":"2024-09-21T02:18:50.339224+0000","last_unstale":"2024-09-21T02:18:53.008565+0000","last_undegraded":"2024-09-21T02:18:53.008565+0000","last_fullsized":"2024-09-21T02:18:53.008565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:35.103374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.340949+0000","last_change":"2024-09-21T02:18:51.365258+0000","last_active":"2024-09-21T02:18:52.340949+0000","last_peered":"2024-09-21T02:18:52.340949+0000","last_clean":"2024-09-21T02:18:52.340949+0000","last_became_active":"2024-09-21T02:18:51.364913+0000","last_became_peered":"2024-09-21T02:18:51.364913+0000","last_unstale":"2024-09-21T02:18:52.340949+0000","last_undegraded":"2024-09-21T02:18:52.340949+0000","last_fullsized":"2024-09-21T02:18:52.340949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:00.730636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"102'1","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336246+0000","last_change":"2024-09-21T02:18:50.312672+0000","last_active":"2024-09-21T02:18:52.336246+0000","last_peered":"2024-09-21T02:18:52.336246+0000","last_clean":"2024-09-21T02:18:52.336246+0000","last_became_active":"2024-09-21T02:18:50.312205+0000","last_became_peered":"2024-09-21T02:18:50.312205+0000","last_unstale":"2024-09-21T02:18:52.336246+0000","last_undegraded":"2024-09-21T02:18:52.336246+0000","last_fullsized":"2024-09-21T02:18:52.336246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:26.380603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753844+0000","last_change":"2024-09-21T02:18:51.370755+0000","last_active":"2024-09-21T02:18:52.753844+0000","last_peered":"2024-09-21T02:18:52.753844+0000","last_clean":"2024-09-21T02:18:52.753844+0000","last_became_active":"2024-09-21T02:18:51.370115+0000","last_became_peered":"2024-09-21T02:18:51.370115+0000","last_unstale":"2024-09-21T02:18:52.753844+0000","last_undegraded":"2024-09-21T02:18:52.753844+0000","last_fullsized":"2024-09-21T02:18:52.753844+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:01.173763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008387+0000","last_change":"2024-09-21T02:18:50.330416+0000","last_active":"2024-09-21T02:18:53.008387+0000","last_peered":"2024-09-21T02:18:53.008387+0000","last_clean":"2024-09-21T02:18:53.008387+0000","last_became_active":"2024-09-21T02:18:50.329563+0000","last_became_peered":"2024-09-21T02:18:50.329563+0000","last_unstale":"2024-09-21T02:18:53.008387+0000","last_undegraded":"2024-09-21T02:18:53.008387+0000","last_fullsized":"2024-09-21T02:18:53.008387+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:12.381672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011332+0000","last_change":"2024-09-21T02:18:51.372338+0000","last_active":"2024-09-21T02:18:53.011332+0000","last_peered":"2024-09-21T02:18:53.011332+0000","last_clean":"2024-09-21T02:18:53.011332+0000","last_became_active":"2024-09-21T02:18:51.358808+0000","last_became_peered":"2024-09-21T02:18:51.358808+0000","last_unstale":"2024-09-21T02:18:53.011332+0000","last_undegraded":"2024-09-21T02:18:53.011332+0000","last_fullsized":"2024-09-21T02:18:53.011332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:27.819855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008158+0000","last_change":"2024-09-21T02:18:50.326586+0000","last_active":"2024-09-21T02:18:53.008158+0000","last_peered":"2024-09-21T02:18:53.008158+0000","last_clean":"2024-09-21T02:18:53.008158+0000","last_became_active":"2024-09-21T02:18:50.325718+0000","last_became_peered":"2024-09-21T02:18:50.325718+0000","last_unstale":"2024-09-21T02:18:53.008158+0000","last_undegraded":"2024-09-21T02:18:53.008158+0000","last_fullsized":"2024-09-21T02:18:53.008158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:39.655636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.943549+0000","last_change":"2024-09-21T02:18:51.382674+0000","last_active":"2024-09-21T02:18:52.943549+0000","last_peered":"2024-09-21T02:18:52.943549+0000","last_clean":"2024-09-21T02:18:52.943549+0000","last_became_active":"2024-09-21T02:18:51.382258+0000","last_became_peered":"2024-09-21T02:18:51.382258+0000","last_unstale":"2024-09-21T02:18:52.943549+0000","last_undegraded":"2024-09-21T02:18:52.943549+0000","last_fullsized":"2024-09-21T02:18:52.943549+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:36:42.692409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.336625+0000","last_change":"2024-09-21T02:18:50.313069+0000","last_active":"2024-09-21T02:18:52.336625+0000","last_peered":"2024-09-21T02:18:52.336625+0000","last_clean":"2024-09-21T02:18:52.336625+0000","last_became_active":"2024-09-21T02:18:50.312597+0000","last_became_peered":"2024-09-21T02:18:50.312597+0000","last_unstale":"2024-09-21T02:18:52.336625+0000","last_undegraded":"2024-09-21T02:18:52.336625+0000","last_fullsized":"2024-09-21T02:18:52.336625+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:26.423876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.011027+0000","last_change":"2024-09-21T02:18:51.373621+0000","last_active":"2024-09-21T02:18:53.011027+0000","last_peered":"2024-09-21T02:18:53.011027+0000","last_clean":"2024-09-21T02:18:53.011027+0000","last_became_active":"2024-09-21T02:18:51.372053+0000","last_became_peered":"2024-09-21T02:18:51.372053+0000","last_unstale":"2024-09-21T02:18:53.011027+0000","last_undegraded":"2024-09-21T02:18:53.011027+0000","last_fullsized":"2024-09-21T02:18:53.011027+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:59.720175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009588+0000","last_change":"2024-09-21T02:18:50.337866+0000","last_active":"2024-09-21T02:18:53.009588+0000","last_peered":"2024-09-21T02:18:53.009588+0000","last_clean":"2024-09-21T02:18:53.009588+0000","last_became_active":"2024-09-21T02:18:50.337213+0000","last_became_peered":"2024-09-21T02:18:50.337213+0000","last_unstale":"2024-09-21T02:18:53.009588+0000","last_undegraded":"2024-09-21T02:18:53.009588+0000","last_fullsized":"2024-09-21T02:18:53.009588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:37.075437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.009890+0000","last_change":"2024-09-21T02:18:51.377478+0000","last_active":"2024-09-21T02:18:53.009890+0000","last_peered":"2024-09-21T02:18:53.009890+0000","last_clean":"2024-09-21T02:18:53.009890+0000","last_became_active":"2024-09-21T02:18:51.376738+0000","last_became_peered":"2024-09-21T02:18:51.376738+0000","last_unstale":"2024-09-21T02:18:53.009890+0000","last_undegraded":"2024-09-21T02:18:53.009890+0000","last_fullsized":"2024-09-21T02:18:53.009890+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:32:01.293154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.753502+0000","last_change":"2024-09-21T02:18:50.334715+0000","last_active":"2024-09-21T02:18:52.753502+0000","last_peered":"2024-09-21T02:18:52.753502+0000","last_clean":"2024-09-21T02:18:52.753502+0000","last_became_active":"2024-09-21T02:18:50.333717+0000","last_became_peered":"2024-09-21T02:18:50.333717+0000","last_unstale":"2024-09-21T02:18:52.753502+0000","last_undegraded":"2024-09-21T02:18:52.753502+0000","last_fullsized":"2024-09-21T02:18:52.753502+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:01:10.659304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.340186+0000","last_change":"2024-09-21T02:18:51.368560+0000","last_active":"2024-09-21T02:18:52.340186+0000","last_peered":"2024-09-21T02:18:52.340186+0000","last_clean":"2024-09-21T02:18:52.340186+0000","last_became_active":"2024-09-21T02:18:51.368179+0000","last_became_peered":"2024-09-21T02:18:51.368179+0000","last_unstale":"2024-09-21T02:18:52.340186+0000","last_undegraded":"2024-09-21T02:18:52.340186+0000","last_fullsized":"2024-09-21T02:18:52.340186+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:41.825495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.339628+0000","last_change":"2024-09-21T02:18:50.312150+0000","last_active":"2024-09-21T02:18:52.339628+0000","last_peered":"2024-09-21T02:18:52.339628+0000","last_clean":"2024-09-21T02:18:52.339628+0000","last_became_active":"2024-09-21T02:18:50.311220+0000","last_became_peered":"2024-09-21T02:18:50.311220+0000","last_unstale":"2024-09-21T02:18:52.339628+0000","last_undegraded":"2024-09-21T02:18:52.339628+0000","last_fullsized":"2024-09-21T02:18:52.339628+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:25.251261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.007944+0000","last_change":"2024-09-21T02:18:51.364984+0000","last_active":"2024-09-21T02:18:53.007944+0000","last_peered":"2024-09-21T02:18:53.007944+0000","last_clean":"2024-09-21T02:18:53.007944+0000","last_became_active":"2024-09-21T02:18:51.364425+0000","last_became_peered":"2024-09-21T02:18:51.364425+0000","last_unstale":"2024-09-21T02:18:53.007944+0000","last_undegraded":"2024-09-21T02:18:53.007944+0000","last_fullsized":"2024-09-21T02:18:53.007944+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:50.287929+0000","last_clean_scrub_stamp":"2024-09-21T02:18:50.287929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:40.670689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:53.008621+0000","last_change":"2024-09-21T02:18:50.321012+0000","last_active":"2024-09-21T02:18:53.008621+0000","last_peered":"2024-09-21T02:18:53.008621+0000","last_clean":"2024-09-21T02:18:53.008621+0000","last_became_active":"2024-09-21T02:18:50.320553+0000","last_became_peered":"2024-09-21T02:18:50.320553+0000","last_unstale":"2024-09-21T02:18:53.008621+0000","last_undegraded":"2024-09-21T02:18:53.008621+0000","last_fullsized":"2024-09-21T02:18:53.008621+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:18:49.276479+0000","last_clean_scrub_stamp":"2024-09-21T02:18:49.276479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:30.947988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-21T02:18:52.942684+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:18:52.942684+0000","last_peered":"2024-09-21T02:18:52.942684+0000","last_clean":"2024-09-21T02:18:52.942684+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:18:52.942684+0000","last_undegraded":"2024-09-21T02:18:52.942684+0000","last_fullsized":"2024-09-21T02:18:52.942684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":552625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1261568,"data_stored":550889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1863680,"data_stored":1141349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542216,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":552591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1261568,"data_stored":550889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36592,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335248,"statfs":{"total":96636764160,"available":96599293952,"internally_reserved":0,"allocated":1273856,"data_stored":551527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1855488,"data_stored":1141257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36588,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335252,"statfs":{"total":96636764160,"available":96599298048,"internally_reserved":0,"allocated":1269760,"data_stored":551541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:18:55.915 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:18:55.915 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:18:56.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- 172.21.15.170:0/1752978511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2588003a10 msgr2=0x7f2588003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 --2- 172.21.15.170:0/1752978511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2588003a10 0x7f2588003e80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f259800bbe0 tx=0x7f259801ce80 comp rx=0 tx=0).stop 2024-09-21T02:18:56.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a26f5640 1 -- 172.21.15.170:0/1752978511 reap_dead start 2024-09-21T02:18:56.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- 172.21.15.170:0/1752978511 shutdown_connections 2024-09-21T02:18:56.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- 172.21.15.170:0/1752978511 >> 172.21.15.170:0/1752978511 conn(0x7f259c0d0170 msgr2=0x7f259c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:56.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- 172.21.15.170:0/1752978511 shutdown_connections 2024-09-21T02:18:56.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- 172.21.15.170:0/1752978511 wait complete. 2024-09-21T02:18:56.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 Processor -- start 2024-09-21T02:18:56.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- start start 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2588003a10 0x7f259c1fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 0x7f259c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c09d680 0x7f259c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f259c073ec0 con 0x7f259c09a9c0 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f259c073d40 con 0x7f2588003a10 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a4158640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f259c074040 con 0x7f259c09d680 2024-09-21T02:18:56.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a16f3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 0x7f259c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.111+0000 7f25a16f3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 0x7f259c0c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51374/0 (socket says 172.21.15.170:51374) 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 -- 172.21.15.170:0/201678682 learned_addr learned my addr 172.21.15.170:0/201678682 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a1ef4640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2588003a10 0x7f259c1fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a26f5640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c09d680 0x7f259c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 -- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c09d680 msgr2=0x7f259c0c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c09d680 0x7f259c0c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 -- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2588003a10 msgr2=0x7f259c1fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2588003a10 0x7f259c1fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 -- 172.21.15.170:0/201678682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c09c8c0 con 0x7f259c09a9c0 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a1ef4640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2588003a10 0x7f259c1fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:56.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a26f5640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c09d680 0x7f259c0c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:56.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a16f3640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 0x7f259c0c80b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e990 tx=0x7f258c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:56.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c00cbe0 con 0x7f259c09a9c0 2024-09-21T02:18:56.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f258c005020 con 0x7f259c09a9c0 2024-09-21T02:18:56.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c09cba0 con 0x7f259c09a9c0 2024-09-21T02:18:56.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c005450 con 0x7f259c09a9c0 2024-09-21T02:18:56.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c09d160 con 0x7f259c09a9c0 2024-09-21T02:18:56.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c09af40 con 0x7f259c09a9c0 2024-09-21T02:18:56.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f258c0047b0 con 0x7f259c09a9c0 2024-09-21T02:18:56.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25927fc640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2580073320 0x7f2580075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a1ef4640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2580073320 0x7f2580075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f258c094ec0 con 0x7f259c09a9c0 2024-09-21T02:18:56.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.115+0000 7f25a1ef4640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2580073320 0x7f2580075830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f25980b0000 tx=0x7f2598019c20 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:56.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.119+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f258c016030 con 0x7f259c09a9c0 2024-09-21T02:18:56.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.339+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f259c0c8ba0 con 0x7f259c09a9c0 2024-09-21T02:18:56.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.343+0000 7f25927fc640 1 -- 172.21.15.170:0/201678682 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7f258c05fcc0 con 0x7f259c09a9c0 2024-09-21T02:18:56.347 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:18:56.347 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:18:56.347 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:18:56.347 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:18:56.347 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:18:56.347 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2580073320 msgr2=0x7f2580075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2580073320 0x7f2580075830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f25980b0000 tx=0x7f2598019c20 comp rx=0 tx=0).stop 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 msgr2=0x7f259c0c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 0x7f259c0c80b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e990 tx=0x7f258c00ee50 comp rx=0 tx=0).stop 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 shutdown_connections 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2580073320 0x7f2580075830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c09d680 0x7f259c0c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c09a9c0 0x7f259c0c80b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 --2- 172.21.15.170:0/201678682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2588003a10 0x7f259c1fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 >> 172.21.15.170:0/201678682 conn(0x7f259c0d0170 msgr2=0x7f259c0b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:56.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 shutdown_connections 2024-09-21T02:18:56.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.347+0000 7f25a4158640 1 -- 172.21.15.170:0/201678682 wait complete. 2024-09-21T02:18:56.370 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 -- 172.21.15.170:0/2017850205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 msgr2=0x7fa0fc0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 --2- 172.21.15.170:0/2017850205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0cccd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0d1160 tx=0x7fa0f401cd80 comp rx=0 tx=0).stop 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 -- 172.21.15.170:0/2017850205 shutdown_connections 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 --2- 172.21.15.170:0/2017850205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0cccd0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 --2- 172.21.15.170:0/2017850205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0fc0cb460 0x7fa0fc0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 --2- 172.21.15.170:0/2017850205 >> v2:172.21.15.191:3300/0 conn(0x7fa0fc0cf7e0 0x7fa0fc070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 -- 172.21.15.170:0/2017850205 >> 172.21.15.170:0/2017850205 conn(0x7fa0fc0b1c40 msgr2=0x7fa0fc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 -- 172.21.15.170:0/2017850205 shutdown_connections 2024-09-21T02:18:56.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 -- 172.21.15.170:0/2017850205 wait complete. 2024-09-21T02:18:56.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 Processor -- start 2024-09-21T02:18:56.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.507+0000 7fa103f91640 1 -- start start 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0fc0cb460 0x7fa0fc147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0fc0cf7e0 0x7fa0fc0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc06f840 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc06f6c0 con 0x7fa0fc0cf7e0 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc06f9c0 con 0x7fa0fc0cb460 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51392/0 (socket says 172.21.15.170:51392) 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 -- 172.21.15.170:0/1936508321 learned_addr learned my addr 172.21.15.170:0/1936508321 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa101d2d640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0fc0cb460 0x7fa0fc147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10252e640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0fc0cf7e0 0x7fa0fc0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 -- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0fc0cb460 msgr2=0x7fa0fc147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0fc0cb460 0x7fa0fc147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 -- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0fc0cf7e0 msgr2=0x7fa0fc0b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0fc0cf7e0 0x7fa0fc0b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 -- 172.21.15.170:0/1936508321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc147aa0 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10252e640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0fc0cf7e0 0x7fa0fc0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa10152c640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0b03e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c940 tx=0x7fa0ec00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec004220 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0ec0079b0 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc148670 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0044b0 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc148c30 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc0cccd0 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fa0ec005020 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa0f27fc640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa0e0073320 0x7fa0e0075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:56.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec094980 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.511+0000 7fa101d2d640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa0e0073320 0x7fa0e0075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:56.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.515+0000 7fa101d2d640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa0e0073320 0x7fa0e0075830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800ca60 tx=0x7fa0f800c440 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:56.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.515+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa0ec014030 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.707+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0fc0b2330 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.707+0000 7fa0f27fc640 1 -- 172.21.15.170:0/1936508321 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5237 (secure 0 0 0) 0x7fa0ec05f860 con 0x7fa0fc0cc8c0 2024-09-21T02:18:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:18:56.712 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":71,"btime":"2024-09-21T02:18:54:379135+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:50.719554+0000","modified":"2024-09-21T02:18:53.780500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6801/1778665511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1778665511},{"type":"v1","addr":"172.21.15.191:6801","nonce":1778665511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7107,"qdb_cluster":[7107]},"id":8}]} 2024-09-21T02:18:56.712 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 71 2024-09-21T02:18:56.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa0e0073320 msgr2=0x7fa0e0075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa0e0073320 0x7fa0e0075830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800ca60 tx=0x7fa0f800c440 comp rx=0 tx=0).stop 2024-09-21T02:18:56.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 msgr2=0x7fa0fc0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:56.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0b03e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c940 tx=0x7fa0ec00ce00 comp rx=0 tx=0).stop 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 shutdown_connections 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa0e0073320 0x7fa0e0075830 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0fc0cf7e0 0x7fa0fc0b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0fc0cc8c0 0x7fa0fc0b03e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 --2- 172.21.15.170:0/1936508321 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0fc0cb460 0x7fa0fc147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 >> 172.21.15.170:0/1936508321 conn(0x7fa0fc0b1c40 msgr2=0x7fa0fc15b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 shutdown_connections 2024-09-21T02:18:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:56.711+0000 7fa103f91640 1 -- 172.21.15.170:0/1936508321 wait complete. 2024-09-21T02:18:57.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:18:50.719554+0000', 'modified': '2024-09-21T02:18:53.780500+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7107}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7107': {'gid': 7107, 'name': 'b', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6801/1778665511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6800', 'nonce': 1778665511}, {'type': 'v1', 'addr': '172.21.15.191:6801', 'nonce': 1778665511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7107, 'qdb_cluster': [7107]} 2024-09-21T02:18:57.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:18:57.135 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T02:18:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/3659398981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 msgr2=0x7f4a480cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/3659398981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a480cadf0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4a480b2d00 tx=0x7f4a4001cd50 comp rx=0 tx=0).stop 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/3659398981 shutdown_connections 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/3659398981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a480cadf0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/3659398981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a48070570 0x7f4a480ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/3659398981 >> v2:172.21.15.191:3300/0 conn(0x7f4a4815f990 0x7f4a48070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/3659398981 >> 172.21.15.170:0/3659398981 conn(0x7f4a480b1c40 msgr2=0x7f4a480b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/3659398981 shutdown_connections 2024-09-21T02:18:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/3659398981 wait complete. 2024-09-21T02:18:57.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 Processor -- start 2024-09-21T02:18:57.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- start start 2024-09-21T02:18:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a48070570 0x7f4a481655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a4809cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a4815f990 0x7f4a4809d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48073a40 con 0x7f4a480cea00 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480738c0 con 0x7f4a48070570 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4ce1f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a48073bc0 con 0x7f4a4815f990 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a4809cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a4809cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51412/0 (socket says 172.21.15.170:51412) 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 -- 172.21.15.170:0/24518712 learned_addr learned my addr 172.21.15.170:0/24518712 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a4659c640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a48070570 0x7f4a481655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a4815f990 0x7f4a4809d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 -- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a4815f990 msgr2=0x7f4a4809d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a4815f990 0x7f4a4809d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 -- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a48070570 msgr2=0x7f4a481655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a48070570 0x7f4a481655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.295+0000 7f4a45d9b640 1 -- 172.21.15.170:0/24518712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a48165b00 con 0x7f4a480cea00 2024-09-21T02:18:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a4809cd30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c720 tx=0x7f4a3c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a36ffd640 1 -- 172.21.15.170:0/24518712 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c007bb0 con 0x7f4a480cea00 2024-09-21T02:18:57.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a36ffd640 1 -- 172.21.15.170:0/24518712 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a3c007e40 con 0x7f4a480cea00 2024-09-21T02:18:57.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a481f8e20 con 0x7f4a480cea00 2024-09-21T02:18:57.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a36ffd640 1 -- 172.21.15.170:0/24518712 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c005280 con 0x7f4a480cea00 2024-09-21T02:18:57.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a481f93e0 con 0x7f4a480cea00 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a36ffd640 1 -- 172.21.15.170:0/24518712 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f4a3c005420 con 0x7f4a480cea00 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a36ffd640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a28073320 0x7f4a28075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a4659c640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a28073320 0x7f4a28075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a36ffd640 1 -- 172.21.15.170:0/24518712 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4a3c094890 con 0x7f4a480cea00 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a4659c640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a28073320 0x7f4a28075830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b9a0 tx=0x7f4a38005c30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a540ea000 1 Processor -- start 2024-09-21T02:18:57.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.299+0000 7f4a540ea000 1 -- start start 2024-09-21T02:18:57.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f577950 0x559f7f6c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f6c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559f7f59d410 con 0x559f7f187460 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559f7f4dab40 con 0x559f7f577950 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x559f7f4c9f70 con 0x559f7f560ca0 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51428/0 (socket says 172.21.15.170:51428) 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 -- 172.21.15.170:0/615981188 learned_addr learned my addr 172.21.15.170:0/615981188 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f577950 0x559f7f6c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f6c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 -- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 msgr2=0x559f7f6c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f6c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 -- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f577950 msgr2=0x559f7f6c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f577950 0x559f7f6c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 -- 172.21.15.170:0/615981188 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7f4915b0 con 0x559f7f187460 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a4659c640 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6c86a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800dcc0 tx=0x7f4a38010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60318331a473ebb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a27fff640 1 -- 172.21.15.170:0/615981188 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3801f070 con 0x559f7f187460 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a27fff640 1 -- 172.21.15.170:0/615981188 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a38024750 con 0x559f7f187460 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a27fff640 1 -- 172.21.15.170:0/615981188 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38024b60 con 0x559f7f187460 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- 172.21.15.170:0/615981188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 msgr2=0x559f7f6c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6c86a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800dcc0 tx=0x7f4a38010000 comp rx=0 tx=0).stop 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- 172.21.15.170:0/615981188 shutdown_connections 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f6c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f577950 0x559f7f6c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 --2- 172.21.15.170:0/615981188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6c86a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- 172.21.15.170:0/615981188 >> 172.21.15.170:0/615981188 conn(0x559f7f64dd90 msgr2=0x559f7f67d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- 172.21.15.170:0/615981188 shutdown_connections 2024-09-21T02:18:57.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- 172.21.15.170:0/615981188 wait complete. 2024-09-21T02:18:57.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 -- start start 2024-09-21T02:18:57.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.303+0000 7f4a540ea000 1 Processor -- start 2024-09-21T02:18:57.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f6cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f577950 0x559f7f6d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559f7f4645f0 con 0x559f7f187460 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559f7f4cdd80 con 0x559f7f560ca0 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x559f7f470520 con 0x559f7f577950 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 10 client.0 _ll_register_callbacks cb 0x559f7f474830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51430/0 (socket says 172.21.15.170:51430) 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 -- 172.21.15.170:0/958518225 learned_addr learned my addr 172.21.15.170:0/958518225 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f6cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f577950 0x559f7f6d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 -- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f577950 msgr2=0x559f7f6d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f577950 0x559f7f6d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 -- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 msgr2=0x559f7f6cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f6cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 -- 172.21.15.170:0/958518225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7f468190 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f6cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a4659c640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6cf920 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x559f7f4d47e0 tx=0x7f4a38024b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b0c40b376d0667 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 1 -- 172.21.15.170:0/958518225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3801f050 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 1 -- 172.21.15.170:0/958518225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4a3802c620 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f7f462110 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 1 -- 172.21.15.170:0/958518225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f4a38036050 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 10 client.7230 fetch_fsmap learned FSMap version 71 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x559f7f54ef10 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 1 -- 172.21.15.170:0/958518225 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559f7f462110 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 1 -- 172.21.15.170:0/958518225 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f4a38005020 con 0x559f7f187460 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 10 client.7230 fetch_fsmap finished waiting for FSMap version 71 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 10 client.7230 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:18:57.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 20 client.7230 populate_metadata read hostname 'smithi170' 2024-09-21T02:18:57.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f16c7d0 0x559f7f6d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 4 client.7230 mds_command: new command op to 7107 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:18:57.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 --> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f7f4dca90 con 0x559f7f16c7d0 2024-09-21T02:18:57.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f16c7d0 0x559f7f6d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a257fa640 10 client.7230 ms_handle_connect on v2:172.21.15.191:6800/1778665511 2024-09-21T02:18:57.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.307+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f16c7d0 0x559f7f6d9d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb2d3980dfb0255 server_cookie=b55641c68e9a5e13 in_seq=0 out_seq=0 2024-09-21T02:18:57.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a257fa640 1 -- 172.21.15.170:0/958518225 <== mds.0 v2:172.21.15.191:6800/1778665511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f7f4dca90 con 0x559f7f16c7d0 2024-09-21T02:18:57.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a257fa640 10 client.7230 handle_command_reply: tid=0 2024-09-21T02:18:57.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 client.7230 shutdown 2024-09-21T02:18:57.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 msgr2=0x559f7f6cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6cf920 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x559f7f4d47e0 tx=0x7f4a38024b80 comp rx=0 tx=0).stop 2024-09-21T02:18:57.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 shutdown_connections 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f16c7d0 0x559f7f6d9d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f577950 0x559f7f6d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f6cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 --2- 172.21.15.170:0/958518225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f187460 0x559f7f6cf920 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 >> 172.21.15.170:0/958518225 conn(0x559f7f64dd90 msgr2=0x559f7f684440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a257fa640 0 client.7230 ms_handle_reset on v2:172.21.15.191:6800/1778665511 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 shutdown_connections 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 1 -- 172.21.15.170:0/958518225 wait complete. 2024-09-21T02:18:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.311+0000 7f4a540ea000 20 client.7230 trim_cache size 0 max 16384 2024-09-21T02:18:57.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.327+0000 7f4a540ea000 1 Processor -- start 2024-09-21T02:18:57.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.327+0000 7f4a540ea000 1 -- start start 2024-09-21T02:18:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f187460 0x559f7f73a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f73ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a482053c0 con 0x559f7f577950 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48158650 con 0x559f7f560ca0 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a481579b0 con 0x559f7f187460 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51432/0 (socket says 172.21.15.170:51432) 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 -- 172.21.15.170:0/609294180 learned_addr learned my addr 172.21.15.170:0/609294180 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a4659c640 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f187460 0x559f7f73a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f73ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 -- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f187460 msgr2=0x559f7f73a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f187460 0x559f7f73a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 -- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 msgr2=0x559f7f73ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f73ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 -- 172.21.15.170:0/609294180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7f525db0 con 0x559f7f577950 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a4659c640 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f187460 0x559f7f73a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73b0a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001b400 tx=0x7f4a40019af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662084392911147a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a277fe640 1 -- 172.21.15.170:0/609294180 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400bb020 con 0x559f7f577950 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a277fe640 1 -- 172.21.15.170:0/609294180 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a400b3040 con 0x559f7f577950 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a277fe640 1 -- 172.21.15.170:0/609294180 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400b2070 con 0x559f7f577950 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- 172.21.15.170:0/609294180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 msgr2=0x559f7f73b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73b0a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001b400 tx=0x7f4a40019af0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- 172.21.15.170:0/609294180 shutdown_connections 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73b0a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f560ca0 0x559f7f73ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 --2- 172.21.15.170:0/609294180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f187460 0x559f7f73a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- 172.21.15.170:0/609294180 >> 172.21.15.170:0/609294180 conn(0x559f7f64dd90 msgr2=0x559f7f6c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- 172.21.15.170:0/609294180 shutdown_connections 2024-09-21T02:18:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- 172.21.15.170:0/609294180 wait complete. 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 -- start start 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.331+0000 7f4a540ea000 1 Processor -- start 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f187460 0x559f7f73e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f73eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559f7f4d23b0 con 0x559f7f577950 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559f7f495ee0 con 0x559f7f187460 2024-09-21T02:18:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x559f7f4cd090 con 0x559f7f560ca0 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 10 client.0 _ll_register_callbacks cb 0x7f4a3800dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51438/0 (socket says 172.21.15.170:51438) 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 -- 172.21.15.170:0/4130991037 learned_addr learned my addr 172.21.15.170:0/4130991037 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a4659c640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f187460 0x559f7f73e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f73eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 -- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 msgr2=0x559f7f73eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f73eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 -- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f187460 msgr2=0x559f7f73e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f187460 0x559f7f73e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 -- 172.21.15.170:0/4130991037 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7f490ab0 con 0x559f7f577950 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a4659c640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f187460 0x559f7f73e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a45d9b640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f73eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a46d9d640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73f020 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001cb80 tx=0x7f4a40003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc38e773871d75e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 1 -- 172.21.15.170:0/4130991037 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400b2040 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 1 -- 172.21.15.170:0/4130991037 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f4a40005200 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a400b3040 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 1 -- 172.21.15.170:0/4130991037 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f4a400c7050 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 10 client.7236 fetch_fsmap learned FSMap version 71 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x559f7f46f070 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 1 -- 172.21.15.170:0/4130991037 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a400b3040 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 1 -- 172.21.15.170:0/4130991037 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f4a400ca020 con 0x559f7f577950 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 10 client.7236 fetch_fsmap finished waiting for FSMap version 71 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 10 client.7236 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 20 client.7236 populate_metadata read hostname 'smithi170' 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f15a830 0x559f7f6fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 4 client.7236 mds_command: new command op to 7107 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 --> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] -- command(tid 0: {"prefix": "status"}) -- 0x559f7f4d8540 con 0x559f7f15a830 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a4659c640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f15a830 0x559f7f6fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 10 client.7236 ms_handle_connect on v2:172.21.15.191:6800/1778665511 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a4659c640 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f15a830 0x559f7f6fa640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfe8467155817b server_cookie=2b183a2f5ac7cfbc in_seq=0 out_seq=0 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 1 -- 172.21.15.170:0/4130991037 <== mds.0 v2:172.21.15.191:6800/1778665511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x559f7f4d8540 con 0x559f7f15a830 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 10 client.7236 handle_command_reply: tid=0 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 client.7236 shutdown 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 msgr2=0x559f7f73f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73f020 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4a4001cb80 tx=0x7f4a40003ab0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 shutdown_connections 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:6800/1778665511,v1:172.21.15.191:6801/1778665511] conn(0x559f7f15a830 0x559f7f6fa640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x559f7f560ca0 0x559f7f73eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a257fa640 0 client.7236 ms_handle_reset on v2:172.21.15.191:6800/1778665511 2024-09-21T02:18:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f7f577950 0x559f7f73f020 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 --2- 172.21.15.170:0/4130991037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559f7f187460 0x559f7f73e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 >> 172.21.15.170:0/4130991037 conn(0x559f7f64dd90 msgr2=0x559f7f6d96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 shutdown_connections 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 1 -- 172.21.15.170:0/4130991037 wait complete. 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.335+0000 7f4a540ea000 20 client.7236 trim_cache size 0 max 16384 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7107, 2024-09-21T02:18:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.9792361869999997, 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 71, 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 102, 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 102, 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.52070535 2024-09-21T02:18:57.345 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a28073320 msgr2=0x7f4a28075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a28073320 0x7f4a28075830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b9a0 tx=0x7f4a38005c30 comp rx=0 tx=0).stop 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 msgr2=0x7f4a4809cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a4809cd30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c720 tx=0x7f4a3c00cbe0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 shutdown_connections 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a28073320 0x7f4a28075830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a4815f990 0x7f4a4809d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a480cea00 0x7f4a4809cd30 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 --2- 172.21.15.170:0/24518712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a48070570 0x7f4a481655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 >> 172.21.15.170:0/24518712 conn(0x7f4a480b1c40 msgr2=0x7f4a480b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 shutdown_connections 2024-09-21T02:18:57.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.339+0000 7f4a4ce1f640 1 -- 172.21.15.170:0/24518712 wait complete. 2024-09-21T02:18:57.359 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:18:57.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- 172.21.15.170:0/2764392265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 msgr2=0x7f1b08003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 --2- 172.21.15.170:0/2764392265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b08003e80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c2e0 tx=0x7f1b1001f770 comp rx=0 tx=0).stop 2024-09-21T02:18:57.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1f7fe640 1 -- 172.21.15.170:0/2764392265 reap_dead start 2024-09-21T02:18:57.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- 172.21.15.170:0/2764392265 shutdown_connections 2024-09-21T02:18:57.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- 172.21.15.170:0/2764392265 >> 172.21.15.170:0/2764392265 conn(0x7f1b200cd3c0 msgr2=0x7f1b200cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- 172.21.15.170:0/2764392265 shutdown_connections 2024-09-21T02:18:57.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- 172.21.15.170:0/2764392265 wait complete. 2024-09-21T02:18:57.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 Processor -- start 2024-09-21T02:18:57.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- start start 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b200cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b200aada0 0x7f1b201e9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b201e9590 0x7f1b200a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b200cc000 con 0x7f1b08003a10 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1b200cbe80 con 0x7f1b201e9590 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b25551640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1b200cc180 con 0x7f1b200aada0 2024-09-21T02:18:57.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b200cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b200cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51450/0 (socket says 172.21.15.170:51450) 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 -- 172.21.15.170:0/3344860471 learned_addr learned my addr 172.21.15.170:0/3344860471 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1f7fe640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b201e9590 0x7f1b200a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1e7fc640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b200aada0 0x7f1b201e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 -- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b200aada0 msgr2=0x7f1b201e9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b200aada0 0x7f1b201e9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 -- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b201e9590 msgr2=0x7f1b200a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b201e9590 0x7f1b200a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 -- 172.21.15.170:0/3344860471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b201e99a0 con 0x7f1b08003a10 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.503+0000 7f1b1effd640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b200cf760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c2e0 tx=0x7f1b10001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100c0070 con 0x7f1b08003a10 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b10006420 con 0x7f1b08003a10 2024-09-21T02:18:57.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b201f57a0 con 0x7f1b08003a10 2024-09-21T02:18:57.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100b2030 con 0x7f1b08003a10 2024-09-21T02:18:57.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201f5a60 con 0x7f1b08003a10 2024-09-21T02:18:57.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200cc710 con 0x7f1b08003a10 2024-09-21T02:18:57.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f1b100c3070 con 0x7f1b08003a10 2024-09-21T02:18:57.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b077fe640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b0c073320 0x7f1b0c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:18:57.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f1b101317e0 con 0x7f1b08003a10 2024-09-21T02:18:57.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b1e7fc640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b0c073320 0x7f1b0c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:18:57.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.507+0000 7f1b1e7fc640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b0c073320 0x7f1b0c075830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1b200cca40 tx=0x7f1b140091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:18:57.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.511+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1b201e99a0 con 0x7f1b08003a10 2024-09-21T02:18:57.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.707+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b200c7020 con 0x7f1b08003a10 2024-09-21T02:18:57.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b077fe640 1 -- 172.21.15.170:0/3344860471 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f1b10019490 con 0x7f1b08003a10 2024-09-21T02:18:57.712 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:18:57.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b0c073320 msgr2=0x7f1b0c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b0c073320 0x7f1b0c075830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1b200cca40 tx=0x7f1b140091e0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 msgr2=0x7f1b200cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:18:57.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b200cf760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c2e0 tx=0x7f1b10001d50 comp rx=0 tx=0).stop 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 shutdown_connections 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b0c073320 0x7f1b0c075830 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b200aada0 0x7f1b201e9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b201e9590 0x7f1b200a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 --2- 172.21.15.170:0/3344860471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b08003a10 0x7f1b200cf760 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 >> 172.21.15.170:0/3344860471 conn(0x7f1b200cd3c0 msgr2=0x7f1b2006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 shutdown_connections 2024-09-21T02:18:57.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57.711+0000 7f1b25551640 1 -- 172.21.15.170:0/3344860471 wait complete. 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:18:57.734 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:18:57.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:57.734 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:18:57.740 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:18:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:18:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:18:57.741 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42178sec preferred_lft 42178sec 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:18:57.742 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:18:57.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:57.744 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:18:57.795 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:18:57.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:57.797 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:18:57.847 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:18:57.848 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:18:57.849 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:18:57.849 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:18:57.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:18:57.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:18:57.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:18:57.849 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:18:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:18:57.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:18:57.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:57.975 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:18:58.020 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:18:58.020 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:18:58.020 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:18:58.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:58.021 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:18:58.076 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:18:58.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:18:58.076 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:58.124 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:18:58.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:18:58.125 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:58.172 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:18:58.230 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:18:58.279 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:18:58.280 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:58.281 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:58.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:58.282 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:18:58.332 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:18:58.332 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:18:58.377 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:18:58.391 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:18:58.437 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:18:58.438 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:58.439 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:58.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:58.440 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:18:58.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:18:58.483+0000 7f8510604580 -1 init, newargv = 0x7f84f4004370 newargc=15 2024-09-21T02:18:58.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[44905]: starting ceph client 2024-09-21T02:18:58.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[44905]: starting fuse 2024-09-21T02:18:59.589 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:18:59.606 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:18:59.653 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:18:59.653 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:18:59.654 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:18:59.655 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:18:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:18:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:18:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:18:59.656 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:18:59.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:59.657 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:18:59.705 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:18:59.705 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:18:59.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:18:59.705 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:18:59.705 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:18:59.705 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:18:59.705 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:18:59.706 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:18:59.707 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:18:59.830 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.44905.asok 2024-09-21T02:18:59.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:18:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:18:59.841 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44905.asok 2024-09-21T02:18:59.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:18:59.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44905.asok status 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "44905", 2024-09-21T02:19:00.023 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7251, 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 7251 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3272689248 2024-09-21T02:19:00.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3272689248 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.7251 192.168.0.1:0/3272689248", 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/3272689248", 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 71, 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 102, 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:19:00.025 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:19:00.026 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:19:00.026 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:00.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.039 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:19:00.039 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:19:00.045 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:19:00.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:19:00.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:19:00.099 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:19:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 -- 172.21.15.170:0/188679029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800cb460 msgr2=0x7f07800cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 --2- 172.21.15.170:0/188679029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800cb460 0x7f07800cb870 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0768009840 tx=0x7f076801c820 comp rx=0 tx=0).stop 2024-09-21T02:19:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 -- 172.21.15.170:0/188679029 shutdown_connections 2024-09-21T02:19:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 --2- 172.21.15.170:0/188679029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800cb460 0x7f07800cb870 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 -- 172.21.15.170:0/188679029 >> 172.21.15.170:0/188679029 conn(0x7f07800b1c40 msgr2=0x7f07800b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:00.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 -- 172.21.15.170:0/188679029 shutdown_connections 2024-09-21T02:19:00.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 -- 172.21.15.170:0/188679029 wait complete. 2024-09-21T02:19:00.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 Processor -- start 2024-09-21T02:19:00.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.295+0000 7f0785494640 1 -- start start 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07800cb460 0x7f07801501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 0x7f07800afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07800cf7e0 0x7f07800b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f078016a2f0 con 0x7f07800ce1c0 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f078016a170 con 0x7f07800cb460 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f078016a470 con 0x7f07800cf7e0 2024-09-21T02:19:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 0x7f07800afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 0x7f07800afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51462/0 (socket says 172.21.15.170:51462) 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 -- 172.21.15.170:0/3969413656 learned_addr learned my addr 172.21.15.170:0/3969413656 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f077effd640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07800cb460 0x7f07801501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f077f7fe640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07800cf7e0 0x7f07800b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 -- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07800cf7e0 msgr2=0x7f07800b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07800cf7e0 0x7f07800b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 -- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07800cb460 msgr2=0x7f07801501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07800cb460 0x7f07801501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 -- 172.21.15.170:0/3969413656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078015b330 con 0x7f07800ce1c0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f077f7fe640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07800cf7e0 0x7f07800b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0777fff640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 0x7f07800afd40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0768009790 tx=0x7f0768004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680b2040 con 0x7f07800ce1c0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0768005030 con 0x7f07800ce1c0 2024-09-21T02:19:00.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078015b610 con 0x7f07800ce1c0 2024-09-21T02:19:00.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768005440 con 0x7f07800ce1c0 2024-09-21T02:19:00.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078015bbd0 con 0x7f07800ce1c0 2024-09-21T02:19:00.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078015b0a0 con 0x7f07800ce1c0 2024-09-21T02:19:00.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.299+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f07680bd060 con 0x7f07800ce1c0 2024-09-21T02:19:00.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.303+0000 7f07777fe640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0760073350 0x7f0760075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:00.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.303+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f076812dba0 con 0x7f07800ce1c0 2024-09-21T02:19:00.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.303+0000 7f077effd640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0760073350 0x7f0760075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:00.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.303+0000 7f077effd640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0760073350 0x7f0760075860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f077000b810 tx=0x7f0770008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.307+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f07680f1290 con 0x7f07800ce1c0 2024-09-21T02:19:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.515+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07800ce5d0 con 0x7f07800ce1c0 2024-09-21T02:19:00.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f07777fe640 1 -- 172.21.15.170:0/3969413656 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f0768016410 con 0x7f07800ce1c0 2024-09-21T02:19:00.521 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:19:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0760073350 msgr2=0x7f0760075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0760073350 0x7f0760075860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f077000b810 tx=0x7f0770008040 comp rx=0 tx=0).stop 2024-09-21T02:19:00.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 msgr2=0x7f07800afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 0x7f07800afd40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0768009790 tx=0x7f0768004040 comp rx=0 tx=0).stop 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 shutdown_connections 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0760073350 0x7f0760075860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07800cf7e0 0x7f07800b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07800ce1c0 0x7f07800afd40 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 --2- 172.21.15.170:0/3969413656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07800cb460 0x7f07801501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 >> 172.21.15.170:0/3969413656 conn(0x7f07800b1c40 msgr2=0x7f07800cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 shutdown_connections 2024-09-21T02:19:00.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:00.519+0000 7f0785494640 1 -- 172.21.15.170:0/3969413656 wait complete. 2024-09-21T02:19:00.541 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:19:00.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:19:00.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:19:00.541 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:19:00.542 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:00.542 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:19:00.542 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:19:00.542 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:19:00.542 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:19:00.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.542 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.549 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42169sec preferred_lft 42169sec 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:19:00.550 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:19:00.551 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:19:00.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.552 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:19:00.604 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:19:00.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.606 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:19:00.656 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:19:00.657 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:19:00.657 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:19:00.657 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:19:00.657 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:19:00.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:19:00.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:19:00.657 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:19:00.657 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:19:00.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:19:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:19:00.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:19:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:19:00.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.783 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:19:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:19:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:19:00.788 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:19:00.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:00.789 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:19:00.843 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:19:00.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:19:00.843 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:00.891 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:19:00.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:19:00.891 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:00.939 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:19:00.996 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:19:01.044 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:19:01.044 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:19:01.044 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:19:01.044 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:19:01.044 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:19:01.045 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:01.046 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:01.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:01.047 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:19:01.095 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:19:01.096 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:19:01.140 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:19:01.154 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:19:01.200 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:19:01.201 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:19:01.201 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:19:01.201 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:19:01.201 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:19:01.202 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:01.204 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:01.204 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:01.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:01.205 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:19:01.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:19:01.245+0000 7ffb12760580 -1 init, newargv = 0x7ffafc001260 newargc=15 2024-09-21T02:19:01.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35196]: starting ceph client 2024-09-21T02:19:01.253 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35196]: starting fuse 2024-09-21T02:19:02.352 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:19:02.368 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:19:02.420 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:19:02.421 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:19:02.422 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:02.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:19:02.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:02.423 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:19:02.423 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:02.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:02.424 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:19:02.471 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:19:02.471 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:19:02.472 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:19:02.473 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:19:02.594 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.35196.asok 2024-09-21T02:19:02.597 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:19:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:19:02.605 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35196.asok 2024-09-21T02:19:02.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:02.606 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35196.asok status 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:19:02.786 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "35196", 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5711, 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5711 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:19:02.787 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3543835842 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3543835842 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5711 192.168.0.1:0/3543835842", 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/3543835842", 2024-09-21T02:19:02.788 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:19:02.789 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 71, 2024-09-21T02:19:02.789 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 104, 2024-09-21T02:19:02.789 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:19:02.789 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:19:02.789 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:19:02.789 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:19:02.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:02.802 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:19:02.802 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:02.807 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:19:02.808 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:02.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:02.808 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:02.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:19:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- 172.21.15.170:0/484346555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80706d0 msgr2=0x7fd8c80cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 --2- 172.21.15.170:0/484346555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80706d0 0x7fd8c80cabc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009840 tx=0x7fd8b801c820 comp rx=0 tx=0).stop 2024-09-21T02:19:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- 172.21.15.170:0/484346555 shutdown_connections 2024-09-21T02:19:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 --2- 172.21.15.170:0/484346555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c80735d0 0x7fd8c80739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 --2- 172.21.15.170:0/484346555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80706d0 0x7fd8c80cabc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- 172.21.15.170:0/484346555 >> 172.21.15.170:0/484346555 conn(0x7fd8c80b1c40 msgr2=0x7fd8c80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- 172.21.15.170:0/484346555 shutdown_connections 2024-09-21T02:19:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- 172.21.15.170:0/484346555 wait complete. 2024-09-21T02:19:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 Processor -- start 2024-09-21T02:19:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- start start 2024-09-21T02:19:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80735d0 0x7fd8c80c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 0x7fd8c815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c8165380 0x7fd8c8165790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8168ca0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8168b20 con 0x7fd8c80735d0 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8d0147640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c8168e20 con 0x7fd8c8165380 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8cd6e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 0x7fd8c815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8cd6e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 0x7fd8c815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51476/0 (socket says 172.21.15.170:51476) 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8cd6e2640 1 -- 172.21.15.170:0/4127378088 learned_addr learned my addr 172.21.15.170:0/4127378088 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8cdee3640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80735d0 0x7fd8c80c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.035+0000 7fd8ce6e4640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c8165380 0x7fd8c8165790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cd6e2640 1 -- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c8165380 msgr2=0x7fd8c8165790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cd6e2640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c8165380 0x7fd8c8165790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cd6e2640 1 -- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80735d0 msgr2=0x7fd8c80c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cd6e2640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80735d0 0x7fd8c80c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cd6e2640 1 -- 172.21.15.170:0/4127378088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c8165cd0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cd6e2640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 0x7fd8c815a3b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd8b801aed0 tx=0x7fd8b80a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80bb070 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b8004470 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c81fc1f0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80b2040 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81fc7b0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c80cabc0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fd8b80be070 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8be7fc640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd8ac073320 0x7fd8ac075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fd8b812dd20 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cdee3640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd8ac073320 0x7fd8ac075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.039+0000 7fd8cdee3640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd8ac073320 0x7fd8ac075830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4007fd0 tx=0x7fd8c4009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:03.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.043+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd8c8165cd0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.247+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd8c80b37e0 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.927+0000 7fd8be7fc640 1 -- 172.21.15.170:0/4127378088 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v72) ==== 136+0+0 (secure 0 0 0) 0x7fd8b8016410 con 0x7fd8c80ca7b0 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd8ac073320 msgr2=0x7fd8ac075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd8ac073320 0x7fd8ac075830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4007fd0 tx=0x7fd8c4009040 comp rx=0 tx=0).stop 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 msgr2=0x7fd8c815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 0x7fd8c815a3b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd8b801aed0 tx=0x7fd8b80a5650 comp rx=0 tx=0).stop 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 shutdown_connections 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd8ac073320 0x7fd8ac075830 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c8165380 0x7fd8c8165790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8c80ca7b0 0x7fd8c815a3b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 --2- 172.21.15.170:0/4127378088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c80735d0 0x7fd8c80c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 >> 172.21.15.170:0/4127378088 conn(0x7fd8c80b1c40 msgr2=0x7fd8c80b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 shutdown_connections 2024-09-21T02:19:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:03.931+0000 7fd8d0147640 1 -- 172.21.15.170:0/4127378088 wait complete. 2024-09-21T02:19:03.959 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 -- 172.21.15.170:0/2160078864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 msgr2=0x7f96640b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 --2- 172.21.15.170:0/2160078864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 0x7f96640b0ba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f965c009bc0 tx=0x7f965c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 -- 172.21.15.170:0/2160078864 shutdown_connections 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 --2- 172.21.15.170:0/2160078864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 0x7f96640b0ba0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 --2- 172.21.15.170:0/2160078864 >> v2:172.21.15.191:3300/0 conn(0x7f96640b2900 0x7f966406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 -- 172.21.15.170:0/2160078864 >> 172.21.15.170:0/2160078864 conn(0x7f966409a9c0 msgr2=0x7f96640b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 -- 172.21.15.170:0/2160078864 shutdown_connections 2024-09-21T02:19:04.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.115+0000 7f9669892640 1 -- 172.21.15.170:0/2160078864 wait complete. 2024-09-21T02:19:04.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 Processor -- start 2024-09-21T02:19:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 -- start start 2024-09-21T02:19:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 0x7f96640cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 0x7f96641547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9664157fa0 0x7f96641659b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f966416a020 con 0x7f96640b6720 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9664169ea0 con 0x7f9664157fa0 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9669892640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f966416a1a0 con 0x7f966406d340 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 0x7f96641547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 0x7f96641547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51490/0 (socket says 172.21.15.170:51490) 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 -- 172.21.15.170:0/1048381584 learned_addr learned my addr 172.21.15.170:0/1048381584 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96637fe640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9664157fa0 0x7f96641659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f9662ffd640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 0x7f96640cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 -- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 msgr2=0x7f96640cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 0x7f96640cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 -- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9664157fa0 msgr2=0x7f96641659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9664157fa0 0x7f96641659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.119+0000 7f96627fc640 1 -- 172.21.15.170:0/1048381584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9664154d00 con 0x7f96640b6720 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f96627fc640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 0x7f96641547c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f965800e7d0 tx=0x7f965800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965800ccc0 con 0x7f96640b6720 2024-09-21T02:19:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9658010430 con 0x7f96640b6720 2024-09-21T02:19:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9664158780 con 0x7f96640b6720 2024-09-21T02:19:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658010860 con 0x7f96640b6720 2024-09-21T02:19:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9664158d40 con 0x7f96640b6720 2024-09-21T02:19:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96641584f0 con 0x7f96640b6720 2024-09-21T02:19:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f9658010a00 con 0x7f96640b6720 2024-09-21T02:19:04.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f964bfff640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9650073320 0x7f9650075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f9662ffd640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9650073320 0x7f9650075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f9658094ec0 con 0x7f96640b6720 2024-09-21T02:19:04.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.123+0000 7f9662ffd640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9650073320 0x7f9650075830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9654007fd0 tx=0x7f9654007ec0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:04.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.127+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f965805f630 con 0x7f96640b6720 2024-09-21T02:19:04.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.319+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f96640b0ba0 con 0x7f9650073320 2024-09-21T02:19:04.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f964bfff640 1 -- 172.21.15.170:0/1048381584 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f96640b0ba0 con 0x7f9650073320 2024-09-21T02:19:04.324 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:19:04.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:04.324 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:19:04.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:04.324 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9650073320 msgr2=0x7f9650075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9650073320 0x7f9650075830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9654007fd0 tx=0x7f9654007ec0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 msgr2=0x7f96641547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 0x7f96641547c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f965800e7d0 tx=0x7f965800ec90 comp rx=0 tx=0).stop 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 shutdown_connections 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9650073320 0x7f9650075830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9664157fa0 0x7f96641659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96640b6720 0x7f96641547c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 --2- 172.21.15.170:0/1048381584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966406d340 0x7f96640cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 >> 172.21.15.170:0/1048381584 conn(0x7f966409a9c0 msgr2=0x7f96640b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 shutdown_connections 2024-09-21T02:19:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.323+0000 7f9669892640 1 -- 172.21.15.170:0/1048381584 wait complete. 2024-09-21T02:19:04.344 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 -- 172.21.15.170:0/715550810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480cf7d0 msgr2=0x7f5848073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 --2- 172.21.15.170:0/715550810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480cf7d0 0x7f5848073b60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5838009900 tx=0x7f583801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 -- 172.21.15.170:0/715550810 shutdown_connections 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 --2- 172.21.15.170:0/715550810 >> v2:172.21.15.191:3300/0 conn(0x7f58480744b0 0x7f58480ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 --2- 172.21.15.170:0/715550810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480cf7d0 0x7f5848073b60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 -- 172.21.15.170:0/715550810 >> 172.21.15.170:0/715550810 conn(0x7f58480d0170 msgr2=0x7f58480d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 -- 172.21.15.170:0/715550810 shutdown_connections 2024-09-21T02:19:04.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.487+0000 7f584d3c9640 1 -- 172.21.15.170:0/715550810 wait complete. 2024-09-21T02:19:04.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 Processor -- start 2024-09-21T02:19:04.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 -- start start 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 0x7f58480aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480744b0 0x7f58481fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58480cf7d0 0x7f584815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58480b5cd0 con 0x7f58480740a0 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58480ceed0 con 0x7f58480cf7d0 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5848168cb0 con 0x7f58480744b0 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 0x7f58480aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 0x7f58480aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51498/0 (socket says 172.21.15.170:51498) 2024-09-21T02:19:04.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 -- 172.21.15.170:0/3920781932 learned_addr learned my addr 172.21.15.170:0/3920781932 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f583ffff640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480744b0 0x7f58481fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f58477fe640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58480cf7d0 0x7f584815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 -- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480744b0 msgr2=0x7f58481fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480744b0 0x7f58481fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 -- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58480cf7d0 msgr2=0x7f584815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58480cf7d0 0x7f584815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 -- 172.21.15.170:0/3920781932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58481473c0 con 0x7f58480740a0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f58477fe640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58480cf7d0 0x7f584815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f5846ffd640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 0x7f58480aed90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5838009850 tx=0x7f58380a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380b2040 con 0x7f58480740a0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58380043b0 con 0x7f58480740a0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58481fd1a0 con 0x7f58480740a0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838005290 con 0x7f58480740a0 2024-09-21T02:19:04.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.491+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5848147940 con 0x7f58480740a0 2024-09-21T02:19:04.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.495+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848073b60 con 0x7f58480740a0 2024-09-21T02:19:04.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.495+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f5838005430 con 0x7f58480740a0 2024-09-21T02:19:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.495+0000 7f583f7fe640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5828073350 0x7f5828075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.495+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f583812dc70 con 0x7f58480740a0 2024-09-21T02:19:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.495+0000 7f583ffff640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5828073350 0x7f5828075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.495+0000 7f583ffff640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5828073350 0x7f5828075860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58480700b0 tx=0x7f5830009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:04.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.499+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f58380bc060 con 0x7f58480740a0 2024-09-21T02:19:04.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.659+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f584815e5a0 con 0x7f58480740a0 2024-09-21T02:19:04.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.659+0000 7f583f7fe640 1 -- 172.21.15.170:0/3920781932 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f58380164d0 con 0x7f58480740a0 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5828073350 msgr2=0x7f5828075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5828073350 0x7f5828075860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58480700b0 tx=0x7f5830009040 comp rx=0 tx=0).stop 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 msgr2=0x7f58480aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 0x7f58480aed90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5838009850 tx=0x7f58380a5710 comp rx=0 tx=0).stop 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 shutdown_connections 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5828073350 0x7f5828075860 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58480744b0 0x7f58481fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58480cf7d0 0x7f584815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 --2- 172.21.15.170:0/3920781932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58480740a0 0x7f58480aed90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 >> 172.21.15.170:0/3920781932 conn(0x7f58480d0170 msgr2=0x7f5848156460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 shutdown_connections 2024-09-21T02:19:04.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:04.663+0000 7f584d3c9640 1 -- 172.21.15.170:0/3920781932 wait complete. 2024-09-21T02:19:04.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:04.682 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8) 2024-09-21T02:19:04.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:04.714 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/fake) 2024-09-21T02:19:04.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:04.791 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:19:04.839 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:19:04.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:04.840 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8) 2024-09-21T02:19:04.905 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:04.906 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol29140/.meta 2024-09-21T02:19:04.965 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol29140 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.159+0000 7f7dfec18640 1 -- 172.21.15.170:0/2553629364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80cea80 msgr2=0x7f7df80cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.159+0000 7f7dfec18640 1 --2- 172.21.15.170:0/2553629364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80cea80 0x7f7df80cee90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7df80cca30 tx=0x7f7dec01c8a0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.159+0000 7f7dfec18640 1 -- 172.21.15.170:0/2553629364 shutdown_connections 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.159+0000 7f7dfec18640 1 --2- 172.21.15.170:0/2553629364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80cea80 0x7f7df80cee90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.159+0000 7f7dfec18640 1 --2- 172.21.15.170:0/2553629364 >> v2:172.21.15.191:3300/0 conn(0x7f7df80b2fc0 0x7f7df80ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.159+0000 7f7dfec18640 1 -- 172.21.15.170:0/2553629364 >> 172.21.15.170:0/2553629364 conn(0x7f7df80d0170 msgr2=0x7f7df80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:05.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 -- 172.21.15.170:0/2553629364 shutdown_connections 2024-09-21T02:19:05.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 -- 172.21.15.170:0/2553629364 wait complete. 2024-09-21T02:19:05.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 Processor -- start 2024-09-21T02:19:05.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 -- start start 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df806ef40 0x7f7df8152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 0x7f7df80b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df80cea80 0x7f7df809d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8169f50 con 0x7f7df80b2fc0 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8169dd0 con 0x7f7df80cea80 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfec18640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7df816a0d0 con 0x7f7df806ef40 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 0x7f7df80b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 0x7f7df80b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51520/0 (socket says 172.21.15.170:51520) 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 -- 172.21.15.170:0/3904739142 learned_addr learned my addr 172.21.15.170:0/3904739142 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:05.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfc9b4640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df806ef40 0x7f7df8152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7dfd1b5640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df80cea80 0x7f7df809d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 -- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df806ef40 msgr2=0x7f7df8152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df806ef40 0x7f7df8152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 -- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df80cea80 msgr2=0x7f7df809d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df80cea80 0x7f7df809d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 -- 172.21.15.170:0/3904739142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8152680 con 0x7f7df80b2fc0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df7fff640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 0x7f7df80b1b90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0b0040 tx=0x7f7dec005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0bb070 con 0x7f7df80b2fc0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.163+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dec004350 con 0x7f7df80b2fc0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81f6cc0 con 0x7f7df80b2fc0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0b2040 con 0x7f7df80b2fc0 2024-09-21T02:19:05.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df81f7280 con 0x7f7df80b2fc0 2024-09-21T02:19:05.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df80ca210 con 0x7f7df80b2fc0 2024-09-21T02:19:05.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f7dec0be070 con 0x7f7df80b2fc0 2024-09-21T02:19:05.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7df57fa640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7de4073320 0x7f7de4075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7dfc9b4640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7de4073320 0x7f7de4075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f7dec12dc10 con 0x7f7df80b2fc0 2024-09-21T02:19:05.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.167+0000 7f7dfc9b4640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7de4073320 0x7f7de4075830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005c50 tx=0x7f7df0005be0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:05.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.171+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7dec016490 con 0x7f7df80b2fc0 2024-09-21T02:19:05.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.375+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7f7df80b4d60 con 0x7f7de4073320 2024-09-21T02:19:05.382 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:05.379+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:05.383 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:05.379+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:05.383 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:05.379+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:05.383 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:05.379+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:05.383 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:05.379+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:05.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.527+0000 7f7df57fa640 1 -- 172.21.15.170:0/3904739142 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f7df0009de0 con 0x7f7de4073320 2024-09-21T02:19:05.530 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8 2024-09-21T02:19:05.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7de4073320 msgr2=0x7f7de4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7de4073320 0x7f7de4075830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005c50 tx=0x7f7df0005be0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 msgr2=0x7f7df80b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 0x7f7df80b1b90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0b0040 tx=0x7f7dec005e70 comp rx=0 tx=0).stop 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 shutdown_connections 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7de4073320 0x7f7de4075830 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7df80cea80 0x7f7df809d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7df80b2fc0 0x7f7df80b1b90 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 --2- 172.21.15.170:0/3904739142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7df806ef40 0x7f7df8152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 >> 172.21.15.170:0/3904739142 conn(0x7f7df80d0170 msgr2=0x7f7df8070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 shutdown_connections 2024-09-21T02:19:05.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.531+0000 7f7dfec18640 1 -- 172.21.15.170:0/3904739142 wait complete. 2024-09-21T02:19:05.556 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 -- 172.21.15.170:0/666835798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c15c420 msgr2=0x7fda8c15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 --2- 172.21.15.170:0/666835798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c15c420 0x7fda8c15c830 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fda84009c80 tx=0x7fda8401cda0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 -- 172.21.15.170:0/666835798 shutdown_connections 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 --2- 172.21.15.170:0/666835798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c15c420 0x7fda8c15c830 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 --2- 172.21.15.170:0/666835798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c0b2610 0x7fda8c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 -- 172.21.15.170:0/666835798 >> 172.21.15.170:0/666835798 conn(0x7fda8c0d1280 msgr2=0x7fda8c0d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 -- 172.21.15.170:0/666835798 shutdown_connections 2024-09-21T02:19:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.711+0000 7fda920d7640 1 -- 172.21.15.170:0/666835798 wait complete. 2024-09-21T02:19:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 Processor -- start 2024-09-21T02:19:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- start start 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda8c0b2610 0x7fda8c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 0x7fda8c15d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c15c420 0x7fda8c14a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0cd730 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0cd5b0 con 0x7fda8c0b2610 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c0cd8b0 con 0x7fda8c15c420 2024-09-21T02:19:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 0x7fda8c15d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 0x7fda8c15d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37208/0 (socket says 172.21.15.170:37208) 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 -- 172.21.15.170:0/3194943389 learned_addr learned my addr 172.21.15.170:0/3194943389 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8bfff640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c15c420 0x7fda8c14a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8b7fe640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda8c0b2610 0x7fda8c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 -- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c15c420 msgr2=0x7fda8c14a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c15c420 0x7fda8c14a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 -- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda8c0b2610 msgr2=0x7fda8c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda8c0b2610 0x7fda8c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 -- 172.21.15.170:0/3194943389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c0cb470 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8bfff640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c15c420 0x7fda8c14a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda8affd640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 0x7fda8c15d030 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00d750 tx=0x7fda7c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c017070 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda7c004310 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c0cb750 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c005280 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c156de0 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.715+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c14f8d0 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.719+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fda7c005420 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.719+0000 7fda73fff640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda78073320 0x7fda78075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:05.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.719+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fda7c094f20 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.719+0000 7fda8b7fe640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda78073320 0x7fda78075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:05.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.719+0000 7fda8b7fe640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda78073320 0x7fda78075830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda80001640 tx=0x7fda80009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:05.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.723+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fda7c020020 con 0x7fda8c0ce5c0 2024-09-21T02:19:05.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.919+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fda8c070cb0 con 0x7fda78073320 2024-09-21T02:19:05.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda73fff640 1 -- 172.21.15.170:0/3194943389 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fda8c070cb0 con 0x7fda78073320 2024-09-21T02:19:05.932 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:19:05.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:05.932 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "subvol29140" 2024-09-21T02:19:05.932 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:05.932 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda78073320 msgr2=0x7fda78075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda78073320 0x7fda78075830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda80001640 tx=0x7fda80009040 comp rx=0 tx=0).stop 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 msgr2=0x7fda8c15d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 0x7fda8c15d030 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00d750 tx=0x7fda7c00dc10 comp rx=0 tx=0).stop 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 shutdown_connections 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda78073320 0x7fda78075830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda8c15c420 0x7fda8c14a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda8c0ce5c0 0x7fda8c15d030 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 --2- 172.21.15.170:0/3194943389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda8c0b2610 0x7fda8c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 >> 172.21.15.170:0/3194943389 conn(0x7fda8c0d1280 msgr2=0x7fda8c0ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 shutdown_connections 2024-09-21T02:19:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:05.931+0000 7fda920d7640 1 -- 172.21.15.170:0/3194943389 wait complete. 2024-09-21T02:19:05.954 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol29140 2024-09-21T02:19:06.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 -- 172.21.15.170:0/1086134528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200cb380 msgr2=0x7f5d200ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 --2- 172.21.15.170:0/1086134528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200cb380 0x7f5d200ce070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009900 tx=0x7f5d1001c850 comp rx=0 tx=0).stop 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 -- 172.21.15.170:0/1086134528 shutdown_connections 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 --2- 172.21.15.170:0/1086134528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d200700b0 0x7f5d200ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 --2- 172.21.15.170:0/1086134528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200cb380 0x7f5d200ce070 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 -- 172.21.15.170:0/1086134528 >> 172.21.15.170:0/1086134528 conn(0x7f5d200d0170 msgr2=0x7f5d200d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 -- 172.21.15.170:0/1086134528 shutdown_connections 2024-09-21T02:19:06.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.111+0000 7f5d25d12640 1 -- 172.21.15.170:0/1086134528 wait complete. 2024-09-21T02:19:06.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 Processor -- start 2024-09-21T02:19:06.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 -- start start 2024-09-21T02:19:06.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200700b0 0x7f5d2014c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d200cb380 0x7f5d20161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 0x7f5d2014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20073f20 con 0x7f5d200cb790 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20073da0 con 0x7f5d200700b0 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d25d12640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5d200740a0 con 0x7f5d200cb380 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d1ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 0x7f5d2014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d1ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 0x7f5d2014cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37224/0 (socket says 172.21.15.170:37224) 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d1ffff640 1 -- 172.21.15.170:0/3575413804 learned_addr learned my addr 172.21.15.170:0/3575413804 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d1f7fe640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200700b0 0x7f5d2014c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.115+0000 7f5d1effd640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d200cb380 0x7f5d20161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1ffff640 1 -- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d200cb380 msgr2=0x7f5d20161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1ffff640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d200cb380 0x7f5d20161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1ffff640 1 -- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200700b0 msgr2=0x7f5d2014c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1ffff640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200700b0 0x7f5d2014c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1ffff640 1 -- 172.21.15.170:0/3575413804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d20161fe0 con 0x7f5d200cb790 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1f7fe640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200700b0 0x7f5d2014c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1ffff640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 0x7f5d2014cf00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800eab0 tx=0x7f5d1800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1800cea0 con 0x7f5d200cb790 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d20160ce0 con 0x7f5d200cb790 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d18004b30 con 0x7f5d200cb790 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18005020 con 0x7f5d200cb790 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201612a0 con 0x7f5d200cb790 2024-09-21T02:19:06.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2009d5d0 con 0x7f5d200cb790 2024-09-21T02:19:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f5d180051c0 con 0x7f5d200cb790 2024-09-21T02:19:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d07fff640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d0c073350 0x7f5d0c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d1f7fe640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d0c073350 0x7f5d0c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.119+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f5d180955b0 con 0x7f5d200cb790 2024-09-21T02:19:06.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.123+0000 7f5d1f7fe640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d0c073350 0x7f5d0c075860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d100af000 tx=0x7f5d1001b090 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:06.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.123+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5d18060400 con 0x7f5d200cb790 2024-09-21T02:19:06.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.327+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7f5d2009d860 con 0x7f5d0c073350 2024-09-21T02:19:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.355+0000 7f5d07fff640 1 -- 172.21.15.170:0/3575413804 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+706 (secure 0 0 0) 0x7f5d2009d860 con 0x7f5d0c073350 2024-09-21T02:19:06.359 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "atime": "2024-09-21 02:19:04", 2024-09-21T02:19:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_pcent": "undefined", 2024-09-21T02:19:06.359 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_quota": "infinite", 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_used": 0, 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "created_at": "2024-09-21 02:19:04", 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "ctime": "2024-09-21 02:19:05", 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "data_pool": "cephfs_data", 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "earmark": "", 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "features": [ 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-clone", 2024-09-21T02:19:06.360 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-autoprotect" 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "flavor": 1, 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "gid": 0, 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": 16877, 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "mtime": "2024-09-21 02:19:04", 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "path": "/volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8", 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_namespace": "", 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "complete", 2024-09-21T02:19:06.361 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "subvolume", 2024-09-21T02:19:06.362 INFO:teuthology.orchestra.run.smithi170.stdout: "uid": 0 2024-09-21T02:19:06.362 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d0c073350 msgr2=0x7f5d0c075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d0c073350 0x7f5d0c075860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d100af000 tx=0x7f5d1001b090 comp rx=0 tx=0).stop 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 msgr2=0x7f5d2014cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 0x7f5d2014cf00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800eab0 tx=0x7f5d1800ef70 comp rx=0 tx=0).stop 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 shutdown_connections 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d0c073350 0x7f5d0c075860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d200cb380 0x7f5d20161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d200cb790 0x7f5d2014cf00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 --2- 172.21.15.170:0/3575413804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d200700b0 0x7f5d2014c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 >> 172.21.15.170:0/3575413804 conn(0x7f5d200d0170 msgr2=0x7f5d200afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 shutdown_connections 2024-09-21T02:19:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.359+0000 7f5d25d12640 1 -- 172.21.15.170:0/3575413804 wait complete. 2024-09-21T02:19:06.379 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol29140 10485760 2024-09-21T02:19:06.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 -- 172.21.15.170:0/1162912803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae00b6540 msgr2=0x7feae0074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 --2- 172.21.15.170:0/1162912803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae00b6540 0x7feae0074510 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fead8009c80 tx=0x7fead801cda0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 -- 172.21.15.170:0/1162912803 shutdown_connections 2024-09-21T02:19:06.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 --2- 172.21.15.170:0/1162912803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feae0160170 0x7feae0160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 --2- 172.21.15.170:0/1162912803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae00b6540 0x7feae0074510 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 -- 172.21.15.170:0/1162912803 >> 172.21.15.170:0/1162912803 conn(0x7feae00d0940 msgr2=0x7feae00d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:06.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 -- 172.21.15.170:0/1162912803 shutdown_connections 2024-09-21T02:19:06.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 -- 172.21.15.170:0/1162912803 wait complete. 2024-09-21T02:19:06.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 Processor -- start 2024-09-21T02:19:06.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.527+0000 7feae8338640 1 -- start start 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feae00b6540 0x7feae01fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 0x7feae0155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feae0160ac0 0x7feae0158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feae016a510 con 0x7feae0160170 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feae016a390 con 0x7feae0160ac0 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7feae016a690 con 0x7feae00b6540 2024-09-21T02:19:06.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 0x7feae0155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 0x7feae0155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37242/0 (socket says 172.21.15.170:37242) 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 -- 172.21.15.170:0/167245611 learned_addr learned my addr 172.21.15.170:0/167245611 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae68d5640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feae0160ac0 0x7feae0158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 -- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feae00b6540 msgr2=0x7feae01fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae60d4640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feae00b6540 0x7feae01fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feae00b6540 0x7feae01fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 -- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feae0160ac0 msgr2=0x7feae0158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feae0160ac0 0x7feae0158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 -- 172.21.15.170:0/167245611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae01fd7b0 con 0x7feae0160170 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae68d5640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feae0160ac0 0x7feae0158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae58d3640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 0x7feae0155150 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fead400b420 tx=0x7fead400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:06.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4011830 con 0x7feae0160170 2024-09-21T02:19:06.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fead4016070 con 0x7feae0160170 2024-09-21T02:19:06.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae01467f0 con 0x7feae0160170 2024-09-21T02:19:06.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4011d60 con 0x7feae0160170 2024-09-21T02:19:06.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae00cec40 con 0x7feae0160170 2024-09-21T02:19:06.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae00ce6d0 con 0x7feae0160170 2024-09-21T02:19:06.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fead401b460 con 0x7feae0160170 2024-09-21T02:19:06.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7fead2ffd640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feac4073320 0x7feac4075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae60d4640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feac4073320 0x7feac4075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fead4094cc0 con 0x7feae0160170 2024-09-21T02:19:06.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.531+0000 7feae60d4640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feac4073320 0x7feac4075830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fead80af000 tx=0x7fead801b5e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:06.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.535+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feae01fd7b0 con 0x7feae0160170 2024-09-21T02:19:06.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.731+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol29140", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7feae015ff50 con 0x7feac4073320 2024-09-21T02:19:06.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.763+0000 7fead2ffd640 1 -- 172.21.15.170:0/167245611 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7feae015ff50 con 0x7feac4073320 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_used": 0 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_quota": 10485760 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_pcent": "0.00" 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:06.766 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feac4073320 msgr2=0x7feac4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feac4073320 0x7feac4075830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fead80af000 tx=0x7fead801b5e0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 msgr2=0x7feae0155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 0x7feae0155150 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fead400b420 tx=0x7fead400b8e0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 shutdown_connections 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feac4073320 0x7feac4075830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feae0160ac0 0x7feae0158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feae0160170 0x7feae0155150 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 --2- 172.21.15.170:0/167245611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feae00b6540 0x7feae01fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 >> 172.21.15.170:0/167245611 conn(0x7feae00d0940 msgr2=0x7feae00b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:06.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 shutdown_connections 2024-09-21T02:19:06.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.767+0000 7feae8338640 1 -- 172.21.15.170:0/167245611 wait complete. 2024-09-21T02:19:06.786 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol29140 2024-09-21T02:19:06.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 -- 172.21.15.170:0/3313171924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb406f9a0 msgr2=0x7f6bb406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3313171924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb406f9a0 0x7f6bb406fdb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6bb406dc90 tx=0x7f6bac01ccf0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 -- 172.21.15.170:0/3313171924 shutdown_connections 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3313171924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb406f9a0 0x7f6bb406fdb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3313171924 >> v2:172.21.15.191:3300/0 conn(0x7f6bb40cb790 0x7f6bb40ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3313171924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40cb380 0x7f6bb40ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 -- 172.21.15.170:0/3313171924 >> 172.21.15.170:0/3313171924 conn(0x7f6bb40d0170 msgr2=0x7f6bb40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 -- 172.21.15.170:0/3313171924 shutdown_connections 2024-09-21T02:19:06.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.931+0000 7f6bbb962640 1 -- 172.21.15.170:0/3313171924 wait complete. 2024-09-21T02:19:06.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 Processor -- start 2024-09-21T02:19:06.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 -- start start 2024-09-21T02:19:06.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb406f9a0 0x7f6bb40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40cb380 0x7f6bb40afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 0x7f6bb40b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb415fcc0 con 0x7f6bb40cb790 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb415fb40 con 0x7f6bb40cb380 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bbb962640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb415fe40 con 0x7f6bb406f9a0 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 0x7f6bb40b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 0x7f6bb40b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37258/0 (socket says 172.21.15.170:37258) 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 -- 172.21.15.170:0/3458194929 learned_addr learned my addr 172.21.15.170:0/3458194929 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb8efd640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40cb380 0x7f6bb40afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb96fe640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb406f9a0 0x7f6bb40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 -- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb406f9a0 msgr2=0x7f6bb40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb406f9a0 0x7f6bb40b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 -- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40cb380 msgr2=0x7f6bb40afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40cb380 0x7f6bb40afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb9eff640 1 -- 172.21.15.170:0/3458194929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb40b08a0 con 0x7f6bb40cb790 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.935+0000 7f6bb96fe640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb406f9a0 0x7f6bb40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6bb9eff640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 0x7f6bb40b0110 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009bd0 tx=0x7f6bac005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0bb070 con 0x7f6bb40cb790 2024-09-21T02:19:06.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bac004d40 con 0x7f6bb40cb790 2024-09-21T02:19:06.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0b2040 con 0x7f6bb40cb790 2024-09-21T02:19:06.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb40b0b40 con 0x7f6bb40cb790 2024-09-21T02:19:06.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb4167a50 con 0x7f6bb40cb790 2024-09-21T02:19:06.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb406fdb0 con 0x7f6bb40cb790 2024-09-21T02:19:06.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f6bac0be070 con 0x7f6bb40cb790 2024-09-21T02:19:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6ba9ffb640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b98073320 0x7f6b98075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f6bac12e380 con 0x7f6bb40cb790 2024-09-21T02:19:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.939+0000 7f6bb96fe640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b98073320 0x7f6b98075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.943+0000 7f6bb96fe640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b98073320 0x7f6b98075830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb0005d40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:06.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:06.943+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6bac016970 con 0x7f6bb40cb790 2024-09-21T02:19:07.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.131+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7f6bb40c6b90 con 0x7f6b98073320 2024-09-21T02:19:07.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.151+0000 7f6ba9ffb640 1 -- 172.21.15.170:0/3458194929 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+699 (secure 0 0 0) 0x7f6bb40c6b90 con 0x7f6b98073320 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "atime": "2024-09-21 02:19:04", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_pcent": "0.00", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_quota": 10485760, 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_used": 0, 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "created_at": "2024-09-21 02:19:04", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "ctime": "2024-09-21 02:19:06", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "data_pool": "cephfs_data", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "earmark": "", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "features": [ 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-clone", 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-autoprotect" 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:19:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: "flavor": 1, 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "gid": 0, 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": 16877, 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mtime": "2024-09-21 02:19:04", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "path": "/volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_namespace": "", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "complete", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "subvolume", 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: "uid": 0 2024-09-21T02:19:07.155 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.151+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b98073320 msgr2=0x7f6b98075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b98073320 0x7f6b98075830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb0005d40 comp rx=0 tx=0).stop 2024-09-21T02:19:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 msgr2=0x7f6bb40b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 0x7f6bb40b0110 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009bd0 tx=0x7f6bac005b00 comp rx=0 tx=0).stop 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 shutdown_connections 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b98073320 0x7f6b98075830 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb40cb790 0x7f6bb40b0110 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40cb380 0x7f6bb40afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 --2- 172.21.15.170:0/3458194929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb406f9a0 0x7f6bb40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 >> 172.21.15.170:0/3458194929 conn(0x7f6bb40d0170 msgr2=0x7f6bb40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 shutdown_connections 2024-09-21T02:19:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.155+0000 7f6bbb962640 1 -- 172.21.15.170:0/3458194929 wait complete. 2024-09-21T02:19:07.174 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol29140 --mode 777 --uid 1000 --gid 1000 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2129702720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 msgr2=0x7fd324074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2129702720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd324074390 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd31c009bc0 tx=0x7fd31c01cb10 comp rx=0 tx=0).stop 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2129702720 shutdown_connections 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2129702720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd324074390 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2129702720 >> 172.21.15.170:0/2129702720 conn(0x7fd3240d0170 msgr2=0x7fd3240d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2129702720 shutdown_connections 2024-09-21T02:19:07.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.319+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2129702720 wait complete. 2024-09-21T02:19:07.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 Processor -- start 2024-09-21T02:19:07.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- start start 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd3240b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3240cb380 0x7fd32409cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3240ce780 0x7fd32409d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd324168e60 con 0x7fd3240700b0 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd324168ce0 con 0x7fd3240cb380 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd324168fe0 con 0x7fd3240ce780 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd3240b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd3240b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37274/0 (socket says 172.21.15.170:37274) 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 -- 172.21.15.170:0/2940303298 learned_addr learned my addr 172.21.15.170:0/2940303298 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd328870640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3240ce780 0x7fd32409d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd3237fe640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3240cb380 0x7fd32409cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 -- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3240ce780 msgr2=0x7fd32409d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3240ce780 0x7fd32409d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 -- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3240cb380 msgr2=0x7fd32409cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3240cb380 0x7fd32409cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 -- 172.21.15.170:0/2940303298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3240b09a0 con 0x7fd3240700b0 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd323fff640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd3240b2410 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c3f0 tx=0x7fd31400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314017070 con 0x7fd3240700b0 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3140053c0 con 0x7fd3240700b0 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32409b7a0 con 0x7fd3240700b0 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314003860 con 0x7fd3240700b0 2024-09-21T02:19:07.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3240ca350 con 0x7fd3240700b0 2024-09-21T02:19:07.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.323+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3240ce070 con 0x7fd3240700b0 2024-09-21T02:19:07.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.327+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7fd31401a070 con 0x7fd3240700b0 2024-09-21T02:19:07.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.327+0000 7fd320ff9640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd310073320 0x7fd310075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.327+0000 7fd3237fe640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd310073320 0x7fd310075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.327+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fd314093760 con 0x7fd3240700b0 2024-09-21T02:19:07.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.327+0000 7fd3237fe640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd310073320 0x7fd310075830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd324075eb0 tx=0x7fd318009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:07.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.331+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd31405e5b0 con 0x7fd3240700b0 2024-09-21T02:19:07.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.519+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol29140", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7fd3240761c0 con 0x7fd310073320 2024-09-21T02:19:07.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.551+0000 7fd320ff9640 1 -- 172.21.15.170:0/2940303298 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3240761c0 con 0x7fd310073320 2024-09-21T02:19:07.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd310073320 msgr2=0x7fd310075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd310073320 0x7fd310075830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd324075eb0 tx=0x7fd318009040 comp rx=0 tx=0).stop 2024-09-21T02:19:07.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 msgr2=0x7fd3240b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd3240b2410 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c3f0 tx=0x7fd31400c8b0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 shutdown_connections 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd310073320 0x7fd310075830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3240ce780 0x7fd32409d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3240cb380 0x7fd32409cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 --2- 172.21.15.170:0/2940303298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3240700b0 0x7fd3240b2410 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 >> 172.21.15.170:0/2940303298 conn(0x7fd3240d0170 msgr2=0x7fd3240ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 shutdown_connections 2024-09-21T02:19:07.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.555+0000 7fd32a2d3640 1 -- 172.21.15.170:0/2940303298 wait complete. 2024-09-21T02:19:07.575 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol29140 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 -- 172.21.15.170:0/159304843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00cc8a0 msgr2=0x7f98b00cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/159304843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00cc8a0 0x7f98b00cccb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009900 tx=0x7f98a401c850 comp rx=0 tx=0).stop 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 -- 172.21.15.170:0/159304843 shutdown_connections 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/159304843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00cc8a0 0x7f98b00cccb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/159304843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b00b04e0 0x7f98b00b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 -- 172.21.15.170:0/159304843 >> 172.21.15.170:0/159304843 conn(0x7f98b009a9c0 msgr2=0x7f98b00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 -- 172.21.15.170:0/159304843 shutdown_connections 2024-09-21T02:19:07.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 -- 172.21.15.170:0/159304843 wait complete. 2024-09-21T02:19:07.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 Processor -- start 2024-09-21T02:19:07.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.719+0000 7f98b6b7a640 1 -- start start 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 0x7f98b014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b00cc8a0 0x7f98b01f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b015f470 0x7f98b01f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0169350 con 0x7f98b00b04e0 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01691d0 con 0x7f98b00cc8a0 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f98b01694d0 con 0x7f98b015f470 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 0x7f98b014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 0x7f98b014fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37280/0 (socket says 172.21.15.170:37280) 2024-09-21T02:19:07.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 -- 172.21.15.170:0/1526932908 learned_addr learned my addr 172.21.15.170:0/1526932908 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98affff640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b00cc8a0 0x7f98b01f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b5117640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b015f470 0x7f98b01f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 -- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b015f470 msgr2=0x7f98b01f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b015f470 0x7f98b01f96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 -- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b00cc8a0 msgr2=0x7f98b01f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b00cc8a0 0x7f98b01f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 -- 172.21.15.170:0/1526932908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b00c5090 con 0x7f98b00b04e0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b4916640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 0x7f98b014fc10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f98a000b410 tx=0x7f98a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0015030 con 0x7f98b00b04e0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a0005980 con 0x7f98b00b04e0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b00cb220 con 0x7f98b00b04e0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0005df0 con 0x7f98b00b04e0 2024-09-21T02:19:07.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b0150820 con 0x7f98b00b04e0 2024-09-21T02:19:07.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b00d19b0 con 0x7f98b00b04e0 2024-09-21T02:19:07.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 97336+0+0 (secure 0 0 0) 0x7f98a0018070 con 0x7f98b00b04e0 2024-09-21T02:19:07.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98ad7fa640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f989c073320 0x7f989c075830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:07.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f98a0094060 con 0x7f98b00b04e0 2024-09-21T02:19:07.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98affff640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f989c073320 0x7f989c075830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:07.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.723+0000 7f98affff640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f989c073320 0x7f989c075830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98a40ae000 tx=0x7f98a401b090 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:07.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.727+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f98b00c5090 con 0x7f98b00b04e0 2024-09-21T02:19:07.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.927+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7f98b00af290 con 0x7f989c073320 2024-09-21T02:19:07.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.947+0000 7f98ad7fa640 1 -- 172.21.15.170:0/1526932908 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f98b00af290 con 0x7f989c073320 2024-09-21T02:19:07.950 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f989c073320 msgr2=0x7f989c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f989c073320 0x7f989c075830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98a40ae000 tx=0x7f98a401b090 comp rx=0 tx=0).stop 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 msgr2=0x7f98b014fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 0x7f98b014fc10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f98a000b410 tx=0x7f98a000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 shutdown_connections 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f989c073320 0x7f989c075830 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b015f470 0x7f98b01f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b00cc8a0 0x7f98b01f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 --2- 172.21.15.170:0/1526932908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b00b04e0 0x7f98b014fc10 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:07.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 >> 172.21.15.170:0/1526932908 conn(0x7f98b009a9c0 msgr2=0x7f98b00ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:07.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 shutdown_connections 2024-09-21T02:19:07.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:07.951+0000 7f98b6b7a640 1 -- 172.21.15.170:0/1526932908 wait complete. 2024-09-21T02:19:07.972 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol29140 with 8 files each 1MB size under directory volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8 2024-09-21T02:19:07.972 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:07.972 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:07.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:07.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.051 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.051 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.051 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198725 s, 52.8 MB/s 2024-09-21T02:19:08.052 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.053 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.101 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.186 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.186 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.186 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215689 s, 48.6 MB/s 2024-09-21T02:19:08.187 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.187 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.233 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.318 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.319 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.319 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202519 s, 51.8 MB/s 2024-09-21T02:19:08.319 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.320 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.370 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.454 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.454 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.454 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207483 s, 50.5 MB/s 2024-09-21T02:19:08.455 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.455 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.506 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.586 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175968 s, 59.6 MB/s 2024-09-21T02:19:08.587 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.588 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.638 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.720 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.720 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.720 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195683 s, 53.6 MB/s 2024-09-21T02:19:08.721 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.721 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.770 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.853 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.853 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.853 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194377 s, 53.9 MB/s 2024-09-21T02:19:08.854 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:08.854 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:08.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:08.902 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T02:19:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T02:19:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T02:19:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167161 s, 62.7 MB/s 2024-09-21T02:19:08.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol29140 subvol_snap95950 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 -- 172.21.15.170:0/2813325355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840cd1f0 msgr2=0x7fce840af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 --2- 172.21.15.170:0/2813325355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840cd1f0 0x7fce840af440 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fce74009840 tx=0x7fce7401c790 comp rx=0 tx=0).stop 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 -- 172.21.15.170:0/2813325355 shutdown_connections 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 --2- 172.21.15.170:0/2813325355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840cd1f0 0x7fce840af440 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 -- 172.21.15.170:0/2813325355 >> 172.21.15.170:0/2813325355 conn(0x7fce840b1c40 msgr2=0x7fce840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 -- 172.21.15.170:0/2813325355 shutdown_connections 2024-09-21T02:19:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 -- 172.21.15.170:0/2813325355 wait complete. 2024-09-21T02:19:09.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 Processor -- start 2024-09-21T02:19:09.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.191+0000 7fce8a098640 1 -- start start 2024-09-21T02:19:09.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 0x7fce841506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce840b52d0 0x7fce8414e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8409b540 0x7fce8409b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce8416b090 con 0x7fce840b4d80 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fce8416af10 con 0x7fce8409b540 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fce8416b210 con 0x7fce840b52d0 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 0x7fce841506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 0x7fce841506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37306/0 (socket says 172.21.15.170:37306) 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 -- 172.21.15.170:0/80348679 learned_addr learned my addr 172.21.15.170:0/80348679 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce82ffd640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce840b52d0 0x7fce8414e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce83fff640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8409b540 0x7fce8409b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 -- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce840b52d0 msgr2=0x7fce8414e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce840b52d0 0x7fce8414e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 -- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8409b540 msgr2=0x7fce8409b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8409b540 0x7fce8409b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 -- 172.21.15.170:0/80348679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce84150c10 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce837fe640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 0x7fce841506d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b3f0 tx=0x7fce6c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c015020 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce6c011940 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8409ae90 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c011d90 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8414fed0 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce840af440 con 0x7fce840b4d80 2024-09-21T02:19:09.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 97387+0+0 (secure 0 0 0) 0x7fce6c018020 con 0x7fce840b4d80 2024-09-21T02:19:09.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.195+0000 7fce7bfff640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fce640733b0 0x7fce640758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.199+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fce6c094270 con 0x7fce840b4d80 2024-09-21T02:19:09.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.199+0000 7fce82ffd640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fce640733b0 0x7fce640758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.199+0000 7fce82ffd640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fce640733b0 0x7fce640758c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce740af000 tx=0x7fce7401afd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:09.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.203+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fce6c05ef50 con 0x7fce840b4d80 2024-09-21T02:19:09.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.407+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol29140", "snap_name": "subvol_snap95950", "target": ["mon-mgr", ""]}) -- 0x7fce840d0c70 con 0x7fce640733b0 2024-09-21T02:19:09.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.463+0000 7fce7bfff640 1 -- 172.21.15.170:0/80348679 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce840d0c70 con 0x7fce640733b0 2024-09-21T02:19:09.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fce640733b0 msgr2=0x7fce640758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fce640733b0 0x7fce640758c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fce740af000 tx=0x7fce7401afd0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 msgr2=0x7fce841506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 0x7fce841506d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b3f0 tx=0x7fce6c00b8b0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 shutdown_connections 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fce640733b0 0x7fce640758c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce840b52d0 0x7fce8414e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce8409b540 0x7fce8409b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 --2- 172.21.15.170:0/80348679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce840b4d80 0x7fce841506d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 >> 172.21.15.170:0/80348679 conn(0x7fce840b1c40 msgr2=0x7fce840cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 shutdown_connections 2024-09-21T02:19:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.467+0000 7fce8a098640 1 -- 172.21.15.170:0/80348679 wait complete. 2024-09-21T02:19:09.490 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol29140 subvol_snap95950 subvol_clone31200 2024-09-21T02:19:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f0354eb9640 1 -- 172.21.15.170:0/4113502610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 msgr2=0x7f0338003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4113502610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f0338003eb0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0340006fd0 tx=0x7f034001cf60 comp rx=0 tx=0).stop 2024-09-21T02:19:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f034ed9d640 1 -- 172.21.15.170:0/4113502610 reap_dead start 2024-09-21T02:19:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f0354eb9640 1 -- 172.21.15.170:0/4113502610 shutdown_connections 2024-09-21T02:19:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f0354eb9640 1 -- 172.21.15.170:0/4113502610 >> 172.21.15.170:0/4113502610 conn(0x7f03500d0170 msgr2=0x7f03500d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f0354eb9640 1 -- 172.21.15.170:0/4113502610 shutdown_connections 2024-09-21T02:19:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.651+0000 7f0354eb9640 1 -- 172.21.15.170:0/4113502610 wait complete. 2024-09-21T02:19:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 Processor -- start 2024-09-21T02:19:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 -- start start 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f035009d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035009c720 0x7f035009d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03501f7c50 0x7f03500b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0350073c20 con 0x7f0338003a40 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f035006f650 con 0x7f03501f7c50 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0350073da0 con 0x7f035009c720 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f035009d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f035009d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37326/0 (socket says 172.21.15.170:37326) 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 -- 172.21.15.170:0/4202064629 learned_addr learned my addr 172.21.15.170:0/4202064629 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034ed9d640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03501f7c50 0x7f03500b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034dd9b640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035009c720 0x7f035009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 -- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035009c720 msgr2=0x7f035009d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035009c720 0x7f035009d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 -- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03501f7c50 msgr2=0x7f03500b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03501f7c50 0x7f03500b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 -- 172.21.15.170:0/4202064629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501f4cb0 con 0x7f0338003a40 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f034e59c640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f035009d1d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f034001b830 tx=0x7f0340001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400a5a00 con 0x7f0338003a40 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03400014e0 con 0x7f0338003a40 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501f4f90 con 0x7f0338003a40 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400060d0 con 0x7f0338003a40 2024-09-21T02:19:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.655+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035009cb50 con 0x7f0338003a40 2024-09-21T02:19:09.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.659+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035009dca0 con 0x7f0338003a40 2024-09-21T02:19:09.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.659+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 97387+0+0 (secure 0 0 0) 0x7f0340006270 con 0x7f0338003a40 2024-09-21T02:19:09.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.659+0000 7f033effd640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f03300733d0 0x7f03300758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:09.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.659+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f034012d970 con 0x7f0338003a40 2024-09-21T02:19:09.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.659+0000 7f034dd9b640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f03300733d0 0x7f03300758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:09.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.659+0000 7f034dd9b640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f03300733d0 0x7f03300758e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f034400cc30 tx=0x7f0344005f30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:09.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.663+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0340016d10 con 0x7f0338003a40 2024-09-21T02:19:09.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:09.871+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol29140", "snap_name": "subvol_snap95950", "target_sub_name": "subvol_clone31200", "target": ["mon-mgr", ""]}) -- 0x7f035009cdf0 con 0x7f03300733d0 2024-09-21T02:19:10.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.039+0000 7f033effd640 1 -- 172.21.15.170:0/4202064629 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f035009cdf0 con 0x7f03300733d0 2024-09-21T02:19:10.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f03300733d0 msgr2=0x7f03300758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f03300733d0 0x7f03300758e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f034400cc30 tx=0x7f0344005f30 comp rx=0 tx=0).stop 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 msgr2=0x7f035009d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f035009d1d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f034001b830 tx=0x7f0340001d10 comp rx=0 tx=0).stop 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 shutdown_connections 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f03300733d0 0x7f03300758e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035009c720 0x7f035009d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03501f7c50 0x7f03500b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 --2- 172.21.15.170:0/4202064629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0338003a40 0x7f035009d1d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 >> 172.21.15.170:0/4202064629 conn(0x7f03500d0170 msgr2=0x7f03500d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 shutdown_connections 2024-09-21T02:19:10.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.043+0000 7f0354eb9640 1 -- 172.21.15.170:0/4202064629 wait complete. 2024-09-21T02:19:10.047 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.043+0000 7efba3acd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.047 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.043+0000 7efba3acd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.047 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.043+0000 7efba3acd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.047 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.043+0000 7efba3acd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.047 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.043+0000 7efba3acd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone31200 2024-09-21T02:19:10.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.071+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.071+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.071+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.071+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:10.071+0000 7efba52d0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:10.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0a9ffb640 1 -- 172.21.15.170:0/3746354746 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00a6dd0 con 0x7fc0ac0cea80 2024-09-21T02:19:10.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 -- 172.21.15.170:0/3746354746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 msgr2=0x7fc0ac0cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:10.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/3746354746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 0x7fc0ac0cee90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009840 tx=0x7fc0a001c820 comp rx=0 tx=0).stop 2024-09-21T02:19:10.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 -- 172.21.15.170:0/3746354746 shutdown_connections 2024-09-21T02:19:10.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/3746354746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 0x7fc0ac0cee90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/3746354746 >> v2:172.21.15.191:3300/0 conn(0x7fc0ac0b2fc0 0x7fc0ac0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 -- 172.21.15.170:0/3746354746 >> 172.21.15.170:0/3746354746 conn(0x7fc0ac0d0170 msgr2=0x7fc0ac0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:10.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 -- 172.21.15.170:0/3746354746 shutdown_connections 2024-09-21T02:19:10.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.215+0000 7fc0b14a9640 1 -- 172.21.15.170:0/3746354746 wait complete. 2024-09-21T02:19:10.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 Processor -- start 2024-09-21T02:19:10.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 -- start start 2024-09-21T02:19:10.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0ac06ef40 0x7fc0ac0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:10.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 0x7fc0ac15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:10.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 0x7fc0ac15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:10.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac169fe0 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0b3640 con 0x7fc0ac06ef40 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac16a160 con 0x7fc0ac0cea80 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 0x7fc0ac15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 0x7fc0ac15abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37356/0 (socket says 172.21.15.170:37356) 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 -- 172.21.15.170:0/1854092911 learned_addr learned my addr 172.21.15.170:0/1854092911 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0ab7fe640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 0x7fc0ac15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aaffd640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0ac06ef40 0x7fc0ac0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 -- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 msgr2=0x7fc0ac15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 0x7fc0ac15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 -- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0ac06ef40 msgr2=0x7fc0ac0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0ac06ef40 0x7fc0ac0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 -- 172.21.15.170:0/1854092911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac0b22b0 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0aa7fc640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 0x7fc0ac15abf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009790 tx=0x7fc0a00a66b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00be070 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a0004470 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac0afd30 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00052b0 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.219+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac0b02f0 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.223+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac0b2020 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.223+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 97387+0+0 (secure 0 0 0) 0x7fc0a00c1070 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.223+0000 7fc0937fe640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0980733b0 0x7fc0980758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.223+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fc0a012ea10 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.223+0000 7fc0aaffd640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0980733b0 0x7fc0980758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:10.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.223+0000 7fc0aaffd640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0980733b0 0x7fc0980758c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009ad0 tx=0x7fc09c0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:10.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.227+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc0a0016410 con 0x7fc0ac0b2fc0 2024-09-21T02:19:10.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:10.435+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone31200", "target": ["mon-mgr", ""]}) -- 0x7fc0ac0b2590 con 0x7fc0980733b0 2024-09-21T02:19:11.853 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:11.851+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:11.853 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:11.851+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:11.853 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:11.851+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:11.853 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:11.851+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:11.853 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:11.851+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.875+0000 7fc0937fe640 1 -- 172.21.15.170:0/1854092911 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+165 (secure 0 0 0) 0x7fc09c009720 con 0x7fc0980733b0 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: "status": { 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "pending", 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: "source": { 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: "volume": "cephfs", 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: "subvolume": "subvol29140", 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot": "subvol_snap95950" 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:11.879 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0980733b0 msgr2=0x7fc0980758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0980733b0 0x7fc0980758c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009ad0 tx=0x7fc09c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 msgr2=0x7fc0ac15abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 0x7fc0ac15abf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009790 tx=0x7fc0a00a66b0 comp rx=0 tx=0).stop 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 shutdown_connections 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0980733b0 0x7fc0980758c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0ac0cea80 0x7fc0ac15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac0b2fc0 0x7fc0ac15abf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:11.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 --2- 172.21.15.170:0/1854092911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0ac06ef40 0x7fc0ac0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:11.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 >> 172.21.15.170:0/1854092911 conn(0x7fc0ac0d0170 msgr2=0x7fc0ac15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:11.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 shutdown_connections 2024-09-21T02:19:11.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:11.879+0000 7fc0b14a9640 1 -- 172.21.15.170:0/1854092911 wait complete. 2024-09-21T02:19:11.903 DEBUG:tasks.cephfs.test_volumes:current clone state = pending 2024-09-21T02:19:12.904 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone31200 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 -- 172.21.15.170:0/1527871485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f971c004810 msgr2=0x7f971c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 --2- 172.21.15.170:0/1527871485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f971c004810 0x7f971c004c20 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f97200016f0 tx=0x7f972001c750 comp rx=0 tx=0).stop 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 -- 172.21.15.170:0/1527871485 shutdown_connections 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 --2- 172.21.15.170:0/1527871485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f971c004810 0x7f971c004c20 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 -- 172.21.15.170:0/1527871485 >> 172.21.15.170:0/1527871485 conn(0x7f972c0d0170 msgr2=0x7f972c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 -- 172.21.15.170:0/1527871485 shutdown_connections 2024-09-21T02:19:13.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.071+0000 7f9732ef1640 1 -- 172.21.15.170:0/1527871485 wait complete. 2024-09-21T02:19:13.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 Processor -- start 2024-09-21T02:19:13.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 -- start start 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971c004810 0x7f972c14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f972c1e7930 0x7f972c1f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 0x7f972c09bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0b3550 con 0x7f972c1f7410 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0b33d0 con 0x7f972c1e7930 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f972c0cf060 con 0x7f971c004810 2024-09-21T02:19:13.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 0x7f972c09bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 0x7f972c09bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37390/0 (socket says 172.21.15.170:37390) 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 -- 172.21.15.170:0/256390688 learned_addr learned my addr 172.21.15.170:0/256390688 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9730c8d640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971c004810 0x7f972c14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f972bfff640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f972c1e7930 0x7f972c1f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 -- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971c004810 msgr2=0x7f972c14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971c004810 0x7f972c14a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 -- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f972c1e7930 msgr2=0x7f972c1f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f972c1e7930 0x7f972c1f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 -- 172.21.15.170:0/256390688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c1f7e10 con 0x7f972c1f7410 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f973148e640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 0x7f972c09bc60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f972c0b60f0 tx=0x7f9724010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724004b30 con 0x7f972c1f7410 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9724015070 con 0x7f972c1f7410 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c0b0c40 con 0x7f972c1f7410 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724003be0 con 0x7f972c1f7410 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.075+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c0b0ee0 con 0x7f972c1f7410 2024-09-21T02:19:13.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.079+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972c15e290 con 0x7f972c1f7410 2024-09-21T02:19:13.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.079+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 97489+0+0 (secure 0 0 0) 0x7f9724004030 con 0x7f972c1f7410 2024-09-21T02:19:13.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.079+0000 7f97297fa640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9710073500 0x7f9710075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:13.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.079+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f9724094f00 con 0x7f972c1f7410 2024-09-21T02:19:13.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.079+0000 7f9730c8d640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9710073500 0x7f9710075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:13.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.079+0000 7f9730c8d640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9710073500 0x7f9710075a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9718001250 tx=0x7f97180011e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:13.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.083+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f972405fba0 con 0x7f972c1f7410 2024-09-21T02:19:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:13.295+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone31200", "target": ["mon-mgr", ""]}) -- 0x7f972c157760 con 0x7f9710073500 2024-09-21T02:19:14.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:14.035+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f972405b710 con 0x7f972c1f7410 2024-09-21T02:19:16.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.839+0000 7f97297fa640 1 -- 172.21.15.170:0/256390688 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f972c157760 con 0x7f9710073500 2024-09-21T02:19:16.844 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:16.844 INFO:teuthology.orchestra.run.smithi170.stdout: "status": { 2024-09-21T02:19:16.844 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "complete" 2024-09-21T02:19:16.844 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:16.844 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9710073500 msgr2=0x7f9710075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9710073500 0x7f9710075a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9718001250 tx=0x7f97180011e0 comp rx=0 tx=0).stop 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 msgr2=0x7f972c09bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 0x7f972c09bc60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f972c0b60f0 tx=0x7f9724010f40 comp rx=0 tx=0).stop 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 shutdown_connections 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9710073500 0x7f9710075a10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c1f7410 0x7f972c09bc60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f972c1e7930 0x7f972c1f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 --2- 172.21.15.170:0/256390688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f971c004810 0x7f972c14a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 >> 172.21.15.170:0/256390688 conn(0x7f972c0d0170 msgr2=0x7f972c15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 shutdown_connections 2024-09-21T02:19:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:16.843+0000 7f9732ef1640 1 -- 172.21.15.170:0/256390688 wait complete. 2024-09-21T02:19:16.868 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-21T02:19:16.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:16.868 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone31200/.meta) 2024-09-21T02:19:16.889 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:19:16.889 INFO:teuthology.orchestra.run.smithi170.stdout:version = 2 2024-09-21T02:19:16.889 INFO:teuthology.orchestra.run.smithi170.stdout:type = clone 2024-09-21T02:19:16.889 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330 2024-09-21T02:19:16.889 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:19:16.889 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:16.891 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol29140 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/2286347600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 msgr2=0x7f5d980ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/2286347600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d980ce540 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009870 tx=0x7f5d8801c820 comp rx=0 tx=0).stop 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/2286347600 shutdown_connections 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/2286347600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d980cea80 0x7f5d980cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/2286347600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d980ce540 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/2286347600 >> 172.21.15.170:0/2286347600 conn(0x7f5d980d0170 msgr2=0x7f5d980d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/2286347600 shutdown_connections 2024-09-21T02:19:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/2286347600 wait complete. 2024-09-21T02:19:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 Processor -- start 2024-09-21T02:19:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- start start 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d9806ef40 0x7f5d9814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d98154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d980cea80 0x7f5d981550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98169fe0 con 0x7f5d980b2fc0 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980b35f0 con 0x7f5d9806ef40 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.103+0000 7f5d9d4f6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9816a160 con 0x7f5d980cea80 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d98154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d98154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57640/0 (socket says 172.21.15.170:57640) 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 -- 172.21.15.170:0/3465525627 learned_addr learned my addr 172.21.15.170:0/3465525627 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d977fe640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d980cea80 0x7f5d981550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 -- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d980cea80 msgr2=0x7f5d981550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d980cea80 0x7f5d981550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 -- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d9806ef40 msgr2=0x7f5d9814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d9806ef40 0x7f5d9814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 -- 172.21.15.170:0/3465525627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d981fa0c0 con 0x7f5d980b2fc0 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d967fc640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d98154b90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d720 tx=0x7f5d8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c017070 con 0x7f5d980b2fc0 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8c009b10 con 0x7f5d980b2fc0 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d98151c80 con 0x7f5d980b2fc0 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c010460 con 0x7f5d980b2fc0 2024-09-21T02:19:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d981540b0 con 0x7f5d980b2fc0 2024-09-21T02:19:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d980b25d0 con 0x7f5d980b2fc0 2024-09-21T02:19:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f5d8c01a070 con 0x7f5d980b2fc0 2024-09-21T02:19:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d7f7fe640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d84073560 0x7f5d84075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f5d8c094f20 con 0x7f5d980b2fc0 2024-09-21T02:19:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d96ffd640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d84073560 0x7f5d84075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.107+0000 7f5d96ffd640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d84073560 0x7f5d84075a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d880af000 tx=0x7f5d880195c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.111+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5d8c05f9e0 con 0x7f5d980b2fc0 2024-09-21T02:19:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.315+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7f5d980ca340 con 0x7f5d84073560 2024-09-21T02:19:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.347+0000 7f5d7f7fe640 1 -- 172.21.15.170:0/3465525627 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f5d980ca340 con 0x7f5d84073560 2024-09-21T02:19:17.350 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8 2024-09-21T02:19:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d84073560 msgr2=0x7f5d84075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d84073560 0x7f5d84075a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d880af000 tx=0x7f5d880195c0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 msgr2=0x7f5d98154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d98154b90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d720 tx=0x7f5d8c00dbe0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 shutdown_connections 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d84073560 0x7f5d84075a70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d980cea80 0x7f5d981550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d980b2fc0 0x7f5d98154b90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 --2- 172.21.15.170:0/3465525627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d9806ef40 0x7f5d9814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 >> 172.21.15.170:0/3465525627 conn(0x7f5d980d0170 msgr2=0x7f5d98153050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 shutdown_connections 2024-09-21T02:19:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.351+0000 7f5d9d4f6640 1 -- 172.21.15.170:0/3465525627 wait complete. 2024-09-21T02:19:17.372 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone31200 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 -- 172.21.15.170:0/3402143692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca80d0b30 msgr2=0x7efca80b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 --2- 172.21.15.170:0/3402143692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca80d0b30 0x7efca80b6110 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7efca80ced30 tx=0x7efca001cd80 comp rx=0 tx=0).stop 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 -- 172.21.15.170:0/3402143692 shutdown_connections 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 --2- 172.21.15.170:0/3402143692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca80d0b30 0x7efca80b6110 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 --2- 172.21.15.170:0/3402143692 >> v2:172.21.15.191:3300/0 conn(0x7efca815a400 0x7efca806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 -- 172.21.15.170:0/3402143692 >> 172.21.15.170:0/3402143692 conn(0x7efca809a9c0 msgr2=0x7efca80d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 -- 172.21.15.170:0/3402143692 shutdown_connections 2024-09-21T02:19:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.531+0000 7efcb055a640 1 -- 172.21.15.170:0/3402143692 wait complete. 2024-09-21T02:19:17.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 Processor -- start 2024-09-21T02:19:17.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 -- start start 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 0x7efca8154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca80d0b30 0x7efca80cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca815a400 0x7efca80ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efca815d6f0 con 0x7efca806d340 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efca815d570 con 0x7efca80d0b30 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efca815d870 con 0x7efca815a400 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 0x7efca8154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 0x7efca8154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57652/0 (socket says 172.21.15.170:57652) 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 -- 172.21.15.170:0/4228363902 learned_addr learned my addr 172.21.15.170:0/4228363902 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcadaf5640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca80d0b30 0x7efca80cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcaeaf7640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca815a400 0x7efca80ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 -- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca815a400 msgr2=0x7efca80ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca815a400 0x7efca80ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 -- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca80d0b30 msgr2=0x7efca80cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca80d0b30 0x7efca80cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 -- 172.21.15.170:0/4228363902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca815ba80 con 0x7efca806d340 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcae2f6640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 0x7efca8154430 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7efca400c970 tx=0x7efca400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4007bb0 con 0x7efca806d340 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efca4007e40 con 0x7efca806d340 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca815bd60 con 0x7efca806d340 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4005270 con 0x7efca806d340 2024-09-21T02:19:17.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.535+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca815c320 con 0x7efca806d340 2024-09-21T02:19:17.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.539+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca80b6110 con 0x7efca806d340 2024-09-21T02:19:17.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.539+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7efca4005410 con 0x7efca806d340 2024-09-21T02:19:17.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.539+0000 7efc9effd640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc8c073590 0x7efc8c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:17.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.539+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7efca4094dc0 con 0x7efca806d340 2024-09-21T02:19:17.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.539+0000 7efcadaf5640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc8c073590 0x7efc8c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:17.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.539+0000 7efcadaf5640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc8c073590 0x7efc8c075aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efca81fc140 tx=0x7efc98008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:17.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.543+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efca4014070 con 0x7efca806d340 2024-09-21T02:19:17.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.747+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone31200", "target": ["mon-mgr", ""]}) -- 0x7efca8154970 con 0x7efc8c073590 2024-09-21T02:19:17.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efc9effd640 1 -- 172.21.15.170:0/4228363902 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7efca8154970 con 0x7efc8c073590 2024-09-21T02:19:17.761 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc8c073590 msgr2=0x7efc8c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc8c073590 0x7efc8c075aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efca81fc140 tx=0x7efc98008040 comp rx=0 tx=0).stop 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 msgr2=0x7efca8154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 0x7efca8154430 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7efca400c970 tx=0x7efca400ce30 comp rx=0 tx=0).stop 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 shutdown_connections 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc8c073590 0x7efc8c075aa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efca815a400 0x7efca80ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efca80d0b30 0x7efca80cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 --2- 172.21.15.170:0/4228363902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca806d340 0x7efca8154430 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 >> 172.21.15.170:0/4228363902 conn(0x7efca809a9c0 msgr2=0x7efca80b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 shutdown_connections 2024-09-21T02:19:17.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:17.759+0000 7efcb055a640 1 -- 172.21.15.170:0/4228363902 wait complete. 2024-09-21T02:19:17.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:17.783 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:17.805 INFO:teuthology.orchestra.run.smithi170.stdout:9 2024-09-21T02:19:17.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:17.805 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330) 2024-09-21T02:19:17.860 INFO:teuthology.orchestra.run.smithi170.stdout:9 2024-09-21T02:19:17.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone31200 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 -- 172.21.15.170:0/2570084079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780700b0 msgr2=0x7f82780ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 --2- 172.21.15.170:0/2570084079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780700b0 0x7f82780ce5b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8270009c80 tx=0x7f827001cd10 comp rx=0 tx=0).stop 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 -- 172.21.15.170:0/2570084079 shutdown_connections 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 --2- 172.21.15.170:0/2570084079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780700b0 0x7f82780ce5b0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 --2- 172.21.15.170:0/2570084079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82780cb380 0x7f82780ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 -- 172.21.15.170:0/2570084079 >> 172.21.15.170:0/2570084079 conn(0x7f82780d0170 msgr2=0x7f82780d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 -- 172.21.15.170:0/2570084079 shutdown_connections 2024-09-21T02:19:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.067+0000 7f827ce26640 1 -- 172.21.15.170:0/2570084079 wait complete. 2024-09-21T02:19:18.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 Processor -- start 2024-09-21T02:19:18.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- start start 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82780700b0 0x7f82781fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82780cb380 0x7f8278158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 0x7f827815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82780746a0 con 0x7f82780cb790 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8278074520 con 0x7f82780cb380 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f82781684d0 con 0x7f82780700b0 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 0x7f827815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 0x7f827815c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57660/0 (socket says 172.21.15.170:57660) 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 -- 172.21.15.170:0/5681645 learned_addr learned my addr 172.21.15.170:0/5681645 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827659c640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82780700b0 0x7f82781fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 -- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82780700b0 msgr2=0x7f82781fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82780700b0 0x7f82781fd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 -- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82780cb380 msgr2=0x7f8278158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82780cb380 0x7f8278158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 -- 172.21.15.170:0/5681645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781fc6a0 con 0x7f82780cb790 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8276d9d640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 0x7f827815c760 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8270009bd0 tx=0x7f8270004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82700bb070 con 0x7f82780cb790 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82700a5960 con 0x7f82780cb790 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82700b2040 con 0x7f82780cb790 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827815d190 con 0x7f82780cb790 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781fc940 con 0x7f82780cb790 2024-09-21T02:19:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.071+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827806f9a0 con 0x7f82780cb790 2024-09-21T02:19:18.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.075+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f82700be070 con 0x7f82780cb790 2024-09-21T02:19:18.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.075+0000 7f8266ffd640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8258073560 0x7f8258075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:18.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.075+0000 7f827659c640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8258073560 0x7f8258075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:18.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.075+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f827012e540 con 0x7f82780cb790 2024-09-21T02:19:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.075+0000 7f827659c640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8258073560 0x7f8258075a70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f82680058d0 tx=0x7f8268005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:18.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.079+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8270016990 con 0x7f82780cb790 2024-09-21T02:19:18.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.267+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone31200", "target": ["mon-mgr", ""]}) -- 0x7f8278158770 con 0x7f8258073560 2024-09-21T02:19:18.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f8266ffd640 1 -- 172.21.15.170:0/5681645 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+744 (secure 0 0 0) 0x7f8278158770 con 0x7f8258073560 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "atime": "2024-09-21 02:19:04", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_pcent": "80.00", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_quota": 10485760, 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "bytes_used": 8388608, 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "created_at": "2024-09-21 02:19:09", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "ctime": "2024-09-21 02:19:12", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "data_pool": "cephfs_data", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "earmark": "", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "features": [ 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-clone", 2024-09-21T02:19:18.289 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-autoprotect", 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot-retention" 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "flavor": 2, 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "gid": 1000, 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "mode": 16895, 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "mtime": "2024-09-21 02:19:08", 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "path": "/volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330", 2024-09-21T02:19:18.290 INFO:teuthology.orchestra.run.smithi170.stdout: "pool_namespace": "", 2024-09-21T02:19:18.291 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "complete", 2024-09-21T02:19:18.291 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "clone", 2024-09-21T02:19:18.291 INFO:teuthology.orchestra.run.smithi170.stdout: "uid": 1000 2024-09-21T02:19:18.291 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:18.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8258073560 msgr2=0x7f8258075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:18.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8258073560 0x7f8258075a70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f82680058d0 tx=0x7f8268005860 comp rx=0 tx=0).stop 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 msgr2=0x7f827815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 0x7f827815c760 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8270009bd0 tx=0x7f8270004270 comp rx=0 tx=0).stop 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 shutdown_connections 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8258073560 0x7f8258075a70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82780cb790 0x7f827815c760 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82780cb380 0x7f8278158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 --2- 172.21.15.170:0/5681645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82780700b0 0x7f82781fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 >> 172.21.15.170:0/5681645 conn(0x7f82780d0170 msgr2=0x7f827815b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 shutdown_connections 2024-09-21T02:19:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:18.287+0000 7f827ce26640 1 -- 172.21.15.170:0/5681645 wait complete. 2024-09-21T02:19:18.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.308 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.325 INFO:teuthology.orchestra.run.smithi170.stdout:10485760 2024-09-21T02:19:18.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.385 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:19:18.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.385 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.444 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.508 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.2 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.5 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.4 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.7 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.6 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.0 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.3 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run.smithi170.stdout:volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.1 2024-09-21T02:19:18.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.509 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.561 INFO:teuthology.orchestra.run.smithi170.stdout:41ff 2024-09-21T02:19:18.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.562 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/) 2024-09-21T02:19:18.610 INFO:teuthology.orchestra.run.smithi170.stdout:41ff 2024-09-21T02:19:18.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.611 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.660 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:18.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/) 2024-09-21T02:19:18.710 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:18.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.711 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.762 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:18.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.762 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/) 2024-09-21T02:19:18.810 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:18.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950) 2024-09-21T02:19:18.862 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:18.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.863 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/) 2024-09-21T02:19:18.910 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:18.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.911 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.2) 2024-09-21T02:19:18.961 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:18.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:18.962 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.2) 2024-09-21T02:19:19.017 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:19.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.2) 2024-09-21T02:19:19.074 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:19.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.075 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.2) 2024-09-21T02:19:19.122 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:19.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.2) 2024-09-21T02:19:19.178 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:19.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.179 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.2) 2024-09-21T02:19:19.227 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:19.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.228 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.2) 2024-09-21T02:19:19.282 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:19.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.282 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.2) 2024-09-21T02:19:19.330 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:19.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.331 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.5) 2024-09-21T02:19:19.381 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:19.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.382 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.5) 2024-09-21T02:19:19.432 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:19.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.433 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.5) 2024-09-21T02:19:19.487 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:19.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.487 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.5) 2024-09-21T02:19:19.538 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:19.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.539 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.5) 2024-09-21T02:19:19.588 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:19.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.589 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.5) 2024-09-21T02:19:19.642 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:19.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.643 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.5) 2024-09-21T02:19:19.698 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:19.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.699 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.5) 2024-09-21T02:19:19.746 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:19.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.747 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.4) 2024-09-21T02:19:19.799 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:19.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.800 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.4) 2024-09-21T02:19:19.852 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:19.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.853 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.4) 2024-09-21T02:19:19.907 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:19.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.908 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.4) 2024-09-21T02:19:19.959 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:19.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:19.960 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.4) 2024-09-21T02:19:20.019 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:20.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.020 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.4) 2024-09-21T02:19:20.069 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:20.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.070 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.4) 2024-09-21T02:19:20.123 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:20.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.124 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.4) 2024-09-21T02:19:20.174 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:20.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.176 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.7) 2024-09-21T02:19:20.231 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:20.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.232 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.7) 2024-09-21T02:19:20.284 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:20.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.286 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.7) 2024-09-21T02:19:20.337 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:20.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.338 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.7) 2024-09-21T02:19:20.386 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:20.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.387 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.7) 2024-09-21T02:19:20.438 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:20.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.439 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.7) 2024-09-21T02:19:20.486 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:20.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.487 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.7) 2024-09-21T02:19:20.538 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:20.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.539 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.7) 2024-09-21T02:19:20.586 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:20.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.588 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.6) 2024-09-21T02:19:20.641 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:20.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.642 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.6) 2024-09-21T02:19:20.692 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:20.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.693 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.6) 2024-09-21T02:19:20.747 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:20.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.747 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.6) 2024-09-21T02:19:20.798 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:20.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.799 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.6) 2024-09-21T02:19:20.848 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:20.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.849 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.6) 2024-09-21T02:19:20.898 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:20.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.899 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.6) 2024-09-21T02:19:20.950 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:20.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.950 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.6) 2024-09-21T02:19:20.998 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:20.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:20.999 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.0) 2024-09-21T02:19:21.050 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:21.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.051 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.0) 2024-09-21T02:19:21.101 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:21.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.102 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.0) 2024-09-21T02:19:21.155 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:21.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.0) 2024-09-21T02:19:21.206 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:21.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.207 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.0) 2024-09-21T02:19:21.257 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:21.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.258 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.0) 2024-09-21T02:19:21.306 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:21.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.307 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.0) 2024-09-21T02:19:21.358 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:21.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.358 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.0) 2024-09-21T02:19:21.406 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:21.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.407 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.3) 2024-09-21T02:19:21.458 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:21.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.459 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.3) 2024-09-21T02:19:21.509 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:21.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.510 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.3) 2024-09-21T02:19:21.562 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:21.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.563 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.3) 2024-09-21T02:19:21.610 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:21.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.611 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.3) 2024-09-21T02:19:21.662 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:21.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.662 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.3) 2024-09-21T02:19:21.711 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:21.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.3) 2024-09-21T02:19:21.767 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:21.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.768 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.3) 2024-09-21T02:19:21.819 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:21.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.820 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.1) 2024-09-21T02:19:21.875 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:21.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.876 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.1) 2024-09-21T02:19:21.932 INFO:teuthology.orchestra.run.smithi170.stdout:81b4 2024-09-21T02:19:21.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.933 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.1) 2024-09-21T02:19:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:21.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:21.987 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.1) 2024-09-21T02:19:22.035 INFO:teuthology.orchestra.run.smithi170.stdout:1000 2024-09-21T02:19:22.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:22.035 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.1) 2024-09-21T02:19:22.087 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:22.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:22.087 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.1) 2024-09-21T02:19:22.138 INFO:teuthology.orchestra.run.smithi170.stdout:1271 2024-09-21T02:19:22.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:22.139 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8/.snap/subvol_snap95950/subvolume_file.1) 2024-09-21T02:19:22.190 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:22.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:22.191 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31200/3b698a96-9c84-40bc-9d2b-0f715d046330/subvolume_file.1) 2024-09-21T02:19:22.238 INFO:teuthology.orchestra.run.smithi170.stdout:1726885148 2024-09-21T02:19:22.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol29140 fake subvol_clone92841 2024-09-21T02:19:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8f53d4640 1 -- 172.21.15.170:0/1537892063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc003a10 msgr2=0x7ff8dc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/1537892063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc003a10 0x7ff8dc003e80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c2e0 tx=0x7ff8d801cfc0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8ef7fe640 1 -- 172.21.15.170:0/1537892063 reap_dead start 2024-09-21T02:19:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8f53d4640 1 -- 172.21.15.170:0/1537892063 shutdown_connections 2024-09-21T02:19:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8f53d4640 1 -- 172.21.15.170:0/1537892063 >> 172.21.15.170:0/1537892063 conn(0x7ff8f00d0170 msgr2=0x7ff8f00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8f53d4640 1 -- 172.21.15.170:0/1537892063 shutdown_connections 2024-09-21T02:19:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.450+0000 7ff8f53d4640 1 -- 172.21.15.170:0/1537892063 wait complete. 2024-09-21T02:19:22.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 Processor -- start 2024-09-21T02:19:22.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 -- start start 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc003a10 0x7ff8f00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 0x7ff8f01644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8f015bc50 0x7ff8f01649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0167dc0 con 0x7ff8f015b840 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f00b35e0 con 0x7ff8f015bc50 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f0167f40 con 0x7ff8dc003a10 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 0x7ff8f01644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 0x7ff8f01644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57686/0 (socket says 172.21.15.170:57686) 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/3215821064 learned_addr learned my addr 172.21.15.170:0/3215821064 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8eeffd640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc003a10 0x7ff8f00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ef7fe640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8f015bc50 0x7ff8f01649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc003a10 msgr2=0x7ff8f00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc003a10 0x7ff8f00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8f015bc50 msgr2=0x7ff8f01649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8f015bc50 0x7ff8f01649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/3215821064 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00d0790 con 0x7ff8f015b840 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8ee7fc640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 0x7ff8f01644b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e970 tx=0x7ff8e000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000ccc0 con 0x7ff8f015b840 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8e000ce60 con 0x7ff8f015b840 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f01f9380 con 0x7ff8f015b840 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0005260 con 0x7ff8f015b840 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.454+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f01f9940 con 0x7ff8f015b840 2024-09-21T02:19:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.458+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f00b42f0 con 0x7ff8f015b840 2024-09-21T02:19:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.458+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7ff8e0005400 con 0x7ff8f015b840 2024-09-21T02:19:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.458+0000 7ff8e77fe640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8d0073560 0x7ff8d0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.458+0000 7ff8eeffd640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8d0073560 0x7ff8d0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.458+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7ff8e00957c0 con 0x7ff8f015b840 2024-09-21T02:19:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.458+0000 7ff8eeffd640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8d0073560 0x7ff8d0075a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c2e0 tx=0x7ff8d8019e00 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:22.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.462+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff8e0060280 con 0x7ff8f015b840 2024-09-21T02:19:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.658+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol29140", "snap_name": "fake", "target_sub_name": "subvol_clone92841", "target": ["mon-mgr", ""]}) -- 0x7ff8f015fac0 con 0x7ff8d0073560 2024-09-21T02:19:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.854+0000 7ff8e77fe640 1 -- 172.21.15.170:0/3215821064 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8f015fac0 con 0x7ff8d0073560 2024-09-21T02:19:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8d0073560 msgr2=0x7ff8d0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8d0073560 0x7ff8d0075a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c2e0 tx=0x7ff8d8019e00 comp rx=0 tx=0).stop 2024-09-21T02:19:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 msgr2=0x7ff8f01644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 0x7ff8f01644b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e970 tx=0x7ff8e000ee30 comp rx=0 tx=0).stop 2024-09-21T02:19:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 shutdown_connections 2024-09-21T02:19:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8d0073560 0x7ff8d0075a70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8f015bc50 0x7ff8f01649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f015b840 0x7ff8f01644b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 --2- 172.21.15.170:0/3215821064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc003a10 0x7ff8f00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 >> 172.21.15.170:0/3215821064 conn(0x7ff8f00d0170 msgr2=0x7ff8f015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 shutdown_connections 2024-09-21T02:19:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:22.858+0000 7ff8f53d4640 1 -- 172.21.15.170:0/3215821064 wait complete. 2024-09-21T02:19:22.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone92841 2024-09-21T02:19:23.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.046+0000 7fca39436640 1 -- 172.21.15.170:0/16997548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cccb0 msgr2=0x7fca3415fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:23.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.046+0000 7fca39436640 1 --2- 172.21.15.170:0/16997548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cccb0 0x7fca3415fb80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009840 tx=0x7fca1c01c790 comp rx=0 tx=0).stop 2024-09-21T02:19:23.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- 172.21.15.170:0/16997548 shutdown_connections 2024-09-21T02:19:23.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 --2- 172.21.15.170:0/16997548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cccb0 0x7fca3415fb80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:23.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 --2- 172.21.15.170:0/16997548 >> v2:172.21.15.191:3300/0 conn(0x7fca340b04e0 0x7fca340b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- 172.21.15.170:0/16997548 >> 172.21.15.170:0/16997548 conn(0x7fca3409a9c0 msgr2=0x7fca340d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- 172.21.15.170:0/16997548 shutdown_connections 2024-09-21T02:19:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- 172.21.15.170:0/16997548 wait complete. 2024-09-21T02:19:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 Processor -- start 2024-09-21T02:19:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- start start 2024-09-21T02:19:23.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca340b04e0 0x7fca340b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:23.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cc8a0 0x7fca34159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:23.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 0x7fca34159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:23.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fca34169140 con 0x7fca340cccb0 2024-09-21T02:19:23.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca34168fc0 con 0x7fca340b04e0 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca39436640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fca341692c0 con 0x7fca340cc8a0 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 0x7fca34159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 0x7fca34159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57692/0 (socket says 172.21.15.170:57692) 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 -- 172.21.15.170:0/2527614722 learned_addr learned my addr 172.21.15.170:0/2527614722 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca32ffd640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca340b04e0 0x7fca340b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca327fc640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cc8a0 0x7fca34159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 -- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cc8a0 msgr2=0x7fca34159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cc8a0 0x7fca34159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 -- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca340b04e0 msgr2=0x7fca340b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca340b04e0 0x7fca340b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca337fe640 1 -- 172.21.15.170:0/2527614722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341fc660 con 0x7fca340cccb0 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca32ffd640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca340b04e0 0x7fca340b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.050+0000 7fca327fc640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cc8a0 0x7fca34159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca337fe640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 0x7fca34159c90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fca1c01cc70 tx=0x7fca1c0a6f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:23.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ac070 con 0x7fca340cccb0 2024-09-21T02:19:23.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca1c004320 con 0x7fca340cccb0 2024-09-21T02:19:23.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c005240 con 0x7fca340cccb0 2024-09-21T02:19:23.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341fc940 con 0x7fca340cccb0 2024-09-21T02:19:23.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca340ae5a0 con 0x7fca340cccb0 2024-09-21T02:19:23.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca341fc3d0 con 0x7fca340cccb0 2024-09-21T02:19:23.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7fca1c0af070 con 0x7fca340cccb0 2024-09-21T02:19:23.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca2b7fe640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca14073560 0x7fca14075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:23.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fca1c12e970 con 0x7fca340cccb0 2024-09-21T02:19:23.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca32ffd640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca14073560 0x7fca14075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:23.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.054+0000 7fca32ffd640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca14073560 0x7fca14075a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0095b0 tx=0x7fca2c005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:23.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.058+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fca1c0b6070 con 0x7fca340cccb0 2024-09-21T02:19:23.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:23.270+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone92841", "target": ["mon-mgr", ""]}) -- 0x7fca340d19b0 con 0x7fca14073560 2024-09-21T02:19:24.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:19:24.586 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:19:26.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.866+0000 7fca2b7fe640 1 -- 172.21.15.170:0/2527614722 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7fca340d19b0 con 0x7fca14073560 2024-09-21T02:19:26.870 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:26.870 INFO:teuthology.orchestra.run.smithi170.stdout: "status": { 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "pending", 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: "source": { 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: "volume": "cephfs", 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: "subvolume": "subvol29140", 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot": "fake" 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:26.871 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca14073560 msgr2=0x7fca14075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca14073560 0x7fca14075a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0095b0 tx=0x7fca2c005f40 comp rx=0 tx=0).stop 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 msgr2=0x7fca34159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 0x7fca34159c90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fca1c01cc70 tx=0x7fca1c0a6f70 comp rx=0 tx=0).stop 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 shutdown_connections 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca14073560 0x7fca14075a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca340cc8a0 0x7fca34159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:26.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca340cccb0 0x7fca34159c90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:26.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 --2- 172.21.15.170:0/2527614722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca340b04e0 0x7fca340b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:26.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 >> 172.21.15.170:0/2527614722 conn(0x7fca3409a9c0 msgr2=0x7fca340b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:26.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 shutdown_connections 2024-09-21T02:19:26.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:26.870+0000 7fca39436640 1 -- 172.21.15.170:0/2527614722 wait complete. 2024-09-21T02:19:26.897 DEBUG:tasks.cephfs.test_volumes:current clone state = pending 2024-09-21T02:19:27.898 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone92841 2024-09-21T02:19:28.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 -- 172.21.15.170:0/1260049510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d20005360 msgr2=0x7f3d2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:28.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 --2- 172.21.15.170:0/1260049510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d20005360 0x7f3d2000b9d0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ee0 tx=0x7f3d2401d630 comp rx=0 tx=0).stop 2024-09-21T02:19:28.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 -- 172.21.15.170:0/1260049510 shutdown_connections 2024-09-21T02:19:28.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 --2- 172.21.15.170:0/1260049510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d20005360 0x7f3d2000b9d0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:28.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 -- 172.21.15.170:0/1260049510 >> 172.21.15.170:0/1260049510 conn(0x7f3d3409d8b0 msgr2=0x7f3d340d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:28.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 -- 172.21.15.170:0/1260049510 shutdown_connections 2024-09-21T02:19:28.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.066+0000 7f3d39595640 1 -- 172.21.15.170:0/1260049510 wait complete. 2024-09-21T02:19:28.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 Processor -- start 2024-09-21T02:19:28.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 -- start start 2024-09-21T02:19:28.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d20005360 0x7f3d3409e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:28.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d3406f170 0x7f3d3406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:28.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 0x7f3d341eb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:28.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340cf230 con 0x7f3d341eaf60 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340b3850 con 0x7f3d20005360 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3d340cf3b0 con 0x7f3d3406f170 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 0x7f3d341eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 0x7f3d341eb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37368/0 (socket says 172.21.15.170:37368) 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 -- 172.21.15.170:0/1494662555 learned_addr learned my addr 172.21.15.170:0/1494662555 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d327fc640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d3406f170 0x7f3d3406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d32ffd640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d20005360 0x7f3d3409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 -- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d3406f170 msgr2=0x7f3d3406f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d3406f170 0x7f3d3406f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 -- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d20005360 msgr2=0x7f3d3409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d20005360 0x7f3d3409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 -- 172.21.15.170:0/1494662555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d341d2af0 con 0x7f3d341eaf60 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d32ffd640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d20005360 0x7f3d3409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:28.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d337fe640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 0x7f3d341eb370 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009e00 tx=0x7f3d24004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24009ab0 con 0x7f3d341eaf60 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d240b1070 con 0x7f3d341eaf60 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24004870 con 0x7f3d341eaf60 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341f9090 con 0x7f3d341eaf60 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.070+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341d2f50 con 0x7f3d341eaf60 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.074+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d341d2cc0 con 0x7f3d341eaf60 2024-09-21T02:19:28.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.074+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f3d24004cb0 con 0x7f3d341eaf60 2024-09-21T02:19:28.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.074+0000 7f3d2b7fe640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d14073560 0x7f3d14075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:28.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.074+0000 7f3d32ffd640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d14073560 0x7f3d14075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:28.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.074+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3d2412ee80 con 0x7f3d341eaf60 2024-09-21T02:19:28.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.074+0000 7f3d32ffd640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d14073560 0x7f3d14075a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d340a2380 tx=0x7f3d2c006bd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:28.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.078+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d24017340 con 0x7f3d341eaf60 2024-09-21T02:19:28.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:28.294+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone92841", "target": ["mon-mgr", ""]}) -- 0x7f3d3409e760 con 0x7f3d14073560 2024-09-21T02:19:31.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.838+0000 7f3d2b7fe640 1 -- 172.21.15.170:0/1494662555 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f3d3409e760 con 0x7f3d14073560 2024-09-21T02:19:31.840 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:31.840 INFO:teuthology.orchestra.run.smithi170.stdout: "status": { 2024-09-21T02:19:31.840 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "complete" 2024-09-21T02:19:31.840 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:31.840 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d14073560 msgr2=0x7f3d14075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d14073560 0x7f3d14075a70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d340a2380 tx=0x7f3d2c006bd0 comp rx=0 tx=0).stop 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 msgr2=0x7f3d341eb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 0x7f3d341eb370 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009e00 tx=0x7f3d24004680 comp rx=0 tx=0).stop 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 shutdown_connections 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d14073560 0x7f3d14075a70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d3406f170 0x7f3d3406f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d341eaf60 0x7f3d341eb370 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 --2- 172.21.15.170:0/1494662555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d20005360 0x7f3d3409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 >> 172.21.15.170:0/1494662555 conn(0x7f3d3409d8b0 msgr2=0x7f3d340b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:31.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 shutdown_connections 2024-09-21T02:19:31.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:31.846+0000 7f3d39595640 1 -- 172.21.15.170:0/1494662555 wait complete. 2024-09-21T02:19:31.872 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-21T02:19:31.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:31.872 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone92841/.meta) 2024-09-21T02:19:31.891 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:19:31.892 INFO:teuthology.orchestra.run.smithi170.stdout:version = 2 2024-09-21T02:19:31.892 INFO:teuthology.orchestra.run.smithi170.stdout:type = clone 2024-09-21T02:19:31.892 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol_clone92841/8fd73a2e-669d-4528-9ad0-993343a79229 2024-09-21T02:19:31.892 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:19:31.892 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:31.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol29140 subvol_snap95950 2024-09-21T02:19:32.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 -- 172.21.15.170:0/258798836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113809dc00 msgr2=0x7f113815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 --2- 172.21.15.170:0/258798836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113809dc00 0x7f113815ad30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f11380ce1d0 tx=0x7f113401d1b0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 -- 172.21.15.170:0/258798836 shutdown_connections 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 --2- 172.21.15.170:0/258798836 >> v2:172.21.15.191:3300/0 conn(0x7f11380ca720 0x7f11380cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 --2- 172.21.15.170:0/258798836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113809dc00 0x7f113815ad30 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 -- 172.21.15.170:0/258798836 >> 172.21.15.170:0/258798836 conn(0x7f11380b1c40 msgr2=0x7f11380b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 -- 172.21.15.170:0/258798836 shutdown_connections 2024-09-21T02:19:32.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.110+0000 7f113f611640 1 -- 172.21.15.170:0/258798836 wait complete. 2024-09-21T02:19:32.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 Processor -- start 2024-09-21T02:19:32.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 -- start start 2024-09-21T02:19:32.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f113809dc00 0x7f113815d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11380ca720 0x7f1138157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 0x7f11381f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1138168490 con 0x7f113815e9f0 2024-09-21T02:19:32.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1138168310 con 0x7f113809dc00 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1138168610 con 0x7f11380ca720 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 0x7f11381f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 0x7f11381f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37380/0 (socket says 172.21.15.170:37380) 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 -- 172.21.15.170:0/1513120471 learned_addr learned my addr 172.21.15.170:0/1513120471 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113d3ad640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f113809dc00 0x7f113815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113cbac640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11380ca720 0x7f1138157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 -- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11380ca720 msgr2=0x7f1138157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11380ca720 0x7f1138157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 -- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f113809dc00 msgr2=0x7f113815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f113809dc00 0x7f113815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 -- 172.21.15.170:0/1513120471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11381f9630 con 0x7f113815e9f0 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113dbae640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 0x7f11381f90f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f113000cb10 tx=0x7f113000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:32.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130016020 con 0x7f113815e9f0 2024-09-21T02:19:32.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f113000ce50 con 0x7f113815e9f0 2024-09-21T02:19:32.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113815a200 con 0x7f113815e9f0 2024-09-21T02:19:32.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113000ade0 con 0x7f113815e9f0 2024-09-21T02:19:32.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.114+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113815a7c0 con 0x7f113815e9f0 2024-09-21T02:19:32.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.118+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113815ad30 con 0x7f113815e9f0 2024-09-21T02:19:32.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.118+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f1130019020 con 0x7f113815e9f0 2024-09-21T02:19:32.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.118+0000 7f112dffb640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f111c073590 0x7f111c075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.118+0000 7f113d3ad640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f111c073590 0x7f111c075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.118+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f1130094b00 con 0x7f113815e9f0 2024-09-21T02:19:32.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.118+0000 7f113d3ad640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f111c073590 0x7f111c075aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11340ae000 tx=0x7f113401af90 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:32.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.122+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1130020020 con 0x7f113815e9f0 2024-09-21T02:19:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.326+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol29140", "snap_name": "subvol_snap95950", "target": ["mon-mgr", ""]}) -- 0x7f113815a9c0 con 0x7f111c073590 2024-09-21T02:19:32.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.358+0000 7f112dffb640 1 -- 172.21.15.170:0/1513120471 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7f113815a9c0 con 0x7f111c073590 2024-09-21T02:19:32.360 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:32.360 INFO:teuthology.orchestra.run.smithi170.stdout: "created_at": "2024-09-21 02:19:09.453366", 2024-09-21T02:19:32.360 INFO:teuthology.orchestra.run.smithi170.stdout: "data_pool": "cephfs_data", 2024-09-21T02:19:32.360 INFO:teuthology.orchestra.run.smithi170.stdout: "has_pending_clones": "no" 2024-09-21T02:19:32.360 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:32.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f111c073590 msgr2=0x7f111c075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f111c073590 0x7f111c075aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11340ae000 tx=0x7f113401af90 comp rx=0 tx=0).stop 2024-09-21T02:19:32.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 msgr2=0x7f11381f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 0x7f11381f90f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f113000cb10 tx=0x7f113000cc00 comp rx=0 tx=0).stop 2024-09-21T02:19:32.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 shutdown_connections 2024-09-21T02:19:32.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f111c073590 0x7f111c075aa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11380ca720 0x7f1138157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113815e9f0 0x7f11381f90f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 --2- 172.21.15.170:0/1513120471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f113809dc00 0x7f113815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 >> 172.21.15.170:0/1513120471 conn(0x7f11380b1c40 msgr2=0x7f11380b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:32.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 shutdown_connections 2024-09-21T02:19:32.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.362+0000 7f113f611640 1 -- 172.21.15.170:0/1513120471 wait complete. 2024-09-21T02:19:32.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol29140 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.526+0000 7fb52e4c9640 1 -- 172.21.15.170:0/1643383004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 msgr2=0x7fb5280b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.526+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/1643383004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb5280b1160 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb528073d10 tx=0x7fb51c01c830 comp rx=0 tx=0).stop 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- 172.21.15.170:0/1643383004 shutdown_connections 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/1643383004 >> v2:172.21.15.191:3300/0 conn(0x7fb5280cbcd0 0x7fb5280cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/1643383004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb5280b1160 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/1643383004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5280b3460 0x7fb5280ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- 172.21.15.170:0/1643383004 >> 172.21.15.170:0/1643383004 conn(0x7fb52809a9c0 msgr2=0x7fb5280b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:32.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- 172.21.15.170:0/1643383004 shutdown_connections 2024-09-21T02:19:32.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- 172.21.15.170:0/1643383004 wait complete. 2024-09-21T02:19:32.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 Processor -- start 2024-09-21T02:19:32.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- start start 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb52815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5280b3460 0x7fb52806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5280cbcd0 0x7fb52806d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb528167ee0 con 0x7fb5280b0d50 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280cfed0 con 0x7fb5280cbcd0 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52e4c9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb528168060 con 0x7fb5280b3460 2024-09-21T02:19:32.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb52815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb52815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37392/0 (socket says 172.21.15.170:37392) 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 -- 172.21.15.170:0/3685191335 learned_addr learned my addr 172.21.15.170:0/3685191335 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb5277fe640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5280b3460 0x7fb52806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52ca66640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5280cbcd0 0x7fb52806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 -- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5280b3460 msgr2=0x7fb52806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5280b3460 0x7fb52806ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 -- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5280cbcd0 msgr2=0x7fb52806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5280cbcd0 0x7fb52806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 -- 172.21.15.170:0/3685191335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb528146f90 con 0x7fb5280b0d50 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb52ca66640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5280cbcd0 0x7fb52806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb527fff640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb52815d6c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb52000ea60 tx=0x7fb52000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:32.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.530+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000ce40 con 0x7fb5280b0d50 2024-09-21T02:19:32.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb520003a70 con 0x7fb5280b0d50 2024-09-21T02:19:32.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520003e40 con 0x7fb5280b0d50 2024-09-21T02:19:32.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281440b0 con 0x7fb5280b0d50 2024-09-21T02:19:32.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb528144670 con 0x7fb5280b0d50 2024-09-21T02:19:32.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb528074e40 con 0x7fb5280b0d50 2024-09-21T02:19:32.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7fb520003fe0 con 0x7fb5280b0d50 2024-09-21T02:19:32.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb524ff9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb514073560 0x7fb514075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fb520095d70 con 0x7fb5280b0d50 2024-09-21T02:19:32.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb5277fe640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb514073560 0x7fb514075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.534+0000 7fb5277fe640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb514073560 0x7fb514075a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0af000 tx=0x7fb51c01b070 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.538+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb520026020 con 0x7fb5280b0d50 2024-09-21T02:19:32.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.742+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7fb5280ce610 con 0x7fb514073560 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb524ff9640 1 -- 172.21.15.170:0/3685191335 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7fb5280ce610 con 0x7fb514073560 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "fake" 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "subvol_snap95950" 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:32.773 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb514073560 msgr2=0x7fb514075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb514073560 0x7fb514075a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0af000 tx=0x7fb51c01b070 comp rx=0 tx=0).stop 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 msgr2=0x7fb52815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb52815d6c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb52000ea60 tx=0x7fb52000ef20 comp rx=0 tx=0).stop 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 shutdown_connections 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb514073560 0x7fb514075a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5280b3460 0x7fb52806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5280cbcd0 0x7fb52806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 --2- 172.21.15.170:0/3685191335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5280b0d50 0x7fb52815d6c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.774+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 >> 172.21.15.170:0/3685191335 conn(0x7fb52809a9c0 msgr2=0x7fb528160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:32.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.778+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 shutdown_connections 2024-09-21T02:19:32.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.778+0000 7fb52e4c9640 1 -- 172.21.15.170:0/3685191335 wait complete. 2024-09-21T02:19:32.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol29140 subvol_snap95950 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 -- 172.21.15.170:0/4065604969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c15d930 msgr2=0x7f3e2c0ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 --2- 172.21.15.170:0/4065604969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c15d930 0x7f3e2c0ccbf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009bc0 tx=0x7f3e2401cba0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 -- 172.21.15.170:0/4065604969 shutdown_connections 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 --2- 172.21.15.170:0/4065604969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c15d930 0x7f3e2c0ccbf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 --2- 172.21.15.170:0/4065604969 >> v2:172.21.15.191:3300/0 conn(0x7f3e2c06d810 0x7f3e2c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 -- 172.21.15.170:0/4065604969 >> 172.21.15.170:0/4065604969 conn(0x7f3e2c09a9c0 msgr2=0x7f3e2c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:32.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 -- 172.21.15.170:0/4065604969 shutdown_connections 2024-09-21T02:19:32.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.954+0000 7f3e312b4640 1 -- 172.21.15.170:0/4065604969 wait complete. 2024-09-21T02:19:32.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 Processor -- start 2024-09-21T02:19:32.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 -- start start 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e2c06d810 0x7f3e2c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c0d1c30 0x7f3e2c158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 0x7f3e2c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c15caa0 con 0x7f3e2c15d930 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c15c920 con 0x7f3e2c06d810 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2c15cc20 con 0x7f3e2c0d1c30 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 0x7f3e2c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 0x7f3e2c15f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37400/0 (socket says 172.21.15.170:37400) 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 -- 172.21.15.170:0/29951441 learned_addr learned my addr 172.21.15.170:0/29951441 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:32.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2a7fc640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c0d1c30 0x7f3e2c158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2affd640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e2c06d810 0x7f3e2c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 -- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c0d1c30 msgr2=0x7f3e2c158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c0d1c30 0x7f3e2c158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 -- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e2c06d810 msgr2=0x7f3e2c0c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e2c06d810 0x7f3e2c0c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 -- 172.21.15.170:0/29951441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c0c7b30 con 0x7f3e2c15d930 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e2b7fe640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 0x7f3e2c15f320 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3e2401a7d0 tx=0x7f3e24009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240bb070 con 0x7f3e2c15d930 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e24004d40 con 0x7f3e2c15d930 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c1fc100 con 0x7f3e2c15d930 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240b2040 con 0x7f3e2c15d930 2024-09-21T02:19:32.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.958+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c1fc6c0 con 0x7f3e2c15d930 2024-09-21T02:19:32.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.962+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2c15d3f0 con 0x7f3e2c15d930 2024-09-21T02:19:32.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.962+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f3e240be070 con 0x7f3e2c15d930 2024-09-21T02:19:32.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.962+0000 7f3e137fe640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3e18073560 0x7f3e18075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:32.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.962+0000 7f3e2affd640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3e18073560 0x7f3e18075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:32.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.962+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3e2412e620 con 0x7f3e2c15d930 2024-09-21T02:19:32.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.962+0000 7f3e2affd640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3e18073560 0x7f3e18075a70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c007fd0 tx=0x7f3e1c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:32.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:32.966+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3e24016790 con 0x7f3e2c15d930 2024-09-21T02:19:33.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.170+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol29140", "snap_name": "subvol_snap95950", "target": ["mon-mgr", ""]}) -- 0x7f3e2c0c7720 con 0x7f3e18073560 2024-09-21T02:19:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.242+0000 7f3e137fe640 1 -- 172.21.15.170:0/29951441 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e1c001240 con 0x7f3e18073560 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3e18073560 msgr2=0x7f3e18075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3e18073560 0x7f3e18075a70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c007fd0 tx=0x7f3e1c009040 comp rx=0 tx=0).stop 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 msgr2=0x7f3e2c15f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 0x7f3e2c15f320 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3e2401a7d0 tx=0x7f3e24009280 comp rx=0 tx=0).stop 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 shutdown_connections 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3e18073560 0x7f3e18075a70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e2c0d1c30 0x7f3e2c158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e2c15d930 0x7f3e2c15f320 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 --2- 172.21.15.170:0/29951441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e2c06d810 0x7f3e2c0c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 >> 172.21.15.170:0/29951441 conn(0x7f3e2c09a9c0 msgr2=0x7f3e2c0cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 shutdown_connections 2024-09-21T02:19:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.246+0000 7f3e312b4640 1 -- 172.21.15.170:0/29951441 wait complete. 2024-09-21T02:19:33.264 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol29140 fake 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 -- 172.21.15.170:0/2628241757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b52d0 msgr2=0x7f19e00af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 --2- 172.21.15.170:0/2628241757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b52d0 0x7f19e00af440 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f19e0072900 tx=0x7f19d801c810 comp rx=0 tx=0).stop 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 -- 172.21.15.170:0/2628241757 shutdown_connections 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 --2- 172.21.15.170:0/2628241757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e00cd1f0 0x7f19e006ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 --2- 172.21.15.170:0/2628241757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b52d0 0x7f19e00af440 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 -- 172.21.15.170:0/2628241757 >> 172.21.15.170:0/2628241757 conn(0x7f19e00b1c40 msgr2=0x7f19e00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 -- 172.21.15.170:0/2628241757 shutdown_connections 2024-09-21T02:19:33.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 -- 172.21.15.170:0/2628241757 wait complete. 2024-09-21T02:19:33.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 Processor -- start 2024-09-21T02:19:33.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.430+0000 7f19e592c640 1 -- start start 2024-09-21T02:19:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 0x7f19e009d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e00b52d0 0x7f19e014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e0151550 0x7f19e009b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19e016a000 con 0x7f19e00b4d80 2024-09-21T02:19:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0169e80 con 0x7f19e00b52d0 2024-09-21T02:19:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f19e016a180 con 0x7f19e0151550 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 0x7f19e009d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 0x7f19e009d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37416/0 (socket says 172.21.15.170:37416) 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 -- 172.21.15.170:0/2776195323 learned_addr learned my addr 172.21.15.170:0/2776195323 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19df7fe640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e0151550 0x7f19e009b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 -- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e0151550 msgr2=0x7f19e009b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e0151550 0x7f19e009b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 -- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e00b52d0 msgr2=0x7f19e014f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19de7fc640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e00b52d0 0x7f19e014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e00b52d0 0x7f19e014f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 -- 172.21.15.170:0/2776195323 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e0153d50 con 0x7f19e00b4d80 2024-09-21T02:19:33.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19df7fe640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e0151550 0x7f19e009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19deffd640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 0x7f19e009d3e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000c10 tx=0x7f19d8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80b2040 con 0x7f19e00b4d80 2024-09-21T02:19:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d8004130 con 0x7f19e00b4d80 2024-09-21T02:19:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e0150c50 con 0x7f19e00b4d80 2024-09-21T02:19:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8005020 con 0x7f19e00b4d80 2024-09-21T02:19:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e0151210 con 0x7f19e00b4d80 2024-09-21T02:19:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e00af440 con 0x7f19e00b4d80 2024-09-21T02:19:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f19d80bd020 con 0x7f19e00b4d80 2024-09-21T02:19:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19d7fff640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f19c0073560 0x7f19c0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19de7fc640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f19c0073560 0x7f19c0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f19d812e590 con 0x7f19e00b4d80 2024-09-21T02:19:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.434+0000 7f19de7fc640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f19c0073560 0x7f19c0075a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19e015a6a0 tx=0x7f19c8008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.438+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f19d8016490 con 0x7f19e00b4d80 2024-09-21T02:19:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.642+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol29140", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f19e01509d0 con 0x7f19c0073560 2024-09-21T02:19:33.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.706+0000 7f19d7fff640 1 -- 172.21.15.170:0/2776195323 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19c8004020 con 0x7f19c0073560 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f19c0073560 msgr2=0x7f19c0075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f19c0073560 0x7f19c0075a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f19e015a6a0 tx=0x7f19c8008040 comp rx=0 tx=0).stop 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 msgr2=0x7f19e009d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 0x7f19e009d3e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000c10 tx=0x7f19d8005e10 comp rx=0 tx=0).stop 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 shutdown_connections 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f19c0073560 0x7f19c0075a70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19e0151550 0x7f19e009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19e00b52d0 0x7f19e014f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 --2- 172.21.15.170:0/2776195323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e00b4d80 0x7f19e009d3e0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 >> 172.21.15.170:0/2776195323 conn(0x7f19e00b1c40 msgr2=0x7f19e0152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:33.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 shutdown_connections 2024-09-21T02:19:33.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:33.710+0000 7f19e592c640 1 -- 172.21.15.170:0/2776195323 wait complete. 2024-09-21T02:19:33.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:33.732 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol29140/.meta) 2024-09-21T02:19:36.833 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:19:36.833 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:19:36.833 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:19:36.833 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol29140/97c45822-3a0d-4c96-8c25-bb43648695b8 2024-09-21T02:19:36.833 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:19:36.833 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:36.835 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol29140 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/2403191025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 msgr2=0x7f0fb40b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/2403191025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb40b0b60 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40cfd30 tx=0x7f0fa801c770 comp rx=0 tx=0).stop 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/2403191025 shutdown_connections 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/2403191025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb40b0b60 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/2403191025 >> 172.21.15.170:0/2403191025 conn(0x7f0fb409a9c0 msgr2=0x7f0fb40d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/2403191025 shutdown_connections 2024-09-21T02:19:37.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/2403191025 wait complete. 2024-09-21T02:19:37.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 Processor -- start 2024-09-21T02:19:37.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.006+0000 7f0fb9e1a640 1 -- start start 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fb40cc8a0 0x7f0fb415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fb40cccb0 0x7f0fb40b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb416a150 con 0x7f0fb40b04e0 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4169fd0 con 0x7f0fb40cccb0 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb416a2d0 con 0x7f0fb40cc8a0 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40250/0 (socket says 172.21.15.170:40250) 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 -- 172.21.15.170:0/1642111136 learned_addr learned my addr 172.21.15.170:0/1642111136 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb3fff640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fb40cccb0 0x7f0fb40b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 -- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fb40cc8a0 msgr2=0x7f0fb415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fb40cc8a0 0x7f0fb415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 -- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fb40cccb0 msgr2=0x7f0fb40b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fb40cccb0 0x7f0fb40b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 -- 172.21.15.170:0/1642111136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb415b2a0 con 0x7f0fb40b04e0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb37fe640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb415a660 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0fa801a3a0 tx=0x7f0fa8003600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80be070 con 0x7f0fb40b04e0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fa8003ba0 con 0x7f0fb40b04e0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb415b580 con 0x7f0fb40b04e0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80b9480 con 0x7f0fb40b04e0 2024-09-21T02:19:37.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb415d890 con 0x7f0fb40b04e0 2024-09-21T02:19:37.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.010+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb40b0b60 con 0x7f0fb40b04e0 2024-09-21T02:19:37.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.014+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f0fa80a7420 con 0x7f0fb40b04e0 2024-09-21T02:19:37.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.014+0000 7f0f9bfff640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0fa0073590 0x7f0fa0075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.014+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f0fa812ed70 con 0x7f0fb40b04e0 2024-09-21T02:19:37.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.014+0000 7f0fb2ffd640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0fa0073590 0x7f0fa0075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.014+0000 7f0fb2ffd640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0fa0073590 0x7f0fa0075aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4155270 tx=0x7f0fa4007a40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:37.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.018+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0fa8133070 con 0x7f0fb40b04e0 2024-09-21T02:19:37.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.218+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol29140", "target": ["mon-mgr", ""]}) -- 0x7f0fb40d19b0 con 0x7f0fa0073590 2024-09-21T02:19:37.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.334+0000 7f0f9bfff640 1 -- 172.21.15.170:0/1642111136 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0fb40d19b0 con 0x7f0fa0073590 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0fa0073590 msgr2=0x7f0fa0075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0fa0073590 0x7f0fa0075aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4155270 tx=0x7f0fa4007a40 comp rx=0 tx=0).stop 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 msgr2=0x7f0fb415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb415a660 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0fa801a3a0 tx=0x7f0fa8003600 comp rx=0 tx=0).stop 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 shutdown_connections 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0fa0073590 0x7f0fa0075aa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0fb40cc8a0 0x7f0fb415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fb40cccb0 0x7f0fb40b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 --2- 172.21.15.170:0/1642111136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fb40b04e0 0x7f0fb415a660 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 >> 172.21.15.170:0/1642111136 conn(0x7f0fb409a9c0 msgr2=0x7f0fb4071510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 shutdown_connections 2024-09-21T02:19:37.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.338+0000 7f0fb9e1a640 1 -- 172.21.15.170:0/1642111136 wait complete. 2024-09-21T02:19:37.342 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.342+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.342 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.342+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.342 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.342+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.342 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.342+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.342 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.342+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.362 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone31200 2024-09-21T02:19:37.367 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.366+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.368 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.366+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.368 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.366+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.368 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.366+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.368 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:19:37.366+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.518+0000 7f48a2bb3640 1 -- 172.21.15.170:0/35031396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c15d920 msgr2=0x7f489c071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.518+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/35031396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c15d920 0x7f489c071110 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4894009cb0 tx=0x7f489401cd10 comp rx=0 tx=0).stop 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.518+0000 7f48a2bb3640 1 -- 172.21.15.170:0/35031396 shutdown_connections 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.518+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/35031396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c15d920 0x7f489c071110 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.518+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/35031396 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489c160af0 0x7f489c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.518+0000 7f48a2bb3640 1 -- 172.21.15.170:0/35031396 >> 172.21.15.170:0/35031396 conn(0x7f489c0d2a30 msgr2=0x7f489c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 -- 172.21.15.170:0/35031396 shutdown_connections 2024-09-21T02:19:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 -- 172.21.15.170:0/35031396 wait complete. 2024-09-21T02:19:37.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 Processor -- start 2024-09-21T02:19:37.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 -- start start 2024-09-21T02:19:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489c15d920 0x7f489c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f489c1606e0 0x7f489c0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 0x7f489c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f489c16a040 con 0x7f489c160af0 2024-09-21T02:19:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f489c169ec0 con 0x7f489c1606e0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a2bb3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f489c16a1c0 con 0x7f489c15d920 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 0x7f489c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 0x7f489c0c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40312/0 (socket says 172.21.15.170:40312) 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 -- 172.21.15.170:0/3758764595 learned_addr learned my addr 172.21.15.170:0/3758764595 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a094f640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489c15d920 0x7f489c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f489bfff640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f489c1606e0 0x7f489c0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 -- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489c15d920 msgr2=0x7f489c14d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489c15d920 0x7f489c14d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 -- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f489c1606e0 msgr2=0x7f489c0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f489c1606e0 0x7f489c0c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.522+0000 7f48a1150640 1 -- 172.21.15.170:0/3758764595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c1508b0 con 0x7f489c160af0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48a1150640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 0x7f489c0c6ef0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4894098700 tx=0x7f48940092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940bc070 con 0x7f489c160af0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48940042f0 con 0x7f489c160af0 2024-09-21T02:19:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c150b90 con 0x7f489c160af0 2024-09-21T02:19:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940b2040 con 0x7f489c160af0 2024-09-21T02:19:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c0b5200 con 0x7f489c160af0 2024-09-21T02:19:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c14e280 con 0x7f489c160af0 2024-09-21T02:19:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f48940bf070 con 0x7f489c160af0 2024-09-21T02:19:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48997fa640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4888073560 0x7f4888075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f489412e830 con 0x7f489c160af0 2024-09-21T02:19:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48a094f640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4888073560 0x7f4888075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.526+0000 7f48a094f640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4888073560 0x7f4888075a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f488c001650 tx=0x7f488c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.530+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4894016990 con 0x7f489c160af0 2024-09-21T02:19:37.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.734+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone31200", "target": ["mon-mgr", ""]}) -- 0x7f489c0b5460 con 0x7f4888073560 2024-09-21T02:19:37.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.770+0000 7f48997fa640 1 -- 172.21.15.170:0/3758764595 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f489c0b5460 con 0x7f4888073560 2024-09-21T02:19:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4888073560 msgr2=0x7f4888075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4888073560 0x7f4888075a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f488c001650 tx=0x7f488c009040 comp rx=0 tx=0).stop 2024-09-21T02:19:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 msgr2=0x7f489c0c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 0x7f489c0c6ef0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4894098700 tx=0x7f48940092c0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 shutdown_connections 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4888073560 0x7f4888075a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f489c160af0 0x7f489c0c6ef0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f489c1606e0 0x7f489c0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 --2- 172.21.15.170:0/3758764595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f489c15d920 0x7f489c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 >> 172.21.15.170:0/3758764595 conn(0x7f489c0d2a30 msgr2=0x7f489c0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 shutdown_connections 2024-09-21T02:19:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:37.774+0000 7f48a2bb3640 1 -- 172.21.15.170:0/3758764595 wait complete. 2024-09-21T02:19:37.795 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone92841 2024-09-21T02:19:38.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 -- 172.21.15.170:0/3772301541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 msgr2=0x7f8dbc0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/3772301541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc0ce070 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0ca750 tx=0x7f8db001db60 comp rx=0 tx=0).stop 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 -- 172.21.15.170:0/3772301541 shutdown_connections 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/3772301541 >> v2:172.21.15.191:3300/0 conn(0x7f8dbc0cb790 0x7f8dbc0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/3772301541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc0ce070 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 -- 172.21.15.170:0/3772301541 >> 172.21.15.170:0/3772301541 conn(0x7f8dbc0d0170 msgr2=0x7f8dbc0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 -- 172.21.15.170:0/3772301541 shutdown_connections 2024-09-21T02:19:38.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.006+0000 7f8dc24f3640 1 -- 172.21.15.170:0/3772301541 wait complete. 2024-09-21T02:19:38.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 Processor -- start 2024-09-21T02:19:38.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 -- start start 2024-09-21T02:19:38.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dbc06f9a0 0x7f8dbc0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dbc0cb790 0x7f8dbc1612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0746a0 con 0x7f8dbc0cb380 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc074520 con 0x7f8dbc06f9a0 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc24f3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8dbc16a6e0 con 0x7f8dbc0cb790 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40334/0 (socket says 172.21.15.170:40334) 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 -- 172.21.15.170:0/184548503 learned_addr learned my addr 172.21.15.170:0/184548503 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dc0a90640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dbc0cb790 0x7f8dbc1612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbbfff640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dbc06f9a0 0x7f8dbc0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 -- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dbc0cb790 msgr2=0x7f8dbc1612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dbc0cb790 0x7f8dbc1612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 -- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dbc06f9a0 msgr2=0x7f8dbc0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dbc06f9a0 0x7f8dbc0c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 -- 172.21.15.170:0/184548503 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc09d1a0 con 0x7f8dbc0cb380 2024-09-21T02:19:38.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.010+0000 7f8dbb7fe640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc15ffd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00e7c0 tx=0x7f8dac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:38.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00cc60 con 0x7f8dbc0cb380 2024-09-21T02:19:38.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dac00ce00 con 0x7f8dbc0cb380 2024-09-21T02:19:38.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc09d480 con 0x7f8dbc0cb380 2024-09-21T02:19:38.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac005280 con 0x7f8dbc0cb380 2024-09-21T02:19:38.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc16a2c0 con 0x7f8dbc0cb380 2024-09-21T02:19:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc09cf10 con 0x7f8dbc0cb380 2024-09-21T02:19:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f8dac005420 con 0x7f8dbc0cb380 2024-09-21T02:19:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8db8ff9640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8da8073590 0x7f8da8075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f8dac0953d0 con 0x7f8dbc0cb380 2024-09-21T02:19:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8dbbfff640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8da8073590 0x7f8da8075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.014+0000 7f8dbbfff640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8da8073590 0x7f8da8075aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b0000 tx=0x7f8db001c3a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.018+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8dac05fd50 con 0x7f8dbc0cb380 2024-09-21T02:19:38.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.210+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone92841", "target": ["mon-mgr", ""]}) -- 0x7f8dbc06fdb0 con 0x7f8da8073590 2024-09-21T02:19:38.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.262+0000 7f8db8ff9640 1 -- 172.21.15.170:0/184548503 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8dbc06fdb0 con 0x7f8da8073590 2024-09-21T02:19:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8da8073590 msgr2=0x7f8da8075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8da8073590 0x7f8da8075aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b0000 tx=0x7f8db001c3a0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 msgr2=0x7f8dbc15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc15ffd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00e7c0 tx=0x7f8dac00ec80 comp rx=0 tx=0).stop 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 shutdown_connections 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8da8073590 0x7f8da8075aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dbc0cb790 0x7f8dbc1612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dbc0cb380 0x7f8dbc15ffd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 --2- 172.21.15.170:0/184548503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dbc06f9a0 0x7f8dbc0c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 >> 172.21.15.170:0/184548503 conn(0x7f8dbc0d0170 msgr2=0x7f8dbc0aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 shutdown_connections 2024-09-21T02:19:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.266+0000 7f8dc24f3640 1 -- 172.21.15.170:0/184548503 wait complete. 2024-09-21T02:19:38.286 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:19:38.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:19:38.341 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:19:38.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:19:38.348 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:19:38.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:19:38.473 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:19:38.532 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:19:38.532 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:19:38.532 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:38.532 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:38.538 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:38.538 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:38.584 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 -- 172.21.15.170:0/1297917217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 msgr2=0x7f8bb0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/1297917217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb0070340 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0070cc0 tx=0x7f8ba001c800 comp rx=0 tx=0).stop 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 -- 172.21.15.170:0/1297917217 shutdown_connections 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/1297917217 >> v2:172.21.15.191:3300/0 conn(0x7f8bb00cb460 0x7f8bb00cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/1297917217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb0070340 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 -- 172.21.15.170:0/1297917217 >> 172.21.15.170:0/1297917217 conn(0x7f8bb00b1c40 msgr2=0x7f8bb00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 -- 172.21.15.170:0/1297917217 shutdown_connections 2024-09-21T02:19:38.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.786+0000 7f8bb48b5640 1 -- 172.21.15.170:0/1297917217 wait complete. 2024-09-21T02:19:38.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 Processor -- start 2024-09-21T02:19:38.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 -- start start 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb00cb460 0x7f8bb0165a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb00cc8c0 0x7f8bb01fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb01fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0169120 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0168fa0 con 0x7f8bb00cb460 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bb48b5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb01692a0 con 0x7f8bb00cc8c0 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb01fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb01fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40350/0 (socket says 172.21.15.170:40350) 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 -- 172.21.15.170:0/2433781795 learned_addr learned my addr 172.21.15.170:0/2433781795 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8bae59c640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb00cb460 0x7f8bb0165a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8badd9b640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb00cc8c0 0x7f8bb01fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 -- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb00cc8c0 msgr2=0x7f8bb01fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb00cc8c0 0x7f8bb01fc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 -- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb00cb460 msgr2=0x7f8bb0165a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb00cb460 0x7f8bb0165a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 -- 172.21.15.170:0/2433781795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb009bbb0 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.790+0000 7f8baed9d640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb01fcb20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c730 tx=0x7f8ba800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8003b80 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ba8003d40 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb009be90 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80052d0 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01fa350 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb009c3b0 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f8ba8005680 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8b9effd640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b90073560 0x7f8b90075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:38.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f8ba8095480 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8bae59c640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b90073560 0x7f8b90075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:38.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.794+0000 7f8bae59c640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b90073560 0x7f8b90075a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00af000 tx=0x7f8ba001b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:38.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.798+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8bb009c3b0 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.982+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8bb0070340 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.986+0000 7f8b9effd640 1 -- 172.21.15.170:0/2433781795 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f8ba805ff00 con 0x7f8bb00cf7e0 2024-09-21T02:19:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:19:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b90073560 msgr2=0x7f8b90075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b90073560 0x7f8b90075a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00af000 tx=0x7f8ba001b040 comp rx=0 tx=0).stop 2024-09-21T02:19:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 msgr2=0x7f8bb01fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb01fcb20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c730 tx=0x7f8ba800cbf0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 shutdown_connections 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8b90073560 0x7f8b90075a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb00cc8c0 0x7f8bb01fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bb00cf7e0 0x7f8bb01fcb20 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 --2- 172.21.15.170:0/2433781795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb00cb460 0x7f8bb0165a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 >> 172.21.15.170:0/2433781795 conn(0x7f8bb00b1c40 msgr2=0x7f8bb00cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 shutdown_connections 2024-09-21T02:19:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:38.990+0000 7f8bb48b5640 1 -- 172.21.15.170:0/2433781795 wait complete. 2024-09-21T02:19:39.008 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:19:39.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:39.009 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:19:39.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[44905]: fuse finished with error 0 and tester_r 0 2024-09-21T02:19:39.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:19:39.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:39.021 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:19:39.186 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:19:39.187 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:19:39.187 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:39.193 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:19:39.193 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:39.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 -- 172.21.15.170:0/3003593883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400cfc50 msgr2=0x7f7b400caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/3003593883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400cfc50 0x7f7b400caf90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7b400cf0a0 tx=0x7f7b3401c8b0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 -- 172.21.15.170:0/3003593883 shutdown_connections 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/3003593883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b40070ed0 0x7f7b400712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/3003593883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400cfc50 0x7f7b400caf90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 -- 172.21.15.170:0/3003593883 >> 172.21.15.170:0/3003593883 conn(0x7f7b400cd3c0 msgr2=0x7f7b400cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 -- 172.21.15.170:0/3003593883 shutdown_connections 2024-09-21T02:19:39.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 -- 172.21.15.170:0/3003593883 wait complete. 2024-09-21T02:19:39.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 Processor -- start 2024-09-21T02:19:39.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.398+0000 7f7b44ff5640 1 -- start start 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b40070ed0 0x7f7b400b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 0x7f7b4014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b400cfc50 0x7f7b400ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40168df0 con 0x7f7b400b62e0 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b40168c70 con 0x7f7b40070ed0 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7b40168f70 con 0x7f7b400cfc50 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 0x7f7b4014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 0x7f7b4014eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40370/0 (socket says 172.21.15.170:40370) 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 -- 172.21.15.170:0/2568466230 learned_addr learned my addr 172.21.15.170:0/2568466230 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3ed9d640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b400cfc50 0x7f7b400ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3e59c640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b40070ed0 0x7f7b400b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 -- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b400cfc50 msgr2=0x7f7b400ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b400cfc50 0x7f7b400ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 -- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b40070ed0 msgr2=0x7f7b400b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b40070ed0 0x7f7b400b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 -- 172.21.15.170:0/2568466230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b401fc6b0 con 0x7f7b400b62e0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3dd9b640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 0x7f7b4014eaf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000b3e0 tx=0x7f7b3000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30015020 con 0x7f7b400b62e0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b30011940 con 0x7f7b400b62e0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30011d90 con 0x7f7b400b62e0 2024-09-21T02:19:39.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b401fc990 con 0x7f7b400b62e0 2024-09-21T02:19:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4014f030 con 0x7f7b400b62e0 2024-09-21T02:19:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b400712e0 con 0x7f7b400b62e0 2024-09-21T02:19:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f7b30018020 con 0x7f7b400b62e0 2024-09-21T02:19:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b2effd640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b20073590 0x7f7b20075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f7b30094c30 con 0x7f7b400b62e0 2024-09-21T02:19:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3e59c640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b20073590 0x7f7b20075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.402+0000 7f7b3e59c640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b20073590 0x7f7b20075aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b340af000 tx=0x7f7b3401b0f0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:39.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.406+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7b3001f020 con 0x7f7b400b62e0 2024-09-21T02:19:39.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.578+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b40156ae0 con 0x7f7b400b62e0 2024-09-21T02:19:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b2effd640 1 -- 172.21.15.170:0/2568466230 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f7b3005f840 con 0x7f7b400b62e0 2024-09-21T02:19:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:19:39.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b20073590 msgr2=0x7f7b20075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b20073590 0x7f7b20075aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b340af000 tx=0x7f7b3401b0f0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 msgr2=0x7f7b4014eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 0x7f7b4014eaf0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000b3e0 tx=0x7f7b3000b8a0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 shutdown_connections 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b20073590 0x7f7b20075aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b400cfc50 0x7f7b400ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b400b62e0 0x7f7b4014eaf0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 --2- 172.21.15.170:0/2568466230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b40070ed0 0x7f7b400b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 >> 172.21.15.170:0/2568466230 conn(0x7f7b400cd3c0 msgr2=0x7f7b40155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 shutdown_connections 2024-09-21T02:19:39.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.582+0000 7f7b44ff5640 1 -- 172.21.15.170:0/2568466230 wait complete. 2024-09-21T02:19:39.601 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:19:39.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:39.602 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:19:39.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35196]: fuse finished with error 0 and tester_r 0 2024-09-21T02:19:39.617 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:19:39.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:19:39.618 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:19:39.711 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:19:39.711 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:39.711 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:39.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:19:39.717 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:19:39.717 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:19:39.718 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:39.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:19:39.759 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:19:39.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 -- 172.21.15.170:0/2198497601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cc8c0 msgr2=0x7f6e8c0cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 --2- 172.21.15.170:0/2198497601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cc8c0 0x7f6e8c0cccd0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c072620 tx=0x7f6e8401cd80 comp rx=0 tx=0).stop 2024-09-21T02:19:39.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 -- 172.21.15.170:0/2198497601 shutdown_connections 2024-09-21T02:19:39.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 --2- 172.21.15.170:0/2198497601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cc8c0 0x7f6e8c0cccd0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 --2- 172.21.15.170:0/2198497601 >> v2:172.21.15.191:3300/0 conn(0x7f6e8c0cb460 0x7f6e8c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 --2- 172.21.15.170:0/2198497601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cf7e0 0x7f6e8c070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 -- 172.21.15.170:0/2198497601 >> 172.21.15.170:0/2198497601 conn(0x7f6e8c0b1c40 msgr2=0x7f6e8c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:39.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 -- 172.21.15.170:0/2198497601 shutdown_connections 2024-09-21T02:19:39.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.918+0000 7f6e92dba640 1 -- 172.21.15.170:0/2198497601 wait complete. 2024-09-21T02:19:39.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 Processor -- start 2024-09-21T02:19:39.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 -- start start 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e8c0cb460 0x7f6e8c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cc8c0 0x7f6e8c1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 0x7f6e8c09d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c06fa50 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c06f8d0 con 0x7f6e8c0cb460 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c06fbd0 con 0x7f6e8c0cc8c0 2024-09-21T02:19:39.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 0x7f6e8c09d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 0x7f6e8c09d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40386/0 (socket says 172.21.15.170:40386) 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 -- 172.21.15.170:0/1320660940 learned_addr learned my addr 172.21.15.170:0/1320660940 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e83fff640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cc8c0 0x7f6e8c1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e90b56640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e8c0cb460 0x7f6e8c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 -- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cc8c0 msgr2=0x7f6e8c1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cc8c0 0x7f6e8c1575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 -- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e8c0cb460 msgr2=0x7f6e8c150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e8c0cb460 0x7f6e8c150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 -- 172.21.15.170:0/1320660940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c1513c0 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e83fff640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cc8c0 0x7f6e8c1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e91357640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 0x7f6e8c09d5e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009bd0 tx=0x7f6e84009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840bb070 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e84004d40 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840b2040 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c1f6da0 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.922+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c1f7360 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.926+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c0cccd0 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.926+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 97540+0+0 (secure 0 0 0) 0x7f6e840be070 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.926+0000 7f6e817fa640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e78073560 0x7f6e78075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:39.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.926+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f6e8412e740 con 0x7f6e8c0cf7e0 2024-09-21T02:19:39.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.926+0000 7f6e90b56640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e78073560 0x7f6e78075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:39.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.926+0000 7f6e90b56640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e78073560 0x7f6e78075a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009980 tx=0x7f6e88005ca0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:39.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:39.930+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6e84016970 con 0x7f6e8c0cf7e0 2024-09-21T02:19:40.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.042+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f6e840f4e50 con 0x7f6e8c0cf7e0 2024-09-21T02:19:40.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.146+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e8c070640 con 0x7f6e8c0cf7e0 2024-09-21T02:19:40.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.146+0000 7f6e817fa640 1 -- 172.21.15.170:0/1320660940 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5237 (secure 0 0 0) 0x7f6e840f92e0 con 0x7f6e8c0cf7e0 2024-09-21T02:19:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:40.148 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":72,"btime":"2024-09-21T02:19:03:928784+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:50.719554+0000","modified":"2024-09-21T02:18:53.780500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6801/1778665511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1778665511},{"type":"v1","addr":"172.21.15.191:6801","nonce":1778665511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7107,"qdb_cluster":[7107]},"id":8}]} 2024-09-21T02:19:40.148 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 72 2024-09-21T02:19:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e78073560 msgr2=0x7f6e78075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e78073560 0x7f6e78075a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009980 tx=0x7f6e88005ca0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 msgr2=0x7f6e8c09d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 0x7f6e8c09d5e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009bd0 tx=0x7f6e84009350 comp rx=0 tx=0).stop 2024-09-21T02:19:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 shutdown_connections 2024-09-21T02:19:40.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6e78073560 0x7f6e78075a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e8c0cc8c0 0x7f6e8c1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e8c0cf7e0 0x7f6e8c09d5e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 --2- 172.21.15.170:0/1320660940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e8c0cb460 0x7f6e8c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 >> 172.21.15.170:0/1320660940 conn(0x7f6e8c0b1c40 msgr2=0x7f6e8c0b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:40.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 shutdown_connections 2024-09-21T02:19:40.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.150+0000 7f6e92dba640 1 -- 172.21.15.170:0/1320660940 wait complete. 2024-09-21T02:19:40.170 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:19:40.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 -- 172.21.15.170:0/685196987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 msgr2=0x7fb8bc0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 --2- 172.21.15.170:0/685196987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc0b3350 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc071d80 tx=0x7fb8ac01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 -- 172.21.15.170:0/685196987 shutdown_connections 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 --2- 172.21.15.170:0/685196987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc0b3350 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 --2- 172.21.15.170:0/685196987 >> v2:172.21.15.191:3300/0 conn(0x7fb8bc0b5410 0x7fb8bc06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 -- 172.21.15.170:0/685196987 >> 172.21.15.170:0/685196987 conn(0x7fb8bc0b1c40 msgr2=0x7fb8bc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 -- 172.21.15.170:0/685196987 shutdown_connections 2024-09-21T02:19:40.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 -- 172.21.15.170:0/685196987 wait complete. 2024-09-21T02:19:40.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 Processor -- start 2024-09-21T02:19:40.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.326+0000 7fb8c3892640 1 -- start start 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc09ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8bc0b5410 0x7fb8bc0af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8bc0ccb00 0x7fb8bc159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc169fd0 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc169e50 con 0x7fb8bc0ccb00 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb8bc16a150 con 0x7fb8bc0b5410 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc09ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc09ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40412/0 (socket says 172.21.15.170:40412) 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 -- 172.21.15.170:0/3544483185 learned_addr learned my addr 172.21.15.170:0/3544483185 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c1e2f640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8bc0ccb00 0x7fb8bc159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c0e2d640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8bc0b5410 0x7fb8bc0af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 -- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8bc0b5410 msgr2=0x7fb8bc0af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8bc0b5410 0x7fb8bc0af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 -- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8bc0ccb00 msgr2=0x7fb8bc159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8bc0ccb00 0x7fb8bc159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 -- 172.21.15.170:0/3544483185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc159dc0 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c1e2f640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8bc0ccb00 0x7fb8bc159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c162e640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc09ba10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c9f0 tx=0x7fb8b800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:40.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007bb0 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8b8007e40 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8bc0ae970 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800f690 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8bc09c280 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8bc0b3350 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fb8b800f830 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8b1ffb640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8a0073680 0x7fb8a0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7fb8b8095400 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.330+0000 7fb8c0e2d640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8a0073680 0x7fb8a0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.334+0000 7fb8c0e2d640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8a0073680 0x7fb8a0075b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0af000 tx=0x7fb8ac01b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.338+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb8b8014030 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.546+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8bc0b4950 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.546+0000 7fb8b1ffb640 1 -- 172.21.15.170:0/3544483185 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5237 (secure 0 0 0) 0x7fb8b805fc60 con 0x7fb8bc0b2f40 2024-09-21T02:19:40.548 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:40.548 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":72,"btime":"2024-09-21T02:19:03:928784+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:50.719554+0000","modified":"2024-09-21T02:18:53.780500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6801/1778665511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1778665511},{"type":"v1","addr":"172.21.15.191:6801","nonce":1778665511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7107,"qdb_cluster":[7107]},"id":8}]} 2024-09-21T02:19:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 72 2024-09-21T02:19:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8a0073680 msgr2=0x7fb8a0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8a0073680 0x7fb8a0075b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0af000 tx=0x7fb8ac01b100 comp rx=0 tx=0).stop 2024-09-21T02:19:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 msgr2=0x7fb8bc09ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc09ba10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c9f0 tx=0x7fb8b800ceb0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 shutdown_connections 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8a0073680 0x7fb8a0075b90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8bc0b5410 0x7fb8bc0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8bc0ccb00 0x7fb8bc159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 --2- 172.21.15.170:0/3544483185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8bc0b2f40 0x7fb8bc09ba10 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 >> 172.21.15.170:0/3544483185 conn(0x7fb8bc0b1c40 msgr2=0x7fb8bc0c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 shutdown_connections 2024-09-21T02:19:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.550+0000 7fb8c3892640 1 -- 172.21.15.170:0/3544483185 wait complete. 2024-09-21T02:19:40.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:19:40.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 -- 172.21.15.170:0/3601051831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 msgr2=0x7fd6740b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 --2- 172.21.15.170:0/3601051831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b64f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd6740d10a0 tx=0x7fd66c01ccf0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 -- 172.21.15.170:0/3601051831 shutdown_connections 2024-09-21T02:19:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 --2- 172.21.15.170:0/3601051831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b64f0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 --2- 172.21.15.170:0/3601051831 >> v2:172.21.15.191:3300/0 conn(0x7fd6740ccce0 0x7fd6740cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 --2- 172.21.15.170:0/3601051831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6740b21a0 0x7fd6740b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 -- 172.21.15.170:0/3601051831 >> 172.21.15.170:0/3601051831 conn(0x7fd67409a9c0 msgr2=0x7fd6740d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:40.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 -- 172.21.15.170:0/3601051831 shutdown_connections 2024-09-21T02:19:40.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.726+0000 7fd67ba06640 1 -- 172.21.15.170:0/3601051831 wait complete. 2024-09-21T02:19:40.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 Processor -- start 2024-09-21T02:19:40.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 -- start start 2024-09-21T02:19:40.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6740b21a0 0x7fd674154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6740ccce0 0x7fd6740b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740ce820 con 0x7fd6740b60e0 2024-09-21T02:19:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740ce6a0 con 0x7fd6740b21a0 2024-09-21T02:19:40.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.730+0000 7fd67ba06640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd6740ce9a0 con 0x7fd6740ccce0 2024-09-21T02:19:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40436/0 (socket says 172.21.15.170:40436) 2024-09-21T02:19:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 -- 172.21.15.170:0/4254912559 learned_addr learned my addr 172.21.15.170:0/4254912559 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd6797a2640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6740b21a0 0x7fd674154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd679fa3640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6740ccce0 0x7fd6740b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 -- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6740ccce0 msgr2=0x7fd6740b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6740ccce0 0x7fd6740b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 -- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6740b21a0 msgr2=0x7fd674154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6740b21a0 0x7fd674154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 -- 172.21.15.170:0/4254912559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6741f7c50 con 0x7fd6740b60e0 2024-09-21T02:19:40.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd678fa1640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b52c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c970 tx=0x7fd66400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:40.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664004220 con 0x7fd6740b60e0 2024-09-21T02:19:40.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6640079b0 con 0x7fd6740b60e0 2024-09-21T02:19:40.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6640044b0 con 0x7fd6740b60e0 2024-09-21T02:19:40.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67409bf50 con 0x7fd6740b60e0 2024-09-21T02:19:40.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6741f81d0 con 0x7fd6740b60e0 2024-09-21T02:19:40.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6740cb6d0 con 0x7fd6740b60e0 2024-09-21T02:19:40.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fd664005020 con 0x7fd6740b60e0 2024-09-21T02:19:40.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd669ffb640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6580736b0 0x7fd658075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:40.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd6797a2640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6580736b0 0x7fd658075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:40.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.734+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7fd664095050 con 0x7fd6740b60e0 2024-09-21T02:19:40.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.738+0000 7fd6797a2640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6580736b0 0x7fd658075bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd674074d80 tx=0x7fd670008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:40.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.738+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd664014030 con 0x7fd6740b60e0 2024-09-21T02:19:40.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.914+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6740718b0 con 0x7fd6740b60e0 2024-09-21T02:19:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.914+0000 7fd669ffb640 1 -- 172.21.15.170:0/4254912559 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17010 (secure 0 0 0) 0x7fd66405f920 con 0x7fd6740b60e0 2024-09-21T02:19:40.916 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:40.916 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":106,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:37.998780+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:48.430867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:49.686964+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:19:40.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6580736b0 msgr2=0x7fd658075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6580736b0 0x7fd658075bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd674074d80 tx=0x7fd670008040 comp rx=0 tx=0).stop 2024-09-21T02:19:40.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 msgr2=0x7fd6740b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:40.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b52c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c970 tx=0x7fd66400ce30 comp rx=0 tx=0).stop 2024-09-21T02:19:40.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 shutdown_connections 2024-09-21T02:19:40.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6580736b0 0x7fd658075bc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6740ccce0 0x7fd6740b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6740b60e0 0x7fd6740b52c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 --2- 172.21.15.170:0/4254912559 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6740b21a0 0x7fd674154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 >> 172.21.15.170:0/4254912559 conn(0x7fd67409a9c0 msgr2=0x7fd6740b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 shutdown_connections 2024-09-21T02:19:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:40.918+0000 7fd67ba06640 1 -- 172.21.15.170:0/4254912559 wait complete. 2024-09-21T02:19:40.938 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:19:40.938 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:19:41.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 -- 172.21.15.170:0/660098072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b2240 msgr2=0x7f315c1696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 --2- 172.21.15.170:0/660098072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b2240 0x7f315c1696d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3154009bc0 tx=0x7f315401cb10 comp rx=0 tx=0).stop 2024-09-21T02:19:41.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 -- 172.21.15.170:0/660098072 shutdown_connections 2024-09-21T02:19:41.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 --2- 172.21.15.170:0/660098072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b2240 0x7f315c1696d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 -- 172.21.15.170:0/660098072 >> 172.21.15.170:0/660098072 conn(0x7f315c08bf50 msgr2=0x7f315c09d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:41.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 -- 172.21.15.170:0/660098072 shutdown_connections 2024-09-21T02:19:41.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 -- 172.21.15.170:0/660098072 wait complete. 2024-09-21T02:19:41.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 Processor -- start 2024-09-21T02:19:41.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.090+0000 7f3164691640 1 -- start start 2024-09-21T02:19:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f315c0b2240 0x7f315c0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 0x7f315c0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f315c0d0e50 0x7f315c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f315c16cec0 con 0x7f315c0b3fa0 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f315c16cd40 con 0x7f315c0d0e50 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f315c16d040 con 0x7f315c0b2240 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 0x7f315c0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 0x7f315c0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40452/0 (socket says 172.21.15.170:40452) 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 -- 172.21.15.170:0/1756270164 learned_addr learned my addr 172.21.15.170:0/1756270164 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3162c2e640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f315c0d0e50 0x7f315c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f316242d640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f315c0b2240 0x7f315c0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 -- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f315c0b2240 msgr2=0x7f315c0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f315c0b2240 0x7f315c0cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 -- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f315c0d0e50 msgr2=0x7f315c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f315c0d0e50 0x7f315c0cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 -- 172.21.15.170:0/1756270164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c0ce5b0 con 0x7f315c0b3fa0 2024-09-21T02:19:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3162c2e640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f315c0d0e50 0x7f315c0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f316242d640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f315c0b2240 0x7f315c0cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3161c2c640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 0x7f315c0cd950 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f315800c3c0 tx=0x7f315800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158011790 con 0x7f315c0b3fa0 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3158017070 con 0x7f315c0b3fa0 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c1618e0 con 0x7f315c0b3fa0 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158011d20 con 0x7f315c0b3fa0 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c15df60 con 0x7f315c0b3fa0 2024-09-21T02:19:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c0d1260 con 0x7f315c0b3fa0 2024-09-21T02:19:41.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.094+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f315801c460 con 0x7f315c0b3fa0 2024-09-21T02:19:41.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.098+0000 7f3152ffd640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f31400736b0 0x7f3140075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.098+0000 7f316242d640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f31400736b0 0x7f3140075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.098+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f31580951f0 con 0x7f315c0b3fa0 2024-09-21T02:19:41.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.098+0000 7f316242d640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f31400736b0 0x7f3140075bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f314c00a850 tx=0x7f314c008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:41.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.102+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f315805fc80 con 0x7f315c0b3fa0 2024-09-21T02:19:41.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f315c06ce00 con 0x7f315c0b3fa0 2024-09-21T02:19:41.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3152ffd640 1 -- 172.21.15.170:0/1756270164 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5237 (secure 0 0 0) 0x7f315805f580 con 0x7f315c0b3fa0 2024-09-21T02:19:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:41.294 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":72,"btime":"2024-09-21T02:19:03:928784+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:50.719554+0000","modified":"2024-09-21T02:18:53.780500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6801/1778665511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1778665511},{"type":"v1","addr":"172.21.15.191:6801","nonce":1778665511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7107,"qdb_cluster":[7107]},"id":8}]} 2024-09-21T02:19:41.294 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 72 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f31400736b0 msgr2=0x7f3140075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3164691640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f31400736b0 0x7f3140075bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f314c00a850 tx=0x7f314c008040 comp rx=0 tx=0).stop 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 msgr2=0x7f315c0cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3164691640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 0x7f315c0cd950 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f315800c3c0 tx=0x7f315800c880 comp rx=0 tx=0).stop 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.294+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 shutdown_connections 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f31400736b0 0x7f3140075bc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f315c0d0e50 0x7f315c0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315c0b3fa0 0x7f315c0cd950 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 --2- 172.21.15.170:0/1756270164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f315c0b2240 0x7f315c0cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 >> 172.21.15.170:0/1756270164 conn(0x7f315c08bf50 msgr2=0x7f315c0cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:41.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 shutdown_connections 2024-09-21T02:19:41.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.298+0000 7f3164691640 1 -- 172.21.15.170:0/1756270164 wait complete. 2024-09-21T02:19:41.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:19:41.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 -- 172.21.15.170:0/1709711496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 msgr2=0x7f69e40cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 --2- 172.21.15.170:0/1709711496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40cf950 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f69e40d1240 tx=0x7f69cc01c8a0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 -- 172.21.15.170:0/1709711496 shutdown_connections 2024-09-21T02:19:41.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 --2- 172.21.15.170:0/1709711496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69e4160440 0x7f69e40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 --2- 172.21.15.170:0/1709711496 >> v2:172.21.15.191:3300/0 conn(0x7f69e415faf0 0x7f69e415ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 --2- 172.21.15.170:0/1709711496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40cf950 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 -- 172.21.15.170:0/1709711496 >> 172.21.15.170:0/1709711496 conn(0x7f69e409a9c0 msgr2=0x7f69e40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 -- 172.21.15.170:0/1709711496 shutdown_connections 2024-09-21T02:19:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.454+0000 7f69e98a1640 1 -- 172.21.15.170:0/1709711496 wait complete. 2024-09-21T02:19:41.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 Processor -- start 2024-09-21T02:19:41.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 -- start start 2024-09-21T02:19:41.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e415faf0 0x7f69e41fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69e4160440 0x7f69e41facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40b4bc0 con 0x7f69e40b25f0 2024-09-21T02:19:41.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f69e40b4a40 con 0x7f69e415faf0 2024-09-21T02:19:41.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f69e40b4d40 con 0x7f69e4160440 2024-09-21T02:19:41.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40480/0 (socket says 172.21.15.170:40480) 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 -- 172.21.15.170:0/562641429 learned_addr learned my addr 172.21.15.170:0/562641429 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e37fe640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69e4160440 0x7f69e41facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 -- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69e4160440 msgr2=0x7f69e41facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69e4160440 0x7f69e41facf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e27fc640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e415faf0 0x7f69e41fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 -- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e415faf0 msgr2=0x7f69e41fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e415faf0 0x7f69e41fa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 -- 172.21.15.170:0/562641429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e40b1780 con 0x7f69e40b25f0 2024-09-21T02:19:41.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e2ffd640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40b1240 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c10 tx=0x7f69cc0a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:41.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0ba020 con 0x7f69e40b25f0 2024-09-21T02:19:41.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69cc004430 con 0x7f69e40b25f0 2024-09-21T02:19:41.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc0b2040 con 0x7f69e40b25f0 2024-09-21T02:19:41.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e41fb4c0 con 0x7f69e40b25f0 2024-09-21T02:19:41.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.458+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e409bbb0 con 0x7f69e40b25f0 2024-09-21T02:19:41.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.462+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e415ff00 con 0x7f69e40b25f0 2024-09-21T02:19:41.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.462+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f69cc0bd070 con 0x7f69e40b25f0 2024-09-21T02:19:41.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.462+0000 7f69dbfff640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f69c40736b0 0x7f69c4075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.462+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f69cc12e5b0 con 0x7f69e40b25f0 2024-09-21T02:19:41.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.462+0000 7f69e27fc640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f69c40736b0 0x7f69c4075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.462+0000 7f69e27fc640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f69c40736b0 0x7f69c4075bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005c50 tx=0x7f69dc005be0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:41.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.466+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f69cc0c3020 con 0x7f69e40b25f0 2024-09-21T02:19:41.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.658+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69e40ce9d0 con 0x7f69e40b25f0 2024-09-21T02:19:41.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.658+0000 7f69dbfff640 1 -- 172.21.15.170:0/562641429 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5237 (secure 0 0 0) 0x7f69cc016490 con 0x7f69e40b25f0 2024-09-21T02:19:41.659 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:41.659 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":72,"btime":"2024-09-21T02:19:03:928784+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:18:50.719554+0000","modified":"2024-09-21T02:18:53.780500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7107},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7107":{"gid":7107,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6801/1778665511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1778665511},{"type":"v1","addr":"172.21.15.191:6801","nonce":1778665511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7107,"qdb_cluster":[7107]},"id":8}]} 2024-09-21T02:19:41.660 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 72 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f69c40736b0 msgr2=0x7f69c4075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f69c40736b0 0x7f69c4075bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005c50 tx=0x7f69dc005be0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 msgr2=0x7f69e40b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40b1240 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c10 tx=0x7f69cc0a5800 comp rx=0 tx=0).stop 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 shutdown_connections 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f69c40736b0 0x7f69c4075bc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69e4160440 0x7f69e41facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e415faf0 0x7f69e41fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 --2- 172.21.15.170:0/562641429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69e40b25f0 0x7f69e40b1240 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 >> 172.21.15.170:0/562641429 conn(0x7f69e409a9c0 msgr2=0x7f69e415b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 shutdown_connections 2024-09-21T02:19:41.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.662+0000 7f69e98a1640 1 -- 172.21.15.170:0/562641429 wait complete. 2024-09-21T02:19:41.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:19:41.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 -- 172.21.15.170:0/1245766992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b2f40 msgr2=0x7f5a140b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/1245766992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b2f40 0x7f5a140b3350 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5a140b3a10 tx=0x7f5a0801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 -- 172.21.15.170:0/1245766992 shutdown_connections 2024-09-21T02:19:41.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/1245766992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b2f40 0x7f5a140b3350 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/1245766992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140b5410 0x7f5a1406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 -- 172.21.15.170:0/1245766992 >> 172.21.15.170:0/1245766992 conn(0x7f5a140b1c40 msgr2=0x7f5a140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:41.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 -- 172.21.15.170:0/1245766992 shutdown_connections 2024-09-21T02:19:41.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.834+0000 7f5a18ac4640 1 -- 172.21.15.170:0/1245766992 wait complete. 2024-09-21T02:19:41.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 Processor -- start 2024-09-21T02:19:41.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- start start 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a140b2f40 0x7f5a141f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 0x7f5a141f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140ccb00 0x7f5a14157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1416a060 con 0x7f5a140b5410 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5a140701b0 con 0x7f5a140b2f40 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1416a1e0 con 0x7f5a140ccb00 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 0x7f5a141f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 0x7f5a141f9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40488/0 (socket says 172.21.15.170:40488) 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 -- 172.21.15.170:0/405458237 learned_addr learned my addr 172.21.15.170:0/405458237 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:41.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a1259c640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a140b2f40 0x7f5a141f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a12d9d640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140ccb00 0x7f5a14157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 -- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140ccb00 msgr2=0x7f5a14157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140ccb00 0x7f5a14157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 -- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a140b2f40 msgr2=0x7f5a141f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a140b2f40 0x7f5a141f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 -- 172.21.15.170:0/405458237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a141f73d0 con 0x7f5a140b5410 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a12d9d640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140ccb00 0x7f5a14157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a11d9b640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 0x7f5a141f9fb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5a0801a510 tx=0x7f5a080a6f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080b2040 con 0x7f5a140b5410 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a08004310 con 0x7f5a140b5410 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a141fabc0 con 0x7f5a140b5410 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080052d0 con 0x7f5a140b5410 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a140c7180 con 0x7f5a140b5410 2024-09-21T02:19:41.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.838+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a140b3350 con 0x7f5a140b5410 2024-09-21T02:19:41.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.842+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f5a08005470 con 0x7f5a140b5410 2024-09-21T02:19:41.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.842+0000 7f5a02ffd640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59f4073680 0x7f59f4075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:41.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.842+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f5a0812e890 con 0x7f5a140b5410 2024-09-21T02:19:41.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.842+0000 7f5a1259c640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59f4073680 0x7f59f4075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:41.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.842+0000 7f5a1259c640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59f4073680 0x7f59f4075b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a040058d0 tx=0x7f5a04005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:41.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:41.846+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5a080bc060 con 0x7f5a140b5410 2024-09-21T02:19:42.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.006+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a140ceec0 con 0x7f5a140b5410 2024-09-21T02:19:42.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a02ffd640 1 -- 172.21.15.170:0/405458237 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17010 (secure 0 0 0) 0x7f5a080164d0 con 0x7f5a140b5410 2024-09-21T02:19:42.009 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:42.009 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":106,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:37.998780+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:18:48.430867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T02:18:49.686964+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:19:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59f4073680 msgr2=0x7f59f4075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59f4073680 0x7f59f4075b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a040058d0 tx=0x7f5a04005860 comp rx=0 tx=0).stop 2024-09-21T02:19:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 msgr2=0x7f5a141f9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 0x7f5a141f9fb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5a0801a510 tx=0x7f5a080a6f70 comp rx=0 tx=0).stop 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 shutdown_connections 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59f4073680 0x7f59f4075b90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a140ccb00 0x7f5a14157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a140b5410 0x7f5a141f9fb0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 --2- 172.21.15.170:0/405458237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a140b2f40 0x7f5a141f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 >> 172.21.15.170:0/405458237 conn(0x7f5a140b1c40 msgr2=0x7f5a1415c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 shutdown_connections 2024-09-21T02:19:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.010+0000 7f5a18ac4640 1 -- 172.21.15.170:0/405458237 wait complete. 2024-09-21T02:19:42.031 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b96a8640 1 -- 172.21.15.170:0/3158427827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f059c003a10 msgr2=0x7f059c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3158427827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f059c003a10 0x7f059c003e80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c310 tx=0x7f05a401cfc0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b37fe640 1 -- 172.21.15.170:0/3158427827 reap_dead start 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b96a8640 1 -- 172.21.15.170:0/3158427827 shutdown_connections 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b96a8640 1 -- 172.21.15.170:0/3158427827 >> 172.21.15.170:0/3158427827 conn(0x7f05b40b1c40 msgr2=0x7f05b40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b96a8640 1 -- 172.21.15.170:0/3158427827 shutdown_connections 2024-09-21T02:19:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.186+0000 7f05b96a8640 1 -- 172.21.15.170:0/3158427827 wait complete. 2024-09-21T02:19:42.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 Processor -- start 2024-09-21T02:19:42.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 -- start start 2024-09-21T02:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f059c003a10 0x7f05b4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b4158550 0x7f05b409bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 0x7f05b409c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05b41666a0 con 0x7f05b414fd70 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4166520 con 0x7f05b4158550 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f05b4166820 con 0x7f059c003a10 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 0x7f05b409c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 0x7f05b409c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40500/0 (socket says 172.21.15.170:40500) 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 -- 172.21.15.170:0/3255614926 learned_addr learned my addr 172.21.15.170:0/3255614926 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b27fc640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b4158550 0x7f05b409bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f059c003a10 0x7f05b4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 -- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f059c003a10 msgr2=0x7f05b4154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f059c003a10 0x7f05b4154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 -- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b4158550 msgr2=0x7f05b409bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b4158550 0x7f05b409bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 -- 172.21.15.170:0/3255614926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b4155670 con 0x7f05b414fd70 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f059c003a10 0x7f05b4154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b27fc640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b4158550 0x7f05b409bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b37fe640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 0x7f05b409c4f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f05b40b5250 tx=0x7f05ac00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00ddd0 con 0x7f05b414fd70 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05ac011570 con 0x7f05b414fd70 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b4155950 con 0x7f05b414fd70 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0119c0 con 0x7f05b414fd70 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.190+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b4158030 con 0x7f05b414fd70 2024-09-21T02:19:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.194+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b40cb870 con 0x7f05b414fd70 2024-09-21T02:19:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.194+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f05ac011b60 con 0x7f05b414fd70 2024-09-21T02:19:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.194+0000 7f059b7fe640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05a0073680 0x7f05a0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.194+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f05ac095a90 con 0x7f05b414fd70 2024-09-21T02:19:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.194+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05a0073680 0x7f05a0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.194+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05a0073680 0x7f05a0075b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c310 tx=0x7f05a40af040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:42.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.198+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f05ac0605b0 con 0x7f05b414fd70 2024-09-21T02:19:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.410+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f05b40761c0 con 0x7f05b414fd70 2024-09-21T02:19:42.411 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:19:42.410+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:19:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.414+0000 7f059b7fe640 1 -- 172.21.15.170:0/3255614926 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) ==== 161+0+0 (secure 0 0 0) 0x7f05ac05feb0 con 0x7f05b414fd70 2024-09-21T02:19:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:19:42.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05a0073680 msgr2=0x7f05a0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05a0073680 0x7f05a0075b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c310 tx=0x7f05a40af040 comp rx=0 tx=0).stop 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 msgr2=0x7f05b409c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 0x7f05b409c4f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f05b40b5250 tx=0x7f05ac00fed0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 shutdown_connections 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05a0073680 0x7f05a0075b90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b414fd70 0x7f05b409c4f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b4158550 0x7f05b409bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 --2- 172.21.15.170:0/3255614926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f059c003a10 0x7f05b4154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 >> 172.21.15.170:0/3255614926 conn(0x7f05b40b1c40 msgr2=0x7f05b40ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 shutdown_connections 2024-09-21T02:19:42.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.418+0000 7f05b96a8640 1 -- 172.21.15.170:0/3255614926 wait complete. 2024-09-21T02:19:42.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:19:42.552 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:19:42.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- 172.21.15.170:0/168580331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80cec50 msgr2=0x7fa1f806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 --2- 172.21.15.170:0/168580331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80cec50 0x7fa1f806ff50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009bf0 tx=0x7fa1ec01cb00 comp rx=0 tx=0).stop 2024-09-21T02:19:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- 172.21.15.170:0/168580331 shutdown_connections 2024-09-21T02:19:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 --2- 172.21.15.170:0/168580331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80cec50 0x7fa1f806ff50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- 172.21.15.170:0/168580331 >> 172.21.15.170:0/168580331 conn(0x7fa1f80d0170 msgr2=0x7fa1f80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- 172.21.15.170:0/168580331 shutdown_connections 2024-09-21T02:19:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- 172.21.15.170:0/168580331 wait complete. 2024-09-21T02:19:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 Processor -- start 2024-09-21T02:19:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- start start 2024-09-21T02:19:42.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80b2fc0 0x7fa1f8150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1f80cec50 0x7fa1f80cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 0x7fa1f8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f816b670 con 0x7fa1f80cf060 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f816b4f0 con 0x7fa1f80cec50 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa200562640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f816b7f0 con 0x7fa1f80b2fc0 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa1feaff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 0x7fa1f8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa1feaff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 0x7fa1f8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40510/0 (socket says 172.21.15.170:40510) 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa1feaff640 1 -- 172.21.15.170:0/3217212181 learned_addr learned my addr 172.21.15.170:0/3217212181 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1fdafd640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1f80cec50 0x7fa1f80cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.594+0000 7fa1fe2fe640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80b2fc0 0x7fa1f8150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1feaff640 1 -- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80b2fc0 msgr2=0x7fa1f8150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1feaff640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80b2fc0 0x7fa1f8150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1feaff640 1 -- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1f80cec50 msgr2=0x7fa1f80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1feaff640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1f80cec50 0x7fa1f80cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1feaff640 1 -- 172.21.15.170:0/3217212181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f80cb270 con 0x7fa1f80cf060 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1feaff640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 0x7fa1f8156fd0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c3f0 tx=0x7fa1e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8017070 con 0x7fa1f80cf060 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1e8005400 con 0x7fa1f80cf060 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f80cb550 con 0x7fa1f80cf060 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003860 con 0x7fa1f80cf060 2024-09-21T02:19:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f80b2340 con 0x7fa1f80cf060 2024-09-21T02:19:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f80cba70 con 0x7fa1f80cf060 2024-09-21T02:19:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fa1e801a070 con 0x7fa1f80cf060 2024-09-21T02:19:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1f6ffd640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1e4073680 0x7fa1e4075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1fe2fe640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1e4073680 0x7fa1e4075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5881+0+0 (secure 0 0 0) 0x7fa1e8094650 con 0x7fa1f80cf060 2024-09-21T02:19:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.598+0000 7fa1fe2fe640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1e4073680 0x7fa1e4075b90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000fe0 tx=0x7fa1f0001730 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.602+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa1e805e930 con 0x7fa1f80cf060 2024-09-21T02:19:42.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:42.810+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa1f80ae540 con 0x7fa1f80cf060 2024-09-21T02:19:43.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.422+0000 7fa1f6ffd640 1 -- 172.21.15.170:0/3217212181 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7fa1e8063360 con 0x7fa1f80cf060 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1e4073680 msgr2=0x7fa1e4075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1e4073680 0x7fa1e4075b90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000fe0 tx=0x7fa1f0001730 comp rx=0 tx=0).stop 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 msgr2=0x7fa1f8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 0x7fa1f8156fd0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c3f0 tx=0x7fa1e800c8b0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 shutdown_connections 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1e4073680 0x7fa1e4075b90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1f80cf060 0x7fa1f8156fd0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1f80cec50 0x7fa1f80cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 --2- 172.21.15.170:0/3217212181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1f80b2fc0 0x7fa1f8150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 >> 172.21.15.170:0/3217212181 conn(0x7fa1f80d0170 msgr2=0x7fa1f815de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 shutdown_connections 2024-09-21T02:19:43.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.426+0000 7fa200562640 1 -- 172.21.15.170:0/3217212181 wait complete. 2024-09-21T02:19:43.451 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:19:43.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 -- 172.21.15.170:0/2419446749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31740cdda0 msgr2=0x7f31740ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 --2- 172.21.15.170:0/2419446749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31740cdda0 0x7f31740ce1b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f31740ceef0 tx=0x7f316c01cb10 comp rx=0 tx=0).stop 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 -- 172.21.15.170:0/2419446749 shutdown_connections 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 --2- 172.21.15.170:0/2419446749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31740cdda0 0x7f31740ce1b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 --2- 172.21.15.170:0/2419446749 >> v2:172.21.15.191:3300/0 conn(0x7f317406ce00 0x7f317406d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 -- 172.21.15.170:0/2419446749 >> 172.21.15.170:0/2419446749 conn(0x7f3174074320 msgr2=0x7f31740d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 -- 172.21.15.170:0/2419446749 shutdown_connections 2024-09-21T02:19:43.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.610+0000 7f317bad4640 1 -- 172.21.15.170:0/2419446749 wait complete. 2024-09-21T02:19:43.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 Processor -- start 2024-09-21T02:19:43.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 -- start start 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 0x7f31740aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31740cdda0 0x7f31740c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31740ce6f0 0x7f317415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f31740d22c0 con 0x7f317406ce00 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f31740d2140 con 0x7f31740ce6f0 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f317416c4c0 con 0x7f31740cdda0 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 0x7f31740aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 0x7f31740aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40530/0 (socket says 172.21.15.170:40530) 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 -- 172.21.15.170:0/4149544617 learned_addr learned my addr 172.21.15.170:0/4149544617 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317906f640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31740cdda0 0x7f31740c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:43.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317a071640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31740ce6f0 0x7f317415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 -- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31740cdda0 msgr2=0x7f31740c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31740cdda0 0x7f31740c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 -- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31740ce6f0 msgr2=0x7f317415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31740ce6f0 0x7f317415b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 -- 172.21.15.170:0/4149544617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31740c8110 con 0x7f317406ce00 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f3179870640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 0x7f31740aec70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f31700058d0 tx=0x7f317000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170017070 con 0x7f317406ce00 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3170009d20 con 0x7f317406ce00 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31740c51e0 con 0x7f317406ce00 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31700106c0 con 0x7f317406ce00 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.614+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31740c57a0 con 0x7f317406ce00 2024-09-21T02:19:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.618+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317406d210 con 0x7f317406ce00 2024-09-21T02:19:43.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.618+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f317001a070 con 0x7f317406ce00 2024-09-21T02:19:43.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.618+0000 7f31667fc640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3158073680 0x7f3158075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:43.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.618+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5881+0+0 (secure 0 0 0) 0x7f31700951e0 con 0x7f317406ce00 2024-09-21T02:19:43.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.618+0000 7f317906f640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3158073680 0x7f3158075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:43.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.618+0000 7f317906f640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3158073680 0x7f3158075b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f316c0af000 tx=0x7f316c01b350 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.622+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f317005fa10 con 0x7f317406ce00 2024-09-21T02:19:43.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:43.790+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f317415b9e0 con 0x7f317406ce00 2024-09-21T02:19:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.426+0000 7f31667fc640 1 -- 172.21.15.170:0/4149544617 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f317005f310 con 0x7f317406ce00 2024-09-21T02:19:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3158073680 msgr2=0x7f3158075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3158073680 0x7f3158075b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f316c0af000 tx=0x7f316c01b350 comp rx=0 tx=0).stop 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 msgr2=0x7f31740aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 0x7f31740aec70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f31700058d0 tx=0x7f317000dde0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 shutdown_connections 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3158073680 0x7f3158075b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31740cdda0 0x7f31740c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31740ce6f0 0x7f317415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 --2- 172.21.15.170:0/4149544617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f317406ce00 0x7f31740aec70 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 >> 172.21.15.170:0/4149544617 conn(0x7f3174074320 msgr2=0x7f3174157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 shutdown_connections 2024-09-21T02:19:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.430+0000 7f317bad4640 1 -- 172.21.15.170:0/4149544617 wait complete. 2024-09-21T02:19:44.459 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:19:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 -- 172.21.15.170:0/1242784844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 msgr2=0x7fafa8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 --2- 172.21.15.170:0/1242784844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009b60 tx=0x7fafb401d270 comp rx=0 tx=0).stop 2024-09-21T02:19:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 -- 172.21.15.170:0/1242784844 shutdown_connections 2024-09-21T02:19:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 --2- 172.21.15.170:0/1242784844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 -- 172.21.15.170:0/1242784844 >> 172.21.15.170:0/1242784844 conn(0x7fafc00cdc80 msgr2=0x7fafc00ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 -- 172.21.15.170:0/1242784844 shutdown_connections 2024-09-21T02:19:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.618+0000 7fafc7883640 1 -- 172.21.15.170:0/1242784844 wait complete. 2024-09-21T02:19:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 Processor -- start 2024-09-21T02:19:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- start start 2024-09-21T02:19:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafc00c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00b5670 0x7fafc01f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc01f6900 0x7fafc014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fafc0073fa0 con 0x7fafa8004810 2024-09-21T02:19:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fafc01596c0 con 0x7fafc01f6900 2024-09-21T02:19:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fafc0074120 con 0x7fafc00b5670 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafc00c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafc00c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40556/0 (socket says 172.21.15.170:40556) 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 -- 172.21.15.170:0/3681042070 learned_addr learned my addr 172.21.15.170:0/3681042070 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc4e1e640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00b5670 0x7fafc01f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc5e20640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc01f6900 0x7fafc014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 -- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00b5670 msgr2=0x7fafc01f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00b5670 0x7fafc01f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 -- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc01f6900 msgr2=0x7fafc014d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc01f6900 0x7fafc014d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 -- 172.21.15.170:0/3681042070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00b45f0 con 0x7fafa8004810 2024-09-21T02:19:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc561f640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafc00c4d00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e990 tx=0x7fafb000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0015070 con 0x7fafa8004810 2024-09-21T02:19:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb000cda0 con 0x7fafa8004810 2024-09-21T02:19:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc00b48d0 con 0x7fafa8004810 2024-09-21T02:19:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003be0 con 0x7fafa8004810 2024-09-21T02:19:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc00b4e90 con 0x7fafa8004810 2024-09-21T02:19:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.622+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc0074680 con 0x7fafa8004810 2024-09-21T02:19:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.626+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fafb0023480 con 0x7fafa8004810 2024-09-21T02:19:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.626+0000 7fafbdffb640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafac073680 0x7fafac075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.626+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5414+0+0 (secure 0 0 0) 0x7fafb0095600 con 0x7fafa8004810 2024-09-21T02:19:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.626+0000 7fafc4e1e640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafac073680 0x7fafac075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.626+0000 7fafc4e1e640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafac073680 0x7fafac075b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fafb40ae000 tx=0x7fafb401a010 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.630+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fafb0060080 con 0x7fafa8004810 2024-09-21T02:19:44.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:44.806+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fafc006a1e0 con 0x7fafa8004810 2024-09-21T02:19:45.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.770+0000 7fafbdffb640 1 -- 172.21.15.170:0/3681042070 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7fafb005f980 con 0x7fafa8004810 2024-09-21T02:19:45.769 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:19:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafac073680 msgr2=0x7fafac075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafac073680 0x7fafac075b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fafb40ae000 tx=0x7fafb401a010 comp rx=0 tx=0).stop 2024-09-21T02:19:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 msgr2=0x7fafc00c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafc00c4d00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e990 tx=0x7fafb000ee50 comp rx=0 tx=0).stop 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 shutdown_connections 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafac073680 0x7fafac075b90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00b5670 0x7fafc01f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc01f6900 0x7fafc014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 --2- 172.21.15.170:0/3681042070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafa8004810 0x7fafc00c4d00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 >> 172.21.15.170:0/3681042070 conn(0x7fafc00cdc80 msgr2=0x7fafc0158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 shutdown_connections 2024-09-21T02:19:45.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.774+0000 7fafc7883640 1 -- 172.21.15.170:0/3681042070 wait complete. 2024-09-21T02:19:45.800 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 -- 172.21.15.170:0/3961586598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580cfb50 msgr2=0x7f91580cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 --2- 172.21.15.170:0/3961586598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580cfb50 0x7f91580cdff0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f9148009840 tx=0x7f914801c820 comp rx=0 tx=0).stop 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 -- 172.21.15.170:0/3961586598 shutdown_connections 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 --2- 172.21.15.170:0/3961586598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580cfb50 0x7f91580cdff0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 -- 172.21.15.170:0/3961586598 >> 172.21.15.170:0/3961586598 conn(0x7f91580b1c40 msgr2=0x7f91580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 -- 172.21.15.170:0/3961586598 shutdown_connections 2024-09-21T02:19:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.954+0000 7f915e845640 1 -- 172.21.15.170:0/3961586598 wait complete. 2024-09-21T02:19:45.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 Processor -- start 2024-09-21T02:19:45.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 -- start start 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 0x7f9158147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580cc920 0x7f9158169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91580cfb50 0x7f915816a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f915816d2f0 con 0x7f9158070050 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f915816d170 con 0x7f91580cfb50 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f915816d470 con 0x7f91580cc920 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 0x7f9158147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 0x7f9158147f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51768/0 (socket says 172.21.15.170:51768) 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 -- 172.21.15.170:0/1017170737 learned_addr learned my addr 172.21.15.170:0/1017170737 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915cde2640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91580cfb50 0x7f915816a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f91577fe640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580cc920 0x7f9158169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 -- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580cc920 msgr2=0x7f9158169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580cc920 0x7f9158169c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 -- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91580cfb50 msgr2=0x7f915816a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91580cfb50 0x7f915816a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 -- 172.21.15.170:0/1017170737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9158148470 con 0x7f9158070050 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9157fff640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 0x7f9158147f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f914801a450 tx=0x7f91480a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91480bc070 con 0x7f9158070050 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91480044b0 con 0x7f9158070050 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9158145730 con 0x7f9158070050 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.958+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91480b2040 con 0x7f9158070050 2024-09-21T02:19:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9158145cf0 con 0x7f9158070050 2024-09-21T02:19:45.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91580cdff0 con 0x7f9158070050 2024-09-21T02:19:45.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f9148003860 con 0x7f9158070050 2024-09-21T02:19:45.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f9154ff9640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91440736b0 0x7f9144075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:45.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f914812e350 con 0x7f9158070050 2024-09-21T02:19:45.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f91577fe640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91440736b0 0x7f9144075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:45.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.962+0000 7f91577fe640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91440736b0 0x7f9144075bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91580af050 tx=0x7f914c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:45.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:45.966+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f91480f3290 con 0x7f9158070050 2024-09-21T02:19:46.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.146+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f91580aea50 con 0x7f9158070050 2024-09-21T02:19:46.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.154+0000 7f9154ff9640 1 -- 172.21.15.170:0/1017170737 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f9158148470 con 0x7f9158070050 2024-09-21T02:19:46.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91440736b0 msgr2=0x7f9144075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91440736b0 0x7f9144075bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91580af050 tx=0x7f914c009040 comp rx=0 tx=0).stop 2024-09-21T02:19:46.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 msgr2=0x7f9158147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 0x7f9158147f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f914801a450 tx=0x7f91480a5750 comp rx=0 tx=0).stop 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 shutdown_connections 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91440736b0 0x7f9144075bc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580cc920 0x7f9158169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91580cfb50 0x7f915816a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 --2- 172.21.15.170:0/1017170737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9158070050 0x7f9158147f30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.158+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 >> 172.21.15.170:0/1017170737 conn(0x7f91580b1c40 msgr2=0x7f915806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.162+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 shutdown_connections 2024-09-21T02:19:46.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.162+0000 7f915e845640 1 -- 172.21.15.170:0/1017170737 wait complete. 2024-09-21T02:19:46.182 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-21T02:19:46.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 -- 172.21.15.170:0/3505045860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 msgr2=0x7f165806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 --2- 172.21.15.170:0/3505045860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165806fa30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f16580b3a10 tx=0x7f164801c820 comp rx=0 tx=0).stop 2024-09-21T02:19:46.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 -- 172.21.15.170:0/3505045860 shutdown_connections 2024-09-21T02:19:46.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 --2- 172.21.15.170:0/3505045860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580b2f40 0x7f16580b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 --2- 172.21.15.170:0/3505045860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165806fa30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 -- 172.21.15.170:0/3505045860 >> 172.21.15.170:0/3505045860 conn(0x7f16580b1c40 msgr2=0x7f16580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:46.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 -- 172.21.15.170:0/3505045860 shutdown_connections 2024-09-21T02:19:46.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.330+0000 7f165eef9640 1 -- 172.21.15.170:0/3505045860 wait complete. 2024-09-21T02:19:46.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 Processor -- start 2024-09-21T02:19:46.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 -- start start 2024-09-21T02:19:46.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16580b2f40 0x7f1658144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165809d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580ccb00 0x7f16580c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1658169fd0 con 0x7f16580b5410 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1658169e50 con 0x7f16580b2f40 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f165816a150 con 0x7f16580ccb00 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165809d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165809d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51782/0 (socket says 172.21.15.170:51782) 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 -- 172.21.15.170:0/3860346895 learned_addr learned my addr 172.21.15.170:0/3860346895 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165cc95640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16580b2f40 0x7f1658144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165d496640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580ccb00 0x7f16580c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 -- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580ccb00 msgr2=0x7f16580c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580ccb00 0x7f16580c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 -- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16580b2f40 msgr2=0x7f1658144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16580b2f40 0x7f1658144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 -- 172.21.15.170:0/3860346895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1658145530 con 0x7f16580b5410 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165cc95640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16580b2f40 0x7f1658144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165d496640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580ccb00 0x7f16580c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f1657fff640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165809d010 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f164c00d6a0 tx=0x7f164c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:46.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c017070 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f164c005310 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c009820 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1658147a70 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.334+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1658148030 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.338+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16580b3350 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.338+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f164c0099c0 con 0x7f16580b5410 2024-09-21T02:19:46.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.338+0000 7f16557fa640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1644073680 0x7f1644075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.338+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f164c094910 con 0x7f16580b5410 2024-09-21T02:19:46.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.338+0000 7f165cc95640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1644073680 0x7f1644075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.338+0000 7f165cc95640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1644073680 0x7f1644075b90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16480af000 tx=0x7f164801b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:46.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.342+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f164c05f4f0 con 0x7f16580b5410 2024-09-21T02:19:46.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.582+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f16580c9710 con 0x7f16580b5410 2024-09-21T02:19:46.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.770+0000 7f16557fa640 1 -- 172.21.15.170:0/3860346895 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v335) ==== 149+0+0 (secure 0 0 0) 0x7f164c05edf0 con 0x7f16580b5410 2024-09-21T02:19:46.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1644073680 msgr2=0x7f1644075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1644073680 0x7f1644075b90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16480af000 tx=0x7f164801b060 comp rx=0 tx=0).stop 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 msgr2=0x7f165809d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165809d010 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f164c00d6a0 tx=0x7f164c00db60 comp rx=0 tx=0).stop 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 shutdown_connections 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1644073680 0x7f1644075b90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16580ccb00 0x7f16580c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16580b5410 0x7f165809d010 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 --2- 172.21.15.170:0/3860346895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16580b2f40 0x7f1658144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 >> 172.21.15.170:0/3860346895 conn(0x7f16580b1c40 msgr2=0x7f16580cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 shutdown_connections 2024-09-21T02:19:46.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.774+0000 7f165eef9640 1 -- 172.21.15.170:0/3860346895 wait complete. 2024-09-21T02:19:46.801 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-21T02:19:46.801 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-21T02:19:46.801 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:19:46.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:19:46.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 -- 172.21.15.170:0/1462341455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d0e50 msgr2=0x7fd7100d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 --2- 172.21.15.170:0/1462341455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d0e50 0x7fd7100d1260 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd71008d150 tx=0x7fd70001c620 comp rx=0 tx=0).stop 2024-09-21T02:19:46.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 -- 172.21.15.170:0/1462341455 shutdown_connections 2024-09-21T02:19:46.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 --2- 172.21.15.170:0/1462341455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d0e50 0x7fd7100d1260 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 --2- 172.21.15.170:0/1462341455 >> v2:172.21.15.191:3300/0 conn(0x7fd7100b3fa0 0x7fd7100cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 -- 172.21.15.170:0/1462341455 >> 172.21.15.170:0/1462341455 conn(0x7fd71008bf50 msgr2=0x7fd71009d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:46.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 -- 172.21.15.170:0/1462341455 shutdown_connections 2024-09-21T02:19:46.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.962+0000 7fd7187dc640 1 -- 172.21.15.170:0/1462341455 wait complete. 2024-09-21T02:19:46.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 Processor -- start 2024-09-21T02:19:46.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- start start 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7100b3fa0 0x7fd7101fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7100d0e50 0x7fd7100ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 0x7fd7100b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd71016c400 con 0x7fd7100d2440 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd71016c280 con 0x7fd7100b3fa0 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd71016c580 con 0x7fd7100d0e50 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 0x7fd7100b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 0x7fd7100b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51802/0 (socket says 172.21.15.170:51802) 2024-09-21T02:19:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 -- 172.21.15.170:0/3668708381 learned_addr learned my addr 172.21.15.170:0/3668708381 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716578640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7100b3fa0 0x7fd7101fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd715d77640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7100d0e50 0x7fd7100ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 -- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7100d0e50 msgr2=0x7fd7100ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7100d0e50 0x7fd7100ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 -- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7100b3fa0 msgr2=0x7fd7101fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7100b3fa0 0x7fd7101fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 -- 172.21.15.170:0/3668708381 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100ca8c0 con 0x7fd7100d2440 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd715d77640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7100d0e50 0x7fd7100ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd716d79640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 0x7fd7100b6260 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd70800b9e0 tx=0x7fd70800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708019070 con 0x7fd7100d2440 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd70800cc10 con 0x7fd7100d2440 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71015a3e0 con 0x7fd7100d2440 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708014440 con 0x7fd7100d2440 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71015a8e0 con 0x7fd7100d2440 2024-09-21T02:19:46.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.966+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7100cf970 con 0x7fd7100d2440 2024-09-21T02:19:46.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.970+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fd70801c070 con 0x7fd7100d2440 2024-09-21T02:19:46.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.970+0000 7fd706ffd640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6f4073680 0x7fd6f4075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:46.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.970+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fd708094b10 con 0x7fd7100d2440 2024-09-21T02:19:46.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.970+0000 7fd716578640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6f4073680 0x7fd6f4075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:46.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.970+0000 7fd716578640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6f4073680 0x7fd6f4075b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0058d0 tx=0x7fd70c005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:46.974+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd708023020 con 0x7fd7100d2440 2024-09-21T02:19:47.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.146+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd71015a670 con 0x7fd7100d2440 2024-09-21T02:19:47.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd706ffd640 1 -- 172.21.15.170:0/3668708381 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13092 (secure 0 0 0) 0x7fd70805f6a0 con 0x7fd7100d2440 2024-09-21T02:19:47.149 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:47.149 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":110,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:45.762485+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6801/1778665511":"2024-09-22T02:19:42.413924+0000","172.21.15.191:6800/1778665511":"2024-09-22T02:19:42.413924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:19:47.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6f4073680 msgr2=0x7fd6f4075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6f4073680 0x7fd6f4075b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0058d0 tx=0x7fd70c005860 comp rx=0 tx=0).stop 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 msgr2=0x7fd7100b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 0x7fd7100b6260 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd70800b9e0 tx=0x7fd70800bea0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 shutdown_connections 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6f4073680 0x7fd6f4075b90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7100d0e50 0x7fd7100ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7100d2440 0x7fd7100b6260 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 --2- 172.21.15.170:0/3668708381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7100b3fa0 0x7fd7101fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 >> 172.21.15.170:0/3668708381 conn(0x7fd71008bf50 msgr2=0x7fd71015f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 shutdown_connections 2024-09-21T02:19:47.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.150+0000 7fd7187dc640 1 -- 172.21.15.170:0/3668708381 wait complete. 2024-09-21T02:19:47.172 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:19:47.172 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:19:47.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 -- 172.21.15.170:0/4290715105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 msgr2=0x7fae480d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4290715105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480d4510 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fae38009900 tx=0x7fae3801c700 comp rx=0 tx=0).stop 2024-09-21T02:19:47.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 -- 172.21.15.170:0/4290715105 shutdown_connections 2024-09-21T02:19:47.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4290715105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480d4510 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4290715105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae480d17c0 0x7fae480b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 -- 172.21.15.170:0/4290715105 >> 172.21.15.170:0/4290715105 conn(0x7fae4808bf50 msgr2=0x7fae4809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:47.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 -- 172.21.15.170:0/4290715105 shutdown_connections 2024-09-21T02:19:47.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.334+0000 7fae4d08c640 1 -- 172.21.15.170:0/4290715105 wait complete. 2024-09-21T02:19:47.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 Processor -- start 2024-09-21T02:19:47.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 -- start start 2024-09-21T02:19:47.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae480d17c0 0x7fae48162e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae480d4a50 0x7fae480cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae4816bf40 con 0x7fae480d1bd0 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fae4816bdc0 con 0x7fae480d17c0 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fae4816c0c0 con 0x7fae480d4a50 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51824/0 (socket says 172.21.15.170:51824) 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 -- 172.21.15.170:0/4042523370 learned_addr learned my addr 172.21.15.170:0/4042523370 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4759e640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae480d4a50 0x7fae480cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae46d9d640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae480d17c0 0x7fae48162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 -- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae480d4a50 msgr2=0x7fae480cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae480d4a50 0x7fae480cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 -- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae480d17c0 msgr2=0x7fae48162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae480d17c0 0x7fae48162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 -- 172.21.15.170:0/4042523370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae48162060 con 0x7fae480d1bd0 2024-09-21T02:19:47.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae46d9d640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae480d17c0 0x7fae48162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:47.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4659c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480cc970 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fae3801cfa0 tx=0x7fae380a6c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:47.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380ac070 con 0x7fae480d1bd0 2024-09-21T02:19:47.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae48162340 con 0x7fae480d1bd0 2024-09-21T02:19:47.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae38004120 con 0x7fae480d1bd0 2024-09-21T02:19:47.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38005040 con 0x7fae480d1bd0 2024-09-21T02:19:47.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.338+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae480cbef0 con 0x7fae480d1bd0 2024-09-21T02:19:47.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.342+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4806eb00 con 0x7fae480d1bd0 2024-09-21T02:19:47.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.342+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fae380af070 con 0x7fae480d1bd0 2024-09-21T02:19:47.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.342+0000 7fae3f7fe640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fae28073680 0x7fae28075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.342+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fae3812e320 con 0x7fae480d1bd0 2024-09-21T02:19:47.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.342+0000 7fae46d9d640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fae28073680 0x7fae28075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.342+0000 7fae46d9d640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fae28073680 0x7fae28075b90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae300058d0 tx=0x7fae30005ca0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:47.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.346+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fae380bf070 con 0x7fae480d1bd0 2024-09-21T02:19:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.522+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae481fcdd0 con 0x7fae480d1bd0 2024-09-21T02:19:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.522+0000 7fae3f7fe640 1 -- 172.21.15.170:0/4042523370 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fae480d4e60 con 0x7fae480d1bd0 2024-09-21T02:19:47.521 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:19:47.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.522+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fae28073680 msgr2=0x7fae28075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.522+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fae28073680 0x7fae28075b90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae300058d0 tx=0x7fae30005ca0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.522+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 msgr2=0x7fae480cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.522+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480cc970 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fae3801cfa0 tx=0x7fae380a6c20 comp rx=0 tx=0).stop 2024-09-21T02:19:47.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 shutdown_connections 2024-09-21T02:19:47.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fae28073680 0x7fae28075b90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae480d4a50 0x7fae480cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae480d1bd0 0x7fae480cc970 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 --2- 172.21.15.170:0/4042523370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae480d17c0 0x7fae48162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 >> 172.21.15.170:0/4042523370 conn(0x7fae4808bf50 msgr2=0x7fae480b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:47.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 shutdown_connections 2024-09-21T02:19:47.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.526+0000 7fae4d08c640 1 -- 172.21.15.170:0/4042523370 wait complete. 2024-09-21T02:19:47.545 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-21T02:19:47.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 -- 172.21.15.170:0/2348272639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6300b25f0 msgr2=0x7fa6300cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 --2- 172.21.15.170:0/2348272639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6300b25f0 0x7fa6300cf950 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa6300d1210 tx=0x7fa62001cce0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 -- 172.21.15.170:0/2348272639 shutdown_connections 2024-09-21T02:19:47.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 --2- 172.21.15.170:0/2348272639 >> v2:172.21.15.191:3300/0 conn(0x7fa630160440 0x7fa6300cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 --2- 172.21.15.170:0/2348272639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa63015faf0 0x7fa63015ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 --2- 172.21.15.170:0/2348272639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6300b25f0 0x7fa6300cf950 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 -- 172.21.15.170:0/2348272639 >> 172.21.15.170:0/2348272639 conn(0x7fa63009a9c0 msgr2=0x7fa6300d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:47.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 -- 172.21.15.170:0/2348272639 shutdown_connections 2024-09-21T02:19:47.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 -- 172.21.15.170:0/2348272639 wait complete. 2024-09-21T02:19:47.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 Processor -- start 2024-09-21T02:19:47.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.698+0000 7fa636992640 1 -- start start 2024-09-21T02:19:47.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300b25f0 0x7fa630153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa63015faf0 0x7fa6300afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 0x7fa6301f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6300b4ad0 con 0x7fa630160440 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6300b4950 con 0x7fa6300b25f0 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa6300b4c50 con 0x7fa63015faf0 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 0x7fa6301f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 0x7fa6301f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51840/0 (socket says 172.21.15.170:51840) 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 -- 172.21.15.170:0/3038477716 learned_addr learned my addr 172.21.15.170:0/3038477716 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62ffff640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300b25f0 0x7fa630153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62f7fe640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa63015faf0 0x7fa6300afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 -- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa63015faf0 msgr2=0x7fa6300afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa63015faf0 0x7fa6300afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 -- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300b25f0 msgr2=0x7fa630153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:47.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300b25f0 0x7fa630153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 -- 172.21.15.170:0/3038477716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa630156d30 con 0x7fa630160440 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62ffff640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300b25f0 0x7fa630153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa634f2f640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 0x7fa6301f6cd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d630 tx=0x7fa62800daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628017070 con 0x7fa630160440 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa628005310 con 0x7fa630160440 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa630157010 con 0x7fa630160440 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280047b0 con 0x7fa630160440 2024-09-21T02:19:47.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6301f7620 con 0x7fa630160440 2024-09-21T02:19:47.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6301f7880 con 0x7fa630160440 2024-09-21T02:19:47.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.702+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fa628004950 con 0x7fa630160440 2024-09-21T02:19:47.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.706+0000 7fa62cff9640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6100736b0 0x7fa610075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:47.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.706+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fa628094b90 con 0x7fa630160440 2024-09-21T02:19:47.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.706+0000 7fa62ffff640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6100736b0 0x7fa610075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:47.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.706+0000 7fa62ffff640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6100736b0 0x7fa610075bc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6200af000 tx=0x7fa62001b520 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:47.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.710+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa62805f900 con 0x7fa630160440 2024-09-21T02:19:47.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:47.930+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fa63015ff00 con 0x7fa630160440 2024-09-21T02:19:48.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.782+0000 7fa62cff9640 1 -- 172.21.15.170:0/3038477716 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v337) ==== 151+0+0 (secure 0 0 0) 0x7fa62805f200 con 0x7fa630160440 2024-09-21T02:19:48.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6100736b0 msgr2=0x7fa610075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:48.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6100736b0 0x7fa610075bc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6200af000 tx=0x7fa62001b520 comp rx=0 tx=0).stop 2024-09-21T02:19:48.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 msgr2=0x7fa6301f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:48.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 0x7fa6301f6cd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa62800d630 tx=0x7fa62800daf0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 shutdown_connections 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6100736b0 0x7fa610075bc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa63015faf0 0x7fa6300afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa630160440 0x7fa6301f6cd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 --2- 172.21.15.170:0/3038477716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6300b25f0 0x7fa630153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 >> 172.21.15.170:0/3038477716 conn(0x7fa63009a9c0 msgr2=0x7fa6300b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 shutdown_connections 2024-09-21T02:19:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.786+0000 7fa636992640 1 -- 172.21.15.170:0/3038477716 wait complete. 2024-09-21T02:19:48.812 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:19:48.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 -- 172.21.15.170:0/2139607402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 msgr2=0x7ff8b80d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2139607402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b80d0d80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80b7050 tx=0x7ff8a801c740 comp rx=0 tx=0).stop 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 -- 172.21.15.170:0/2139607402 shutdown_connections 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2139607402 >> v2:172.21.15.191:3300/0 conn(0x7ff8b80d12c0 0x7ff8b80b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2139607402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b80d0d80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 -- 172.21.15.170:0/2139607402 >> 172.21.15.170:0/2139607402 conn(0x7ff8b809d830 msgr2=0x7ff8b808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 -- 172.21.15.170:0/2139607402 shutdown_connections 2024-09-21T02:19:48.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.974+0000 7ff8bd869640 1 -- 172.21.15.170:0/2139607402 wait complete. 2024-09-21T02:19:48.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 Processor -- start 2024-09-21T02:19:48.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 -- start start 2024-09-21T02:19:48.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8b80b6330 0x7ff8b81f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:48.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8b80d12c0 0x7ff8b815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b8164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b816c350 con 0x7ff8b80d3630 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b816c1d0 con 0x7ff8b80b6330 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b816c4d0 con 0x7ff8b80d12c0 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b8164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b8164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51848/0 (socket says 172.21.15.170:51848) 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 -- 172.21.15.170:0/2201188237 learned_addr learned my addr 172.21.15.170:0/2201188237 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b67fc640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8b80d12c0 0x7ff8b815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b6ffd640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8b80b6330 0x7ff8b81f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 -- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8b80d12c0 msgr2=0x7ff8b815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:48.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8b80d12c0 0x7ff8b815e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 -- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8b80b6330 msgr2=0x7ff8b81f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8b80b6330 0x7ff8b81f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 -- 172.21.15.170:0/2201188237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b815eef0 con 0x7ff8b80d3630 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b6ffd640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8b80b6330 0x7ff8b81f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b67fc640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8b80d12c0 0x7ff8b815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8b77fe640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b8164900 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c730 tx=0x7ff8b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0003b80 con 0x7ff8b80d3630 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b0003d40 con 0x7ff8b80d3630 2024-09-21T02:19:48.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00052d0 con 0x7ff8b80d3630 2024-09-21T02:19:48.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b806dff0 con 0x7ff8b80d3630 2024-09-21T02:19:48.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.978+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b806e5b0 con 0x7ff8b80d3630 2024-09-21T02:19:48.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.982+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b80b5df0 con 0x7ff8b80d3630 2024-09-21T02:19:48.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.982+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7ff8b0005680 con 0x7ff8b80d3630 2024-09-21T02:19:48.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.982+0000 7ff89ffff640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8a4073680 0x7ff8a4075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:48.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.982+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7ff8b00944b0 con 0x7ff8b80d3630 2024-09-21T02:19:48.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.982+0000 7ff8b6ffd640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8a4073680 0x7ff8a4075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:48.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.982+0000 7ff8b6ffd640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8a4073680 0x7ff8a4075b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80af000 tx=0x7ff8a801af80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:48.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:48.986+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff8b0020060 con 0x7ff8b80d3630 2024-09-21T02:19:49.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.158+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff8b8167920 con 0x7ff8b80d3630 2024-09-21T02:19:49.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff89ffff640 1 -- 172.21.15.170:0/2201188237 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+2294 (secure 0 0 0) 0x7ff8b005f010 con 0x7ff8b80d3630 2024-09-21T02:19:49.160 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:19:49.161 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:19:49.162 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:19:49.163 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8a4073680 msgr2=0x7ff8a4075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8a4073680 0x7ff8a4075b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80af000 tx=0x7ff8a801af80 comp rx=0 tx=0).stop 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 msgr2=0x7ff8b8164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b8164900 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c730 tx=0x7ff8b000cbf0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 shutdown_connections 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8a4073680 0x7ff8a4075b90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8b80d12c0 0x7ff8b815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8b80d3630 0x7ff8b8164900 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 --2- 172.21.15.170:0/2201188237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8b80b6330 0x7ff8b81f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 >> 172.21.15.170:0/2201188237 conn(0x7ff8b809d830 msgr2=0x7ff8b81605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:49.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 shutdown_connections 2024-09-21T02:19:49.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.162+0000 7ff8bd869640 1 -- 172.21.15.170:0/2201188237 wait complete. 2024-09-21T02:19:49.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1708393152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc06f650 msgr2=0x7fa7cc073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1708393152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc06f650 0x7fa7cc073140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0b2ca0 tx=0x7fa7bc01c830 comp rx=0 tx=0).stop 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1708393152 shutdown_connections 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1708393152 >> v2:172.21.15.191:3300/0 conn(0x7fa7cc070fd0 0x7fa7cc073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1708393152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc06f650 0x7fa7cc073140 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1708393152 >> 172.21.15.170:0/1708393152 conn(0x7fa7cc0d0170 msgr2=0x7fa7cc0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1708393152 shutdown_connections 2024-09-21T02:19:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.342+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1708393152 wait complete. 2024-09-21T02:19:49.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 Processor -- start 2024-09-21T02:19:49.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- start start 2024-09-21T02:19:49.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7cc06f650 0x7fa7cc14e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 0x7fa7cc1fb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7cc0ce070 0x7fa7cc0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc1693d0 con 0x7fa7cc070fd0 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc074a40 con 0x7fa7cc06f650 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc169550 con 0x7fa7cc0ce070 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 0x7fa7cc1fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 0x7fa7cc1fb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51862/0 (socket says 172.21.15.170:51862) 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 -- 172.21.15.170:0/1018495367 learned_addr learned my addr 172.21.15.170:0/1018495367 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d0c9d640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7cc06f650 0x7fa7cc14e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 -- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7cc0ce070 msgr2=0x7fa7cc0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7cc0ce070 0x7fa7cc0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 -- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7cc06f650 msgr2=0x7fa7cc14e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7cc06f650 0x7fa7cc14e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 -- 172.21.15.170:0/1018495367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc1fb590 con 0x7fa7cc070fd0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7cbfff640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 0x7fa7cc1fb050 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0b0040 tx=0x7fa7bc0a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0b2040 con 0x7fa7cc070fd0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7bc004430 con 0x7fa7cc070fd0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc157a40 con 0x7fa7cc070fd0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc003860 con 0x7fa7cc070fd0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc0c3dc0 con 0x7fa7cc070fd0 2024-09-21T02:19:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.346+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc073140 con 0x7fa7cc070fd0 2024-09-21T02:19:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.350+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fa7bc0bf060 con 0x7fa7cc070fd0 2024-09-21T02:19:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.350+0000 7fa7c97fa640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa7b8073680 0x7fa7b8075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.350+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fa7bc12e6d0 con 0x7fa7cc070fd0 2024-09-21T02:19:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.350+0000 7fa7d0c9d640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa7b8073680 0x7fa7b8075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.350+0000 7fa7d0c9d640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa7b8073680 0x7fa7b8075b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000a790 tx=0x7fa7c0008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:49.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.354+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa7cc1fb590 con 0x7fa7cc070fd0 2024-09-21T02:19:49.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.526+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa7cc157790 con 0x7fa7cc070fd0 2024-09-21T02:19:49.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.530+0000 7fa7c97fa640 1 -- 172.21.15.170:0/1018495367 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fa7bc0164b0 con 0x7fa7cc070fd0 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa7b8073680 msgr2=0x7fa7b8075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa7b8073680 0x7fa7b8075b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000a790 tx=0x7fa7c0008040 comp rx=0 tx=0).stop 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 msgr2=0x7fa7cc1fb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 0x7fa7cc1fb050 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0b0040 tx=0x7fa7bc0a5650 comp rx=0 tx=0).stop 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 shutdown_connections 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa7b8073680 0x7fa7b8075b90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7cc0ce070 0x7fa7cc0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7cc070fd0 0x7fa7cc1fb050 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 --2- 172.21.15.170:0/1018495367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7cc06f650 0x7fa7cc14e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 >> 172.21.15.170:0/1018495367 conn(0x7fa7cc0d0170 msgr2=0x7fa7cc14eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 shutdown_connections 2024-09-21T02:19:49.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.534+0000 7fa7d2f01640 1 -- 172.21.15.170:0/1018495367 wait complete. 2024-09-21T02:19:49.553 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:19:49.554 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:19:49.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:19:49.696 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:19:49.696 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:49.696 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:49.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:19:49.702 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:19:49.702 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:49.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:19:49.708 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:19:49.708 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:49.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:19:49.748 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:19:49.748 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:19:49.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:19:49.755 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:19:49.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6c5b77640 1 -- 172.21.15.170:0/1866076362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6a8004010 msgr2=0x7fa6a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/1866076362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6a8004010 0x7fa6a8004810 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40016c0 tx=0x7fa6b401c790 comp rx=0 tx=0).stop 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6bffff640 1 -- 172.21.15.170:0/1866076362 reap_dead start 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6c5b77640 1 -- 172.21.15.170:0/1866076362 shutdown_connections 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6c5b77640 1 -- 172.21.15.170:0/1866076362 >> 172.21.15.170:0/1866076362 conn(0x7fa6c009a9c0 msgr2=0x7fa6c00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6c5b77640 1 -- 172.21.15.170:0/1866076362 shutdown_connections 2024-09-21T02:19:49.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.950+0000 7fa6c5b77640 1 -- 172.21.15.170:0/1866076362 wait complete. 2024-09-21T02:19:49.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 Processor -- start 2024-09-21T02:19:49.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 -- start start 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6a8004010 0x7fa6c0154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 0x7fa6c01f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c01f9c60 0x7fa6c0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00cfdf0 con 0x7fa6c0154fd0 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00cfc70 con 0x7fa6c01f9c60 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c00cff70 con 0x7fa6a8004010 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 0x7fa6c01f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 0x7fa6c01f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51878/0 (socket says 172.21.15.170:51878) 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 -- 172.21.15.170:0/415521421 learned_addr learned my addr 172.21.15.170:0/415521421 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:49.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6bf7fe640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6a8004010 0x7fa6c0154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6bffff640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c01f9c60 0x7fa6c0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 -- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6a8004010 msgr2=0x7fa6c0154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6a8004010 0x7fa6c0154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 -- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c01f9c60 msgr2=0x7fa6c0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c01f9c60 0x7fa6c0155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 -- 172.21.15.170:0/415521421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01fa3f0 con 0x7fa6c0154fd0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6beffd640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 0x7fa6c01f9720 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa6b401b060 tx=0x7fa6b40013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4003600 con 0x7fa6c0154fd0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6b4003890 con 0x7fa6c0154fd0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c015b0e0 con 0x7fa6c0154fd0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4003cc0 con 0x7fa6c0154fd0 2024-09-21T02:19:49.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.954+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c015b6a0 con 0x7fa6c0154fd0 2024-09-21T02:19:49.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.958+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c015ff00 con 0x7fa6c0154fd0 2024-09-21T02:19:49.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.958+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fa6b40a55f0 con 0x7fa6c0154fd0 2024-09-21T02:19:49.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.958+0000 7fa6a7fff640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6ac0736b0 0x7fa6ac075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:49.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.958+0000 7fa6bf7fe640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6ac0736b0 0x7fa6ac075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:49.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.958+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7fa6b412dd40 con 0x7fa6c0154fd0 2024-09-21T02:19:49.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.958+0000 7fa6bf7fe640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6ac0736b0 0x7fa6ac075bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009f50 tx=0x7fa6b8009a10 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:49.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:49.962+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa6b40bf020 con 0x7fa6c0154fd0 2024-09-21T02:19:50.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.166+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6c00ce9d0 con 0x7fa6c0154fd0 2024-09-21T02:19:50.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.166+0000 7fa6a7fff640 1 -- 172.21.15.170:0/415521421 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+3923 (secure 0 0 0) 0x7fa6b40164f0 con 0x7fa6c0154fd0 2024-09-21T02:19:50.167 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:50.167 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":74,"btime":"2024-09-21T02:19:43:417625+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1509065283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1509065283},{"type":"v1","addr":"172.21.15.191:6835","nonce":1509065283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3061554766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3061554766},{"type":"v1","addr":"172.21.15.191:6801","nonce":3061554766}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74},{"gid":6016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2203048702","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2203048702},{"type":"v1","addr":"172.21.15.170:6833","nonce":2203048702}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2260987204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2260987204},{"type":"v1","addr":"172.21.15.170:6835","nonce":2260987204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[]} 2024-09-21T02:19:50.167 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 74 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6ac0736b0 msgr2=0x7fa6ac075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6ac0736b0 0x7fa6ac075bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009f50 tx=0x7fa6b8009a10 comp rx=0 tx=0).stop 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 msgr2=0x7fa6c01f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 0x7fa6c01f9720 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa6b401b060 tx=0x7fa6b40013a0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 shutdown_connections 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6ac0736b0 0x7fa6ac075bc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c01f9c60 0x7fa6c0155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c0154fd0 0x7fa6c01f9720 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 --2- 172.21.15.170:0/415521421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6a8004010 0x7fa6c0154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 >> 172.21.15.170:0/415521421 conn(0x7fa6c009a9c0 msgr2=0x7fa6c00b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:50.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 shutdown_connections 2024-09-21T02:19:50.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.170+0000 7fa6c5b77640 1 -- 172.21.15.170:0/415521421 wait complete. 2024-09-21T02:19:50.188 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:19:50.188 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:19:50.188 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:19:50.189 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:19:50.189 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:19:50.189 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:19:50.189 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:19:50.189 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:19:50.189 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:19:50.189 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:19:50.190 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:19:50.190 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:19:50.190 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:19:50.190 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:19:50.190 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:19:50.190 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:19:50.204 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:19:50.204 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:19:50.205 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:19:50.205 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:19:50.213 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:19:50.213 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:19:50.215 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:19:50.215 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:19:50.249 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:19:50.249 DEBUG:teuthology.parallel:result is None 2024-09-21T02:19:50.250 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:19:50.250 DEBUG:teuthology.parallel:result is None 2024-09-21T02:19:50.259 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:19:50.259 DEBUG:teuthology.parallel:result is None 2024-09-21T02:19:50.260 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:19:50.261 DEBUG:teuthology.parallel:result is None 2024-09-21T02:19:50.261 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:19:50.335 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:19:50.335 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:19:50.359 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:19:50.359 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 -- 172.21.15.170:0/3276941881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 msgr2=0x7f0a200b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/3276941881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a200b1640 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0a2008d150 tx=0x7f0a1801c650 comp rx=0 tx=0).stop 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 -- 172.21.15.170:0/3276941881 shutdown_connections 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/3276941881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a200b1640 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/3276941881 >> v2:172.21.15.191:3300/0 conn(0x7f0a200b3fa0 0x7f0a200cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 -- 172.21.15.170:0/3276941881 >> 172.21.15.170:0/3276941881 conn(0x7f0a2008bf50 msgr2=0x7f0a2009d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 -- 172.21.15.170:0/3276941881 shutdown_connections 2024-09-21T02:19:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 -- 172.21.15.170:0/3276941881 wait complete. 2024-09-21T02:19:50.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 Processor -- start 2024-09-21T02:19:50.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.406+0000 7f0a24fb7640 1 -- start start 2024-09-21T02:19:50.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a201fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:50.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a200b3fa0 0x7f0a200b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:50.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a200d0e50 0x7f0a2006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:50.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2016bfc0 con 0x7f0a200b1230 2024-09-21T02:19:50.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2016be40 con 0x7f0a200d0e50 2024-09-21T02:19:50.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2016c140 con 0x7f0a200b3fa0 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a201fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a201fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51918/0 (socket says 172.21.15.170:51918) 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 -- 172.21.15.170:0/4088817357 learned_addr learned my addr 172.21.15.170:0/4088817357 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1dd9b640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a200b3fa0 0x7f0a200b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1ed9d640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a200d0e50 0x7f0a2006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 -- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a200b3fa0 msgr2=0x7f0a200b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a200b3fa0 0x7f0a200b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 -- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a200d0e50 msgr2=0x7f0a2006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a200d0e50 0x7f0a2006dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 -- 172.21.15.170:0/4088817357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a201fd090 con 0x7f0a200b1230 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1ed9d640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a200d0e50 0x7f0a2006dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1e59c640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a201fcb50 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000ea20 tx=0x7f0a1000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:50.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000cde0 con 0x7f0a200b1230 2024-09-21T02:19:50.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a10004c20 con 0x7f0a200b1230 2024-09-21T02:19:50.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10005020 con 0x7f0a200b1230 2024-09-21T02:19:50.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2015aca0 con 0x7f0a200b1230 2024-09-21T02:19:50.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a201f8c60 con 0x7f0a200b1230 2024-09-21T02:19:50.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a200b1640 con 0x7f0a200b1230 2024-09-21T02:19:50.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f0a100051c0 con 0x7f0a200b1230 2024-09-21T02:19:50.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a0effd640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a00073680 0x7f0a00075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:50.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1dd9b640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a00073680 0x7f0a00075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:50.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4943+0+0 (secure 0 0 0) 0x7f0a100955a0 con 0x7f0a200b1230 2024-09-21T02:19:50.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.410+0000 7f0a1dd9b640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a00073680 0x7f0a00075b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a20162470 tx=0x7f0a14009060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:50.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.414+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0a201fd090 con 0x7f0a200b1230 2024-09-21T02:19:50.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.590+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0a200ce8c0 con 0x7f0a200b1230 2024-09-21T02:19:50.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.814+0000 7f0a0effd640 1 -- 172.21.15.170:0/4088817357 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) ==== 97+0+0 (secure 0 0 0) 0x7f0a10016030 con 0x7f0a200b1230 2024-09-21T02:19:50.814 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:19:50.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a00073680 msgr2=0x7f0a00075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a00073680 0x7f0a00075b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a20162470 tx=0x7f0a14009060 comp rx=0 tx=0).stop 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 msgr2=0x7f0a201fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a201fcb50 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000ea20 tx=0x7f0a1000eee0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 shutdown_connections 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a00073680 0x7f0a00075b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a200b3fa0 0x7f0a200b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a200d0e50 0x7f0a2006dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 --2- 172.21.15.170:0/4088817357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a200b1230 0x7f0a201fcb50 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 >> 172.21.15.170:0/4088817357 conn(0x7f0a2008bf50 msgr2=0x7f0a2015ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 shutdown_connections 2024-09-21T02:19:50.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.818+0000 7f0a24fb7640 1 -- 172.21.15.170:0/4088817357 wait complete. 2024-09-21T02:19:50.840 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:19:50.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 -- 172.21.15.170:0/3789505316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b2f40 msgr2=0x7fe1600b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:50.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 --2- 172.21.15.170:0/3789505316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b2f40 0x7fe1600b3350 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe160071d80 tx=0x7fe14801c800 comp rx=0 tx=0).stop 2024-09-21T02:19:50.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 -- 172.21.15.170:0/3789505316 shutdown_connections 2024-09-21T02:19:50.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 --2- 172.21.15.170:0/3789505316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b2f40 0x7fe1600b3350 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 --2- 172.21.15.170:0/3789505316 >> v2:172.21.15.191:3300/0 conn(0x7fe1600b5410 0x7fe16006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:50.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 -- 172.21.15.170:0/3789505316 >> 172.21.15.170:0/3789505316 conn(0x7fe1600b1c40 msgr2=0x7fe1600b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:50.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 -- 172.21.15.170:0/3789505316 shutdown_connections 2024-09-21T02:19:50.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 -- 172.21.15.170:0/3789505316 wait complete. 2024-09-21T02:19:50.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 Processor -- start 2024-09-21T02:19:50.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:50.998+0000 7fe16538f640 1 -- start start 2024-09-21T02:19:51.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1600b2f40 0x7fe1600c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 0x7fe1600b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1600ccb00 0x7fe16014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe160169fd0 con 0x7fe1600b5410 2024-09-21T02:19:51.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe160169e50 con 0x7fe1600b2f40 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe16016a150 con 0x7fe1600ccb00 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 0x7fe1600b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 0x7fe1600b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51922/0 (socket says 172.21.15.170:51922) 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 -- 172.21.15.170:0/4175379647 learned_addr learned my addr 172.21.15.170:0/4175379647 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15effd640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1600b2f40 0x7fe1600c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15f7fe640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1600ccb00 0x7fe16014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 -- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1600ccb00 msgr2=0x7fe16014fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1600ccb00 0x7fe16014fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 -- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1600b2f40 msgr2=0x7fe1600c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1600b2f40 0x7fe1600c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 -- 172.21.15.170:0/4175379647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1600c5d70 con 0x7fe1600b5410 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15effd640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1600b2f40 0x7fe1600c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe15e7fc640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 0x7fe1600b3c50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe1480982f0 tx=0x7fe148005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b2040 con 0x7fe1600b5410 2024-09-21T02:19:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1480bb070 con 0x7fe1600b5410 2024-09-21T02:19:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480a5bb0 con 0x7fe1600b5410 2024-09-21T02:19:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601f77e0 con 0x7fe1600b5410 2024-09-21T02:19:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1600c5a90 con 0x7fe1600b5410 2024-09-21T02:19:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1601f7550 con 0x7fe1600b5410 2024-09-21T02:19:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fe148003600 con 0x7fe1600b5410 2024-09-21T02:19:51.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.002+0000 7fe1577fe640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe140073680 0x7fe140075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.006+0000 7fe15effd640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe140073680 0x7fe140075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.006+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7fe14812ebd0 con 0x7fe1600b5410 2024-09-21T02:19:51.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.006+0000 7fe15effd640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe140073680 0x7fe140075b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe150009d90 tx=0x7fe150009320 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:51.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.010+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe1480163f0 con 0x7fe1600b5410 2024-09-21T02:19:51.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe1600b3350 con 0x7fe1600b5410 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe1577fe640 1 -- 172.21.15.170:0/4175379647 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) ==== 80+0+5304 (secure 0 0 0) 0x7fe1480f9930 con 0x7fe1600b5410 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.245 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:19:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.249 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:19:51.251 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:19:51.253 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.254 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:19:51.255 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.256 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe140073680 msgr2=0x7fe140075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe140073680 0x7fe140075b90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe150009d90 tx=0x7fe150009320 comp rx=0 tx=0).stop 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 msgr2=0x7fe1600b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 0x7fe1600b3c50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe1480982f0 tx=0x7fe148005760 comp rx=0 tx=0).stop 2024-09-21T02:19:51.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 shutdown_connections 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe140073680 0x7fe140075b90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1600ccb00 0x7fe16014fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1600b5410 0x7fe1600b3c50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 --2- 172.21.15.170:0/4175379647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1600b2f40 0x7fe1600c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 >> 172.21.15.170:0/4175379647 conn(0x7fe1600b1c40 msgr2=0x7fe1600c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 shutdown_connections 2024-09-21T02:19:51.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.246+0000 7fe16538f640 1 -- 172.21.15.170:0/4175379647 wait complete. 2024-09-21T02:19:51.268 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- 172.21.15.170:0/3542874879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91280ce4c0 msgr2=0x7f9110004b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 --2- 172.21.15.170:0/3542874879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91280ce4c0 0x7f9110004b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f911c0016c0 tx=0x7f911c01c790 comp rx=0 tx=0).stop 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f9126d9d640 1 -- 172.21.15.170:0/3542874879 reap_dead start 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- 172.21.15.170:0/3542874879 shutdown_connections 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- 172.21.15.170:0/3542874879 >> 172.21.15.170:0/3542874879 conn(0x7f91280b1c40 msgr2=0x7f91280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- 172.21.15.170:0/3542874879 shutdown_connections 2024-09-21T02:19:51.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- 172.21.15.170:0/3542874879 wait complete. 2024-09-21T02:19:51.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 Processor -- start 2024-09-21T02:19:51.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- start start 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 0x7f912809cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f912809d2e0 0x7f91281f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91281f41f0 0x7f91280c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f912809dd70 con 0x7f91280ce4c0 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9128074f00 con 0x7f91281f41f0 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.418+0000 7f912c9d3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f91280ce8d0 con 0x7f912809d2e0 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 0x7f912809cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 0x7f912809cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51944/0 (socket says 172.21.15.170:51944) 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 -- 172.21.15.170:0/2981988175 learned_addr learned my addr 172.21.15.170:0/2981988175 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9126d9d640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91281f41f0 0x7f91280c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9125d9b640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f912809d2e0 0x7f91281f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 -- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f912809d2e0 msgr2=0x7f91281f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f912809d2e0 0x7f91281f3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 -- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91281f41f0 msgr2=0x7f91280c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91281f41f0 0x7f91280c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 -- 172.21.15.170:0/2981988175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91281f49e0 con 0x7f91280ce4c0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912659c640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 0x7f912809cda0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f911c006040 tx=0x7f911c0013b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c003810 con 0x7f91280ce4c0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f911c003aa0 con 0x7f91280ce4c0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91280c3dc0 con 0x7f91280ce4c0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c0a55f0 con 0x7f91280ce4c0 2024-09-21T02:19:51.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91280c4380 con 0x7f91280ce4c0 2024-09-21T02:19:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9128070030 con 0x7f91280ce4c0 2024-09-21T02:19:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f911c0a5790 con 0x7f91280ce4c0 2024-09-21T02:19:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9116ffd640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9108073680 0x7f9108075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9125d9b640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9108073680 0x7f9108075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f911c12e760 con 0x7f91280ce4c0 2024-09-21T02:19:51.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.422+0000 7f9125d9b640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9108073680 0x7f9108075b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f91281eada0 tx=0x7f91180091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:51.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.430+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f911c0164f0 con 0x7f91280ce4c0 2024-09-21T02:19:51.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.598+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f91280ce2a0 con 0x7f91280ce4c0 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.598+0000 7f9116ffd640 1 -- 172.21.15.170:0/2981988175 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7f911c0f9310 con 0x7f91280ce4c0 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:19:51.600 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:19:51.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9108073680 msgr2=0x7f9108075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9108073680 0x7f9108075b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f91281eada0 tx=0x7f91180091e0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 msgr2=0x7f912809cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 0x7f912809cda0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f911c006040 tx=0x7f911c0013b0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 shutdown_connections 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9108073680 0x7f9108075b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f912809d2e0 0x7f91281f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91281f41f0 0x7f91280c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 --2- 172.21.15.170:0/2981988175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91280ce4c0 0x7f912809cda0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 >> 172.21.15.170:0/2981988175 conn(0x7f91280b1c40 msgr2=0x7f91280b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 shutdown_connections 2024-09-21T02:19:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.602+0000 7f912c9d3640 1 -- 172.21.15.170:0/2981988175 wait complete. 2024-09-21T02:19:51.621 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:19:51.621 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:19:51.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 -- 172.21.15.170:0/1112364408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc09dc00 msgr2=0x7f06fc15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 --2- 172.21.15.170:0/1112364408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc09dc00 0x7f06fc15ad30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0ce1d0 tx=0x7f06f801d300 comp rx=0 tx=0).stop 2024-09-21T02:19:51.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 -- 172.21.15.170:0/1112364408 shutdown_connections 2024-09-21T02:19:51.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 --2- 172.21.15.170:0/1112364408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06fc15e9f0 0x7f06fc15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 --2- 172.21.15.170:0/1112364408 >> v2:172.21.15.191:3300/0 conn(0x7f06fc0ca720 0x7f06fc0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 --2- 172.21.15.170:0/1112364408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc09dc00 0x7f06fc15ad30 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 -- 172.21.15.170:0/1112364408 >> 172.21.15.170:0/1112364408 conn(0x7f06fc0b1c40 msgr2=0x7f06fc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:51.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 -- 172.21.15.170:0/1112364408 shutdown_connections 2024-09-21T02:19:51.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.766+0000 7f0703d10640 1 -- 172.21.15.170:0/1112364408 wait complete. 2024-09-21T02:19:51.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 Processor -- start 2024-09-21T02:19:51.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- start start 2024-09-21T02:19:51.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06fc09dc00 0x7f06fc14a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 0x7f06fc15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06fc15e9f0 0x7f06fc09cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc0701d0 con 0x7f06fc0ca720 2024-09-21T02:19:51.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc070050 con 0x7f06fc09dc00 2024-09-21T02:19:51.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc070350 con 0x7f06fc15e9f0 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 0x7f06fc15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 0x7f06fc15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51956/0 (socket says 172.21.15.170:51956) 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 -- 172.21.15.170:0/701481732 learned_addr learned my addr 172.21.15.170:0/701481732 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0701aac640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06fc09dc00 0x7f06fc14a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07022ad640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06fc15e9f0 0x7f06fc09cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 -- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06fc15e9f0 msgr2=0x7f06fc09cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06fc15e9f0 0x7f06fc09cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 -- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06fc09dc00 msgr2=0x7f06fc14a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06fc09dc00 0x7f06fc14a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 -- 172.21.15.170:0/701481732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc15f170 con 0x7f06fc0ca720 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0701aac640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06fc09dc00 0x7f06fc14a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f07012ab640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 0x7f06fc15dd00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00c7a0 tx=0x7f06ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:51.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec007bb0 con 0x7f06fc0ca720 2024-09-21T02:19:51.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06ec007e40 con 0x7f06fc0ca720 2024-09-21T02:19:51.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc14cd00 con 0x7f06fc0ca720 2024-09-21T02:19:51.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec005280 con 0x7f06fc0ca720 2024-09-21T02:19:51.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc14d2c0 con 0x7f06fc0ca720 2024-09-21T02:19:51.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.770+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc0cab30 con 0x7f06fc0ca720 2024-09-21T02:19:51.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.774+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f06ec005420 con 0x7f06fc0ca720 2024-09-21T02:19:51.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.774+0000 7f06f27fc640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06e0073680 0x7f06e0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:51.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.774+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4857+0+0 (secure 0 0 0) 0x7f06ec094ba0 con 0x7f06fc0ca720 2024-09-21T02:19:51.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.774+0000 7f0701aac640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06e0073680 0x7f06e0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:51.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.774+0000 7f0701aac640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06e0073680 0x7f06e0075b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06f80af000 tx=0x7f06f801b0e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:51.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.778+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f06ec014030 con 0x7f06fc0ca720 2024-09-21T02:19:51.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:51.954+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f06fc165b40 con 0x7f06fc0ca720 2024-09-21T02:19:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.854+0000 7f06f27fc640 1 -- 172.21.15.170:0/701481732 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f06ec05f860 con 0x7f06fc0ca720 2024-09-21T02:19:52.854 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06e0073680 msgr2=0x7f06e0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06e0073680 0x7f06e0075b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06f80af000 tx=0x7f06f801b0e0 comp rx=0 tx=0).stop 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 msgr2=0x7f06fc15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 0x7f06fc15dd00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00c7a0 tx=0x7f06ec00cc60 comp rx=0 tx=0).stop 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 shutdown_connections 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f06e0073680 0x7f06e0075b90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06fc15e9f0 0x7f06fc09cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06fc0ca720 0x7f06fc15dd00 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 --2- 172.21.15.170:0/701481732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06fc09dc00 0x7f06fc14a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 >> 172.21.15.170:0/701481732 conn(0x7f06fc0b1c40 msgr2=0x7f06fc073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:52.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 shutdown_connections 2024-09-21T02:19:52.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:52.858+0000 7f0703d10640 1 -- 172.21.15.170:0/701481732 wait complete. 2024-09-21T02:19:52.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:19:53.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- 172.21.15.170:0/3636684261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b66e0 msgr2=0x7f1f4c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:53.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/3636684261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b66e0 0x7f1f4c06e650 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c08d510 tx=0x7f1f4401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- 172.21.15.170:0/3636684261 shutdown_connections 2024-09-21T02:19:53.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/3636684261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b66e0 0x7f1f4c06e650 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/3636684261 >> v2:172.21.15.191:3300/0 conn(0x7f1f4c0d4c60 0x7f1f4c0d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- 172.21.15.170:0/3636684261 >> 172.21.15.170:0/3636684261 conn(0x7f1f4c09d8c0 msgr2=0x7f1f4c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:53.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- 172.21.15.170:0/3636684261 shutdown_connections 2024-09-21T02:19:53.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- 172.21.15.170:0/3636684261 wait complete. 2024-09-21T02:19:53.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 Processor -- start 2024-09-21T02:19:53.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- start start 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 0x7f1f4c151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0d4c60 0x7f1f4c161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0b4690 0x7f1f4c161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c16c160 con 0x7f1f4c0b9660 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c16bfe0 con 0x7f1f4c0b4690 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f51d6f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c16c2e0 con 0x7f1f4c0d4c60 2024-09-21T02:19:53.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 0x7f1f4c151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 0x7f1f4c151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51972/0 (socket says 172.21.15.170:51972) 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 -- 172.21.15.170:0/1524002003 learned_addr learned my addr 172.21.15.170:0/1524002003 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 -- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0d4c60 msgr2=0x7f1f4c161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4affd640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0d4c60 0x7f1f4c161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4bfff640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0b4690 0x7f1f4c161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0d4c60 0x7f1f4c161550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 -- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0b4690 msgr2=0x7f1f4c161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0b4690 0x7f1f4c161a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.050+0000 7f1f4b7fe640 1 -- 172.21.15.170:0/1524002003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c0b4d80 con 0x7f1f4c0b9660 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f4b7fe640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 0x7f1f4c151a50 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00ba50 tx=0x7f1f3c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c019070 con 0x7f1f4c0b9660 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f3c00ccf0 con 0x7f1f4c0b9660 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c150c90 con 0x7f1f4c0b9660 2024-09-21T02:19:53.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0146a0 con 0x7f1f4c0b9660 2024-09-21T02:19:53.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c151250 con 0x7f1f4c0b9660 2024-09-21T02:19:53.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c06e650 con 0x7f1f4c0b9660 2024-09-21T02:19:53.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f1f3c01c070 con 0x7f1f4c0b9660 2024-09-21T02:19:53.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f33fff640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f380736b0 0x7f1f38075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:53.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5227+0+0 (secure 0 0 0) 0x7f1f3c094d40 con 0x7f1f4c0b9660 2024-09-21T02:19:53.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f4affd640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f380736b0 0x7f1f38075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:53.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.054+0000 7f1f4affd640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f380736b0 0x7f1f38075bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0b5e60 tx=0x7f1f40008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:53.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.058+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1f3c023070 con 0x7f1f4c0b9660 2024-09-21T02:19:53.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.218+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1f4c0b6e80 con 0x7f1f4c0b9660 2024-09-21T02:19:53.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.866+0000 7f1f33fff640 1 -- 172.21.15.170:0/1524002003 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7f1f3c05f990 con 0x7f1f4c0b9660 2024-09-21T02:19:53.866 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:19:53.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f380736b0 msgr2=0x7f1f38075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:53.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f380736b0 0x7f1f38075bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0b5e60 tx=0x7f1f40008040 comp rx=0 tx=0).stop 2024-09-21T02:19:53.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 msgr2=0x7f1f4c151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:53.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 0x7f1f4c151a50 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00ba50 tx=0x7f1f3c00bf10 comp rx=0 tx=0).stop 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 shutdown_connections 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f380736b0 0x7f1f38075bc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0d4c60 0x7f1f4c161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0b4690 0x7f1f4c161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 --2- 172.21.15.170:0/1524002003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f4c0b9660 0x7f1f4c151a50 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 >> 172.21.15.170:0/1524002003 conn(0x7f1f4c09d8c0 msgr2=0x7f1f4c0b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 shutdown_connections 2024-09-21T02:19:53.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:53.870+0000 7f1f51d6f640 1 -- 172.21.15.170:0/1524002003 wait complete. 2024-09-21T02:19:53.911 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 -- 172.21.15.170:0/5394689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140d4c20 msgr2=0x7f71141694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 --2- 172.21.15.170:0/5394689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140d4c20 0x7f71141694f0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f710c009c80 tx=0x7f710c01cda0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 -- 172.21.15.170:0/5394689 shutdown_connections 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 --2- 172.21.15.170:0/5394689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140d4c20 0x7f71141694f0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 -- 172.21.15.170:0/5394689 >> 172.21.15.170:0/5394689 conn(0x7f711408bf50 msgr2=0x7f711409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 -- 172.21.15.170:0/5394689 shutdown_connections 2024-09-21T02:19:54.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.070+0000 7f711c4dd640 1 -- 172.21.15.170:0/5394689 wait complete. 2024-09-21T02:19:54.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 Processor -- start 2024-09-21T02:19:54.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 -- start start 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71140d17c0 0x7f71140b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71140d1bd0 0x7f71140cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 0x7f71140cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f711416d460 con 0x7f71140b7c20 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f711416d2e0 con 0x7f71140d17c0 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f711416d5e0 con 0x7f71140d1bd0 2024-09-21T02:19:54.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 0x7f71140cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 0x7f71140cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51980/0 (socket says 172.21.15.170:51980) 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 -- 172.21.15.170:0/2547678112 learned_addr learned my addr 172.21.15.170:0/2547678112 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711a279640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71140d17c0 0x7f71140b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f7119a78640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71140d1bd0 0x7f71140cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 -- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71140d1bd0 msgr2=0x7f71140cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71140d1bd0 0x7f71140cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 -- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71140d17c0 msgr2=0x7f71140b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71140d17c0 0x7f71140b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 -- 172.21.15.170:0/2547678112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711415a420 con 0x7f71140b7c20 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711a279640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71140d17c0 0x7f71140b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711aa7a640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 0x7f71140cd9f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f710c01a9d0 tx=0x7f710c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0bc070 con 0x7f71140b7c20 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f710c0042f0 con 0x7f71140b7c20 2024-09-21T02:19:54.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71140b8450 con 0x7f71140b7c20 2024-09-21T02:19:54.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0b2040 con 0x7f71140b7c20 2024-09-21T02:19:54.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.074+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71140b8a10 con 0x7f71140b7c20 2024-09-21T02:19:54.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.078+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71140b81c0 con 0x7f71140b7c20 2024-09-21T02:19:54.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.078+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f710c0ad440 con 0x7f71140b7c20 2024-09-21T02:19:54.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.078+0000 7f710affd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70f80736b0 0x7f70f8075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:54.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.078+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5609+0+0 (secure 0 0 0) 0x7f710c12ecb0 con 0x7f71140b7c20 2024-09-21T02:19:54.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.078+0000 7f711a279640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70f80736b0 0x7f70f8075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:54.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.078+0000 7f711a279640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70f80736b0 0x7f70f8075bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71040016b0 tx=0x7f7104009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:54.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.082+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f710c0c5080 con 0x7f71140b7c20 2024-09-21T02:19:54.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.298+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f71140b3a10 con 0x7f71140b7c20 2024-09-21T02:19:54.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:19:54.634 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:19:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.870+0000 7f710affd640 1 -- 172.21.15.170:0/2547678112 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v77) ==== 174+0+0 (secure 0 0 0) 0x7f710c016990 con 0x7f71140b7c20 2024-09-21T02:19:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 18 and data pool 19 2024-09-21T02:19:54.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70f80736b0 msgr2=0x7f70f8075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:54.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70f80736b0 0x7f70f8075bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71040016b0 tx=0x7f7104009040 comp rx=0 tx=0).stop 2024-09-21T02:19:54.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 msgr2=0x7f71140cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:54.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 0x7f71140cd9f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f710c01a9d0 tx=0x7f710c0093d0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 shutdown_connections 2024-09-21T02:19:54.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70f80736b0 0x7f70f8075bc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71140d1bd0 0x7f71140cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71140b7c20 0x7f71140cd9f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 --2- 172.21.15.170:0/2547678112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71140d17c0 0x7f71140b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:54.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.874+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 >> 172.21.15.170:0/2547678112 conn(0x7f711408bf50 msgr2=0x7f71140b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:54.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.878+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 shutdown_connections 2024-09-21T02:19:54.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:54.878+0000 7f711c4dd640 1 -- 172.21.15.170:0/2547678112 wait complete. 2024-09-21T02:19:54.927 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:19:55.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 -- 172.21.15.170:0/775779282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 msgr2=0x7fb0100cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 --2- 172.21.15.170:0/775779282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 0x7fb0100cff60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0100d3180 tx=0x7faff801c820 comp rx=0 tx=0).stop 2024-09-21T02:19:55.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 -- 172.21.15.170:0/775779282 shutdown_connections 2024-09-21T02:19:55.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 --2- 172.21.15.170:0/775779282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 0x7fb0100cff60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 --2- 172.21.15.170:0/775779282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0100b9670 0x7fb0101620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 -- 172.21.15.170:0/775779282 >> 172.21.15.170:0/775779282 conn(0x7fb01009d850 msgr2=0x7fb01008bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:55.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 -- 172.21.15.170:0/775779282 shutdown_connections 2024-09-21T02:19:55.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 -- 172.21.15.170:0/775779282 wait complete. 2024-09-21T02:19:55.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 Processor -- start 2024-09-21T02:19:55.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.078+0000 7fb015d2d640 1 -- start start 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 0x7fb01014e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0100b9670 0x7fb01015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 0x7fb01015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb01016bef0 con 0x7fb01006dec0 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb01006e7e0 con 0x7fb0100cfb50 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb01016c070 con 0x7fb0100b9670 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 0x7fb01014e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 0x7fb01014e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51986/0 (socket says 172.21.15.170:51986) 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 -- 172.21.15.170:0/3804886457 learned_addr learned my addr 172.21.15.170:0/3804886457 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00effd640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0100b9670 0x7fb01015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00ffff640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 0x7fb01015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 -- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0100b9670 msgr2=0x7fb01015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0100b9670 0x7fb01015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 -- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 msgr2=0x7fb01015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 0x7fb01015e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 -- 172.21.15.170:0/3804886457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0100ce760 con 0x7fb01006dec0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb00f7fe640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 0x7fb01014e180 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb00800ee50 tx=0x7fb00800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00801b070 con 0x7fb01006dec0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb00800b570 con 0x7fb01006dec0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01014e6c0 con 0x7fb01006dec0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00800b9c0 con 0x7fb01006dec0 2024-09-21T02:19:55.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb010158220 con 0x7fb01006dec0 2024-09-21T02:19:55.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0101620c0 con 0x7fb01006dec0 2024-09-21T02:19:55.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.082+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fb00801e070 con 0x7fb01006dec0 2024-09-21T02:19:55.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.086+0000 7fb007fff640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faff00736b0 0x7faff0075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.086+0000 7fb00effd640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faff00736b0 0x7faff0075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.086+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7fb008095150 con 0x7fb01006dec0 2024-09-21T02:19:55.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.086+0000 7fb00effd640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faff00736b0 0x7faff0075bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faff80af000 tx=0x7faff801b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:55.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.090+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb00805f350 con 0x7fb01006dec0 2024-09-21T02:19:55.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.250+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0101613c0 con 0x7fb01006dec0 2024-09-21T02:19:55.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb007fff640 1 -- 172.21.15.170:0/3804886457 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16748 (secure 0 0 0) 0x7fb008063d80 con 0x7fb01006dec0 2024-09-21T02:19:55.253 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:55.254 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":117,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:54.300195+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:19:51.966032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T02:19:53.233775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faff00736b0 msgr2=0x7faff0075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faff00736b0 0x7faff0075bc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faff80af000 tx=0x7faff801b060 comp rx=0 tx=0).stop 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 msgr2=0x7fb01014e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 0x7fb01014e180 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb00800ee50 tx=0x7fb00800c740 comp rx=0 tx=0).stop 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 shutdown_connections 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faff00736b0 0x7faff0075bc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0100b9670 0x7fb01015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0100cfb50 0x7fb01015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 --2- 172.21.15.170:0/3804886457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01006dec0 0x7fb01014e180 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.254+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 >> 172.21.15.170:0/3804886457 conn(0x7fb01009d850 msgr2=0x7fb0100b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:55.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.258+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 shutdown_connections 2024-09-21T02:19:55.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.258+0000 7fb015d2d640 1 -- 172.21.15.170:0/3804886457 wait complete. 2024-09-21T02:19:55.275 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:19:55.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 -- 172.21.15.170:0/2921970734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d3630 msgr2=0x7ff9280d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 --2- 172.21.15.170:0/2921970734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d3630 0x7ff9280d0d80 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff92808cec0 tx=0x7ff92001ccc0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 -- 172.21.15.170:0/2921970734 shutdown_connections 2024-09-21T02:19:55.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 --2- 172.21.15.170:0/2921970734 >> v2:172.21.15.191:3300/0 conn(0x7ff9280b6500 0x7ff9281691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 --2- 172.21.15.170:0/2921970734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d3630 0x7ff9280d0d80 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 -- 172.21.15.170:0/2921970734 >> 172.21.15.170:0/2921970734 conn(0x7ff92809d830 msgr2=0x7ff92808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:55.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 -- 172.21.15.170:0/2921970734 shutdown_connections 2024-09-21T02:19:55.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.426+0000 7ff92f15a640 1 -- 172.21.15.170:0/2921970734 wait complete. 2024-09-21T02:19:55.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 Processor -- start 2024-09-21T02:19:55.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 -- start start 2024-09-21T02:19:55.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9280b6500 0x7ff92815fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 0x7ff9280b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9280d3630 0x7ff9280b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff92816c090 con 0x7ff9280d12c0 2024-09-21T02:19:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff92816bf10 con 0x7ff9280b6500 2024-09-21T02:19:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff92816c210 con 0x7ff9280d3630 2024-09-21T02:19:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 0x7ff9280b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92cef6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9280b6500 0x7ff92815fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 0x7ff9280b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59008/0 (socket says 172.21.15.170:59008) 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 -- 172.21.15.170:0/3095979965 learned_addr learned my addr 172.21.15.170:0/3095979965 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92d6f7640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9280d3630 0x7ff9280b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 -- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9280d3630 msgr2=0x7ff9280b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9280d3630 0x7ff9280b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 -- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9280b6500 msgr2=0x7ff92815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9280b6500 0x7ff92815fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 -- 172.21.15.170:0/3095979965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff928164860 con 0x7ff9280d12c0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff927fff640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 0x7ff9280b18b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b3e0 tx=0x7ff91000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910015020 con 0x7ff9280d12c0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff910011920 con 0x7ff9280d12c0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910011db0 con 0x7ff9280d12c0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff928166d10 con 0x7ff9280d12c0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.430+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff928160eb0 con 0x7ff9280d12c0 2024-09-21T02:19:55.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.434+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9280d0d80 con 0x7ff9280d12c0 2024-09-21T02:19:55.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.434+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7ff910018020 con 0x7ff9280d12c0 2024-09-21T02:19:55.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.434+0000 7ff9257fa640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9080736b0 0x7ff908075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.434+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7ff910094980 con 0x7ff9280d12c0 2024-09-21T02:19:55.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.434+0000 7ff92cef6640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9080736b0 0x7ff908075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.434+0000 7ff92cef6640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9080736b0 0x7ff908075bc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9200af000 tx=0x7ff92001b500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:55.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.438+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff91001f030 con 0x7ff9280d12c0 2024-09-21T02:19:55.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.614+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9280b9120 con 0x7ff9280d12c0 2024-09-21T02:19:55.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.614+0000 7ff9257fa640 1 -- 172.21.15.170:0/3095979965 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16748 (secure 0 0 0) 0x7ff91007e180 con 0x7ff9280d12c0 2024-09-21T02:19:55.614 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:55.614 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":117,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:54.300195+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:19:51.966032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T02:19:53.233775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9080736b0 msgr2=0x7ff908075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9080736b0 0x7ff908075bc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9200af000 tx=0x7ff92001b500 comp rx=0 tx=0).stop 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 msgr2=0x7ff9280b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 0x7ff9280b18b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b3e0 tx=0x7ff91000b8a0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 shutdown_connections 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9080736b0 0x7ff908075bc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9280d3630 0x7ff9280b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9280d12c0 0x7ff9280b18b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 --2- 172.21.15.170:0/3095979965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9280b6500 0x7ff92815fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 >> 172.21.15.170:0/3095979965 conn(0x7ff92809d830 msgr2=0x7ff928162630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 shutdown_connections 2024-09-21T02:19:55.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.618+0000 7ff92f15a640 1 -- 172.21.15.170:0/3095979965 wait complete. 2024-09-21T02:19:55.638 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:19:55.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 -- 172.21.15.170:0/2643897878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d1bd0 msgr2=0x7f50900d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 --2- 172.21.15.170:0/2643897878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d1bd0 0x7f50900d4510 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5084009900 tx=0x7f508401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 -- 172.21.15.170:0/2643897878 shutdown_connections 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 --2- 172.21.15.170:0/2643897878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d1bd0 0x7f50900d4510 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 --2- 172.21.15.170:0/2643897878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50900d17c0 0x7f50900b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 -- 172.21.15.170:0/2643897878 >> 172.21.15.170:0/2643897878 conn(0x7f509008bf50 msgr2=0x7f509009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 -- 172.21.15.170:0/2643897878 shutdown_connections 2024-09-21T02:19:55.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 -- 172.21.15.170:0/2643897878 wait complete. 2024-09-21T02:19:55.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 Processor -- start 2024-09-21T02:19:55.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.786+0000 7f50968dc640 1 -- start start 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50900d17c0 0x7f5090158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50900d1bd0 0x7f50900cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 0x7f50900cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f509016bf90 con 0x7f50900d4a50 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f509016be10 con 0x7f50900d1bd0 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f509016c110 con 0x7f50900d17c0 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 0x7f50900cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 0x7f50900cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59032/0 (socket says 172.21.15.170:59032) 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 -- 172.21.15.170:0/1337613493 learned_addr learned my addr 172.21.15.170:0/1337613493 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:55.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f508ffff640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50900d17c0 0x7f5090158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 -- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50900d17c0 msgr2=0x7f5090158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f508f7fe640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50900d1bd0 0x7f50900cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50900d17c0 0x7f5090158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 -- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50900d1bd0 msgr2=0x7f50900cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50900d1bd0 0x7f50900cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 -- 172.21.15.170:0/1337613493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50901598c0 con 0x7f50900d4a50 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f5094e79640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 0x7f50900cb9d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f508800c6b0 tx=0x7f508800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088017070 con 0x7f50900d4a50 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5088003c70 con 0x7f50900d4a50 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50900b85c0 con 0x7f50900d4a50 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088005350 con 0x7f50900d4a50 2024-09-21T02:19:55.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5090162160 con 0x7f50900d4a50 2024-09-21T02:19:55.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.790+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5090159d50 con 0x7f50900d4a50 2024-09-21T02:19:55.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.794+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f508801a070 con 0x7f50900d4a50 2024-09-21T02:19:55.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.794+0000 7f508cff9640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f507c0736b0 0x7f507c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:55.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.794+0000 7f508ffff640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f507c0736b0 0x7f507c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:55.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.794+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5709+0+0 (secure 0 0 0) 0x7f5088094520 con 0x7f50900d4a50 2024-09-21T02:19:55.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.794+0000 7f508ffff640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f507c0736b0 0x7f507c075bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5080005fd0 tx=0x7f508000c460 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:55.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.798+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5088021020 con 0x7f50900d4a50 2024-09-21T02:19:55.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:55.994+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f50900b3a10 con 0x7f50900d4a50 2024-09-21T02:19:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.926+0000 7f508cff9640 1 -- 172.21.15.170:0/1337613493 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7f50900d4e60 con 0x7f50900d4a50 2024-09-21T02:19:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f507c0736b0 msgr2=0x7f507c075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f507c0736b0 0x7f507c075bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5080005fd0 tx=0x7f508000c460 comp rx=0 tx=0).stop 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 msgr2=0x7f50900cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 0x7f50900cb9d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f508800c6b0 tx=0x7f508800cb70 comp rx=0 tx=0).stop 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 shutdown_connections 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f507c0736b0 0x7f507c075bc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50900d4a50 0x7f50900cb9d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50900d1bd0 0x7f50900cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 --2- 172.21.15.170:0/1337613493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50900d17c0 0x7f5090158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 >> 172.21.15.170:0/1337613493 conn(0x7f509008bf50 msgr2=0x7f50900b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 shutdown_connections 2024-09-21T02:19:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:56.930+0000 7f50968dc640 1 -- 172.21.15.170:0/1337613493 wait complete. 2024-09-21T02:19:56.952 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:19:56.952 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:19:57.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 -- 172.21.15.170:0/1938834125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 msgr2=0x7effb00744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:57.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 --2- 172.21.15.170:0/1938834125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb00744b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7effb00726b0 tx=0x7eff9801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 -- 172.21.15.170:0/1938834125 shutdown_connections 2024-09-21T02:19:57.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 --2- 172.21.15.170:0/1938834125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effb00ce070 0x7effb00ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 --2- 172.21.15.170:0/1938834125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb00744b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 --2- 172.21.15.170:0/1938834125 >> v2:172.21.15.191:3300/0 conn(0x7effb00cf7d0 0x7effb0073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 -- 172.21.15.170:0/1938834125 >> 172.21.15.170:0/1938834125 conn(0x7effb00d0170 msgr2=0x7effb00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:57.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 -- 172.21.15.170:0/1938834125 shutdown_connections 2024-09-21T02:19:57.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 -- 172.21.15.170:0/1938834125 wait complete. 2024-09-21T02:19:57.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 Processor -- start 2024-09-21T02:19:57.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.114+0000 7effb5de1640 1 -- start start 2024-09-21T02:19:57.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb0152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:57.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effb00ce070 0x7effb01f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:57.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effb00cf7d0 0x7effb015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:57.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7effb00cbe60 con 0x7effb00740a0 2024-09-21T02:19:57.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7effb00cbce0 con 0x7effb00ce070 2024-09-21T02:19:57.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7effb00cbfe0 con 0x7effb00cf7d0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb0152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb0152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59044/0 (socket says 172.21.15.170:59044) 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 -- 172.21.15.170:0/2511951132 learned_addr learned my addr 172.21.15.170:0/2511951132 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effa7fff640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effb00ce070 0x7effb01f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaffff640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effb00cf7d0 0x7effb015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 -- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effb00cf7d0 msgr2=0x7effb015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effb00cf7d0 0x7effb015d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 -- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effb00ce070 msgr2=0x7effb01f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effb00ce070 0x7effb01f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 -- 172.21.15.170:0/2511951132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb015e4c0 con 0x7effb00740a0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effaf7fe640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb0152d50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7effa000ca20 tx=0x7effa000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0007c70 con 0x7effb00740a0 2024-09-21T02:19:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effa000f450 con 0x7effb00740a0 2024-09-21T02:19:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb015cb50 con 0x7effb00740a0 2024-09-21T02:19:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa000f8a0 con 0x7effb00740a0 2024-09-21T02:19:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb00b1890 con 0x7effb00740a0 2024-09-21T02:19:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.118+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb00ce480 con 0x7effb00740a0 2024-09-21T02:19:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.122+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7effa0018400 con 0x7effb00740a0 2024-09-21T02:19:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.122+0000 7effacff9640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff900736b0 0x7eff90075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:57.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.122+0000 7effa7fff640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff900736b0 0x7eff90075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:57.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.122+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7effa00958b0 con 0x7effb00740a0 2024-09-21T02:19:57.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.122+0000 7effa7fff640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff900736b0 0x7eff90075bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff980af000 tx=0x7eff9801b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:57.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.126+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7effa0026020 con 0x7effb00740a0 2024-09-21T02:19:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.334+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7effb0156340 con 0x7effb00740a0 2024-09-21T02:19:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.934+0000 7effacff9640 1 -- 172.21.15.170:0/2511951132 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7effa0014030 con 0x7effb00740a0 2024-09-21T02:19:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff900736b0 msgr2=0x7eff90075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff900736b0 0x7eff90075bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7eff980af000 tx=0x7eff9801b100 comp rx=0 tx=0).stop 2024-09-21T02:19:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 msgr2=0x7effb0152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb0152d50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7effa000ca20 tx=0x7effa000cee0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 shutdown_connections 2024-09-21T02:19:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff900736b0 0x7eff90075bc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effb00cf7d0 0x7effb015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effb00ce070 0x7effb01f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 --2- 172.21.15.170:0/2511951132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effb00740a0 0x7effb0152d50 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 >> 172.21.15.170:0/2511951132 conn(0x7effb00d0170 msgr2=0x7effb0159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 shutdown_connections 2024-09-21T02:19:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:57.938+0000 7effb5de1640 1 -- 172.21.15.170:0/2511951132 wait complete. 2024-09-21T02:19:57.965 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:19:57.965 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:19:58.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/2051891761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf20004810 msgr2=0x7fcf20004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/2051891761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf20004810 0x7fcf20004c20 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fcf380d3bf0 tx=0x7fcf2c01ce80 comp rx=0 tx=0).stop 2024-09-21T02:19:58.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf37fff640 1 -- 172.21.15.170:0/2051891761 reap_dead start 2024-09-21T02:19:58.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/2051891761 shutdown_connections 2024-09-21T02:19:58.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/2051891761 >> 172.21.15.170:0/2051891761 conn(0x7fcf3809d830 msgr2=0x7fcf3808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:58.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/2051891761 shutdown_connections 2024-09-21T02:19:58.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.122+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/2051891761 wait complete. 2024-09-21T02:19:58.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 Processor -- start 2024-09-21T02:19:58.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 -- start start 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf20004810 0x7fcf381fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 0x7fcf381f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf38157770 0x7fcf3815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3816b0e0 con 0x7fcf380ce200 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf380b3f30 con 0x7fcf20004810 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3816b260 con 0x7fcf38157770 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 0x7fcf381f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 0x7fcf381f9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59056/0 (socket says 172.21.15.170:59056) 2024-09-21T02:19:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 -- 172.21.15.170:0/1960487540 learned_addr learned my addr 172.21.15.170:0/1960487540 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf377fe640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf20004810 0x7fcf381fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf37fff640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf38157770 0x7fcf3815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 -- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf38157770 msgr2=0x7fcf3815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf38157770 0x7fcf3815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 -- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf20004810 msgr2=0x7fcf381fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf20004810 0x7fcf381fd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 -- 172.21.15.170:0/1960487540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf380cd420 con 0x7fcf380ce200 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf36ffd640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 0x7fcf381f9b00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c01b5e0 tx=0x7fcf2c006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c006150 con 0x7fcf380ce200 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf2c0062f0 con 0x7fcf380ce200 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf380cd700 con 0x7fcf380ce200 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c003600 con 0x7fcf380ce200 2024-09-21T02:19:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.126+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf381f95e0 con 0x7fcf380ce200 2024-09-21T02:19:58.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.130+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf380cd940 con 0x7fcf380ce200 2024-09-21T02:19:58.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.130+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fcf2c0037a0 con 0x7fcf380ce200 2024-09-21T02:19:58.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.130+0000 7fcf1ffff640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcf240736b0 0x7fcf24075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.130+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fcf2c12ef20 con 0x7fcf380ce200 2024-09-21T02:19:58.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.130+0000 7fcf377fe640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcf240736b0 0x7fcf24075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.130+0000 7fcf377fe640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcf240736b0 0x7fcf24075bc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009ab0 tx=0x7fcf280091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:58.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.134+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcf380cd940 con 0x7fcf380ce200 2024-09-21T02:19:58.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.334+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf380b5df0 con 0x7fcf380ce200 2024-09-21T02:19:58.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.334+0000 7fcf1ffff640 1 -- 172.21.15.170:0/1960487540 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5228 (secure 0 0 0) 0x7fcf2c0c7080 con 0x7fcf380ce200 2024-09-21T02:19:58.336 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:58.336 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":81,"btime":"2024-09-21T02:19:57:928053+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:19:54.300110+0000","modified":"2024-09-21T02:19:57.337649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7614":{"gid":7614,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/300226457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":300226457},{"type":"v1","addr":"172.21.15.170:6835","nonce":300226457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7614,"qdb_cluster":[7614]},"id":9}]} 2024-09-21T02:19:58.336 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 81 2024-09-21T02:19:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcf240736b0 msgr2=0x7fcf24075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcf240736b0 0x7fcf24075bc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009ab0 tx=0x7fcf280091e0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 msgr2=0x7fcf381f9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 0x7fcf381f9b00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c01b5e0 tx=0x7fcf2c006c80 comp rx=0 tx=0).stop 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 shutdown_connections 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcf240736b0 0x7fcf24075bc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf38157770 0x7fcf3815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf380ce200 0x7fcf381f9b00 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 --2- 172.21.15.170:0/1960487540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf20004810 0x7fcf381fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 >> 172.21.15.170:0/1960487540 conn(0x7fcf3809d830 msgr2=0x7fcf38075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 shutdown_connections 2024-09-21T02:19:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.338+0000 7fcf3e0d0640 1 -- 172.21.15.170:0/1960487540 wait complete. 2024-09-21T02:19:58.356 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:19:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 -- 172.21.15.170:0/2174902273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 msgr2=0x7fc890070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 --2- 172.21.15.170:0/2174902273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 0x7fc890070340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009840 tx=0x7fc88c01c820 comp rx=0 tx=0).stop 2024-09-21T02:19:58.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 -- 172.21.15.170:0/2174902273 shutdown_connections 2024-09-21T02:19:58.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 --2- 172.21.15.170:0/2174902273 >> v2:172.21.15.191:3300/0 conn(0x7fc8900cb460 0x7fc8900cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 --2- 172.21.15.170:0/2174902273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 0x7fc890070340 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 -- 172.21.15.170:0/2174902273 >> 172.21.15.170:0/2174902273 conn(0x7fc8900b1c40 msgr2=0x7fc8900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:58.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 -- 172.21.15.170:0/2174902273 shutdown_connections 2024-09-21T02:19:58.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.510+0000 7fc897c53640 1 -- 172.21.15.170:0/2174902273 wait complete. 2024-09-21T02:19:58.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 Processor -- start 2024-09-21T02:19:58.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 -- start start 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 0x7fc8901466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8900cc8c0 0x7fc8901594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 0x7fc8900b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc890169180 con 0x7fc8900cb460 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc890169000 con 0x7fc8900cc8c0 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc897c53640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc890169300 con 0x7fc8900cf7e0 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 0x7fc8901466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 0x7fc8901466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59064/0 (socket says 172.21.15.170:59064) 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 -- 172.21.15.170:0/331572783 learned_addr learned my addr 172.21.15.170:0/331572783 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:58.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8951ee640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8900cc8c0 0x7fc8901594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8961f0640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 0x7fc8900b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 -- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 msgr2=0x7fc8900b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 0x7fc8900b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 -- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8900cc8c0 msgr2=0x7fc8901594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8900cc8c0 0x7fc8901594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.514+0000 7fc8959ef640 1 -- 172.21.15.170:0/331572783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8900c8cb0 con 0x7fc8900cb460 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8959ef640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 0x7fc8901466f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fc88c01aed0 tx=0x7fc88c0a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0bb070 con 0x7fc8900cb460 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc88c004470 con 0x7fc8900cb460 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc890146fa0 con 0x7fc8900cb460 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0b2040 con 0x7fc8900cb460 2024-09-21T02:19:58.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc890147560 con 0x7fc8900cb460 2024-09-21T02:19:58.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8900cb870 con 0x7fc8900cb460 2024-09-21T02:19:58.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fc88c0be070 con 0x7fc8900cb460 2024-09-21T02:19:58.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8867fc640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8740736b0 0x7fc874075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8951ee640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8740736b0 0x7fc874075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fc88c12e270 con 0x7fc8900cb460 2024-09-21T02:19:58.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.518+0000 7fc8951ee640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8740736b0 0x7fc874075bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8901f8eb0 tx=0x7fc880009260 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:58.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.522+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc88c016410 con 0x7fc8900cb460 2024-09-21T02:19:58.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.698+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc89006de10 con 0x7fc8900cb460 2024-09-21T02:19:58.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.698+0000 7fc8867fc640 1 -- 172.21.15.170:0/331572783 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16735 (secure 0 0 0) 0x7fc88c0f8ad0 con 0x7fc8900cb460 2024-09-21T02:19:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:58.699 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":118,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:55.871538+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:19:51.966032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T02:19:53.233775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8740736b0 msgr2=0x7fc874075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8740736b0 0x7fc874075bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc8901f8eb0 tx=0x7fc880009260 comp rx=0 tx=0).stop 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 msgr2=0x7fc8901466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 0x7fc8901466f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fc88c01aed0 tx=0x7fc88c0a5670 comp rx=0 tx=0).stop 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 shutdown_connections 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8740736b0 0x7fc874075bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8900cf7e0 0x7fc8900b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8900cc8c0 0x7fc8901594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 --2- 172.21.15.170:0/331572783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8900cb460 0x7fc8901466f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 >> 172.21.15.170:0/331572783 conn(0x7fc8900b1c40 msgr2=0x7fc8900cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:58.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 shutdown_connections 2024-09-21T02:19:58.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.702+0000 7fc897c53640 1 -- 172.21.15.170:0/331572783 wait complete. 2024-09-21T02:19:58.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:19:58.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:19:58.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff54ee9640 1 -- 172.21.15.170:0/1225943532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff38005160 msgr2=0x7eff3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff54ee9640 1 --2- 172.21.15.170:0/1225943532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff38005160 0x7eff3800b9d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c610 tx=0x7eff4801d300 comp rx=0 tx=0).stop 2024-09-21T02:19:58.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff4ed9d640 1 -- 172.21.15.170:0/1225943532 reap_dead start 2024-09-21T02:19:58.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff54ee9640 1 -- 172.21.15.170:0/1225943532 shutdown_connections 2024-09-21T02:19:58.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff54ee9640 1 -- 172.21.15.170:0/1225943532 >> 172.21.15.170:0/1225943532 conn(0x7eff500b1c40 msgr2=0x7eff500b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:58.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff54ee9640 1 -- 172.21.15.170:0/1225943532 shutdown_connections 2024-09-21T02:19:58.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.862+0000 7eff54ee9640 1 -- 172.21.15.170:0/1225943532 wait complete. 2024-09-21T02:19:58.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 Processor -- start 2024-09-21T02:19:58.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 -- start start 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff38005160 0x7eff5014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 0x7eff5015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff501f19b0 0x7eff501f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff500cb500 con 0x7eff5009ba10 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7eff50074f00 con 0x7eff501f19b0 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7eff500cb680 con 0x7eff38005160 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 0x7eff5015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 0x7eff5015ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59068/0 (socket says 172.21.15.170:59068) 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 -- 172.21.15.170:0/382506536 learned_addr learned my addr 172.21.15.170:0/382506536 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4e59c640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff38005160 0x7eff5014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4ed9d640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff501f19b0 0x7eff501f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 -- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff38005160 msgr2=0x7eff5014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff38005160 0x7eff5014dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 -- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff501f19b0 msgr2=0x7eff501f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff501f19b0 0x7eff501f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 -- 172.21.15.170:0/382506536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5009c220 con 0x7eff5009ba10 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff4dd9b640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 0x7eff5015ec40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7eff440107e0 tx=0x7eff44010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400ec80 con 0x7eff5009ba10 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff44010e90 con 0x7eff5009ba10 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5009c500 con 0x7eff5009ba10 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44005020 con 0x7eff5009ba10 2024-09-21T02:19:58.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.866+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5009cac0 con 0x7eff5009ba10 2024-09-21T02:19:58.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.870+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff500702b0 con 0x7eff5009ba10 2024-09-21T02:19:58.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.870+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7eff440051c0 con 0x7eff5009ba10 2024-09-21T02:19:58.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.870+0000 7eff3effd640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff300736e0 0x7eff30075bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:58.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.870+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7eff44095650 con 0x7eff5009ba10 2024-09-21T02:19:58.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.870+0000 7eff4e59c640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff300736e0 0x7eff30075bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:58.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.870+0000 7eff4e59c640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff300736e0 0x7eff30075bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff400058d0 tx=0x7eff4000a540 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:58.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:58.874+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7eff44060040 con 0x7eff5009ba10 2024-09-21T02:19:59.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.042+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff5009bfb0 con 0x7eff300736e0 2024-09-21T02:19:59.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.046+0000 7eff3effd640 1 -- 172.21.15.170:0/382506536 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247256 (secure 0 0 0) 0x7eff5009bfb0 con 0x7eff300736e0 2024-09-21T02:19:59.049 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:59.051 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:19:59.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff300736e0 msgr2=0x7eff30075bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff300736e0 0x7eff30075bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff400058d0 tx=0x7eff4000a540 comp rx=0 tx=0).stop 2024-09-21T02:19:59.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 msgr2=0x7eff5015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 0x7eff5015ec40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7eff440107e0 tx=0x7eff44010ca0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 shutdown_connections 2024-09-21T02:19:59.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7eff300736e0 0x7eff30075bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff501f19b0 0x7eff501f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5009ba10 0x7eff5015ec40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 --2- 172.21.15.170:0/382506536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff38005160 0x7eff5014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 >> 172.21.15.170:0/382506536 conn(0x7eff500b1c40 msgr2=0x7eff500b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:59.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 shutdown_connections 2024-09-21T02:19:59.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.050+0000 7eff54ee9640 1 -- 172.21.15.170:0/382506536 wait complete. 2024-09-21T02:19:59.069 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-21T02:19:58.894985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282740,"kb_used_data":12412,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691980,"statfs":{"total":773094113280,"available":772804587520,"internally_reserved":0,"allocated":12709888,"data_stored":6228517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002214"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879824+0000","last_change":"2024-09-21T02:19:54.879824+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879824+0000","last_undegraded":"2024-09-21T02:19:54.879824+0000","last_fullsized":"2024-09-21T02:19:54.879824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879361+0000","last_change":"2024-09-21T02:19:54.879361+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879361+0000","last_undegraded":"2024-09-21T02:19:54.879361+0000","last_fullsized":"2024-09-21T02:19:54.879361+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885776+0000","last_change":"2024-09-21T02:19:53.885776+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885776+0000","last_undegraded":"2024-09-21T02:19:53.885776+0000","last_fullsized":"2024-09-21T02:19:53.885776+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":4,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.886129+0000","last_change":"2024-09-21T02:19:54.880458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.886129+0000","last_undegraded":"2024-09-21T02:19:54.886129+0000","last_fullsized":"2024-09-21T02:19:54.886129+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886975+0000","last_change":"2024-09-21T02:19:53.886975+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886975+0000","last_undegraded":"2024-09-21T02:19:53.886975+0000","last_fullsized":"2024-09-21T02:19:53.886975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889655+0000","last_change":"2024-09-21T02:19:53.889655+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889655+0000","last_undegraded":"2024-09-21T02:19:53.889655+0000","last_fullsized":"2024-09-21T02:19:53.889655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886780+0000","last_change":"2024-09-21T02:19:53.886780+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886780+0000","last_undegraded":"2024-09-21T02:19:53.886780+0000","last_fullsized":"2024-09-21T02:19:53.886780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887128+0000","last_change":"2024-09-21T02:19:53.887128+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887128+0000","last_undegraded":"2024-09-21T02:19:53.887128+0000","last_fullsized":"2024-09-21T02:19:53.887128+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885049+0000","last_change":"2024-09-21T02:19:53.885049+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885049+0000","last_undegraded":"2024-09-21T02:19:53.885049+0000","last_fullsized":"2024-09-21T02:19:53.885049+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878396+0000","last_change":"2024-09-21T02:19:54.878396+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878396+0000","last_undegraded":"2024-09-21T02:19:54.878396+0000","last_fullsized":"2024-09-21T02:19:54.878396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877485+0000","last_change":"2024-09-21T02:19:54.877485+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877485+0000","last_undegraded":"2024-09-21T02:19:54.877485+0000","last_fullsized":"2024-09-21T02:19:54.877485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882109+0000","last_change":"2024-09-21T02:19:53.882109+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882109+0000","last_undegraded":"2024-09-21T02:19:53.882109+0000","last_fullsized":"2024-09-21T02:19:53.882109+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876172+0000","last_change":"2024-09-21T02:19:54.876172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876172+0000","last_undegraded":"2024-09-21T02:19:54.876172+0000","last_fullsized":"2024-09-21T02:19:54.876172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892017+0000","last_change":"2024-09-21T02:19:53.892017+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892017+0000","last_undegraded":"2024-09-21T02:19:53.892017+0000","last_fullsized":"2024-09-21T02:19:53.892017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897505+0000","last_change":"2024-09-21T02:19:53.897505+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897505+0000","last_undegraded":"2024-09-21T02:19:53.897505+0000","last_fullsized":"2024-09-21T02:19:53.897505+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902786+0000","last_change":"2024-09-21T02:19:53.902786+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902786+0000","last_undegraded":"2024-09-21T02:19:53.902786+0000","last_fullsized":"2024-09-21T02:19:53.902786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883879+0000","last_change":"2024-09-21T02:19:53.883879+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883879+0000","last_undegraded":"2024-09-21T02:19:53.883879+0000","last_fullsized":"2024-09-21T02:19:53.883879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897882+0000","last_change":"2024-09-21T02:19:53.897882+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897882+0000","last_undegraded":"2024-09-21T02:19:53.897882+0000","last_fullsized":"2024-09-21T02:19:53.897882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886319+0000","last_change":"2024-09-21T02:19:53.886319+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886319+0000","last_undegraded":"2024-09-21T02:19:53.886319+0000","last_fullsized":"2024-09-21T02:19:53.886319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899311+0000","last_change":"2024-09-21T02:19:53.899311+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899311+0000","last_undegraded":"2024-09-21T02:19:53.899311+0000","last_fullsized":"2024-09-21T02:19:53.899311+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889310+0000","last_change":"2024-09-21T02:19:53.889310+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889310+0000","last_undegraded":"2024-09-21T02:19:53.889310+0000","last_fullsized":"2024-09-21T02:19:53.889310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886303+0000","last_change":"2024-09-21T02:19:53.886303+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886303+0000","last_undegraded":"2024-09-21T02:19:53.886303+0000","last_fullsized":"2024-09-21T02:19:53.886303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902222+0000","last_change":"2024-09-21T02:19:53.902222+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902222+0000","last_undegraded":"2024-09-21T02:19:53.902222+0000","last_fullsized":"2024-09-21T02:19:53.902222+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.900530+0000","last_change":"2024-09-21T02:19:53.900530+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.900530+0000","last_undegraded":"2024-09-21T02:19:53.900530+0000","last_fullsized":"2024-09-21T02:19:53.900530+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876179+0000","last_change":"2024-09-21T02:19:54.876179+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876179+0000","last_undegraded":"2024-09-21T02:19:54.876179+0000","last_fullsized":"2024-09-21T02:19:54.876179+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877410+0000","last_change":"2024-09-21T02:19:54.877410+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877410+0000","last_undegraded":"2024-09-21T02:19:54.877410+0000","last_fullsized":"2024-09-21T02:19:54.877410+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892439+0000","last_change":"2024-09-21T02:19:53.892439+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892439+0000","last_undegraded":"2024-09-21T02:19:53.892439+0000","last_fullsized":"2024-09-21T02:19:53.892439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.903671+0000","last_change":"2024-09-21T02:19:53.903671+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.903671+0000","last_undegraded":"2024-09-21T02:19:53.903671+0000","last_fullsized":"2024-09-21T02:19:53.903671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876878+0000","last_change":"2024-09-21T02:19:54.876878+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876878+0000","last_undegraded":"2024-09-21T02:19:54.876878+0000","last_fullsized":"2024-09-21T02:19:54.876878+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883817+0000","last_change":"2024-09-21T02:19:53.883817+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883817+0000","last_undegraded":"2024-09-21T02:19:53.883817+0000","last_fullsized":"2024-09-21T02:19:53.883817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.895765+0000","last_change":"2024-09-21T02:19:53.895765+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.895765+0000","last_undegraded":"2024-09-21T02:19:53.895765+0000","last_fullsized":"2024-09-21T02:19:53.895765+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889849+0000","last_change":"2024-09-21T02:19:53.889849+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889849+0000","last_undegraded":"2024-09-21T02:19:53.889849+0000","last_fullsized":"2024-09-21T02:19:53.889849+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902590+0000","last_change":"2024-09-21T02:19:53.902590+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902590+0000","last_undegraded":"2024-09-21T02:19:53.902590+0000","last_fullsized":"2024-09-21T02:19:53.902590+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901157+0000","last_change":"2024-09-21T02:19:53.899077+0000","last_active":"2024-09-21T02:19:53.901157+0000","last_peered":"2024-09-21T02:19:53.901157+0000","last_clean":"2024-09-21T02:19:53.901157+0000","last_became_active":"2024-09-21T02:19:53.898374+0000","last_became_peered":"2024-09-21T02:19:53.898374+0000","last_unstale":"2024-09-21T02:19:53.901157+0000","last_undegraded":"2024-09-21T02:19:53.901157+0000","last_fullsized":"2024-09-21T02:19:53.901157+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:20.344188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882611+0000","last_change":"2024-09-21T02:19:53.882611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882611+0000","last_undegraded":"2024-09-21T02:19:53.882611+0000","last_fullsized":"2024-09-21T02:19:53.882611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.895668+0000","last_change":"2024-09-21T02:19:53.894376+0000","last_active":"2024-09-21T02:19:53.895668+0000","last_peered":"2024-09-21T02:19:53.895668+0000","last_clean":"2024-09-21T02:19:53.895668+0000","last_became_active":"2024-09-21T02:19:53.893849+0000","last_became_peered":"2024-09-21T02:19:53.893849+0000","last_unstale":"2024-09-21T02:19:53.895668+0000","last_undegraded":"2024-09-21T02:19:53.895668+0000","last_fullsized":"2024-09-21T02:19:53.895668+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:07.797675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880476+0000","last_change":"2024-09-21T02:19:53.880476+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880476+0000","last_undegraded":"2024-09-21T02:19:53.880476+0000","last_fullsized":"2024-09-21T02:19:53.880476+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886324+0000","last_change":"2024-09-21T02:19:53.913322+0000","last_active":"2024-09-21T02:19:54.886324+0000","last_peered":"2024-09-21T02:19:54.886324+0000","last_clean":"2024-09-21T02:19:54.886324+0000","last_became_active":"2024-09-21T02:19:53.912089+0000","last_became_peered":"2024-09-21T02:19:53.912089+0000","last_unstale":"2024-09-21T02:19:54.886324+0000","last_undegraded":"2024-09-21T02:19:54.886324+0000","last_fullsized":"2024-09-21T02:19:54.886324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:17.721839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885419+0000","last_change":"2024-09-21T02:19:53.885419+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885419+0000","last_undegraded":"2024-09-21T02:19:53.885419+0000","last_fullsized":"2024-09-21T02:19:53.885419+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880481+0000","last_change":"2024-09-21T02:19:53.911987+0000","last_active":"2024-09-21T02:19:54.880481+0000","last_peered":"2024-09-21T02:19:54.880481+0000","last_clean":"2024-09-21T02:19:54.880481+0000","last_became_active":"2024-09-21T02:19:53.911176+0000","last_became_peered":"2024-09-21T02:19:53.911176+0000","last_unstale":"2024-09-21T02:19:54.880481+0000","last_undegraded":"2024-09-21T02:19:54.880481+0000","last_fullsized":"2024-09-21T02:19:54.880481+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:55.290071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899523+0000","last_change":"2024-09-21T02:19:53.899523+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899523+0000","last_undegraded":"2024-09-21T02:19:53.899523+0000","last_fullsized":"2024-09-21T02:19:53.899523+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.877925+0000","last_change":"2024-09-21T02:19:53.906525+0000","last_active":"2024-09-21T02:19:54.877925+0000","last_peered":"2024-09-21T02:19:54.877925+0000","last_clean":"2024-09-21T02:19:54.877925+0000","last_became_active":"2024-09-21T02:19:53.905607+0000","last_became_peered":"2024-09-21T02:19:53.905607+0000","last_unstale":"2024-09-21T02:19:54.877925+0000","last_undegraded":"2024-09-21T02:19:54.877925+0000","last_fullsized":"2024-09-21T02:19:54.877925+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.829774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888730+0000","last_change":"2024-09-21T02:19:53.888730+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888730+0000","last_undegraded":"2024-09-21T02:19:53.888730+0000","last_fullsized":"2024-09-21T02:19:53.888730+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.921790+0000","last_change":"2024-09-21T02:19:53.920704+0000","last_active":"2024-09-21T02:19:53.921790+0000","last_peered":"2024-09-21T02:19:53.921790+0000","last_clean":"2024-09-21T02:19:53.921790+0000","last_became_active":"2024-09-21T02:19:53.920013+0000","last_became_peered":"2024-09-21T02:19:53.920013+0000","last_unstale":"2024-09-21T02:19:53.921790+0000","last_undegraded":"2024-09-21T02:19:53.921790+0000","last_fullsized":"2024-09-21T02:19:53.921790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:45.115276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.884740+0000","last_change":"2024-09-21T02:19:53.884740+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.884740+0000","last_undegraded":"2024-09-21T02:19:53.884740+0000","last_fullsized":"2024-09-21T02:19:53.884740+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901049+0000","last_change":"2024-09-21T02:19:53.900369+0000","last_active":"2024-09-21T02:19:53.901049+0000","last_peered":"2024-09-21T02:19:53.901049+0000","last_clean":"2024-09-21T02:19:53.901049+0000","last_became_active":"2024-09-21T02:19:53.899961+0000","last_became_peered":"2024-09-21T02:19:53.899961+0000","last_unstale":"2024-09-21T02:19:53.901049+0000","last_undegraded":"2024-09-21T02:19:53.901049+0000","last_fullsized":"2024-09-21T02:19:53.901049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:58.476667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877317+0000","last_change":"2024-09-21T02:19:54.877317+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877317+0000","last_undegraded":"2024-09-21T02:19:54.877317+0000","last_fullsized":"2024-09-21T02:19:54.877317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876958+0000","last_change":"2024-09-21T02:19:53.906244+0000","last_active":"2024-09-21T02:19:54.876958+0000","last_peered":"2024-09-21T02:19:54.876958+0000","last_clean":"2024-09-21T02:19:54.876958+0000","last_became_active":"2024-09-21T02:19:53.905406+0000","last_became_peered":"2024-09-21T02:19:53.905406+0000","last_unstale":"2024-09-21T02:19:54.876958+0000","last_undegraded":"2024-09-21T02:19:54.876958+0000","last_fullsized":"2024-09-21T02:19:54.876958+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:09.984572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876431+0000","last_change":"2024-09-21T02:19:54.876431+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876431+0000","last_undegraded":"2024-09-21T02:19:54.876431+0000","last_fullsized":"2024-09-21T02:19:54.876431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899290+0000","last_change":"2024-09-21T02:19:53.898084+0000","last_active":"2024-09-21T02:19:53.899290+0000","last_peered":"2024-09-21T02:19:53.899290+0000","last_clean":"2024-09-21T02:19:53.899290+0000","last_became_active":"2024-09-21T02:19:53.897635+0000","last_became_peered":"2024-09-21T02:19:53.897635+0000","last_unstale":"2024-09-21T02:19:53.899290+0000","last_undegraded":"2024-09-21T02:19:53.899290+0000","last_fullsized":"2024-09-21T02:19:53.899290+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:34.034584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887925+0000","last_change":"2024-09-21T02:19:53.887925+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887925+0000","last_undegraded":"2024-09-21T02:19:53.887925+0000","last_fullsized":"2024-09-21T02:19:53.887925+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.919831+0000","last_change":"2024-09-21T02:19:53.914573+0000","last_active":"2024-09-21T02:19:53.919831+0000","last_peered":"2024-09-21T02:19:53.919831+0000","last_clean":"2024-09-21T02:19:53.919831+0000","last_became_active":"2024-09-21T02:19:53.913541+0000","last_became_peered":"2024-09-21T02:19:53.913541+0000","last_unstale":"2024-09-21T02:19:53.919831+0000","last_undegraded":"2024-09-21T02:19:53.919831+0000","last_fullsized":"2024-09-21T02:19:53.919831+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:47.109959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880707+0000","last_change":"2024-09-21T02:19:53.880707+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880707+0000","last_undegraded":"2024-09-21T02:19:53.880707+0000","last_fullsized":"2024-09-21T02:19:53.880707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.900789+0000","last_change":"2024-09-21T02:19:53.899480+0000","last_active":"2024-09-21T02:19:53.900789+0000","last_peered":"2024-09-21T02:19:53.900789+0000","last_clean":"2024-09-21T02:19:53.900789+0000","last_became_active":"2024-09-21T02:19:53.899037+0000","last_became_peered":"2024-09-21T02:19:53.899037+0000","last_unstale":"2024-09-21T02:19:53.900789+0000","last_undegraded":"2024-09-21T02:19:53.900789+0000","last_fullsized":"2024-09-21T02:19:53.900789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:38.427447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887372+0000","last_change":"2024-09-21T02:19:53.887372+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887372+0000","last_undegraded":"2024-09-21T02:19:53.887372+0000","last_fullsized":"2024-09-21T02:19:53.887372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.906801+0000","last_change":"2024-09-21T02:19:53.904993+0000","last_active":"2024-09-21T02:19:53.906801+0000","last_peered":"2024-09-21T02:19:53.906801+0000","last_clean":"2024-09-21T02:19:53.906801+0000","last_became_active":"2024-09-21T02:19:53.904509+0000","last_became_peered":"2024-09-21T02:19:53.904509+0000","last_unstale":"2024-09-21T02:19:53.906801+0000","last_undegraded":"2024-09-21T02:19:53.906801+0000","last_fullsized":"2024-09-21T02:19:53.906801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:48.851528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889611+0000","last_change":"2024-09-21T02:19:53.889611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889611+0000","last_undegraded":"2024-09-21T02:19:53.889611+0000","last_fullsized":"2024-09-21T02:19:53.889611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917550+0000","last_change":"2024-09-21T02:19:53.903917+0000","last_active":"2024-09-21T02:19:53.917550+0000","last_peered":"2024-09-21T02:19:53.917550+0000","last_clean":"2024-09-21T02:19:53.917550+0000","last_became_active":"2024-09-21T02:19:53.903089+0000","last_became_peered":"2024-09-21T02:19:53.903089+0000","last_unstale":"2024-09-21T02:19:53.917550+0000","last_undegraded":"2024-09-21T02:19:53.917550+0000","last_fullsized":"2024-09-21T02:19:53.917550+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:34.884559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892462+0000","last_change":"2024-09-21T02:19:53.892462+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892462+0000","last_undegraded":"2024-09-21T02:19:53.892462+0000","last_fullsized":"2024-09-21T02:19:53.892462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878927+0000","last_change":"2024-09-21T02:19:53.896469+0000","last_active":"2024-09-21T02:19:54.878927+0000","last_peered":"2024-09-21T02:19:54.878927+0000","last_clean":"2024-09-21T02:19:54.878927+0000","last_became_active":"2024-09-21T02:19:53.895529+0000","last_became_peered":"2024-09-21T02:19:53.895529+0000","last_unstale":"2024-09-21T02:19:54.878927+0000","last_undegraded":"2024-09-21T02:19:54.878927+0000","last_fullsized":"2024-09-21T02:19:54.878927+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:30.370234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882720+0000","last_change":"2024-09-21T02:19:53.882720+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882720+0000","last_undegraded":"2024-09-21T02:19:53.882720+0000","last_fullsized":"2024-09-21T02:19:53.882720+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920349+0000","last_change":"2024-09-21T02:19:53.919682+0000","last_active":"2024-09-21T02:19:53.920349+0000","last_peered":"2024-09-21T02:19:53.920349+0000","last_clean":"2024-09-21T02:19:53.920349+0000","last_became_active":"2024-09-21T02:19:53.919315+0000","last_became_peered":"2024-09-21T02:19:53.919315+0000","last_unstale":"2024-09-21T02:19:53.920349+0000","last_undegraded":"2024-09-21T02:19:53.920349+0000","last_fullsized":"2024-09-21T02:19:53.920349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:33.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887258+0000","last_change":"2024-09-21T02:19:53.887258+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887258+0000","last_undegraded":"2024-09-21T02:19:53.887258+0000","last_fullsized":"2024-09-21T02:19:53.887258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.908256+0000","last_change":"2024-09-21T02:19:53.907585+0000","last_active":"2024-09-21T02:19:53.908256+0000","last_peered":"2024-09-21T02:19:53.908256+0000","last_clean":"2024-09-21T02:19:53.908256+0000","last_became_active":"2024-09-21T02:19:53.906977+0000","last_became_peered":"2024-09-21T02:19:53.906977+0000","last_unstale":"2024-09-21T02:19:53.908256+0000","last_undegraded":"2024-09-21T02:19:53.908256+0000","last_fullsized":"2024-09-21T02:19:53.908256+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:18.433443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886728+0000","last_change":"2024-09-21T02:19:53.886728+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886728+0000","last_undegraded":"2024-09-21T02:19:53.886728+0000","last_fullsized":"2024-09-21T02:19:53.886728+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886814+0000","last_change":"2024-09-21T02:19:53.904893+0000","last_active":"2024-09-21T02:19:54.886814+0000","last_peered":"2024-09-21T02:19:54.886814+0000","last_clean":"2024-09-21T02:19:54.886814+0000","last_became_active":"2024-09-21T02:19:53.904064+0000","last_became_peered":"2024-09-21T02:19:53.904064+0000","last_unstale":"2024-09-21T02:19:54.886814+0000","last_undegraded":"2024-09-21T02:19:54.886814+0000","last_fullsized":"2024-09-21T02:19:54.886814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:33.874089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879257+0000","last_change":"2024-09-21T02:19:54.879257+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879257+0000","last_undegraded":"2024-09-21T02:19:54.879257+0000","last_fullsized":"2024-09-21T02:19:54.879257+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899521+0000","last_change":"2024-09-21T02:19:53.898390+0000","last_active":"2024-09-21T02:19:53.899521+0000","last_peered":"2024-09-21T02:19:53.899521+0000","last_clean":"2024-09-21T02:19:53.899521+0000","last_became_active":"2024-09-21T02:19:53.897822+0000","last_became_peered":"2024-09-21T02:19:53.897822+0000","last_unstale":"2024-09-21T02:19:53.899521+0000","last_undegraded":"2024-09-21T02:19:53.899521+0000","last_fullsized":"2024-09-21T02:19:53.899521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:11.038001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.881173+0000","last_change":"2024-09-21T02:19:54.881173+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.881173+0000","last_undegraded":"2024-09-21T02:19:54.881173+0000","last_fullsized":"2024-09-21T02:19:54.881173+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898678+0000","last_change":"2024-09-21T02:19:53.889955+0000","last_active":"2024-09-21T02:19:53.898678+0000","last_peered":"2024-09-21T02:19:53.898678+0000","last_clean":"2024-09-21T02:19:53.898678+0000","last_became_active":"2024-09-21T02:19:53.889393+0000","last_became_peered":"2024-09-21T02:19:53.889393+0000","last_unstale":"2024-09-21T02:19:53.898678+0000","last_undegraded":"2024-09-21T02:19:53.898678+0000","last_fullsized":"2024-09-21T02:19:53.898678+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:32.922018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881657+0000","last_change":"2024-09-21T02:19:53.881657+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881657+0000","last_undegraded":"2024-09-21T02:19:53.881657+0000","last_fullsized":"2024-09-21T02:19:53.881657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898735+0000","last_change":"2024-09-21T02:19:53.897209+0000","last_active":"2024-09-21T02:19:53.898735+0000","last_peered":"2024-09-21T02:19:53.898735+0000","last_clean":"2024-09-21T02:19:53.898735+0000","last_became_active":"2024-09-21T02:19:53.896800+0000","last_became_peered":"2024-09-21T02:19:53.896800+0000","last_unstale":"2024-09-21T02:19:53.898735+0000","last_undegraded":"2024-09-21T02:19:53.898735+0000","last_fullsized":"2024-09-21T02:19:53.898735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:31.737184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.904140+0000","last_change":"2024-09-21T02:19:53.904140+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.904140+0000","last_undegraded":"2024-09-21T02:19:53.904140+0000","last_fullsized":"2024-09-21T02:19:53.904140+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920415+0000","last_change":"2024-09-21T02:19:53.915141+0000","last_active":"2024-09-21T02:19:53.920415+0000","last_peered":"2024-09-21T02:19:53.920415+0000","last_clean":"2024-09-21T02:19:53.920415+0000","last_became_active":"2024-09-21T02:19:53.914156+0000","last_became_peered":"2024-09-21T02:19:53.914156+0000","last_unstale":"2024-09-21T02:19:53.920415+0000","last_undegraded":"2024-09-21T02:19:53.920415+0000","last_fullsized":"2024-09-21T02:19:53.920415+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:52.051938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890763+0000","last_change":"2024-09-21T02:19:53.890763+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890763+0000","last_undegraded":"2024-09-21T02:19:53.890763+0000","last_fullsized":"2024-09-21T02:19:53.890763+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917649+0000","last_change":"2024-09-21T02:19:53.906690+0000","last_active":"2024-09-21T02:19:53.917649+0000","last_peered":"2024-09-21T02:19:53.917649+0000","last_clean":"2024-09-21T02:19:53.917649+0000","last_became_active":"2024-09-21T02:19:53.905289+0000","last_became_peered":"2024-09-21T02:19:53.905289+0000","last_unstale":"2024-09-21T02:19:53.917649+0000","last_undegraded":"2024-09-21T02:19:53.917649+0000","last_fullsized":"2024-09-21T02:19:53.917649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:25.254722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881976+0000","last_change":"2024-09-21T02:19:53.881976+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881976+0000","last_undegraded":"2024-09-21T02:19:53.881976+0000","last_fullsized":"2024-09-21T02:19:53.881976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876219+0000","last_change":"2024-09-21T02:19:53.904489+0000","last_active":"2024-09-21T02:19:54.876219+0000","last_peered":"2024-09-21T02:19:54.876219+0000","last_clean":"2024-09-21T02:19:54.876219+0000","last_became_active":"2024-09-21T02:19:53.903679+0000","last_became_peered":"2024-09-21T02:19:53.903679+0000","last_unstale":"2024-09-21T02:19:54.876219+0000","last_undegraded":"2024-09-21T02:19:54.876219+0000","last_fullsized":"2024-09-21T02:19:54.876219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:00.611818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.894458+0000","last_change":"2024-09-21T02:19:53.894458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.894458+0000","last_undegraded":"2024-09-21T02:19:53.894458+0000","last_fullsized":"2024-09-21T02:19:53.894458+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920396+0000","last_change":"2024-09-21T02:19:53.917476+0000","last_active":"2024-09-21T02:19:53.920396+0000","last_peered":"2024-09-21T02:19:53.920396+0000","last_clean":"2024-09-21T02:19:53.920396+0000","last_became_active":"2024-09-21T02:19:53.916672+0000","last_became_peered":"2024-09-21T02:19:53.916672+0000","last_unstale":"2024-09-21T02:19:53.920396+0000","last_undegraded":"2024-09-21T02:19:53.920396+0000","last_fullsized":"2024-09-21T02:19:53.920396+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:04.729381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878762+0000","last_change":"2024-09-21T02:19:54.878762+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878762+0000","last_undegraded":"2024-09-21T02:19:54.878762+0000","last_fullsized":"2024-09-21T02:19:54.878762+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901861+0000","last_change":"2024-09-21T02:19:53.900842+0000","last_active":"2024-09-21T02:19:53.901861+0000","last_peered":"2024-09-21T02:19:53.901861+0000","last_clean":"2024-09-21T02:19:53.901861+0000","last_became_active":"2024-09-21T02:19:53.900353+0000","last_became_peered":"2024-09-21T02:19:53.900353+0000","last_unstale":"2024-09-21T02:19:53.901861+0000","last_undegraded":"2024-09-21T02:19:53.901861+0000","last_fullsized":"2024-09-21T02:19:53.901861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.302971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890952+0000","last_change":"2024-09-21T02:19:53.890952+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890952+0000","last_undegraded":"2024-09-21T02:19:53.890952+0000","last_fullsized":"2024-09-21T02:19:53.890952+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920113+0000","last_change":"2024-09-21T02:19:53.916530+0000","last_active":"2024-09-21T02:19:53.920113+0000","last_peered":"2024-09-21T02:19:53.920113+0000","last_clean":"2024-09-21T02:19:53.920113+0000","last_became_active":"2024-09-21T02:19:53.915419+0000","last_became_peered":"2024-09-21T02:19:53.915419+0000","last_unstale":"2024-09-21T02:19:53.920113+0000","last_undegraded":"2024-09-21T02:19:53.920113+0000","last_fullsized":"2024-09-21T02:19:53.920113+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:28.827382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881579+0000","last_change":"2024-09-21T02:19:53.881579+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881579+0000","last_undegraded":"2024-09-21T02:19:53.881579+0000","last_fullsized":"2024-09-21T02:19:53.881579+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.918803+0000","last_change":"2024-09-21T02:19:53.911432+0000","last_active":"2024-09-21T02:19:53.918803+0000","last_peered":"2024-09-21T02:19:53.918803+0000","last_clean":"2024-09-21T02:19:53.918803+0000","last_became_active":"2024-09-21T02:19:53.910658+0000","last_became_peered":"2024-09-21T02:19:53.910658+0000","last_unstale":"2024-09-21T02:19:53.918803+0000","last_undegraded":"2024-09-21T02:19:53.918803+0000","last_fullsized":"2024-09-21T02:19:53.918803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:56.513763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889059+0000","last_change":"2024-09-21T02:19:53.889059+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889059+0000","last_undegraded":"2024-09-21T02:19:53.889059+0000","last_fullsized":"2024-09-21T02:19:53.889059+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878522+0000","last_change":"2024-09-21T02:19:53.909822+0000","last_active":"2024-09-21T02:19:54.878522+0000","last_peered":"2024-09-21T02:19:54.878522+0000","last_clean":"2024-09-21T02:19:54.878522+0000","last_became_active":"2024-09-21T02:19:53.908919+0000","last_became_peered":"2024-09-21T02:19:53.908919+0000","last_unstale":"2024-09-21T02:19:54.878522+0000","last_undegraded":"2024-09-21T02:19:54.878522+0000","last_fullsized":"2024-09-21T02:19:54.878522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.216241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888015+0000","last_change":"2024-09-21T02:19:53.888015+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888015+0000","last_undegraded":"2024-09-21T02:19:53.888015+0000","last_fullsized":"2024-09-21T02:19:53.888015+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.916193+0000","last_change":"2024-09-21T02:19:53.915286+0000","last_active":"2024-09-21T02:19:53.916193+0000","last_peered":"2024-09-21T02:19:53.916193+0000","last_clean":"2024-09-21T02:19:53.916193+0000","last_became_active":"2024-09-21T02:19:53.914782+0000","last_became_peered":"2024-09-21T02:19:53.914782+0000","last_unstale":"2024-09-21T02:19:53.916193+0000","last_undegraded":"2024-09-21T02:19:53.916193+0000","last_fullsized":"2024-09-21T02:19:53.916193+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:39.555502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899037+0000","last_change":"2024-09-21T02:19:53.899037+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899037+0000","last_undegraded":"2024-09-21T02:19:53.899037+0000","last_fullsized":"2024-09-21T02:19:53.899037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880986+0000","last_change":"2024-09-21T02:19:53.896536+0000","last_active":"2024-09-21T02:19:54.880986+0000","last_peered":"2024-09-21T02:19:54.880986+0000","last_clean":"2024-09-21T02:19:54.880986+0000","last_became_active":"2024-09-21T02:19:53.895589+0000","last_became_peered":"2024-09-21T02:19:53.895589+0000","last_unstale":"2024-09-21T02:19:54.880986+0000","last_undegraded":"2024-09-21T02:19:54.880986+0000","last_fullsized":"2024-09-21T02:19:54.880986+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:17.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890723+0000","last_change":"2024-09-21T02:19:53.890723+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890723+0000","last_undegraded":"2024-09-21T02:19:53.890723+0000","last_fullsized":"2024-09-21T02:19:53.890723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917794+0000","last_change":"2024-09-21T02:19:53.908787+0000","last_active":"2024-09-21T02:19:53.917794+0000","last_peered":"2024-09-21T02:19:53.917794+0000","last_clean":"2024-09-21T02:19:53.917794+0000","last_became_active":"2024-09-21T02:19:53.907954+0000","last_became_peered":"2024-09-21T02:19:53.907954+0000","last_unstale":"2024-09-21T02:19:53.917794+0000","last_undegraded":"2024-09-21T02:19:53.917794+0000","last_fullsized":"2024-09-21T02:19:53.917794+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:20.813712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883172+0000","last_change":"2024-09-21T02:19:53.883172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883172+0000","last_undegraded":"2024-09-21T02:19:53.883172+0000","last_fullsized":"2024-09-21T02:19:53.883172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.915086+0000","last_change":"2024-09-21T02:19:53.913344+0000","last_active":"2024-09-21T02:19:53.915086+0000","last_peered":"2024-09-21T02:19:53.915086+0000","last_clean":"2024-09-21T02:19:53.915086+0000","last_became_active":"2024-09-21T02:19:53.912800+0000","last_became_peered":"2024-09-21T02:19:53.912800+0000","last_unstale":"2024-09-21T02:19:53.915086+0000","last_undegraded":"2024-09-21T02:19:53.915086+0000","last_fullsized":"2024-09-21T02:19:53.915086+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:13.374558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":308,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.904728+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:19:53.904728+0000","last_peered":"2024-09-21T02:19:53.904728+0000","last_clean":"2024-09-21T02:19:53.904728+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:19:53.904728+0000","last_undegraded":"2024-09-21T02:19:53.904728+0000","last_fullsized":"2024-09-21T02:19:53.904728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":625482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41500,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330340,"statfs":{"total":96636764160,"available":96594268160,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:19:59.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:19:59.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 -- 172.21.15.170:0/2787478417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 msgr2=0x7ff2d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 --2- 172.21.15.170:0/2787478417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2d800b9d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009e90 tx=0x7ff2e801d5f0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 -- 172.21.15.170:0/2787478417 shutdown_connections 2024-09-21T02:19:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 --2- 172.21.15.170:0/2787478417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2d800b9d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 -- 172.21.15.170:0/2787478417 >> 172.21.15.170:0/2787478417 conn(0x7ff2f008bf50 msgr2=0x7ff2f009d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 -- 172.21.15.170:0/2787478417 shutdown_connections 2024-09-21T02:19:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.250+0000 7ff2f6765640 1 -- 172.21.15.170:0/2787478417 wait complete. 2024-09-21T02:19:59.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 Processor -- start 2024-09-21T02:19:59.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 -- start start 2024-09-21T02:19:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2f00c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2f01f9ba0 0x7ff2f014d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2f01f8fe0 0x7ff2f01f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f00cf970 con 0x7ff2d8005360 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f00d16e0 con 0x7ff2f01f8fe0 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f00cfaf0 con 0x7ff2f01f9ba0 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2f00c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2f00c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59086/0 (socket says 172.21.15.170:59086) 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 -- 172.21.15.170:0/214209201 learned_addr learned my addr 172.21.15.170:0/214209201 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f4d02640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2f01f8fe0 0x7ff2f01f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2ef7fe640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2f01f9ba0 0x7ff2f014d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 -- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2f01f9ba0 msgr2=0x7ff2f014d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2f01f9ba0 0x7ff2f014d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 -- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2f01f8fe0 msgr2=0x7ff2f01f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2f01f8fe0 0x7ff2f01f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 -- 172.21.15.170:0/214209201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f01f9930 con 0x7ff2d8005360 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2ef7fe640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2f01f9ba0 0x7ff2f014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2effff640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2f00c8270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e570 tx=0x7ff2e000ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0012850 con 0x7ff2d8005360 2024-09-21T02:19:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2e0012ae0 con 0x7ff2d8005360 2024-09-21T02:19:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f00c9c10 con 0x7ff2d8005360 2024-09-21T02:19:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001a400 con 0x7ff2d8005360 2024-09-21T02:19:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.254+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f00c8e50 con 0x7ff2d8005360 2024-09-21T02:19:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.258+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f00b1230 con 0x7ff2d8005360 2024-09-21T02:19:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.258+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7ff2e000d030 con 0x7ff2d8005360 2024-09-21T02:19:59.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.258+0000 7ff2ecff9640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2dc0736b0 0x7ff2dc075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.258+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ff2e00982f0 con 0x7ff2d8005360 2024-09-21T02:19:59.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.258+0000 7ff2ef7fe640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2dc0736b0 0x7ff2dc075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.258+0000 7ff2ef7fe640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2dc0736b0 0x7ff2dc075bc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4000c10 tx=0x7ff2e4006b80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.262+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff2e0029080 con 0x7ff2d8005360 2024-09-21T02:19:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.418+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2f006f490 con 0x7ff2dc0736b0 2024-09-21T02:19:59.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.422+0000 7ff2ecff9640 1 -- 172.21.15.170:0/214209201 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247256 (secure 0 0 0) 0x7ff2f006f490 con 0x7ff2dc0736b0 2024-09-21T02:19:59.424 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:59.427 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2dc0736b0 msgr2=0x7ff2dc075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2dc0736b0 0x7ff2dc075bc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4000c10 tx=0x7ff2e4006b80 comp rx=0 tx=0).stop 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 msgr2=0x7ff2f00c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2f00c8270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e570 tx=0x7ff2e000ea30 comp rx=0 tx=0).stop 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 shutdown_connections 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2dc0736b0 0x7ff2dc075bc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2f01f9ba0 0x7ff2f014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2f01f8fe0 0x7ff2f01f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 --2- 172.21.15.170:0/214209201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d8005360 0x7ff2f00c8270 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 >> 172.21.15.170:0/214209201 conn(0x7ff2f008bf50 msgr2=0x7ff2f00b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:59.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 shutdown_connections 2024-09-21T02:19:59.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.426+0000 7ff2f6765640 1 -- 172.21.15.170:0/214209201 wait complete. 2024-09-21T02:19:59.446 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-21T02:19:58.894985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282740,"kb_used_data":12412,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691980,"statfs":{"total":773094113280,"available":772804587520,"internally_reserved":0,"allocated":12709888,"data_stored":6228517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002214"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879824+0000","last_change":"2024-09-21T02:19:54.879824+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879824+0000","last_undegraded":"2024-09-21T02:19:54.879824+0000","last_fullsized":"2024-09-21T02:19:54.879824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879361+0000","last_change":"2024-09-21T02:19:54.879361+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879361+0000","last_undegraded":"2024-09-21T02:19:54.879361+0000","last_fullsized":"2024-09-21T02:19:54.879361+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885776+0000","last_change":"2024-09-21T02:19:53.885776+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885776+0000","last_undegraded":"2024-09-21T02:19:53.885776+0000","last_fullsized":"2024-09-21T02:19:53.885776+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":4,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.886129+0000","last_change":"2024-09-21T02:19:54.880458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.886129+0000","last_undegraded":"2024-09-21T02:19:54.886129+0000","last_fullsized":"2024-09-21T02:19:54.886129+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886975+0000","last_change":"2024-09-21T02:19:53.886975+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886975+0000","last_undegraded":"2024-09-21T02:19:53.886975+0000","last_fullsized":"2024-09-21T02:19:53.886975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889655+0000","last_change":"2024-09-21T02:19:53.889655+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889655+0000","last_undegraded":"2024-09-21T02:19:53.889655+0000","last_fullsized":"2024-09-21T02:19:53.889655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886780+0000","last_change":"2024-09-21T02:19:53.886780+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886780+0000","last_undegraded":"2024-09-21T02:19:53.886780+0000","last_fullsized":"2024-09-21T02:19:53.886780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887128+0000","last_change":"2024-09-21T02:19:53.887128+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887128+0000","last_undegraded":"2024-09-21T02:19:53.887128+0000","last_fullsized":"2024-09-21T02:19:53.887128+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885049+0000","last_change":"2024-09-21T02:19:53.885049+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885049+0000","last_undegraded":"2024-09-21T02:19:53.885049+0000","last_fullsized":"2024-09-21T02:19:53.885049+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878396+0000","last_change":"2024-09-21T02:19:54.878396+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878396+0000","last_undegraded":"2024-09-21T02:19:54.878396+0000","last_fullsized":"2024-09-21T02:19:54.878396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877485+0000","last_change":"2024-09-21T02:19:54.877485+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877485+0000","last_undegraded":"2024-09-21T02:19:54.877485+0000","last_fullsized":"2024-09-21T02:19:54.877485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882109+0000","last_change":"2024-09-21T02:19:53.882109+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882109+0000","last_undegraded":"2024-09-21T02:19:53.882109+0000","last_fullsized":"2024-09-21T02:19:53.882109+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876172+0000","last_change":"2024-09-21T02:19:54.876172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876172+0000","last_undegraded":"2024-09-21T02:19:54.876172+0000","last_fullsized":"2024-09-21T02:19:54.876172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892017+0000","last_change":"2024-09-21T02:19:53.892017+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892017+0000","last_undegraded":"2024-09-21T02:19:53.892017+0000","last_fullsized":"2024-09-21T02:19:53.892017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897505+0000","last_change":"2024-09-21T02:19:53.897505+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897505+0000","last_undegraded":"2024-09-21T02:19:53.897505+0000","last_fullsized":"2024-09-21T02:19:53.897505+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902786+0000","last_change":"2024-09-21T02:19:53.902786+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902786+0000","last_undegraded":"2024-09-21T02:19:53.902786+0000","last_fullsized":"2024-09-21T02:19:53.902786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883879+0000","last_change":"2024-09-21T02:19:53.883879+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883879+0000","last_undegraded":"2024-09-21T02:19:53.883879+0000","last_fullsized":"2024-09-21T02:19:53.883879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897882+0000","last_change":"2024-09-21T02:19:53.897882+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897882+0000","last_undegraded":"2024-09-21T02:19:53.897882+0000","last_fullsized":"2024-09-21T02:19:53.897882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886319+0000","last_change":"2024-09-21T02:19:53.886319+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886319+0000","last_undegraded":"2024-09-21T02:19:53.886319+0000","last_fullsized":"2024-09-21T02:19:53.886319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899311+0000","last_change":"2024-09-21T02:19:53.899311+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899311+0000","last_undegraded":"2024-09-21T02:19:53.899311+0000","last_fullsized":"2024-09-21T02:19:53.899311+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889310+0000","last_change":"2024-09-21T02:19:53.889310+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889310+0000","last_undegraded":"2024-09-21T02:19:53.889310+0000","last_fullsized":"2024-09-21T02:19:53.889310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886303+0000","last_change":"2024-09-21T02:19:53.886303+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886303+0000","last_undegraded":"2024-09-21T02:19:53.886303+0000","last_fullsized":"2024-09-21T02:19:53.886303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902222+0000","last_change":"2024-09-21T02:19:53.902222+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902222+0000","last_undegraded":"2024-09-21T02:19:53.902222+0000","last_fullsized":"2024-09-21T02:19:53.902222+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.900530+0000","last_change":"2024-09-21T02:19:53.900530+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.900530+0000","last_undegraded":"2024-09-21T02:19:53.900530+0000","last_fullsized":"2024-09-21T02:19:53.900530+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876179+0000","last_change":"2024-09-21T02:19:54.876179+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876179+0000","last_undegraded":"2024-09-21T02:19:54.876179+0000","last_fullsized":"2024-09-21T02:19:54.876179+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877410+0000","last_change":"2024-09-21T02:19:54.877410+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877410+0000","last_undegraded":"2024-09-21T02:19:54.877410+0000","last_fullsized":"2024-09-21T02:19:54.877410+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892439+0000","last_change":"2024-09-21T02:19:53.892439+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892439+0000","last_undegraded":"2024-09-21T02:19:53.892439+0000","last_fullsized":"2024-09-21T02:19:53.892439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.903671+0000","last_change":"2024-09-21T02:19:53.903671+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.903671+0000","last_undegraded":"2024-09-21T02:19:53.903671+0000","last_fullsized":"2024-09-21T02:19:53.903671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876878+0000","last_change":"2024-09-21T02:19:54.876878+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876878+0000","last_undegraded":"2024-09-21T02:19:54.876878+0000","last_fullsized":"2024-09-21T02:19:54.876878+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883817+0000","last_change":"2024-09-21T02:19:53.883817+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883817+0000","last_undegraded":"2024-09-21T02:19:53.883817+0000","last_fullsized":"2024-09-21T02:19:53.883817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.895765+0000","last_change":"2024-09-21T02:19:53.895765+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.895765+0000","last_undegraded":"2024-09-21T02:19:53.895765+0000","last_fullsized":"2024-09-21T02:19:53.895765+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889849+0000","last_change":"2024-09-21T02:19:53.889849+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889849+0000","last_undegraded":"2024-09-21T02:19:53.889849+0000","last_fullsized":"2024-09-21T02:19:53.889849+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902590+0000","last_change":"2024-09-21T02:19:53.902590+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902590+0000","last_undegraded":"2024-09-21T02:19:53.902590+0000","last_fullsized":"2024-09-21T02:19:53.902590+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901157+0000","last_change":"2024-09-21T02:19:53.899077+0000","last_active":"2024-09-21T02:19:53.901157+0000","last_peered":"2024-09-21T02:19:53.901157+0000","last_clean":"2024-09-21T02:19:53.901157+0000","last_became_active":"2024-09-21T02:19:53.898374+0000","last_became_peered":"2024-09-21T02:19:53.898374+0000","last_unstale":"2024-09-21T02:19:53.901157+0000","last_undegraded":"2024-09-21T02:19:53.901157+0000","last_fullsized":"2024-09-21T02:19:53.901157+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:20.344188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882611+0000","last_change":"2024-09-21T02:19:53.882611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882611+0000","last_undegraded":"2024-09-21T02:19:53.882611+0000","last_fullsized":"2024-09-21T02:19:53.882611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.895668+0000","last_change":"2024-09-21T02:19:53.894376+0000","last_active":"2024-09-21T02:19:53.895668+0000","last_peered":"2024-09-21T02:19:53.895668+0000","last_clean":"2024-09-21T02:19:53.895668+0000","last_became_active":"2024-09-21T02:19:53.893849+0000","last_became_peered":"2024-09-21T02:19:53.893849+0000","last_unstale":"2024-09-21T02:19:53.895668+0000","last_undegraded":"2024-09-21T02:19:53.895668+0000","last_fullsized":"2024-09-21T02:19:53.895668+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:07.797675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880476+0000","last_change":"2024-09-21T02:19:53.880476+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880476+0000","last_undegraded":"2024-09-21T02:19:53.880476+0000","last_fullsized":"2024-09-21T02:19:53.880476+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886324+0000","last_change":"2024-09-21T02:19:53.913322+0000","last_active":"2024-09-21T02:19:54.886324+0000","last_peered":"2024-09-21T02:19:54.886324+0000","last_clean":"2024-09-21T02:19:54.886324+0000","last_became_active":"2024-09-21T02:19:53.912089+0000","last_became_peered":"2024-09-21T02:19:53.912089+0000","last_unstale":"2024-09-21T02:19:54.886324+0000","last_undegraded":"2024-09-21T02:19:54.886324+0000","last_fullsized":"2024-09-21T02:19:54.886324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:17.721839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885419+0000","last_change":"2024-09-21T02:19:53.885419+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885419+0000","last_undegraded":"2024-09-21T02:19:53.885419+0000","last_fullsized":"2024-09-21T02:19:53.885419+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880481+0000","last_change":"2024-09-21T02:19:53.911987+0000","last_active":"2024-09-21T02:19:54.880481+0000","last_peered":"2024-09-21T02:19:54.880481+0000","last_clean":"2024-09-21T02:19:54.880481+0000","last_became_active":"2024-09-21T02:19:53.911176+0000","last_became_peered":"2024-09-21T02:19:53.911176+0000","last_unstale":"2024-09-21T02:19:54.880481+0000","last_undegraded":"2024-09-21T02:19:54.880481+0000","last_fullsized":"2024-09-21T02:19:54.880481+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:55.290071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899523+0000","last_change":"2024-09-21T02:19:53.899523+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899523+0000","last_undegraded":"2024-09-21T02:19:53.899523+0000","last_fullsized":"2024-09-21T02:19:53.899523+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.877925+0000","last_change":"2024-09-21T02:19:53.906525+0000","last_active":"2024-09-21T02:19:54.877925+0000","last_peered":"2024-09-21T02:19:54.877925+0000","last_clean":"2024-09-21T02:19:54.877925+0000","last_became_active":"2024-09-21T02:19:53.905607+0000","last_became_peered":"2024-09-21T02:19:53.905607+0000","last_unstale":"2024-09-21T02:19:54.877925+0000","last_undegraded":"2024-09-21T02:19:54.877925+0000","last_fullsized":"2024-09-21T02:19:54.877925+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.829774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888730+0000","last_change":"2024-09-21T02:19:53.888730+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888730+0000","last_undegraded":"2024-09-21T02:19:53.888730+0000","last_fullsized":"2024-09-21T02:19:53.888730+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.921790+0000","last_change":"2024-09-21T02:19:53.920704+0000","last_active":"2024-09-21T02:19:53.921790+0000","last_peered":"2024-09-21T02:19:53.921790+0000","last_clean":"2024-09-21T02:19:53.921790+0000","last_became_active":"2024-09-21T02:19:53.920013+0000","last_became_peered":"2024-09-21T02:19:53.920013+0000","last_unstale":"2024-09-21T02:19:53.921790+0000","last_undegraded":"2024-09-21T02:19:53.921790+0000","last_fullsized":"2024-09-21T02:19:53.921790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:45.115276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.884740+0000","last_change":"2024-09-21T02:19:53.884740+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.884740+0000","last_undegraded":"2024-09-21T02:19:53.884740+0000","last_fullsized":"2024-09-21T02:19:53.884740+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901049+0000","last_change":"2024-09-21T02:19:53.900369+0000","last_active":"2024-09-21T02:19:53.901049+0000","last_peered":"2024-09-21T02:19:53.901049+0000","last_clean":"2024-09-21T02:19:53.901049+0000","last_became_active":"2024-09-21T02:19:53.899961+0000","last_became_peered":"2024-09-21T02:19:53.899961+0000","last_unstale":"2024-09-21T02:19:53.901049+0000","last_undegraded":"2024-09-21T02:19:53.901049+0000","last_fullsized":"2024-09-21T02:19:53.901049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:58.476667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877317+0000","last_change":"2024-09-21T02:19:54.877317+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877317+0000","last_undegraded":"2024-09-21T02:19:54.877317+0000","last_fullsized":"2024-09-21T02:19:54.877317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876958+0000","last_change":"2024-09-21T02:19:53.906244+0000","last_active":"2024-09-21T02:19:54.876958+0000","last_peered":"2024-09-21T02:19:54.876958+0000","last_clean":"2024-09-21T02:19:54.876958+0000","last_became_active":"2024-09-21T02:19:53.905406+0000","last_became_peered":"2024-09-21T02:19:53.905406+0000","last_unstale":"2024-09-21T02:19:54.876958+0000","last_undegraded":"2024-09-21T02:19:54.876958+0000","last_fullsized":"2024-09-21T02:19:54.876958+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:09.984572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876431+0000","last_change":"2024-09-21T02:19:54.876431+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876431+0000","last_undegraded":"2024-09-21T02:19:54.876431+0000","last_fullsized":"2024-09-21T02:19:54.876431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899290+0000","last_change":"2024-09-21T02:19:53.898084+0000","last_active":"2024-09-21T02:19:53.899290+0000","last_peered":"2024-09-21T02:19:53.899290+0000","last_clean":"2024-09-21T02:19:53.899290+0000","last_became_active":"2024-09-21T02:19:53.897635+0000","last_became_peered":"2024-09-21T02:19:53.897635+0000","last_unstale":"2024-09-21T02:19:53.899290+0000","last_undegraded":"2024-09-21T02:19:53.899290+0000","last_fullsized":"2024-09-21T02:19:53.899290+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:34.034584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887925+0000","last_change":"2024-09-21T02:19:53.887925+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887925+0000","last_undegraded":"2024-09-21T02:19:53.887925+0000","last_fullsized":"2024-09-21T02:19:53.887925+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.919831+0000","last_change":"2024-09-21T02:19:53.914573+0000","last_active":"2024-09-21T02:19:53.919831+0000","last_peered":"2024-09-21T02:19:53.919831+0000","last_clean":"2024-09-21T02:19:53.919831+0000","last_became_active":"2024-09-21T02:19:53.913541+0000","last_became_peered":"2024-09-21T02:19:53.913541+0000","last_unstale":"2024-09-21T02:19:53.919831+0000","last_undegraded":"2024-09-21T02:19:53.919831+0000","last_fullsized":"2024-09-21T02:19:53.919831+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:47.109959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880707+0000","last_change":"2024-09-21T02:19:53.880707+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880707+0000","last_undegraded":"2024-09-21T02:19:53.880707+0000","last_fullsized":"2024-09-21T02:19:53.880707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.900789+0000","last_change":"2024-09-21T02:19:53.899480+0000","last_active":"2024-09-21T02:19:53.900789+0000","last_peered":"2024-09-21T02:19:53.900789+0000","last_clean":"2024-09-21T02:19:53.900789+0000","last_became_active":"2024-09-21T02:19:53.899037+0000","last_became_peered":"2024-09-21T02:19:53.899037+0000","last_unstale":"2024-09-21T02:19:53.900789+0000","last_undegraded":"2024-09-21T02:19:53.900789+0000","last_fullsized":"2024-09-21T02:19:53.900789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:38.427447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887372+0000","last_change":"2024-09-21T02:19:53.887372+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887372+0000","last_undegraded":"2024-09-21T02:19:53.887372+0000","last_fullsized":"2024-09-21T02:19:53.887372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.906801+0000","last_change":"2024-09-21T02:19:53.904993+0000","last_active":"2024-09-21T02:19:53.906801+0000","last_peered":"2024-09-21T02:19:53.906801+0000","last_clean":"2024-09-21T02:19:53.906801+0000","last_became_active":"2024-09-21T02:19:53.904509+0000","last_became_peered":"2024-09-21T02:19:53.904509+0000","last_unstale":"2024-09-21T02:19:53.906801+0000","last_undegraded":"2024-09-21T02:19:53.906801+0000","last_fullsized":"2024-09-21T02:19:53.906801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:48.851528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889611+0000","last_change":"2024-09-21T02:19:53.889611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889611+0000","last_undegraded":"2024-09-21T02:19:53.889611+0000","last_fullsized":"2024-09-21T02:19:53.889611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917550+0000","last_change":"2024-09-21T02:19:53.903917+0000","last_active":"2024-09-21T02:19:53.917550+0000","last_peered":"2024-09-21T02:19:53.917550+0000","last_clean":"2024-09-21T02:19:53.917550+0000","last_became_active":"2024-09-21T02:19:53.903089+0000","last_became_peered":"2024-09-21T02:19:53.903089+0000","last_unstale":"2024-09-21T02:19:53.917550+0000","last_undegraded":"2024-09-21T02:19:53.917550+0000","last_fullsized":"2024-09-21T02:19:53.917550+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:34.884559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892462+0000","last_change":"2024-09-21T02:19:53.892462+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892462+0000","last_undegraded":"2024-09-21T02:19:53.892462+0000","last_fullsized":"2024-09-21T02:19:53.892462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878927+0000","last_change":"2024-09-21T02:19:53.896469+0000","last_active":"2024-09-21T02:19:54.878927+0000","last_peered":"2024-09-21T02:19:54.878927+0000","last_clean":"2024-09-21T02:19:54.878927+0000","last_became_active":"2024-09-21T02:19:53.895529+0000","last_became_peered":"2024-09-21T02:19:53.895529+0000","last_unstale":"2024-09-21T02:19:54.878927+0000","last_undegraded":"2024-09-21T02:19:54.878927+0000","last_fullsized":"2024-09-21T02:19:54.878927+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:30.370234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882720+0000","last_change":"2024-09-21T02:19:53.882720+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882720+0000","last_undegraded":"2024-09-21T02:19:53.882720+0000","last_fullsized":"2024-09-21T02:19:53.882720+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920349+0000","last_change":"2024-09-21T02:19:53.919682+0000","last_active":"2024-09-21T02:19:53.920349+0000","last_peered":"2024-09-21T02:19:53.920349+0000","last_clean":"2024-09-21T02:19:53.920349+0000","last_became_active":"2024-09-21T02:19:53.919315+0000","last_became_peered":"2024-09-21T02:19:53.919315+0000","last_unstale":"2024-09-21T02:19:53.920349+0000","last_undegraded":"2024-09-21T02:19:53.920349+0000","last_fullsized":"2024-09-21T02:19:53.920349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:33.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887258+0000","last_change":"2024-09-21T02:19:53.887258+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887258+0000","last_undegraded":"2024-09-21T02:19:53.887258+0000","last_fullsized":"2024-09-21T02:19:53.887258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.908256+0000","last_change":"2024-09-21T02:19:53.907585+0000","last_active":"2024-09-21T02:19:53.908256+0000","last_peered":"2024-09-21T02:19:53.908256+0000","last_clean":"2024-09-21T02:19:53.908256+0000","last_became_active":"2024-09-21T02:19:53.906977+0000","last_became_peered":"2024-09-21T02:19:53.906977+0000","last_unstale":"2024-09-21T02:19:53.908256+0000","last_undegraded":"2024-09-21T02:19:53.908256+0000","last_fullsized":"2024-09-21T02:19:53.908256+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:18.433443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886728+0000","last_change":"2024-09-21T02:19:53.886728+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886728+0000","last_undegraded":"2024-09-21T02:19:53.886728+0000","last_fullsized":"2024-09-21T02:19:53.886728+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886814+0000","last_change":"2024-09-21T02:19:53.904893+0000","last_active":"2024-09-21T02:19:54.886814+0000","last_peered":"2024-09-21T02:19:54.886814+0000","last_clean":"2024-09-21T02:19:54.886814+0000","last_became_active":"2024-09-21T02:19:53.904064+0000","last_became_peered":"2024-09-21T02:19:53.904064+0000","last_unstale":"2024-09-21T02:19:54.886814+0000","last_undegraded":"2024-09-21T02:19:54.886814+0000","last_fullsized":"2024-09-21T02:19:54.886814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:33.874089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879257+0000","last_change":"2024-09-21T02:19:54.879257+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879257+0000","last_undegraded":"2024-09-21T02:19:54.879257+0000","last_fullsized":"2024-09-21T02:19:54.879257+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899521+0000","last_change":"2024-09-21T02:19:53.898390+0000","last_active":"2024-09-21T02:19:53.899521+0000","last_peered":"2024-09-21T02:19:53.899521+0000","last_clean":"2024-09-21T02:19:53.899521+0000","last_became_active":"2024-09-21T02:19:53.897822+0000","last_became_peered":"2024-09-21T02:19:53.897822+0000","last_unstale":"2024-09-21T02:19:53.899521+0000","last_undegraded":"2024-09-21T02:19:53.899521+0000","last_fullsized":"2024-09-21T02:19:53.899521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:11.038001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.881173+0000","last_change":"2024-09-21T02:19:54.881173+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.881173+0000","last_undegraded":"2024-09-21T02:19:54.881173+0000","last_fullsized":"2024-09-21T02:19:54.881173+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898678+0000","last_change":"2024-09-21T02:19:53.889955+0000","last_active":"2024-09-21T02:19:53.898678+0000","last_peered":"2024-09-21T02:19:53.898678+0000","last_clean":"2024-09-21T02:19:53.898678+0000","last_became_active":"2024-09-21T02:19:53.889393+0000","last_became_peered":"2024-09-21T02:19:53.889393+0000","last_unstale":"2024-09-21T02:19:53.898678+0000","last_undegraded":"2024-09-21T02:19:53.898678+0000","last_fullsized":"2024-09-21T02:19:53.898678+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:32.922018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881657+0000","last_change":"2024-09-21T02:19:53.881657+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881657+0000","last_undegraded":"2024-09-21T02:19:53.881657+0000","last_fullsized":"2024-09-21T02:19:53.881657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898735+0000","last_change":"2024-09-21T02:19:53.897209+0000","last_active":"2024-09-21T02:19:53.898735+0000","last_peered":"2024-09-21T02:19:53.898735+0000","last_clean":"2024-09-21T02:19:53.898735+0000","last_became_active":"2024-09-21T02:19:53.896800+0000","last_became_peered":"2024-09-21T02:19:53.896800+0000","last_unstale":"2024-09-21T02:19:53.898735+0000","last_undegraded":"2024-09-21T02:19:53.898735+0000","last_fullsized":"2024-09-21T02:19:53.898735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:31.737184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.904140+0000","last_change":"2024-09-21T02:19:53.904140+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.904140+0000","last_undegraded":"2024-09-21T02:19:53.904140+0000","last_fullsized":"2024-09-21T02:19:53.904140+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920415+0000","last_change":"2024-09-21T02:19:53.915141+0000","last_active":"2024-09-21T02:19:53.920415+0000","last_peered":"2024-09-21T02:19:53.920415+0000","last_clean":"2024-09-21T02:19:53.920415+0000","last_became_active":"2024-09-21T02:19:53.914156+0000","last_became_peered":"2024-09-21T02:19:53.914156+0000","last_unstale":"2024-09-21T02:19:53.920415+0000","last_undegraded":"2024-09-21T02:19:53.920415+0000","last_fullsized":"2024-09-21T02:19:53.920415+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:52.051938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890763+0000","last_change":"2024-09-21T02:19:53.890763+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890763+0000","last_undegraded":"2024-09-21T02:19:53.890763+0000","last_fullsized":"2024-09-21T02:19:53.890763+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917649+0000","last_change":"2024-09-21T02:19:53.906690+0000","last_active":"2024-09-21T02:19:53.917649+0000","last_peered":"2024-09-21T02:19:53.917649+0000","last_clean":"2024-09-21T02:19:53.917649+0000","last_became_active":"2024-09-21T02:19:53.905289+0000","last_became_peered":"2024-09-21T02:19:53.905289+0000","last_unstale":"2024-09-21T02:19:53.917649+0000","last_undegraded":"2024-09-21T02:19:53.917649+0000","last_fullsized":"2024-09-21T02:19:53.917649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:25.254722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881976+0000","last_change":"2024-09-21T02:19:53.881976+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881976+0000","last_undegraded":"2024-09-21T02:19:53.881976+0000","last_fullsized":"2024-09-21T02:19:53.881976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876219+0000","last_change":"2024-09-21T02:19:53.904489+0000","last_active":"2024-09-21T02:19:54.876219+0000","last_peered":"2024-09-21T02:19:54.876219+0000","last_clean":"2024-09-21T02:19:54.876219+0000","last_became_active":"2024-09-21T02:19:53.903679+0000","last_became_peered":"2024-09-21T02:19:53.903679+0000","last_unstale":"2024-09-21T02:19:54.876219+0000","last_undegraded":"2024-09-21T02:19:54.876219+0000","last_fullsized":"2024-09-21T02:19:54.876219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:00.611818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.894458+0000","last_change":"2024-09-21T02:19:53.894458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.894458+0000","last_undegraded":"2024-09-21T02:19:53.894458+0000","last_fullsized":"2024-09-21T02:19:53.894458+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920396+0000","last_change":"2024-09-21T02:19:53.917476+0000","last_active":"2024-09-21T02:19:53.920396+0000","last_peered":"2024-09-21T02:19:53.920396+0000","last_clean":"2024-09-21T02:19:53.920396+0000","last_became_active":"2024-09-21T02:19:53.916672+0000","last_became_peered":"2024-09-21T02:19:53.916672+0000","last_unstale":"2024-09-21T02:19:53.920396+0000","last_undegraded":"2024-09-21T02:19:53.920396+0000","last_fullsized":"2024-09-21T02:19:53.920396+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:04.729381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878762+0000","last_change":"2024-09-21T02:19:54.878762+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878762+0000","last_undegraded":"2024-09-21T02:19:54.878762+0000","last_fullsized":"2024-09-21T02:19:54.878762+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901861+0000","last_change":"2024-09-21T02:19:53.900842+0000","last_active":"2024-09-21T02:19:53.901861+0000","last_peered":"2024-09-21T02:19:53.901861+0000","last_clean":"2024-09-21T02:19:53.901861+0000","last_became_active":"2024-09-21T02:19:53.900353+0000","last_became_peered":"2024-09-21T02:19:53.900353+0000","last_unstale":"2024-09-21T02:19:53.901861+0000","last_undegraded":"2024-09-21T02:19:53.901861+0000","last_fullsized":"2024-09-21T02:19:53.901861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.302971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890952+0000","last_change":"2024-09-21T02:19:53.890952+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890952+0000","last_undegraded":"2024-09-21T02:19:53.890952+0000","last_fullsized":"2024-09-21T02:19:53.890952+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920113+0000","last_change":"2024-09-21T02:19:53.916530+0000","last_active":"2024-09-21T02:19:53.920113+0000","last_peered":"2024-09-21T02:19:53.920113+0000","last_clean":"2024-09-21T02:19:53.920113+0000","last_became_active":"2024-09-21T02:19:53.915419+0000","last_became_peered":"2024-09-21T02:19:53.915419+0000","last_unstale":"2024-09-21T02:19:53.920113+0000","last_undegraded":"2024-09-21T02:19:53.920113+0000","last_fullsized":"2024-09-21T02:19:53.920113+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:28.827382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881579+0000","last_change":"2024-09-21T02:19:53.881579+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881579+0000","last_undegraded":"2024-09-21T02:19:53.881579+0000","last_fullsized":"2024-09-21T02:19:53.881579+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.918803+0000","last_change":"2024-09-21T02:19:53.911432+0000","last_active":"2024-09-21T02:19:53.918803+0000","last_peered":"2024-09-21T02:19:53.918803+0000","last_clean":"2024-09-21T02:19:53.918803+0000","last_became_active":"2024-09-21T02:19:53.910658+0000","last_became_peered":"2024-09-21T02:19:53.910658+0000","last_unstale":"2024-09-21T02:19:53.918803+0000","last_undegraded":"2024-09-21T02:19:53.918803+0000","last_fullsized":"2024-09-21T02:19:53.918803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:56.513763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889059+0000","last_change":"2024-09-21T02:19:53.889059+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889059+0000","last_undegraded":"2024-09-21T02:19:53.889059+0000","last_fullsized":"2024-09-21T02:19:53.889059+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878522+0000","last_change":"2024-09-21T02:19:53.909822+0000","last_active":"2024-09-21T02:19:54.878522+0000","last_peered":"2024-09-21T02:19:54.878522+0000","last_clean":"2024-09-21T02:19:54.878522+0000","last_became_active":"2024-09-21T02:19:53.908919+0000","last_became_peered":"2024-09-21T02:19:53.908919+0000","last_unstale":"2024-09-21T02:19:54.878522+0000","last_undegraded":"2024-09-21T02:19:54.878522+0000","last_fullsized":"2024-09-21T02:19:54.878522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.216241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888015+0000","last_change":"2024-09-21T02:19:53.888015+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888015+0000","last_undegraded":"2024-09-21T02:19:53.888015+0000","last_fullsized":"2024-09-21T02:19:53.888015+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.916193+0000","last_change":"2024-09-21T02:19:53.915286+0000","last_active":"2024-09-21T02:19:53.916193+0000","last_peered":"2024-09-21T02:19:53.916193+0000","last_clean":"2024-09-21T02:19:53.916193+0000","last_became_active":"2024-09-21T02:19:53.914782+0000","last_became_peered":"2024-09-21T02:19:53.914782+0000","last_unstale":"2024-09-21T02:19:53.916193+0000","last_undegraded":"2024-09-21T02:19:53.916193+0000","last_fullsized":"2024-09-21T02:19:53.916193+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:39.555502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899037+0000","last_change":"2024-09-21T02:19:53.899037+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899037+0000","last_undegraded":"2024-09-21T02:19:53.899037+0000","last_fullsized":"2024-09-21T02:19:53.899037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880986+0000","last_change":"2024-09-21T02:19:53.896536+0000","last_active":"2024-09-21T02:19:54.880986+0000","last_peered":"2024-09-21T02:19:54.880986+0000","last_clean":"2024-09-21T02:19:54.880986+0000","last_became_active":"2024-09-21T02:19:53.895589+0000","last_became_peered":"2024-09-21T02:19:53.895589+0000","last_unstale":"2024-09-21T02:19:54.880986+0000","last_undegraded":"2024-09-21T02:19:54.880986+0000","last_fullsized":"2024-09-21T02:19:54.880986+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:17.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890723+0000","last_change":"2024-09-21T02:19:53.890723+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890723+0000","last_undegraded":"2024-09-21T02:19:53.890723+0000","last_fullsized":"2024-09-21T02:19:53.890723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917794+0000","last_change":"2024-09-21T02:19:53.908787+0000","last_active":"2024-09-21T02:19:53.917794+0000","last_peered":"2024-09-21T02:19:53.917794+0000","last_clean":"2024-09-21T02:19:53.917794+0000","last_became_active":"2024-09-21T02:19:53.907954+0000","last_became_peered":"2024-09-21T02:19:53.907954+0000","last_unstale":"2024-09-21T02:19:53.917794+0000","last_undegraded":"2024-09-21T02:19:53.917794+0000","last_fullsized":"2024-09-21T02:19:53.917794+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:20.813712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883172+0000","last_change":"2024-09-21T02:19:53.883172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883172+0000","last_undegraded":"2024-09-21T02:19:53.883172+0000","last_fullsized":"2024-09-21T02:19:53.883172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.915086+0000","last_change":"2024-09-21T02:19:53.913344+0000","last_active":"2024-09-21T02:19:53.915086+0000","last_peered":"2024-09-21T02:19:53.915086+0000","last_clean":"2024-09-21T02:19:53.915086+0000","last_became_active":"2024-09-21T02:19:53.912800+0000","last_became_peered":"2024-09-21T02:19:53.912800+0000","last_unstale":"2024-09-21T02:19:53.915086+0000","last_undegraded":"2024-09-21T02:19:53.915086+0000","last_fullsized":"2024-09-21T02:19:53.915086+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:13.374558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":308,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.904728+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:19:53.904728+0000","last_peered":"2024-09-21T02:19:53.904728+0000","last_clean":"2024-09-21T02:19:53.904728+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:19:53.904728+0000","last_undegraded":"2024-09-21T02:19:53.904728+0000","last_fullsized":"2024-09-21T02:19:53.904728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":625482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41500,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330340,"statfs":{"total":96636764160,"available":96594268160,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:19:59.450 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:19:59.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- 172.21.15.170:0/514890825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa248004810 msgr2=0x7fa248004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 --2- 172.21.15.170:0/514890825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa248004810 0x7fa248004c20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0d3bf0 tx=0x7fa24c01d0e0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 -- 172.21.15.170:0/514890825 reap_dead start 2024-09-21T02:19:59.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- 172.21.15.170:0/514890825 shutdown_connections 2024-09-21T02:19:59.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- 172.21.15.170:0/514890825 >> 172.21.15.170:0/514890825 conn(0x7fa25c09d830 msgr2=0x7fa25c08bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:59.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- 172.21.15.170:0/514890825 shutdown_connections 2024-09-21T02:19:59.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- 172.21.15.170:0/514890825 wait complete. 2024-09-21T02:19:59.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 Processor -- start 2024-09-21T02:19:59.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- start start 2024-09-21T02:19:59.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa248004810 0x7fa25c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25c157c60 0x7fa25c156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 0x7fa25c160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c16ad60 con 0x7fa25c1576a0 2024-09-21T02:19:59.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c16abe0 con 0x7fa248004810 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa263fa1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa25c16aee0 con 0x7fa25c157c60 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 0x7fa25c160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 0x7fa25c160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59110/0 (socket says 172.21.15.170:59110) 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 -- 172.21.15.170:0/398915571 learned_addr learned my addr 172.21.15.170:0/398915571 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26153c640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25c157c60 0x7fa25c156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa261d3d640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa248004810 0x7fa25c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 -- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25c157c60 msgr2=0x7fa25c156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25c157c60 0x7fa25c156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 -- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa248004810 msgr2=0x7fa25c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa248004810 0x7fa25c0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 -- 172.21.15.170:0/398915571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c15f470 con 0x7fa25c1576a0 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26153c640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25c157c60 0x7fa25c156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa261d3d640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa248004810 0x7fa25c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:19:59.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.646+0000 7fa26253e640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 0x7fa25c160510 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0d07f0 tx=0x7fa25400eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:59.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25400ca10 con 0x7fa25c1576a0 2024-09-21T02:19:59.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa254017070 con 0x7fa25c1576a0 2024-09-21T02:19:59.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c15f750 con 0x7fa25c1576a0 2024-09-21T02:19:59.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254011460 con 0x7fa25c1576a0 2024-09-21T02:19:59.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c15fd10 con 0x7fa25c1576a0 2024-09-21T02:19:59.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25c15f1e0 con 0x7fa25c1576a0 2024-09-21T02:19:59.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fa254011600 con 0x7fa25c1576a0 2024-09-21T02:19:59.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa2527fc640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa2400736b0 0x7fa240075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:19:59.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fa254094ea0 con 0x7fa25c1576a0 2024-09-21T02:19:59.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa261d3d640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa2400736b0 0x7fa240075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:19:59.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.650+0000 7fa261d3d640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa2400736b0 0x7fa240075bc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa258000c10 tx=0x7fa258006b80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:19:59.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.654+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa25405f820 con 0x7fa25c1576a0 2024-09-21T02:19:59.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.822+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa25c169fa0 con 0x7fa2400736b0 2024-09-21T02:19:59.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.826+0000 7fa2527fc640 1 -- 172.21.15.170:0/398915571 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247256 (secure 0 0 0) 0x7fa25c169fa0 con 0x7fa2400736b0 2024-09-21T02:19:59.829 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:19:59.830 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:19:59.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa2400736b0 msgr2=0x7fa240075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa2400736b0 0x7fa240075bc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa258000c10 tx=0x7fa258006b80 comp rx=0 tx=0).stop 2024-09-21T02:19:59.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 msgr2=0x7fa25c160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:19:59.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 0x7fa25c160510 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0d07f0 tx=0x7fa25400eb40 comp rx=0 tx=0).stop 2024-09-21T02:19:59.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 shutdown_connections 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa2400736b0 0x7fa240075bc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa25c157c60 0x7fa25c156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa25c1576a0 0x7fa25c160510 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 --2- 172.21.15.170:0/398915571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa248004810 0x7fa25c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 >> 172.21.15.170:0/398915571 conn(0x7fa25c09d830 msgr2=0x7fa25c075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 shutdown_connections 2024-09-21T02:19:59.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:19:59.830+0000 7fa263fa1640 1 -- 172.21.15.170:0/398915571 wait complete. 2024-09-21T02:19:59.849 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-21T02:19:58.894985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282740,"kb_used_data":12412,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691980,"statfs":{"total":773094113280,"available":772804587520,"internally_reserved":0,"allocated":12709888,"data_stored":6228517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002214"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879824+0000","last_change":"2024-09-21T02:19:54.879824+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879824+0000","last_undegraded":"2024-09-21T02:19:54.879824+0000","last_fullsized":"2024-09-21T02:19:54.879824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879361+0000","last_change":"2024-09-21T02:19:54.879361+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879361+0000","last_undegraded":"2024-09-21T02:19:54.879361+0000","last_fullsized":"2024-09-21T02:19:54.879361+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885776+0000","last_change":"2024-09-21T02:19:53.885776+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885776+0000","last_undegraded":"2024-09-21T02:19:53.885776+0000","last_fullsized":"2024-09-21T02:19:53.885776+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":4,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.886129+0000","last_change":"2024-09-21T02:19:54.880458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.886129+0000","last_undegraded":"2024-09-21T02:19:54.886129+0000","last_fullsized":"2024-09-21T02:19:54.886129+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886975+0000","last_change":"2024-09-21T02:19:53.886975+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886975+0000","last_undegraded":"2024-09-21T02:19:53.886975+0000","last_fullsized":"2024-09-21T02:19:53.886975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889655+0000","last_change":"2024-09-21T02:19:53.889655+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889655+0000","last_undegraded":"2024-09-21T02:19:53.889655+0000","last_fullsized":"2024-09-21T02:19:53.889655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886780+0000","last_change":"2024-09-21T02:19:53.886780+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886780+0000","last_undegraded":"2024-09-21T02:19:53.886780+0000","last_fullsized":"2024-09-21T02:19:53.886780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887128+0000","last_change":"2024-09-21T02:19:53.887128+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887128+0000","last_undegraded":"2024-09-21T02:19:53.887128+0000","last_fullsized":"2024-09-21T02:19:53.887128+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885049+0000","last_change":"2024-09-21T02:19:53.885049+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885049+0000","last_undegraded":"2024-09-21T02:19:53.885049+0000","last_fullsized":"2024-09-21T02:19:53.885049+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878396+0000","last_change":"2024-09-21T02:19:54.878396+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878396+0000","last_undegraded":"2024-09-21T02:19:54.878396+0000","last_fullsized":"2024-09-21T02:19:54.878396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877485+0000","last_change":"2024-09-21T02:19:54.877485+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877485+0000","last_undegraded":"2024-09-21T02:19:54.877485+0000","last_fullsized":"2024-09-21T02:19:54.877485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882109+0000","last_change":"2024-09-21T02:19:53.882109+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882109+0000","last_undegraded":"2024-09-21T02:19:53.882109+0000","last_fullsized":"2024-09-21T02:19:53.882109+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876172+0000","last_change":"2024-09-21T02:19:54.876172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876172+0000","last_undegraded":"2024-09-21T02:19:54.876172+0000","last_fullsized":"2024-09-21T02:19:54.876172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892017+0000","last_change":"2024-09-21T02:19:53.892017+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892017+0000","last_undegraded":"2024-09-21T02:19:53.892017+0000","last_fullsized":"2024-09-21T02:19:53.892017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897505+0000","last_change":"2024-09-21T02:19:53.897505+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897505+0000","last_undegraded":"2024-09-21T02:19:53.897505+0000","last_fullsized":"2024-09-21T02:19:53.897505+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902786+0000","last_change":"2024-09-21T02:19:53.902786+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902786+0000","last_undegraded":"2024-09-21T02:19:53.902786+0000","last_fullsized":"2024-09-21T02:19:53.902786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883879+0000","last_change":"2024-09-21T02:19:53.883879+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883879+0000","last_undegraded":"2024-09-21T02:19:53.883879+0000","last_fullsized":"2024-09-21T02:19:53.883879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897882+0000","last_change":"2024-09-21T02:19:53.897882+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897882+0000","last_undegraded":"2024-09-21T02:19:53.897882+0000","last_fullsized":"2024-09-21T02:19:53.897882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886319+0000","last_change":"2024-09-21T02:19:53.886319+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886319+0000","last_undegraded":"2024-09-21T02:19:53.886319+0000","last_fullsized":"2024-09-21T02:19:53.886319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899311+0000","last_change":"2024-09-21T02:19:53.899311+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899311+0000","last_undegraded":"2024-09-21T02:19:53.899311+0000","last_fullsized":"2024-09-21T02:19:53.899311+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889310+0000","last_change":"2024-09-21T02:19:53.889310+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889310+0000","last_undegraded":"2024-09-21T02:19:53.889310+0000","last_fullsized":"2024-09-21T02:19:53.889310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886303+0000","last_change":"2024-09-21T02:19:53.886303+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886303+0000","last_undegraded":"2024-09-21T02:19:53.886303+0000","last_fullsized":"2024-09-21T02:19:53.886303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902222+0000","last_change":"2024-09-21T02:19:53.902222+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902222+0000","last_undegraded":"2024-09-21T02:19:53.902222+0000","last_fullsized":"2024-09-21T02:19:53.902222+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.900530+0000","last_change":"2024-09-21T02:19:53.900530+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.900530+0000","last_undegraded":"2024-09-21T02:19:53.900530+0000","last_fullsized":"2024-09-21T02:19:53.900530+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876179+0000","last_change":"2024-09-21T02:19:54.876179+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876179+0000","last_undegraded":"2024-09-21T02:19:54.876179+0000","last_fullsized":"2024-09-21T02:19:54.876179+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877410+0000","last_change":"2024-09-21T02:19:54.877410+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877410+0000","last_undegraded":"2024-09-21T02:19:54.877410+0000","last_fullsized":"2024-09-21T02:19:54.877410+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892439+0000","last_change":"2024-09-21T02:19:53.892439+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892439+0000","last_undegraded":"2024-09-21T02:19:53.892439+0000","last_fullsized":"2024-09-21T02:19:53.892439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.903671+0000","last_change":"2024-09-21T02:19:53.903671+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.903671+0000","last_undegraded":"2024-09-21T02:19:53.903671+0000","last_fullsized":"2024-09-21T02:19:53.903671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876878+0000","last_change":"2024-09-21T02:19:54.876878+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876878+0000","last_undegraded":"2024-09-21T02:19:54.876878+0000","last_fullsized":"2024-09-21T02:19:54.876878+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883817+0000","last_change":"2024-09-21T02:19:53.883817+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883817+0000","last_undegraded":"2024-09-21T02:19:53.883817+0000","last_fullsized":"2024-09-21T02:19:53.883817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.895765+0000","last_change":"2024-09-21T02:19:53.895765+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.895765+0000","last_undegraded":"2024-09-21T02:19:53.895765+0000","last_fullsized":"2024-09-21T02:19:53.895765+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889849+0000","last_change":"2024-09-21T02:19:53.889849+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889849+0000","last_undegraded":"2024-09-21T02:19:53.889849+0000","last_fullsized":"2024-09-21T02:19:53.889849+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902590+0000","last_change":"2024-09-21T02:19:53.902590+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902590+0000","last_undegraded":"2024-09-21T02:19:53.902590+0000","last_fullsized":"2024-09-21T02:19:53.902590+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901157+0000","last_change":"2024-09-21T02:19:53.899077+0000","last_active":"2024-09-21T02:19:53.901157+0000","last_peered":"2024-09-21T02:19:53.901157+0000","last_clean":"2024-09-21T02:19:53.901157+0000","last_became_active":"2024-09-21T02:19:53.898374+0000","last_became_peered":"2024-09-21T02:19:53.898374+0000","last_unstale":"2024-09-21T02:19:53.901157+0000","last_undegraded":"2024-09-21T02:19:53.901157+0000","last_fullsized":"2024-09-21T02:19:53.901157+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:20.344188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882611+0000","last_change":"2024-09-21T02:19:53.882611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882611+0000","last_undegraded":"2024-09-21T02:19:53.882611+0000","last_fullsized":"2024-09-21T02:19:53.882611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.895668+0000","last_change":"2024-09-21T02:19:53.894376+0000","last_active":"2024-09-21T02:19:53.895668+0000","last_peered":"2024-09-21T02:19:53.895668+0000","last_clean":"2024-09-21T02:19:53.895668+0000","last_became_active":"2024-09-21T02:19:53.893849+0000","last_became_peered":"2024-09-21T02:19:53.893849+0000","last_unstale":"2024-09-21T02:19:53.895668+0000","last_undegraded":"2024-09-21T02:19:53.895668+0000","last_fullsized":"2024-09-21T02:19:53.895668+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:07.797675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880476+0000","last_change":"2024-09-21T02:19:53.880476+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880476+0000","last_undegraded":"2024-09-21T02:19:53.880476+0000","last_fullsized":"2024-09-21T02:19:53.880476+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886324+0000","last_change":"2024-09-21T02:19:53.913322+0000","last_active":"2024-09-21T02:19:54.886324+0000","last_peered":"2024-09-21T02:19:54.886324+0000","last_clean":"2024-09-21T02:19:54.886324+0000","last_became_active":"2024-09-21T02:19:53.912089+0000","last_became_peered":"2024-09-21T02:19:53.912089+0000","last_unstale":"2024-09-21T02:19:54.886324+0000","last_undegraded":"2024-09-21T02:19:54.886324+0000","last_fullsized":"2024-09-21T02:19:54.886324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:17.721839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885419+0000","last_change":"2024-09-21T02:19:53.885419+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885419+0000","last_undegraded":"2024-09-21T02:19:53.885419+0000","last_fullsized":"2024-09-21T02:19:53.885419+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880481+0000","last_change":"2024-09-21T02:19:53.911987+0000","last_active":"2024-09-21T02:19:54.880481+0000","last_peered":"2024-09-21T02:19:54.880481+0000","last_clean":"2024-09-21T02:19:54.880481+0000","last_became_active":"2024-09-21T02:19:53.911176+0000","last_became_peered":"2024-09-21T02:19:53.911176+0000","last_unstale":"2024-09-21T02:19:54.880481+0000","last_undegraded":"2024-09-21T02:19:54.880481+0000","last_fullsized":"2024-09-21T02:19:54.880481+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:55.290071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899523+0000","last_change":"2024-09-21T02:19:53.899523+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899523+0000","last_undegraded":"2024-09-21T02:19:53.899523+0000","last_fullsized":"2024-09-21T02:19:53.899523+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.877925+0000","last_change":"2024-09-21T02:19:53.906525+0000","last_active":"2024-09-21T02:19:54.877925+0000","last_peered":"2024-09-21T02:19:54.877925+0000","last_clean":"2024-09-21T02:19:54.877925+0000","last_became_active":"2024-09-21T02:19:53.905607+0000","last_became_peered":"2024-09-21T02:19:53.905607+0000","last_unstale":"2024-09-21T02:19:54.877925+0000","last_undegraded":"2024-09-21T02:19:54.877925+0000","last_fullsized":"2024-09-21T02:19:54.877925+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.829774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888730+0000","last_change":"2024-09-21T02:19:53.888730+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888730+0000","last_undegraded":"2024-09-21T02:19:53.888730+0000","last_fullsized":"2024-09-21T02:19:53.888730+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.921790+0000","last_change":"2024-09-21T02:19:53.920704+0000","last_active":"2024-09-21T02:19:53.921790+0000","last_peered":"2024-09-21T02:19:53.921790+0000","last_clean":"2024-09-21T02:19:53.921790+0000","last_became_active":"2024-09-21T02:19:53.920013+0000","last_became_peered":"2024-09-21T02:19:53.920013+0000","last_unstale":"2024-09-21T02:19:53.921790+0000","last_undegraded":"2024-09-21T02:19:53.921790+0000","last_fullsized":"2024-09-21T02:19:53.921790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:45.115276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.884740+0000","last_change":"2024-09-21T02:19:53.884740+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.884740+0000","last_undegraded":"2024-09-21T02:19:53.884740+0000","last_fullsized":"2024-09-21T02:19:53.884740+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901049+0000","last_change":"2024-09-21T02:19:53.900369+0000","last_active":"2024-09-21T02:19:53.901049+0000","last_peered":"2024-09-21T02:19:53.901049+0000","last_clean":"2024-09-21T02:19:53.901049+0000","last_became_active":"2024-09-21T02:19:53.899961+0000","last_became_peered":"2024-09-21T02:19:53.899961+0000","last_unstale":"2024-09-21T02:19:53.901049+0000","last_undegraded":"2024-09-21T02:19:53.901049+0000","last_fullsized":"2024-09-21T02:19:53.901049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:58.476667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877317+0000","last_change":"2024-09-21T02:19:54.877317+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877317+0000","last_undegraded":"2024-09-21T02:19:54.877317+0000","last_fullsized":"2024-09-21T02:19:54.877317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876958+0000","last_change":"2024-09-21T02:19:53.906244+0000","last_active":"2024-09-21T02:19:54.876958+0000","last_peered":"2024-09-21T02:19:54.876958+0000","last_clean":"2024-09-21T02:19:54.876958+0000","last_became_active":"2024-09-21T02:19:53.905406+0000","last_became_peered":"2024-09-21T02:19:53.905406+0000","last_unstale":"2024-09-21T02:19:54.876958+0000","last_undegraded":"2024-09-21T02:19:54.876958+0000","last_fullsized":"2024-09-21T02:19:54.876958+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:09.984572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876431+0000","last_change":"2024-09-21T02:19:54.876431+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876431+0000","last_undegraded":"2024-09-21T02:19:54.876431+0000","last_fullsized":"2024-09-21T02:19:54.876431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899290+0000","last_change":"2024-09-21T02:19:53.898084+0000","last_active":"2024-09-21T02:19:53.899290+0000","last_peered":"2024-09-21T02:19:53.899290+0000","last_clean":"2024-09-21T02:19:53.899290+0000","last_became_active":"2024-09-21T02:19:53.897635+0000","last_became_peered":"2024-09-21T02:19:53.897635+0000","last_unstale":"2024-09-21T02:19:53.899290+0000","last_undegraded":"2024-09-21T02:19:53.899290+0000","last_fullsized":"2024-09-21T02:19:53.899290+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:34.034584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887925+0000","last_change":"2024-09-21T02:19:53.887925+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887925+0000","last_undegraded":"2024-09-21T02:19:53.887925+0000","last_fullsized":"2024-09-21T02:19:53.887925+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.919831+0000","last_change":"2024-09-21T02:19:53.914573+0000","last_active":"2024-09-21T02:19:53.919831+0000","last_peered":"2024-09-21T02:19:53.919831+0000","last_clean":"2024-09-21T02:19:53.919831+0000","last_became_active":"2024-09-21T02:19:53.913541+0000","last_became_peered":"2024-09-21T02:19:53.913541+0000","last_unstale":"2024-09-21T02:19:53.919831+0000","last_undegraded":"2024-09-21T02:19:53.919831+0000","last_fullsized":"2024-09-21T02:19:53.919831+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:47.109959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880707+0000","last_change":"2024-09-21T02:19:53.880707+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880707+0000","last_undegraded":"2024-09-21T02:19:53.880707+0000","last_fullsized":"2024-09-21T02:19:53.880707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.900789+0000","last_change":"2024-09-21T02:19:53.899480+0000","last_active":"2024-09-21T02:19:53.900789+0000","last_peered":"2024-09-21T02:19:53.900789+0000","last_clean":"2024-09-21T02:19:53.900789+0000","last_became_active":"2024-09-21T02:19:53.899037+0000","last_became_peered":"2024-09-21T02:19:53.899037+0000","last_unstale":"2024-09-21T02:19:53.900789+0000","last_undegraded":"2024-09-21T02:19:53.900789+0000","last_fullsized":"2024-09-21T02:19:53.900789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:38.427447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887372+0000","last_change":"2024-09-21T02:19:53.887372+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887372+0000","last_undegraded":"2024-09-21T02:19:53.887372+0000","last_fullsized":"2024-09-21T02:19:53.887372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.906801+0000","last_change":"2024-09-21T02:19:53.904993+0000","last_active":"2024-09-21T02:19:53.906801+0000","last_peered":"2024-09-21T02:19:53.906801+0000","last_clean":"2024-09-21T02:19:53.906801+0000","last_became_active":"2024-09-21T02:19:53.904509+0000","last_became_peered":"2024-09-21T02:19:53.904509+0000","last_unstale":"2024-09-21T02:19:53.906801+0000","last_undegraded":"2024-09-21T02:19:53.906801+0000","last_fullsized":"2024-09-21T02:19:53.906801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:48.851528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889611+0000","last_change":"2024-09-21T02:19:53.889611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889611+0000","last_undegraded":"2024-09-21T02:19:53.889611+0000","last_fullsized":"2024-09-21T02:19:53.889611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917550+0000","last_change":"2024-09-21T02:19:53.903917+0000","last_active":"2024-09-21T02:19:53.917550+0000","last_peered":"2024-09-21T02:19:53.917550+0000","last_clean":"2024-09-21T02:19:53.917550+0000","last_became_active":"2024-09-21T02:19:53.903089+0000","last_became_peered":"2024-09-21T02:19:53.903089+0000","last_unstale":"2024-09-21T02:19:53.917550+0000","last_undegraded":"2024-09-21T02:19:53.917550+0000","last_fullsized":"2024-09-21T02:19:53.917550+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:34.884559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892462+0000","last_change":"2024-09-21T02:19:53.892462+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892462+0000","last_undegraded":"2024-09-21T02:19:53.892462+0000","last_fullsized":"2024-09-21T02:19:53.892462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878927+0000","last_change":"2024-09-21T02:19:53.896469+0000","last_active":"2024-09-21T02:19:54.878927+0000","last_peered":"2024-09-21T02:19:54.878927+0000","last_clean":"2024-09-21T02:19:54.878927+0000","last_became_active":"2024-09-21T02:19:53.895529+0000","last_became_peered":"2024-09-21T02:19:53.895529+0000","last_unstale":"2024-09-21T02:19:54.878927+0000","last_undegraded":"2024-09-21T02:19:54.878927+0000","last_fullsized":"2024-09-21T02:19:54.878927+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:30.370234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882720+0000","last_change":"2024-09-21T02:19:53.882720+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882720+0000","last_undegraded":"2024-09-21T02:19:53.882720+0000","last_fullsized":"2024-09-21T02:19:53.882720+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920349+0000","last_change":"2024-09-21T02:19:53.919682+0000","last_active":"2024-09-21T02:19:53.920349+0000","last_peered":"2024-09-21T02:19:53.920349+0000","last_clean":"2024-09-21T02:19:53.920349+0000","last_became_active":"2024-09-21T02:19:53.919315+0000","last_became_peered":"2024-09-21T02:19:53.919315+0000","last_unstale":"2024-09-21T02:19:53.920349+0000","last_undegraded":"2024-09-21T02:19:53.920349+0000","last_fullsized":"2024-09-21T02:19:53.920349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:33.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887258+0000","last_change":"2024-09-21T02:19:53.887258+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887258+0000","last_undegraded":"2024-09-21T02:19:53.887258+0000","last_fullsized":"2024-09-21T02:19:53.887258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.908256+0000","last_change":"2024-09-21T02:19:53.907585+0000","last_active":"2024-09-21T02:19:53.908256+0000","last_peered":"2024-09-21T02:19:53.908256+0000","last_clean":"2024-09-21T02:19:53.908256+0000","last_became_active":"2024-09-21T02:19:53.906977+0000","last_became_peered":"2024-09-21T02:19:53.906977+0000","last_unstale":"2024-09-21T02:19:53.908256+0000","last_undegraded":"2024-09-21T02:19:53.908256+0000","last_fullsized":"2024-09-21T02:19:53.908256+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:18.433443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886728+0000","last_change":"2024-09-21T02:19:53.886728+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886728+0000","last_undegraded":"2024-09-21T02:19:53.886728+0000","last_fullsized":"2024-09-21T02:19:53.886728+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886814+0000","last_change":"2024-09-21T02:19:53.904893+0000","last_active":"2024-09-21T02:19:54.886814+0000","last_peered":"2024-09-21T02:19:54.886814+0000","last_clean":"2024-09-21T02:19:54.886814+0000","last_became_active":"2024-09-21T02:19:53.904064+0000","last_became_peered":"2024-09-21T02:19:53.904064+0000","last_unstale":"2024-09-21T02:19:54.886814+0000","last_undegraded":"2024-09-21T02:19:54.886814+0000","last_fullsized":"2024-09-21T02:19:54.886814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:33.874089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879257+0000","last_change":"2024-09-21T02:19:54.879257+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879257+0000","last_undegraded":"2024-09-21T02:19:54.879257+0000","last_fullsized":"2024-09-21T02:19:54.879257+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899521+0000","last_change":"2024-09-21T02:19:53.898390+0000","last_active":"2024-09-21T02:19:53.899521+0000","last_peered":"2024-09-21T02:19:53.899521+0000","last_clean":"2024-09-21T02:19:53.899521+0000","last_became_active":"2024-09-21T02:19:53.897822+0000","last_became_peered":"2024-09-21T02:19:53.897822+0000","last_unstale":"2024-09-21T02:19:53.899521+0000","last_undegraded":"2024-09-21T02:19:53.899521+0000","last_fullsized":"2024-09-21T02:19:53.899521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:11.038001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.881173+0000","last_change":"2024-09-21T02:19:54.881173+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.881173+0000","last_undegraded":"2024-09-21T02:19:54.881173+0000","last_fullsized":"2024-09-21T02:19:54.881173+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898678+0000","last_change":"2024-09-21T02:19:53.889955+0000","last_active":"2024-09-21T02:19:53.898678+0000","last_peered":"2024-09-21T02:19:53.898678+0000","last_clean":"2024-09-21T02:19:53.898678+0000","last_became_active":"2024-09-21T02:19:53.889393+0000","last_became_peered":"2024-09-21T02:19:53.889393+0000","last_unstale":"2024-09-21T02:19:53.898678+0000","last_undegraded":"2024-09-21T02:19:53.898678+0000","last_fullsized":"2024-09-21T02:19:53.898678+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:32.922018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881657+0000","last_change":"2024-09-21T02:19:53.881657+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881657+0000","last_undegraded":"2024-09-21T02:19:53.881657+0000","last_fullsized":"2024-09-21T02:19:53.881657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898735+0000","last_change":"2024-09-21T02:19:53.897209+0000","last_active":"2024-09-21T02:19:53.898735+0000","last_peered":"2024-09-21T02:19:53.898735+0000","last_clean":"2024-09-21T02:19:53.898735+0000","last_became_active":"2024-09-21T02:19:53.896800+0000","last_became_peered":"2024-09-21T02:19:53.896800+0000","last_unstale":"2024-09-21T02:19:53.898735+0000","last_undegraded":"2024-09-21T02:19:53.898735+0000","last_fullsized":"2024-09-21T02:19:53.898735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:31.737184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.904140+0000","last_change":"2024-09-21T02:19:53.904140+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.904140+0000","last_undegraded":"2024-09-21T02:19:53.904140+0000","last_fullsized":"2024-09-21T02:19:53.904140+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920415+0000","last_change":"2024-09-21T02:19:53.915141+0000","last_active":"2024-09-21T02:19:53.920415+0000","last_peered":"2024-09-21T02:19:53.920415+0000","last_clean":"2024-09-21T02:19:53.920415+0000","last_became_active":"2024-09-21T02:19:53.914156+0000","last_became_peered":"2024-09-21T02:19:53.914156+0000","last_unstale":"2024-09-21T02:19:53.920415+0000","last_undegraded":"2024-09-21T02:19:53.920415+0000","last_fullsized":"2024-09-21T02:19:53.920415+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:52.051938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890763+0000","last_change":"2024-09-21T02:19:53.890763+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890763+0000","last_undegraded":"2024-09-21T02:19:53.890763+0000","last_fullsized":"2024-09-21T02:19:53.890763+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917649+0000","last_change":"2024-09-21T02:19:53.906690+0000","last_active":"2024-09-21T02:19:53.917649+0000","last_peered":"2024-09-21T02:19:53.917649+0000","last_clean":"2024-09-21T02:19:53.917649+0000","last_became_active":"2024-09-21T02:19:53.905289+0000","last_became_peered":"2024-09-21T02:19:53.905289+0000","last_unstale":"2024-09-21T02:19:53.917649+0000","last_undegraded":"2024-09-21T02:19:53.917649+0000","last_fullsized":"2024-09-21T02:19:53.917649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:25.254722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881976+0000","last_change":"2024-09-21T02:19:53.881976+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881976+0000","last_undegraded":"2024-09-21T02:19:53.881976+0000","last_fullsized":"2024-09-21T02:19:53.881976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876219+0000","last_change":"2024-09-21T02:19:53.904489+0000","last_active":"2024-09-21T02:19:54.876219+0000","last_peered":"2024-09-21T02:19:54.876219+0000","last_clean":"2024-09-21T02:19:54.876219+0000","last_became_active":"2024-09-21T02:19:53.903679+0000","last_became_peered":"2024-09-21T02:19:53.903679+0000","last_unstale":"2024-09-21T02:19:54.876219+0000","last_undegraded":"2024-09-21T02:19:54.876219+0000","last_fullsized":"2024-09-21T02:19:54.876219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:00.611818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.894458+0000","last_change":"2024-09-21T02:19:53.894458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.894458+0000","last_undegraded":"2024-09-21T02:19:53.894458+0000","last_fullsized":"2024-09-21T02:19:53.894458+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920396+0000","last_change":"2024-09-21T02:19:53.917476+0000","last_active":"2024-09-21T02:19:53.920396+0000","last_peered":"2024-09-21T02:19:53.920396+0000","last_clean":"2024-09-21T02:19:53.920396+0000","last_became_active":"2024-09-21T02:19:53.916672+0000","last_became_peered":"2024-09-21T02:19:53.916672+0000","last_unstale":"2024-09-21T02:19:53.920396+0000","last_undegraded":"2024-09-21T02:19:53.920396+0000","last_fullsized":"2024-09-21T02:19:53.920396+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:04.729381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878762+0000","last_change":"2024-09-21T02:19:54.878762+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878762+0000","last_undegraded":"2024-09-21T02:19:54.878762+0000","last_fullsized":"2024-09-21T02:19:54.878762+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901861+0000","last_change":"2024-09-21T02:19:53.900842+0000","last_active":"2024-09-21T02:19:53.901861+0000","last_peered":"2024-09-21T02:19:53.901861+0000","last_clean":"2024-09-21T02:19:53.901861+0000","last_became_active":"2024-09-21T02:19:53.900353+0000","last_became_peered":"2024-09-21T02:19:53.900353+0000","last_unstale":"2024-09-21T02:19:53.901861+0000","last_undegraded":"2024-09-21T02:19:53.901861+0000","last_fullsized":"2024-09-21T02:19:53.901861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.302971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890952+0000","last_change":"2024-09-21T02:19:53.890952+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890952+0000","last_undegraded":"2024-09-21T02:19:53.890952+0000","last_fullsized":"2024-09-21T02:19:53.890952+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920113+0000","last_change":"2024-09-21T02:19:53.916530+0000","last_active":"2024-09-21T02:19:53.920113+0000","last_peered":"2024-09-21T02:19:53.920113+0000","last_clean":"2024-09-21T02:19:53.920113+0000","last_became_active":"2024-09-21T02:19:53.915419+0000","last_became_peered":"2024-09-21T02:19:53.915419+0000","last_unstale":"2024-09-21T02:19:53.920113+0000","last_undegraded":"2024-09-21T02:19:53.920113+0000","last_fullsized":"2024-09-21T02:19:53.920113+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:28.827382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881579+0000","last_change":"2024-09-21T02:19:53.881579+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881579+0000","last_undegraded":"2024-09-21T02:19:53.881579+0000","last_fullsized":"2024-09-21T02:19:53.881579+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.918803+0000","last_change":"2024-09-21T02:19:53.911432+0000","last_active":"2024-09-21T02:19:53.918803+0000","last_peered":"2024-09-21T02:19:53.918803+0000","last_clean":"2024-09-21T02:19:53.918803+0000","last_became_active":"2024-09-21T02:19:53.910658+0000","last_became_peered":"2024-09-21T02:19:53.910658+0000","last_unstale":"2024-09-21T02:19:53.918803+0000","last_undegraded":"2024-09-21T02:19:53.918803+0000","last_fullsized":"2024-09-21T02:19:53.918803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:56.513763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889059+0000","last_change":"2024-09-21T02:19:53.889059+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889059+0000","last_undegraded":"2024-09-21T02:19:53.889059+0000","last_fullsized":"2024-09-21T02:19:53.889059+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878522+0000","last_change":"2024-09-21T02:19:53.909822+0000","last_active":"2024-09-21T02:19:54.878522+0000","last_peered":"2024-09-21T02:19:54.878522+0000","last_clean":"2024-09-21T02:19:54.878522+0000","last_became_active":"2024-09-21T02:19:53.908919+0000","last_became_peered":"2024-09-21T02:19:53.908919+0000","last_unstale":"2024-09-21T02:19:54.878522+0000","last_undegraded":"2024-09-21T02:19:54.878522+0000","last_fullsized":"2024-09-21T02:19:54.878522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.216241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888015+0000","last_change":"2024-09-21T02:19:53.888015+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888015+0000","last_undegraded":"2024-09-21T02:19:53.888015+0000","last_fullsized":"2024-09-21T02:19:53.888015+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.916193+0000","last_change":"2024-09-21T02:19:53.915286+0000","last_active":"2024-09-21T02:19:53.916193+0000","last_peered":"2024-09-21T02:19:53.916193+0000","last_clean":"2024-09-21T02:19:53.916193+0000","last_became_active":"2024-09-21T02:19:53.914782+0000","last_became_peered":"2024-09-21T02:19:53.914782+0000","last_unstale":"2024-09-21T02:19:53.916193+0000","last_undegraded":"2024-09-21T02:19:53.916193+0000","last_fullsized":"2024-09-21T02:19:53.916193+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:39.555502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899037+0000","last_change":"2024-09-21T02:19:53.899037+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899037+0000","last_undegraded":"2024-09-21T02:19:53.899037+0000","last_fullsized":"2024-09-21T02:19:53.899037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880986+0000","last_change":"2024-09-21T02:19:53.896536+0000","last_active":"2024-09-21T02:19:54.880986+0000","last_peered":"2024-09-21T02:19:54.880986+0000","last_clean":"2024-09-21T02:19:54.880986+0000","last_became_active":"2024-09-21T02:19:53.895589+0000","last_became_peered":"2024-09-21T02:19:53.895589+0000","last_unstale":"2024-09-21T02:19:54.880986+0000","last_undegraded":"2024-09-21T02:19:54.880986+0000","last_fullsized":"2024-09-21T02:19:54.880986+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:17.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890723+0000","last_change":"2024-09-21T02:19:53.890723+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890723+0000","last_undegraded":"2024-09-21T02:19:53.890723+0000","last_fullsized":"2024-09-21T02:19:53.890723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917794+0000","last_change":"2024-09-21T02:19:53.908787+0000","last_active":"2024-09-21T02:19:53.917794+0000","last_peered":"2024-09-21T02:19:53.917794+0000","last_clean":"2024-09-21T02:19:53.917794+0000","last_became_active":"2024-09-21T02:19:53.907954+0000","last_became_peered":"2024-09-21T02:19:53.907954+0000","last_unstale":"2024-09-21T02:19:53.917794+0000","last_undegraded":"2024-09-21T02:19:53.917794+0000","last_fullsized":"2024-09-21T02:19:53.917794+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:20.813712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883172+0000","last_change":"2024-09-21T02:19:53.883172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883172+0000","last_undegraded":"2024-09-21T02:19:53.883172+0000","last_fullsized":"2024-09-21T02:19:53.883172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.915086+0000","last_change":"2024-09-21T02:19:53.913344+0000","last_active":"2024-09-21T02:19:53.915086+0000","last_peered":"2024-09-21T02:19:53.915086+0000","last_clean":"2024-09-21T02:19:53.915086+0000","last_became_active":"2024-09-21T02:19:53.912800+0000","last_became_peered":"2024-09-21T02:19:53.912800+0000","last_unstale":"2024-09-21T02:19:53.915086+0000","last_undegraded":"2024-09-21T02:19:53.915086+0000","last_fullsized":"2024-09-21T02:19:53.915086+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:13.374558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":308,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.904728+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:19:53.904728+0000","last_peered":"2024-09-21T02:19:53.904728+0000","last_clean":"2024-09-21T02:19:53.904728+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:19:53.904728+0000","last_undegraded":"2024-09-21T02:19:53.904728+0000","last_fullsized":"2024-09-21T02:19:53.904728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":625482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41500,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330340,"statfs":{"total":96636764160,"available":96594268160,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:19:59.852 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3f is not active+clean 2024-09-21T02:19:59.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.879824+0000', 'last_change': '2024-09-21T02:19:54.879824+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.879824+0000', 'last_undegraded': '2024-09-21T02:19:54.879824+0000', 'last_fullsized': '2024-09-21T02:19:54.879824+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.852 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3e is not active+clean 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.879361+0000', 'last_change': '2024-09-21T02:19:54.879361+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.879361+0000', 'last_undegraded': '2024-09-21T02:19:54.879361+0000', 'last_fullsized': '2024-09-21T02:19:54.879361+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3d is not active+clean 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.885776+0000', 'last_change': '2024-09-21T02:19:53.885776+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.885776+0000', 'last_undegraded': '2024-09-21T02:19:53.885776+0000', 'last_fullsized': '2024-09-21T02:19:53.885776+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3c is not active+clean 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.886129+0000', 'last_change': '2024-09-21T02:19:54.880458+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.886129+0000', 'last_undegraded': '2024-09-21T02:19:54.886129+0000', 'last_fullsized': '2024-09-21T02:19:54.886129+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3b is not active+clean 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.886975+0000', 'last_change': '2024-09-21T02:19:53.886975+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.886975+0000', 'last_undegraded': '2024-09-21T02:19:53.886975+0000', 'last_fullsized': '2024-09-21T02:19:53.886975+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3a is not active+clean 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.889655+0000', 'last_change': '2024-09-21T02:19:53.889655+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.889655+0000', 'last_undegraded': '2024-09-21T02:19:53.889655+0000', 'last_fullsized': '2024-09-21T02:19:53.889655+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:PG 19.39 is not active+clean 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.886780+0000', 'last_change': '2024-09-21T02:19:53.886780+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.886780+0000', 'last_undegraded': '2024-09-21T02:19:53.886780+0000', 'last_fullsized': '2024-09-21T02:19:53.886780+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:19:59.853 INFO:tasks.ceph.ceph_manager.ceph:PG 19.38 is not active+clean 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.887128+0000', 'last_change': '2024-09-21T02:19:53.887128+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.887128+0000', 'last_undegraded': '2024-09-21T02:19:53.887128+0000', 'last_fullsized': '2024-09-21T02:19:53.887128+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:PG 19.37 is not active+clean 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.885049+0000', 'last_change': '2024-09-21T02:19:53.885049+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.885049+0000', 'last_undegraded': '2024-09-21T02:19:53.885049+0000', 'last_fullsized': '2024-09-21T02:19:53.885049+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:PG 19.36 is not active+clean 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.878396+0000', 'last_change': '2024-09-21T02:19:54.878396+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.878396+0000', 'last_undegraded': '2024-09-21T02:19:54.878396+0000', 'last_fullsized': '2024-09-21T02:19:54.878396+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:PG 19.35 is not active+clean 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.877485+0000', 'last_change': '2024-09-21T02:19:54.877485+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.877485+0000', 'last_undegraded': '2024-09-21T02:19:54.877485+0000', 'last_fullsized': '2024-09-21T02:19:54.877485+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:PG 19.34 is not active+clean 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.882109+0000', 'last_change': '2024-09-21T02:19:53.882109+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.882109+0000', 'last_undegraded': '2024-09-21T02:19:53.882109+0000', 'last_fullsized': '2024-09-21T02:19:53.882109+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:PG 19.33 is not active+clean 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.876172+0000', 'last_change': '2024-09-21T02:19:54.876172+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.876172+0000', 'last_undegraded': '2024-09-21T02:19:54.876172+0000', 'last_fullsized': '2024-09-21T02:19:54.876172+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:19:59.854 INFO:tasks.ceph.ceph_manager.ceph:PG 19.32 is not active+clean 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.892017+0000', 'last_change': '2024-09-21T02:19:53.892017+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.892017+0000', 'last_undegraded': '2024-09-21T02:19:53.892017+0000', 'last_fullsized': '2024-09-21T02:19:53.892017+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:PG 19.31 is not active+clean 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.897505+0000', 'last_change': '2024-09-21T02:19:53.897505+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.897505+0000', 'last_undegraded': '2024-09-21T02:19:53.897505+0000', 'last_fullsized': '2024-09-21T02:19:53.897505+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:PG 19.30 is not active+clean 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.902786+0000', 'last_change': '2024-09-21T02:19:53.902786+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.902786+0000', 'last_undegraded': '2024-09-21T02:19:53.902786+0000', 'last_fullsized': '2024-09-21T02:19:53.902786+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2f is not active+clean 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.883879+0000', 'last_change': '2024-09-21T02:19:53.883879+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.883879+0000', 'last_undegraded': '2024-09-21T02:19:53.883879+0000', 'last_fullsized': '2024-09-21T02:19:53.883879+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2e is not active+clean 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.897882+0000', 'last_change': '2024-09-21T02:19:53.897882+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.897882+0000', 'last_undegraded': '2024-09-21T02:19:53.897882+0000', 'last_fullsized': '2024-09-21T02:19:53.897882+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2d is not active+clean 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.886319+0000', 'last_change': '2024-09-21T02:19:53.886319+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.886319+0000', 'last_undegraded': '2024-09-21T02:19:53.886319+0000', 'last_fullsized': '2024-09-21T02:19:53.886319+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.855 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2c is not active+clean 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.899311+0000', 'last_change': '2024-09-21T02:19:53.899311+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.899311+0000', 'last_undegraded': '2024-09-21T02:19:53.899311+0000', 'last_fullsized': '2024-09-21T02:19:53.899311+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2b is not active+clean 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.889310+0000', 'last_change': '2024-09-21T02:19:53.889310+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.889310+0000', 'last_undegraded': '2024-09-21T02:19:53.889310+0000', 'last_fullsized': '2024-09-21T02:19:53.889310+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2a is not active+clean 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.886303+0000', 'last_change': '2024-09-21T02:19:53.886303+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.886303+0000', 'last_undegraded': '2024-09-21T02:19:53.886303+0000', 'last_fullsized': '2024-09-21T02:19:53.886303+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:PG 19.29 is not active+clean 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.902222+0000', 'last_change': '2024-09-21T02:19:53.902222+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.902222+0000', 'last_undegraded': '2024-09-21T02:19:53.902222+0000', 'last_fullsized': '2024-09-21T02:19:53.902222+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:PG 19.28 is not active+clean 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.900530+0000', 'last_change': '2024-09-21T02:19:53.900530+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.900530+0000', 'last_undegraded': '2024-09-21T02:19:53.900530+0000', 'last_fullsized': '2024-09-21T02:19:53.900530+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:PG 19.27 is not active+clean 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.876179+0000', 'last_change': '2024-09-21T02:19:54.876179+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.876179+0000', 'last_undegraded': '2024-09-21T02:19:54.876179+0000', 'last_fullsized': '2024-09-21T02:19:54.876179+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T02:19:59.856 INFO:tasks.ceph.ceph_manager.ceph:PG 19.26 is not active+clean 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.877410+0000', 'last_change': '2024-09-21T02:19:54.877410+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.877410+0000', 'last_undegraded': '2024-09-21T02:19:54.877410+0000', 'last_fullsized': '2024-09-21T02:19:54.877410+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': ['4', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:PG 19.25 is not active+clean 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.892439+0000', 'last_change': '2024-09-21T02:19:53.892439+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.892439+0000', 'last_undegraded': '2024-09-21T02:19:53.892439+0000', 'last_fullsized': '2024-09-21T02:19:53.892439+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:PG 19.24 is not active+clean 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.903671+0000', 'last_change': '2024-09-21T02:19:53.903671+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.903671+0000', 'last_undegraded': '2024-09-21T02:19:53.903671+0000', 'last_fullsized': '2024-09-21T02:19:53.903671+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:PG 19.23 is not active+clean 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.876878+0000', 'last_change': '2024-09-21T02:19:54.876878+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.876878+0000', 'last_undegraded': '2024-09-21T02:19:54.876878+0000', 'last_fullsized': '2024-09-21T02:19:54.876878+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:PG 19.22 is not active+clean 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.883817+0000', 'last_change': '2024-09-21T02:19:53.883817+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.883817+0000', 'last_undegraded': '2024-09-21T02:19:53.883817+0000', 'last_fullsized': '2024-09-21T02:19:53.883817+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:PG 19.21 is not active+clean 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.895765+0000', 'last_change': '2024-09-21T02:19:53.895765+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.895765+0000', 'last_undegraded': '2024-09-21T02:19:53.895765+0000', 'last_fullsized': '2024-09-21T02:19:53.895765+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:19:59.857 INFO:tasks.ceph.ceph_manager.ceph:PG 19.20 is not active+clean 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.889849+0000', 'last_change': '2024-09-21T02:19:53.889849+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.889849+0000', 'last_undegraded': '2024-09-21T02:19:53.889849+0000', 'last_fullsized': '2024-09-21T02:19:53.889849+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1e is not active+clean 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.902590+0000', 'last_change': '2024-09-21T02:19:53.902590+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.902590+0000', 'last_undegraded': '2024-09-21T02:19:53.902590+0000', 'last_fullsized': '2024-09-21T02:19:53.902590+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1f is not active+clean 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.882611+0000', 'last_change': '2024-09-21T02:19:53.882611+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.882611+0000', 'last_undegraded': '2024-09-21T02:19:53.882611+0000', 'last_fullsized': '2024-09-21T02:19:53.882611+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1c is not active+clean 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.880476+0000', 'last_change': '2024-09-21T02:19:53.880476+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.880476+0000', 'last_undegraded': '2024-09-21T02:19:53.880476+0000', 'last_fullsized': '2024-09-21T02:19:53.880476+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1d is not active+clean 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.885419+0000', 'last_change': '2024-09-21T02:19:53.885419+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.885419+0000', 'last_undegraded': '2024-09-21T02:19:53.885419+0000', 'last_fullsized': '2024-09-21T02:19:53.885419+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1a is not active+clean 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.899523+0000', 'last_change': '2024-09-21T02:19:53.899523+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.899523+0000', 'last_undegraded': '2024-09-21T02:19:53.899523+0000', 'last_fullsized': '2024-09-21T02:19:53.899523+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:19:59.858 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1b is not active+clean 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.888730+0000', 'last_change': '2024-09-21T02:19:53.888730+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.888730+0000', 'last_undegraded': '2024-09-21T02:19:53.888730+0000', 'last_fullsized': '2024-09-21T02:19:53.888730+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:PG 19.18 is not active+clean 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.884740+0000', 'last_change': '2024-09-21T02:19:53.884740+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.884740+0000', 'last_undegraded': '2024-09-21T02:19:53.884740+0000', 'last_fullsized': '2024-09-21T02:19:53.884740+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:PG 19.19 is not active+clean 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.877317+0000', 'last_change': '2024-09-21T02:19:54.877317+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.877317+0000', 'last_undegraded': '2024-09-21T02:19:54.877317+0000', 'last_fullsized': '2024-09-21T02:19:54.877317+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:PG 19.16 is not active+clean 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.876431+0000', 'last_change': '2024-09-21T02:19:54.876431+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.876431+0000', 'last_undegraded': '2024-09-21T02:19:54.876431+0000', 'last_fullsized': '2024-09-21T02:19:54.876431+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:PG 19.17 is not active+clean 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.887925+0000', 'last_change': '2024-09-21T02:19:53.887925+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.887925+0000', 'last_undegraded': '2024-09-21T02:19:53.887925+0000', 'last_fullsized': '2024-09-21T02:19:53.887925+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:PG 19.14 is not active+clean 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.880707+0000', 'last_change': '2024-09-21T02:19:53.880707+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.880707+0000', 'last_undegraded': '2024-09-21T02:19:53.880707+0000', 'last_fullsized': '2024-09-21T02:19:53.880707+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.859 INFO:tasks.ceph.ceph_manager.ceph:PG 19.15 is not active+clean 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.887372+0000', 'last_change': '2024-09-21T02:19:53.887372+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.887372+0000', 'last_undegraded': '2024-09-21T02:19:53.887372+0000', 'last_fullsized': '2024-09-21T02:19:53.887372+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:PG 19.13 is not active+clean 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.889611+0000', 'last_change': '2024-09-21T02:19:53.889611+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.889611+0000', 'last_undegraded': '2024-09-21T02:19:53.889611+0000', 'last_fullsized': '2024-09-21T02:19:53.889611+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:PG 19.10 is not active+clean 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.892462+0000', 'last_change': '2024-09-21T02:19:53.892462+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.892462+0000', 'last_undegraded': '2024-09-21T02:19:53.892462+0000', 'last_fullsized': '2024-09-21T02:19:53.892462+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:PG 19.11 is not active+clean 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.882720+0000', 'last_change': '2024-09-21T02:19:53.882720+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.882720+0000', 'last_undegraded': '2024-09-21T02:19:53.882720+0000', 'last_fullsized': '2024-09-21T02:19:53.882720+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:PG 19.e is not active+clean 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.887258+0000', 'last_change': '2024-09-21T02:19:53.887258+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.887258+0000', 'last_undegraded': '2024-09-21T02:19:53.887258+0000', 'last_fullsized': '2024-09-21T02:19:53.887258+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:PG 19.f is not active+clean 2024-09-21T02:19:59.860 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.886728+0000', 'last_change': '2024-09-21T02:19:53.886728+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.886728+0000', 'last_undegraded': '2024-09-21T02:19:53.886728+0000', 'last_fullsized': '2024-09-21T02:19:53.886728+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:PG 19.c is not active+clean 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.879257+0000', 'last_change': '2024-09-21T02:19:54.879257+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.879257+0000', 'last_undegraded': '2024-09-21T02:19:54.879257+0000', 'last_fullsized': '2024-09-21T02:19:54.879257+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:PG 19.d is not active+clean 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.881173+0000', 'last_change': '2024-09-21T02:19:54.881173+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.881173+0000', 'last_undegraded': '2024-09-21T02:19:54.881173+0000', 'last_fullsized': '2024-09-21T02:19:54.881173+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:PG 19.a is not active+clean 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.881657+0000', 'last_change': '2024-09-21T02:19:53.881657+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.881657+0000', 'last_undegraded': '2024-09-21T02:19:53.881657+0000', 'last_fullsized': '2024-09-21T02:19:53.881657+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:PG 19.b is not active+clean 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.904140+0000', 'last_change': '2024-09-21T02:19:53.904140+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.904140+0000', 'last_undegraded': '2024-09-21T02:19:53.904140+0000', 'last_fullsized': '2024-09-21T02:19:53.904140+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:PG 19.8 is not active+clean 2024-09-21T02:19:59.861 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.890763+0000', 'last_change': '2024-09-21T02:19:53.890763+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.890763+0000', 'last_undegraded': '2024-09-21T02:19:53.890763+0000', 'last_fullsized': '2024-09-21T02:19:53.890763+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.9 is not active+clean 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.881976+0000', 'last_change': '2024-09-21T02:19:53.881976+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.881976+0000', 'last_undegraded': '2024-09-21T02:19:53.881976+0000', 'last_fullsized': '2024-09-21T02:19:53.881976+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.6 is not active+clean 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.894458+0000', 'last_change': '2024-09-21T02:19:53.894458+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.894458+0000', 'last_undegraded': '2024-09-21T02:19:53.894458+0000', 'last_fullsized': '2024-09-21T02:19:53.894458+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.7 is not active+clean 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 117, 'state': 'creating+activating', 'last_fresh': '2024-09-21T02:19:54.878762+0000', 'last_change': '2024-09-21T02:19:54.878762+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:54.878762+0000', 'last_undegraded': '2024-09-21T02:19:54.878762+0000', 'last_fullsized': '2024-09-21T02:19:54.878762+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.4 is not active+clean 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.890952+0000', 'last_change': '2024-09-21T02:19:53.890952+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.890952+0000', 'last_undegraded': '2024-09-21T02:19:53.890952+0000', 'last_fullsized': '2024-09-21T02:19:53.890952+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.5 is not active+clean 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.881579+0000', 'last_change': '2024-09-21T02:19:53.881579+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.881579+0000', 'last_undegraded': '2024-09-21T02:19:53.881579+0000', 'last_fullsized': '2024-09-21T02:19:53.881579+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2 is not active+clean 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.889059+0000', 'last_change': '2024-09-21T02:19:53.889059+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.889059+0000', 'last_undegraded': '2024-09-21T02:19:53.889059+0000', 'last_fullsized': '2024-09-21T02:19:53.889059+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.862 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3 is not active+clean 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.888015+0000', 'last_change': '2024-09-21T02:19:53.888015+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.888015+0000', 'last_undegraded': '2024-09-21T02:19:53.888015+0000', 'last_fullsized': '2024-09-21T02:19:53.888015+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:PG 19.0 is not active+clean 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.899037+0000', 'last_change': '2024-09-21T02:19:53.899037+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.899037+0000', 'last_undegraded': '2024-09-21T02:19:53.899037+0000', 'last_fullsized': '2024-09-21T02:19:53.899037+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1 is not active+clean 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.890723+0000', 'last_change': '2024-09-21T02:19:53.890723+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.890723+0000', 'last_undegraded': '2024-09-21T02:19:53.890723+0000', 'last_fullsized': '2024-09-21T02:19:53.890723+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:PG 19.12 is not active+clean 2024-09-21T02:19:59.863 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-21T02:19:53.883172+0000', 'last_change': '2024-09-21T02:19:53.883172+0000', 'last_active': '2024-09-21T02:19:53.856540+0000', 'last_peered': '2024-09-21T02:19:53.856540+0000', 'last_clean': '2024-09-21T02:19:53.856540+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T02:19:53.883172+0000', 'last_undegraded': '2024-09-21T02:19:53.883172+0000', 'last_fullsized': '2024-09-21T02:19:53.883172+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'last_clean_scrub_stamp': '2024-09-21T02:19:53.856540+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T02:19:59.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 -- 172.21.15.170:0/1403350429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00cf280 msgr2=0x7f2de00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/1403350429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00cf280 0x7f2de00cf690 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2de00b7050 tx=0x7f2dd801cac0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 -- 172.21.15.170:0/1403350429 shutdown_connections 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/1403350429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00cf280 0x7f2de00cf690 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/1403350429 >> v2:172.21.15.191:3300/0 conn(0x7f2de00d1490 0x7f2de00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 -- 172.21.15.170:0/1403350429 >> 172.21.15.170:0/1403350429 conn(0x7f2de009d830 msgr2=0x7f2de008bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:00.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 -- 172.21.15.170:0/1403350429 shutdown_connections 2024-09-21T02:20:00.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 -- 172.21.15.170:0/1403350429 wait complete. 2024-09-21T02:20:00.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 Processor -- start 2024-09-21T02:20:00.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.038+0000 7f2de4dd5640 1 -- start start 2024-09-21T02:20:00.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de00cf280 0x7f2de015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 0x7f2de0159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00d3630 0x7f2de0160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2de016bd60 con 0x7f2de00d1490 2024-09-21T02:20:00.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2de016bbe0 con 0x7f2de00d3630 2024-09-21T02:20:00.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2de016bee0 con 0x7f2de00cf280 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 0x7f2de0159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 0x7f2de0159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59122/0 (socket says 172.21.15.170:59122) 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 -- 172.21.15.170:0/3175287779 learned_addr learned my addr 172.21.15.170:0/3175287779 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dded9d640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00d3630 0x7f2de0160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dde59c640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de00cf280 0x7f2de015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 -- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de00cf280 msgr2=0x7f2de015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de00cf280 0x7f2de015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 -- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00d3630 msgr2=0x7f2de0160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00d3630 0x7f2de0160e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 -- 172.21.15.170:0/3175287779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de00b2fe0 con 0x7f2de00d1490 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dddd9b640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 0x7f2de0159b30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000c970 tx=0x7f2dd000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0007bb0 con 0x7f2de00d1490 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd0005020 con 0x7f2de00d1490 2024-09-21T02:20:00.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0007e60 con 0x7f2de00d1490 2024-09-21T02:20:00.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de015cc10 con 0x7f2de00d1490 2024-09-21T02:20:00.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de015e500 con 0x7f2de00d1490 2024-09-21T02:20:00.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de00cf690 con 0x7f2de00d1490 2024-09-21T02:20:00.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f2dd0005470 con 0x7f2de00d1490 2024-09-21T02:20:00.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dceffd640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dc0073680 0x7f2dc0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f2dd0095030 con 0x7f2de00d1490 2024-09-21T02:20:00.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dde59c640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dc0073680 0x7f2dc0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.042+0000 7f2dde59c640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dc0073680 0x7f2dc0075b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4001570 tx=0x7f2dd4009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:00.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.046+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2dd0014030 con 0x7f2de00d1490 2024-09-21T02:20:00.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2de015c430 con 0x7f2de00d1490 2024-09-21T02:20:00.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2dceffd640 1 -- 172.21.15.170:0/3175287779 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1287 (secure 0 0 0) 0x7f2dd005f910 con 0x7f2de00d1490 2024-09-21T02:20:00.269 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:00.269 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":420,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":118,"num_osds":8,"num_up_osds":8,"osd_up_since":1726884793,"num_in_osds":8,"osd_in_since":1726884783,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":50},{"state_name":"active+clean","count":33},{"state_name":"creating+activating","count":14}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":289525760,"bytes_avail":772804587520,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":81,"btime":"2024-09-21T02:19:57:928053+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":7614}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-21T02:19:48.891915+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T02:20:00.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dc0073680 msgr2=0x7f2dc0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dc0073680 0x7f2dc0075b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4001570 tx=0x7f2dd4009040 comp rx=0 tx=0).stop 2024-09-21T02:20:00.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 msgr2=0x7f2de0159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 0x7f2de0159b30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000c970 tx=0x7f2dd000ce30 comp rx=0 tx=0).stop 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 shutdown_connections 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dc0073680 0x7f2dc0075b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2de00d3630 0x7f2de0160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2de00d1490 0x7f2de0159b30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 --2- 172.21.15.170:0/3175287779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2de00cf280 0x7f2de015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 >> 172.21.15.170:0/3175287779 conn(0x7f2de009d830 msgr2=0x7f2de00b67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 shutdown_connections 2024-09-21T02:20:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.270+0000 7f2de4dd5640 1 -- 172.21.15.170:0/3175287779 wait complete. 2024-09-21T02:20:00.290 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T02:20:00.291 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 -- 172.21.15.170:0/1475434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 msgr2=0x7fea840b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/1475434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea840b3350 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fea84071d80 tx=0x7fea7401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 -- 172.21.15.170:0/1475434320 shutdown_connections 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/1475434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea840b3350 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/1475434320 >> v2:172.21.15.191:3300/0 conn(0x7fea840b5410 0x7fea8406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 -- 172.21.15.170:0/1475434320 >> 172.21.15.170:0/1475434320 conn(0x7fea840b1c40 msgr2=0x7fea840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 -- 172.21.15.170:0/1475434320 shutdown_connections 2024-09-21T02:20:00.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 -- 172.21.15.170:0/1475434320 wait complete. 2024-09-21T02:20:00.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 Processor -- start 2024-09-21T02:20:00.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.446+0000 7fea8bbdc640 1 -- start start 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea84148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea840b5410 0x7fea841fa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea840ccb00 0x7fea84144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea84169fd0 con 0x7fea840b2f40 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea84169e50 con 0x7fea840ccb00 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fea8416a150 con 0x7fea840b5410 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea84148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea84148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59146/0 (socket says 172.21.15.170:59146) 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 -- 172.21.15.170:0/3069937863 learned_addr learned my addr 172.21.15.170:0/3069937863 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8a179640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea840ccb00 0x7fea84144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89177640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea840b5410 0x7fea841fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 -- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea840b5410 msgr2=0x7fea841fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea840b5410 0x7fea841fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 -- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea840ccb00 msgr2=0x7fea84144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea840ccb00 0x7fea84144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 -- 172.21.15.170:0/3069937863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8406a8a0 con 0x7fea840b2f40 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89177640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea840b5410 0x7fea841fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8a179640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea840ccb00 0x7fea84144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea89978640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea84148510 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fea800058d0 tx=0x7fea8000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80017070 con 0x7fea840b2f40 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea80009cc0 con 0x7fea840b2f40 2024-09-21T02:20:00.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea841fac20 con 0x7fea840b2f40 2024-09-21T02:20:00.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800106c0 con 0x7fea840b2f40 2024-09-21T02:20:00.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea84147d10 con 0x7fea840b2f40 2024-09-21T02:20:00.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea84148140 con 0x7fea840b2f40 2024-09-21T02:20:00.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.450+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fea8001a070 con 0x7fea840b2f40 2024-09-21T02:20:00.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.454+0000 7fea7a7fc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fea68073680 0x7fea68075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:00.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.454+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fea80094cd0 con 0x7fea840b2f40 2024-09-21T02:20:00.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.454+0000 7fea89177640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fea68073680 0x7fea68075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:00.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.454+0000 7fea89177640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fea68073680 0x7fea68075b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea740af000 tx=0x7fea7401b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:00.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.458+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fea8009b080 con 0x7fea840b2f40 2024-09-21T02:20:00.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.618+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea840c3a10 con 0x7fea68073680 2024-09-21T02:20:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.630+0000 7fea7a7fc640 1 -- 172.21.15.170:0/3069937863 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247256 (secure 0 0 0) 0x7fea840c3a10 con 0x7fea68073680 2024-09-21T02:20:00.631 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fea68073680 msgr2=0x7fea68075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fea68073680 0x7fea68075b90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea740af000 tx=0x7fea7401b100 comp rx=0 tx=0).stop 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 msgr2=0x7fea84148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea84148510 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fea800058d0 tx=0x7fea8000dde0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 shutdown_connections 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fea68073680 0x7fea68075b90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea840b5410 0x7fea841fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea840ccb00 0x7fea84144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 --2- 172.21.15.170:0/3069937863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea840b2f40 0x7fea84148510 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 >> 172.21.15.170:0/3069937863 conn(0x7fea840b1c40 msgr2=0x7fea840c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 shutdown_connections 2024-09-21T02:20:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:00.634+0000 7fea8bbdc640 1 -- 172.21.15.170:0/3069937863 wait complete. 2024-09-21T02:20:00.652 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-21T02:19:58.894985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282740,"kb_used_data":12412,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691980,"statfs":{"total":773094113280,"available":772804587520,"internally_reserved":0,"allocated":12709888,"data_stored":6228517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002214"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879824+0000","last_change":"2024-09-21T02:19:54.879824+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879824+0000","last_undegraded":"2024-09-21T02:19:54.879824+0000","last_fullsized":"2024-09-21T02:19:54.879824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879361+0000","last_change":"2024-09-21T02:19:54.879361+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879361+0000","last_undegraded":"2024-09-21T02:19:54.879361+0000","last_fullsized":"2024-09-21T02:19:54.879361+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885776+0000","last_change":"2024-09-21T02:19:53.885776+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885776+0000","last_undegraded":"2024-09-21T02:19:53.885776+0000","last_fullsized":"2024-09-21T02:19:53.885776+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":4,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.886129+0000","last_change":"2024-09-21T02:19:54.880458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.886129+0000","last_undegraded":"2024-09-21T02:19:54.886129+0000","last_fullsized":"2024-09-21T02:19:54.886129+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886975+0000","last_change":"2024-09-21T02:19:53.886975+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886975+0000","last_undegraded":"2024-09-21T02:19:53.886975+0000","last_fullsized":"2024-09-21T02:19:53.886975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889655+0000","last_change":"2024-09-21T02:19:53.889655+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889655+0000","last_undegraded":"2024-09-21T02:19:53.889655+0000","last_fullsized":"2024-09-21T02:19:53.889655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886780+0000","last_change":"2024-09-21T02:19:53.886780+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886780+0000","last_undegraded":"2024-09-21T02:19:53.886780+0000","last_fullsized":"2024-09-21T02:19:53.886780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887128+0000","last_change":"2024-09-21T02:19:53.887128+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887128+0000","last_undegraded":"2024-09-21T02:19:53.887128+0000","last_fullsized":"2024-09-21T02:19:53.887128+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885049+0000","last_change":"2024-09-21T02:19:53.885049+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885049+0000","last_undegraded":"2024-09-21T02:19:53.885049+0000","last_fullsized":"2024-09-21T02:19:53.885049+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878396+0000","last_change":"2024-09-21T02:19:54.878396+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878396+0000","last_undegraded":"2024-09-21T02:19:54.878396+0000","last_fullsized":"2024-09-21T02:19:54.878396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877485+0000","last_change":"2024-09-21T02:19:54.877485+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877485+0000","last_undegraded":"2024-09-21T02:19:54.877485+0000","last_fullsized":"2024-09-21T02:19:54.877485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882109+0000","last_change":"2024-09-21T02:19:53.882109+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882109+0000","last_undegraded":"2024-09-21T02:19:53.882109+0000","last_fullsized":"2024-09-21T02:19:53.882109+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876172+0000","last_change":"2024-09-21T02:19:54.876172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876172+0000","last_undegraded":"2024-09-21T02:19:54.876172+0000","last_fullsized":"2024-09-21T02:19:54.876172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892017+0000","last_change":"2024-09-21T02:19:53.892017+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892017+0000","last_undegraded":"2024-09-21T02:19:53.892017+0000","last_fullsized":"2024-09-21T02:19:53.892017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897505+0000","last_change":"2024-09-21T02:19:53.897505+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897505+0000","last_undegraded":"2024-09-21T02:19:53.897505+0000","last_fullsized":"2024-09-21T02:19:53.897505+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902786+0000","last_change":"2024-09-21T02:19:53.902786+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902786+0000","last_undegraded":"2024-09-21T02:19:53.902786+0000","last_fullsized":"2024-09-21T02:19:53.902786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883879+0000","last_change":"2024-09-21T02:19:53.883879+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883879+0000","last_undegraded":"2024-09-21T02:19:53.883879+0000","last_fullsized":"2024-09-21T02:19:53.883879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.897882+0000","last_change":"2024-09-21T02:19:53.897882+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.897882+0000","last_undegraded":"2024-09-21T02:19:53.897882+0000","last_fullsized":"2024-09-21T02:19:53.897882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886319+0000","last_change":"2024-09-21T02:19:53.886319+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886319+0000","last_undegraded":"2024-09-21T02:19:53.886319+0000","last_fullsized":"2024-09-21T02:19:53.886319+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899311+0000","last_change":"2024-09-21T02:19:53.899311+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899311+0000","last_undegraded":"2024-09-21T02:19:53.899311+0000","last_fullsized":"2024-09-21T02:19:53.899311+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889310+0000","last_change":"2024-09-21T02:19:53.889310+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889310+0000","last_undegraded":"2024-09-21T02:19:53.889310+0000","last_fullsized":"2024-09-21T02:19:53.889310+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886303+0000","last_change":"2024-09-21T02:19:53.886303+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886303+0000","last_undegraded":"2024-09-21T02:19:53.886303+0000","last_fullsized":"2024-09-21T02:19:53.886303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902222+0000","last_change":"2024-09-21T02:19:53.902222+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902222+0000","last_undegraded":"2024-09-21T02:19:53.902222+0000","last_fullsized":"2024-09-21T02:19:53.902222+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.900530+0000","last_change":"2024-09-21T02:19:53.900530+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.900530+0000","last_undegraded":"2024-09-21T02:19:53.900530+0000","last_fullsized":"2024-09-21T02:19:53.900530+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876179+0000","last_change":"2024-09-21T02:19:54.876179+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876179+0000","last_undegraded":"2024-09-21T02:19:54.876179+0000","last_fullsized":"2024-09-21T02:19:54.876179+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877410+0000","last_change":"2024-09-21T02:19:54.877410+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877410+0000","last_undegraded":"2024-09-21T02:19:54.877410+0000","last_fullsized":"2024-09-21T02:19:54.877410+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892439+0000","last_change":"2024-09-21T02:19:53.892439+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892439+0000","last_undegraded":"2024-09-21T02:19:53.892439+0000","last_fullsized":"2024-09-21T02:19:53.892439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.903671+0000","last_change":"2024-09-21T02:19:53.903671+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.903671+0000","last_undegraded":"2024-09-21T02:19:53.903671+0000","last_fullsized":"2024-09-21T02:19:53.903671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876878+0000","last_change":"2024-09-21T02:19:54.876878+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876878+0000","last_undegraded":"2024-09-21T02:19:54.876878+0000","last_fullsized":"2024-09-21T02:19:54.876878+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883817+0000","last_change":"2024-09-21T02:19:53.883817+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883817+0000","last_undegraded":"2024-09-21T02:19:53.883817+0000","last_fullsized":"2024-09-21T02:19:53.883817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.895765+0000","last_change":"2024-09-21T02:19:53.895765+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.895765+0000","last_undegraded":"2024-09-21T02:19:53.895765+0000","last_fullsized":"2024-09-21T02:19:53.895765+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889849+0000","last_change":"2024-09-21T02:19:53.889849+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889849+0000","last_undegraded":"2024-09-21T02:19:53.889849+0000","last_fullsized":"2024-09-21T02:19:53.889849+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.902590+0000","last_change":"2024-09-21T02:19:53.902590+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.902590+0000","last_undegraded":"2024-09-21T02:19:53.902590+0000","last_fullsized":"2024-09-21T02:19:53.902590+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901157+0000","last_change":"2024-09-21T02:19:53.899077+0000","last_active":"2024-09-21T02:19:53.901157+0000","last_peered":"2024-09-21T02:19:53.901157+0000","last_clean":"2024-09-21T02:19:53.901157+0000","last_became_active":"2024-09-21T02:19:53.898374+0000","last_became_peered":"2024-09-21T02:19:53.898374+0000","last_unstale":"2024-09-21T02:19:53.901157+0000","last_undegraded":"2024-09-21T02:19:53.901157+0000","last_fullsized":"2024-09-21T02:19:53.901157+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:20.344188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882611+0000","last_change":"2024-09-21T02:19:53.882611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882611+0000","last_undegraded":"2024-09-21T02:19:53.882611+0000","last_fullsized":"2024-09-21T02:19:53.882611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.895668+0000","last_change":"2024-09-21T02:19:53.894376+0000","last_active":"2024-09-21T02:19:53.895668+0000","last_peered":"2024-09-21T02:19:53.895668+0000","last_clean":"2024-09-21T02:19:53.895668+0000","last_became_active":"2024-09-21T02:19:53.893849+0000","last_became_peered":"2024-09-21T02:19:53.893849+0000","last_unstale":"2024-09-21T02:19:53.895668+0000","last_undegraded":"2024-09-21T02:19:53.895668+0000","last_fullsized":"2024-09-21T02:19:53.895668+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:07.797675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880476+0000","last_change":"2024-09-21T02:19:53.880476+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880476+0000","last_undegraded":"2024-09-21T02:19:53.880476+0000","last_fullsized":"2024-09-21T02:19:53.880476+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886324+0000","last_change":"2024-09-21T02:19:53.913322+0000","last_active":"2024-09-21T02:19:54.886324+0000","last_peered":"2024-09-21T02:19:54.886324+0000","last_clean":"2024-09-21T02:19:54.886324+0000","last_became_active":"2024-09-21T02:19:53.912089+0000","last_became_peered":"2024-09-21T02:19:53.912089+0000","last_unstale":"2024-09-21T02:19:54.886324+0000","last_undegraded":"2024-09-21T02:19:54.886324+0000","last_fullsized":"2024-09-21T02:19:54.886324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:17.721839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.885419+0000","last_change":"2024-09-21T02:19:53.885419+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.885419+0000","last_undegraded":"2024-09-21T02:19:53.885419+0000","last_fullsized":"2024-09-21T02:19:53.885419+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880481+0000","last_change":"2024-09-21T02:19:53.911987+0000","last_active":"2024-09-21T02:19:54.880481+0000","last_peered":"2024-09-21T02:19:54.880481+0000","last_clean":"2024-09-21T02:19:54.880481+0000","last_became_active":"2024-09-21T02:19:53.911176+0000","last_became_peered":"2024-09-21T02:19:53.911176+0000","last_unstale":"2024-09-21T02:19:54.880481+0000","last_undegraded":"2024-09-21T02:19:54.880481+0000","last_fullsized":"2024-09-21T02:19:54.880481+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:55.290071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899523+0000","last_change":"2024-09-21T02:19:53.899523+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899523+0000","last_undegraded":"2024-09-21T02:19:53.899523+0000","last_fullsized":"2024-09-21T02:19:53.899523+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.877925+0000","last_change":"2024-09-21T02:19:53.906525+0000","last_active":"2024-09-21T02:19:54.877925+0000","last_peered":"2024-09-21T02:19:54.877925+0000","last_clean":"2024-09-21T02:19:54.877925+0000","last_became_active":"2024-09-21T02:19:53.905607+0000","last_became_peered":"2024-09-21T02:19:53.905607+0000","last_unstale":"2024-09-21T02:19:54.877925+0000","last_undegraded":"2024-09-21T02:19:54.877925+0000","last_fullsized":"2024-09-21T02:19:54.877925+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.829774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888730+0000","last_change":"2024-09-21T02:19:53.888730+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888730+0000","last_undegraded":"2024-09-21T02:19:53.888730+0000","last_fullsized":"2024-09-21T02:19:53.888730+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.921790+0000","last_change":"2024-09-21T02:19:53.920704+0000","last_active":"2024-09-21T02:19:53.921790+0000","last_peered":"2024-09-21T02:19:53.921790+0000","last_clean":"2024-09-21T02:19:53.921790+0000","last_became_active":"2024-09-21T02:19:53.920013+0000","last_became_peered":"2024-09-21T02:19:53.920013+0000","last_unstale":"2024-09-21T02:19:53.921790+0000","last_undegraded":"2024-09-21T02:19:53.921790+0000","last_fullsized":"2024-09-21T02:19:53.921790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:45.115276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.884740+0000","last_change":"2024-09-21T02:19:53.884740+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.884740+0000","last_undegraded":"2024-09-21T02:19:53.884740+0000","last_fullsized":"2024-09-21T02:19:53.884740+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901049+0000","last_change":"2024-09-21T02:19:53.900369+0000","last_active":"2024-09-21T02:19:53.901049+0000","last_peered":"2024-09-21T02:19:53.901049+0000","last_clean":"2024-09-21T02:19:53.901049+0000","last_became_active":"2024-09-21T02:19:53.899961+0000","last_became_peered":"2024-09-21T02:19:53.899961+0000","last_unstale":"2024-09-21T02:19:53.901049+0000","last_undegraded":"2024-09-21T02:19:53.901049+0000","last_fullsized":"2024-09-21T02:19:53.901049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:58.476667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.877317+0000","last_change":"2024-09-21T02:19:54.877317+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.877317+0000","last_undegraded":"2024-09-21T02:19:54.877317+0000","last_fullsized":"2024-09-21T02:19:54.877317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876958+0000","last_change":"2024-09-21T02:19:53.906244+0000","last_active":"2024-09-21T02:19:54.876958+0000","last_peered":"2024-09-21T02:19:54.876958+0000","last_clean":"2024-09-21T02:19:54.876958+0000","last_became_active":"2024-09-21T02:19:53.905406+0000","last_became_peered":"2024-09-21T02:19:53.905406+0000","last_unstale":"2024-09-21T02:19:54.876958+0000","last_undegraded":"2024-09-21T02:19:54.876958+0000","last_fullsized":"2024-09-21T02:19:54.876958+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:09.984572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.876431+0000","last_change":"2024-09-21T02:19:54.876431+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.876431+0000","last_undegraded":"2024-09-21T02:19:54.876431+0000","last_fullsized":"2024-09-21T02:19:54.876431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899290+0000","last_change":"2024-09-21T02:19:53.898084+0000","last_active":"2024-09-21T02:19:53.899290+0000","last_peered":"2024-09-21T02:19:53.899290+0000","last_clean":"2024-09-21T02:19:53.899290+0000","last_became_active":"2024-09-21T02:19:53.897635+0000","last_became_peered":"2024-09-21T02:19:53.897635+0000","last_unstale":"2024-09-21T02:19:53.899290+0000","last_undegraded":"2024-09-21T02:19:53.899290+0000","last_fullsized":"2024-09-21T02:19:53.899290+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:34.034584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887925+0000","last_change":"2024-09-21T02:19:53.887925+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887925+0000","last_undegraded":"2024-09-21T02:19:53.887925+0000","last_fullsized":"2024-09-21T02:19:53.887925+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.919831+0000","last_change":"2024-09-21T02:19:53.914573+0000","last_active":"2024-09-21T02:19:53.919831+0000","last_peered":"2024-09-21T02:19:53.919831+0000","last_clean":"2024-09-21T02:19:53.919831+0000","last_became_active":"2024-09-21T02:19:53.913541+0000","last_became_peered":"2024-09-21T02:19:53.913541+0000","last_unstale":"2024-09-21T02:19:53.919831+0000","last_undegraded":"2024-09-21T02:19:53.919831+0000","last_fullsized":"2024-09-21T02:19:53.919831+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:47.109959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.880707+0000","last_change":"2024-09-21T02:19:53.880707+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.880707+0000","last_undegraded":"2024-09-21T02:19:53.880707+0000","last_fullsized":"2024-09-21T02:19:53.880707+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.900789+0000","last_change":"2024-09-21T02:19:53.899480+0000","last_active":"2024-09-21T02:19:53.900789+0000","last_peered":"2024-09-21T02:19:53.900789+0000","last_clean":"2024-09-21T02:19:53.900789+0000","last_became_active":"2024-09-21T02:19:53.899037+0000","last_became_peered":"2024-09-21T02:19:53.899037+0000","last_unstale":"2024-09-21T02:19:53.900789+0000","last_undegraded":"2024-09-21T02:19:53.900789+0000","last_fullsized":"2024-09-21T02:19:53.900789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:38.427447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887372+0000","last_change":"2024-09-21T02:19:53.887372+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887372+0000","last_undegraded":"2024-09-21T02:19:53.887372+0000","last_fullsized":"2024-09-21T02:19:53.887372+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.906801+0000","last_change":"2024-09-21T02:19:53.904993+0000","last_active":"2024-09-21T02:19:53.906801+0000","last_peered":"2024-09-21T02:19:53.906801+0000","last_clean":"2024-09-21T02:19:53.906801+0000","last_became_active":"2024-09-21T02:19:53.904509+0000","last_became_peered":"2024-09-21T02:19:53.904509+0000","last_unstale":"2024-09-21T02:19:53.906801+0000","last_undegraded":"2024-09-21T02:19:53.906801+0000","last_fullsized":"2024-09-21T02:19:53.906801+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:48.851528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889611+0000","last_change":"2024-09-21T02:19:53.889611+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889611+0000","last_undegraded":"2024-09-21T02:19:53.889611+0000","last_fullsized":"2024-09-21T02:19:53.889611+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917550+0000","last_change":"2024-09-21T02:19:53.903917+0000","last_active":"2024-09-21T02:19:53.917550+0000","last_peered":"2024-09-21T02:19:53.917550+0000","last_clean":"2024-09-21T02:19:53.917550+0000","last_became_active":"2024-09-21T02:19:53.903089+0000","last_became_peered":"2024-09-21T02:19:53.903089+0000","last_unstale":"2024-09-21T02:19:53.917550+0000","last_undegraded":"2024-09-21T02:19:53.917550+0000","last_fullsized":"2024-09-21T02:19:53.917550+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:34.884559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.892462+0000","last_change":"2024-09-21T02:19:53.892462+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.892462+0000","last_undegraded":"2024-09-21T02:19:53.892462+0000","last_fullsized":"2024-09-21T02:19:53.892462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878927+0000","last_change":"2024-09-21T02:19:53.896469+0000","last_active":"2024-09-21T02:19:54.878927+0000","last_peered":"2024-09-21T02:19:54.878927+0000","last_clean":"2024-09-21T02:19:54.878927+0000","last_became_active":"2024-09-21T02:19:53.895529+0000","last_became_peered":"2024-09-21T02:19:53.895529+0000","last_unstale":"2024-09-21T02:19:54.878927+0000","last_undegraded":"2024-09-21T02:19:54.878927+0000","last_fullsized":"2024-09-21T02:19:54.878927+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:30.370234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.882720+0000","last_change":"2024-09-21T02:19:53.882720+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.882720+0000","last_undegraded":"2024-09-21T02:19:53.882720+0000","last_fullsized":"2024-09-21T02:19:53.882720+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920349+0000","last_change":"2024-09-21T02:19:53.919682+0000","last_active":"2024-09-21T02:19:53.920349+0000","last_peered":"2024-09-21T02:19:53.920349+0000","last_clean":"2024-09-21T02:19:53.920349+0000","last_became_active":"2024-09-21T02:19:53.919315+0000","last_became_peered":"2024-09-21T02:19:53.919315+0000","last_unstale":"2024-09-21T02:19:53.920349+0000","last_undegraded":"2024-09-21T02:19:53.920349+0000","last_fullsized":"2024-09-21T02:19:53.920349+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:33.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.887258+0000","last_change":"2024-09-21T02:19:53.887258+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.887258+0000","last_undegraded":"2024-09-21T02:19:53.887258+0000","last_fullsized":"2024-09-21T02:19:53.887258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.908256+0000","last_change":"2024-09-21T02:19:53.907585+0000","last_active":"2024-09-21T02:19:53.908256+0000","last_peered":"2024-09-21T02:19:53.908256+0000","last_clean":"2024-09-21T02:19:53.908256+0000","last_became_active":"2024-09-21T02:19:53.906977+0000","last_became_peered":"2024-09-21T02:19:53.906977+0000","last_unstale":"2024-09-21T02:19:53.908256+0000","last_undegraded":"2024-09-21T02:19:53.908256+0000","last_fullsized":"2024-09-21T02:19:53.908256+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:18.433443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.886728+0000","last_change":"2024-09-21T02:19:53.886728+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.886728+0000","last_undegraded":"2024-09-21T02:19:53.886728+0000","last_fullsized":"2024-09-21T02:19:53.886728+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.886814+0000","last_change":"2024-09-21T02:19:53.904893+0000","last_active":"2024-09-21T02:19:54.886814+0000","last_peered":"2024-09-21T02:19:54.886814+0000","last_clean":"2024-09-21T02:19:54.886814+0000","last_became_active":"2024-09-21T02:19:53.904064+0000","last_became_peered":"2024-09-21T02:19:53.904064+0000","last_unstale":"2024-09-21T02:19:54.886814+0000","last_undegraded":"2024-09-21T02:19:54.886814+0000","last_fullsized":"2024-09-21T02:19:54.886814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:33.874089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.879257+0000","last_change":"2024-09-21T02:19:54.879257+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.879257+0000","last_undegraded":"2024-09-21T02:19:54.879257+0000","last_fullsized":"2024-09-21T02:19:54.879257+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.899521+0000","last_change":"2024-09-21T02:19:53.898390+0000","last_active":"2024-09-21T02:19:53.899521+0000","last_peered":"2024-09-21T02:19:53.899521+0000","last_clean":"2024-09-21T02:19:53.899521+0000","last_became_active":"2024-09-21T02:19:53.897822+0000","last_became_peered":"2024-09-21T02:19:53.897822+0000","last_unstale":"2024-09-21T02:19:53.899521+0000","last_undegraded":"2024-09-21T02:19:53.899521+0000","last_fullsized":"2024-09-21T02:19:53.899521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:11.038001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.881173+0000","last_change":"2024-09-21T02:19:54.881173+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.881173+0000","last_undegraded":"2024-09-21T02:19:54.881173+0000","last_fullsized":"2024-09-21T02:19:54.881173+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898678+0000","last_change":"2024-09-21T02:19:53.889955+0000","last_active":"2024-09-21T02:19:53.898678+0000","last_peered":"2024-09-21T02:19:53.898678+0000","last_clean":"2024-09-21T02:19:53.898678+0000","last_became_active":"2024-09-21T02:19:53.889393+0000","last_became_peered":"2024-09-21T02:19:53.889393+0000","last_unstale":"2024-09-21T02:19:53.898678+0000","last_undegraded":"2024-09-21T02:19:53.898678+0000","last_fullsized":"2024-09-21T02:19:53.898678+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:32.922018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881657+0000","last_change":"2024-09-21T02:19:53.881657+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881657+0000","last_undegraded":"2024-09-21T02:19:53.881657+0000","last_fullsized":"2024-09-21T02:19:53.881657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.898735+0000","last_change":"2024-09-21T02:19:53.897209+0000","last_active":"2024-09-21T02:19:53.898735+0000","last_peered":"2024-09-21T02:19:53.898735+0000","last_clean":"2024-09-21T02:19:53.898735+0000","last_became_active":"2024-09-21T02:19:53.896800+0000","last_became_peered":"2024-09-21T02:19:53.896800+0000","last_unstale":"2024-09-21T02:19:53.898735+0000","last_undegraded":"2024-09-21T02:19:53.898735+0000","last_fullsized":"2024-09-21T02:19:53.898735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:31.737184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.904140+0000","last_change":"2024-09-21T02:19:53.904140+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.904140+0000","last_undegraded":"2024-09-21T02:19:53.904140+0000","last_fullsized":"2024-09-21T02:19:53.904140+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920415+0000","last_change":"2024-09-21T02:19:53.915141+0000","last_active":"2024-09-21T02:19:53.920415+0000","last_peered":"2024-09-21T02:19:53.920415+0000","last_clean":"2024-09-21T02:19:53.920415+0000","last_became_active":"2024-09-21T02:19:53.914156+0000","last_became_peered":"2024-09-21T02:19:53.914156+0000","last_unstale":"2024-09-21T02:19:53.920415+0000","last_undegraded":"2024-09-21T02:19:53.920415+0000","last_fullsized":"2024-09-21T02:19:53.920415+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:56:52.051938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890763+0000","last_change":"2024-09-21T02:19:53.890763+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890763+0000","last_undegraded":"2024-09-21T02:19:53.890763+0000","last_fullsized":"2024-09-21T02:19:53.890763+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917649+0000","last_change":"2024-09-21T02:19:53.906690+0000","last_active":"2024-09-21T02:19:53.917649+0000","last_peered":"2024-09-21T02:19:53.917649+0000","last_clean":"2024-09-21T02:19:53.917649+0000","last_became_active":"2024-09-21T02:19:53.905289+0000","last_became_peered":"2024-09-21T02:19:53.905289+0000","last_unstale":"2024-09-21T02:19:53.917649+0000","last_undegraded":"2024-09-21T02:19:53.917649+0000","last_fullsized":"2024-09-21T02:19:53.917649+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:25.254722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881976+0000","last_change":"2024-09-21T02:19:53.881976+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881976+0000","last_undegraded":"2024-09-21T02:19:53.881976+0000","last_fullsized":"2024-09-21T02:19:53.881976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.876219+0000","last_change":"2024-09-21T02:19:53.904489+0000","last_active":"2024-09-21T02:19:54.876219+0000","last_peered":"2024-09-21T02:19:54.876219+0000","last_clean":"2024-09-21T02:19:54.876219+0000","last_became_active":"2024-09-21T02:19:53.903679+0000","last_became_peered":"2024-09-21T02:19:53.903679+0000","last_unstale":"2024-09-21T02:19:54.876219+0000","last_undegraded":"2024-09-21T02:19:54.876219+0000","last_fullsized":"2024-09-21T02:19:54.876219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:00.611818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.894458+0000","last_change":"2024-09-21T02:19:53.894458+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.894458+0000","last_undegraded":"2024-09-21T02:19:53.894458+0000","last_fullsized":"2024-09-21T02:19:53.894458+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920396+0000","last_change":"2024-09-21T02:19:53.917476+0000","last_active":"2024-09-21T02:19:53.920396+0000","last_peered":"2024-09-21T02:19:53.920396+0000","last_clean":"2024-09-21T02:19:53.920396+0000","last_became_active":"2024-09-21T02:19:53.916672+0000","last_became_peered":"2024-09-21T02:19:53.916672+0000","last_unstale":"2024-09-21T02:19:53.920396+0000","last_undegraded":"2024-09-21T02:19:53.920396+0000","last_fullsized":"2024-09-21T02:19:53.920396+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:04.729381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":3,"reported_epoch":117,"state":"creating+activating","last_fresh":"2024-09-21T02:19:54.878762+0000","last_change":"2024-09-21T02:19:54.878762+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:54.878762+0000","last_undegraded":"2024-09-21T02:19:54.878762+0000","last_fullsized":"2024-09-21T02:19:54.878762+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.901861+0000","last_change":"2024-09-21T02:19:53.900842+0000","last_active":"2024-09-21T02:19:53.901861+0000","last_peered":"2024-09-21T02:19:53.901861+0000","last_clean":"2024-09-21T02:19:53.901861+0000","last_became_active":"2024-09-21T02:19:53.900353+0000","last_became_peered":"2024-09-21T02:19:53.900353+0000","last_unstale":"2024-09-21T02:19:53.901861+0000","last_undegraded":"2024-09-21T02:19:53.901861+0000","last_fullsized":"2024-09-21T02:19:53.901861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:57.302971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890952+0000","last_change":"2024-09-21T02:19:53.890952+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890952+0000","last_undegraded":"2024-09-21T02:19:53.890952+0000","last_fullsized":"2024-09-21T02:19:53.890952+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.920113+0000","last_change":"2024-09-21T02:19:53.916530+0000","last_active":"2024-09-21T02:19:53.920113+0000","last_peered":"2024-09-21T02:19:53.920113+0000","last_clean":"2024-09-21T02:19:53.920113+0000","last_became_active":"2024-09-21T02:19:53.915419+0000","last_became_peered":"2024-09-21T02:19:53.915419+0000","last_unstale":"2024-09-21T02:19:53.920113+0000","last_undegraded":"2024-09-21T02:19:53.920113+0000","last_fullsized":"2024-09-21T02:19:53.920113+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:28.827382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.881579+0000","last_change":"2024-09-21T02:19:53.881579+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.881579+0000","last_undegraded":"2024-09-21T02:19:53.881579+0000","last_fullsized":"2024-09-21T02:19:53.881579+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.918803+0000","last_change":"2024-09-21T02:19:53.911432+0000","last_active":"2024-09-21T02:19:53.918803+0000","last_peered":"2024-09-21T02:19:53.918803+0000","last_clean":"2024-09-21T02:19:53.918803+0000","last_became_active":"2024-09-21T02:19:53.910658+0000","last_became_peered":"2024-09-21T02:19:53.910658+0000","last_unstale":"2024-09-21T02:19:53.918803+0000","last_undegraded":"2024-09-21T02:19:53.918803+0000","last_fullsized":"2024-09-21T02:19:53.918803+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:56.513763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.889059+0000","last_change":"2024-09-21T02:19:53.889059+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.889059+0000","last_undegraded":"2024-09-21T02:19:53.889059+0000","last_fullsized":"2024-09-21T02:19:53.889059+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.878522+0000","last_change":"2024-09-21T02:19:53.909822+0000","last_active":"2024-09-21T02:19:54.878522+0000","last_peered":"2024-09-21T02:19:54.878522+0000","last_clean":"2024-09-21T02:19:54.878522+0000","last_became_active":"2024-09-21T02:19:53.908919+0000","last_became_peered":"2024-09-21T02:19:53.908919+0000","last_unstale":"2024-09-21T02:19:54.878522+0000","last_undegraded":"2024-09-21T02:19:54.878522+0000","last_fullsized":"2024-09-21T02:19:54.878522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.216241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.888015+0000","last_change":"2024-09-21T02:19:53.888015+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.888015+0000","last_undegraded":"2024-09-21T02:19:53.888015+0000","last_fullsized":"2024-09-21T02:19:53.888015+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.916193+0000","last_change":"2024-09-21T02:19:53.915286+0000","last_active":"2024-09-21T02:19:53.916193+0000","last_peered":"2024-09-21T02:19:53.916193+0000","last_clean":"2024-09-21T02:19:53.916193+0000","last_became_active":"2024-09-21T02:19:53.914782+0000","last_became_peered":"2024-09-21T02:19:53.914782+0000","last_unstale":"2024-09-21T02:19:53.916193+0000","last_undegraded":"2024-09-21T02:19:53.916193+0000","last_fullsized":"2024-09-21T02:19:53.916193+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:39.555502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.899037+0000","last_change":"2024-09-21T02:19:53.899037+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.899037+0000","last_undegraded":"2024-09-21T02:19:53.899037+0000","last_fullsized":"2024-09-21T02:19:53.899037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-21T02:19:54.880986+0000","last_change":"2024-09-21T02:19:53.896536+0000","last_active":"2024-09-21T02:19:54.880986+0000","last_peered":"2024-09-21T02:19:54.880986+0000","last_clean":"2024-09-21T02:19:54.880986+0000","last_became_active":"2024-09-21T02:19:53.895589+0000","last_became_peered":"2024-09-21T02:19:53.895589+0000","last_unstale":"2024-09-21T02:19:54.880986+0000","last_undegraded":"2024-09-21T02:19:54.880986+0000","last_fullsized":"2024-09-21T02:19:54.880986+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:17.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.890723+0000","last_change":"2024-09-21T02:19:53.890723+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.890723+0000","last_undegraded":"2024-09-21T02:19:53.890723+0000","last_fullsized":"2024-09-21T02:19:53.890723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.917794+0000","last_change":"2024-09-21T02:19:53.908787+0000","last_active":"2024-09-21T02:19:53.917794+0000","last_peered":"2024-09-21T02:19:53.917794+0000","last_clean":"2024-09-21T02:19:53.917794+0000","last_became_active":"2024-09-21T02:19:53.907954+0000","last_became_peered":"2024-09-21T02:19:53.907954+0000","last_unstale":"2024-09-21T02:19:53.917794+0000","last_undegraded":"2024-09-21T02:19:53.917794+0000","last_fullsized":"2024-09-21T02:19:53.917794+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:20.813712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T02:19:53.883172+0000","last_change":"2024-09-21T02:19:53.883172+0000","last_active":"2024-09-21T02:19:53.856540+0000","last_peered":"2024-09-21T02:19:53.856540+0000","last_clean":"2024-09-21T02:19:53.856540+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T02:19:53.883172+0000","last_undegraded":"2024-09-21T02:19:53.883172+0000","last_fullsized":"2024-09-21T02:19:53.883172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.915086+0000","last_change":"2024-09-21T02:19:53.913344+0000","last_active":"2024-09-21T02:19:53.915086+0000","last_peered":"2024-09-21T02:19:53.915086+0000","last_clean":"2024-09-21T02:19:53.915086+0000","last_became_active":"2024-09-21T02:19:53.912800+0000","last_became_peered":"2024-09-21T02:19:53.912800+0000","last_unstale":"2024-09-21T02:19:53.915086+0000","last_undegraded":"2024-09-21T02:19:53.915086+0000","last_fullsized":"2024-09-21T02:19:53.915086+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:13.374558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":308,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T02:19:53.904728+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:19:53.904728+0000","last_peered":"2024-09-21T02:19:53.904728+0000","last_clean":"2024-09-21T02:19:53.904728+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:19:53.904728+0000","last_undegraded":"2024-09-21T02:19:53.904728+0000","last_fullsized":"2024-09-21T02:19:53.904728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":625482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41500,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330340,"statfs":{"total":96636764160,"available":96594268160,"internally_reserved":0,"allocated":2035712,"data_stored":1222125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1441792,"data_stored":631757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:20:00.812 INFO:tasks.ceph.mds.a.smithi170.stderr:2024-09-21T02:20:00.814+0000 7f036f222640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:20:03.656 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 -- 172.21.15.170:0/1866857178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b040cd210 msgr2=0x7f7b040b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 --2- 172.21.15.170:0/1866857178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b040cd210 0x7f7b040b3350 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f7b040b3a10 tx=0x7f7afc01cda0 comp rx=0 tx=0).stop 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 -- 172.21.15.170:0/1866857178 shutdown_connections 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 --2- 172.21.15.170:0/1866857178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b040cd210 0x7f7b040b3350 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 --2- 172.21.15.170:0/1866857178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 0x7f7b0406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 -- 172.21.15.170:0/1866857178 >> 172.21.15.170:0/1866857178 conn(0x7f7b040b1c40 msgr2=0x7f7b040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 -- 172.21.15.170:0/1866857178 shutdown_connections 2024-09-21T02:20:03.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 -- 172.21.15.170:0/1866857178 wait complete. 2024-09-21T02:20:03.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 Processor -- start 2024-09-21T02:20:03.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 -- start start 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b040b2f40 0x7f7b04154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.826+0000 7f7b09005640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 0x7f7b0415fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 0x7f7b0415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0416a060 con 0x7f7b04159a50 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b040701b0 con 0x7f7b040b2f40 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0416a1e0 con 0x7f7b040b5410 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 0x7f7b0415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 0x7f7b0415dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59168/0 (socket says 172.21.15.170:59168) 2024-09-21T02:20:03.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 -- 172.21.15.170:0/4168627748 learned_addr learned my addr 172.21.15.170:0/4168627748 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0259c640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 0x7f7b0415fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b02d9d640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b040b2f40 0x7f7b04154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 -- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 msgr2=0x7f7b0415fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 0x7f7b0415fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 -- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b040b2f40 msgr2=0x7f7b04154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b040b2f40 0x7f7b04154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 -- 172.21.15.170:0/4168627748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b04155460 con 0x7f7b04159a50 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0259c640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 0x7f7b0415fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b0359e640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 0x7f7b0415dae0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009bd0 tx=0x7f7afc003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0bc070 con 0x7f7b04159a50 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7afc0042f0 con 0x7f7b04159a50 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b04155740 con 0x7f7b04159a50 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0b2040 con 0x7f7b04159a50 2024-09-21T02:20:03.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b04155d00 con 0x7f7b04159a50 2024-09-21T02:20:03.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0406fa30 con 0x7f7b04159a50 2024-09-21T02:20:03.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f7afc0bf070 con 0x7f7b04159a50 2024-09-21T02:20:03.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7aeb7fe640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073680 0x7f7af0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:03.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f7afc12eae0 con 0x7f7b04159a50 2024-09-21T02:20:03.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.830+0000 7f7b02d9d640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073680 0x7f7af0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:03.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.834+0000 7f7b02d9d640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073680 0x7f7af0075b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005fd0 tx=0x7f7af400c460 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:03.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:03.838+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7afc016990 con 0x7f7b04159a50 2024-09-21T02:20:04.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.006+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b040c3a10 con 0x7f7af0073680 2024-09-21T02:20:04.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.018+0000 7f7aeb7fe640 1 -- 172.21.15.170:0/4168627748 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252470 (secure 0 0 0) 0x7f7b040c3a10 con 0x7f7af0073680 2024-09-21T02:20:04.019 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:04.020 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:20:04.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073680 msgr2=0x7f7af0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073680 0x7f7af0075b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005fd0 tx=0x7f7af400c460 comp rx=0 tx=0).stop 2024-09-21T02:20:04.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 msgr2=0x7f7b0415dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 0x7f7b0415dae0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009bd0 tx=0x7f7afc003f80 comp rx=0 tx=0).stop 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 shutdown_connections 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7af0073680 0x7f7af0075b90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b040b5410 0x7f7b0415fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b04159a50 0x7f7b0415dae0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 --2- 172.21.15.170:0/4168627748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b040b2f40 0x7f7b04154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 >> 172.21.15.170:0/4168627748 conn(0x7f7b040b1c40 msgr2=0x7f7b040c57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 shutdown_connections 2024-09-21T02:20:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.022+0000 7f7b09005640 1 -- 172.21.15.170:0/4168627748 wait complete. 2024-09-21T02:20:04.039 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-21T02:20:02.896492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283008,"kb_used_data":12680,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691712,"statfs":{"total":773094113280,"available":772804313088,"internally_reserved":0,"allocated":12984320,"data_stored":6344212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.022909"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032011+0000","last_change":"2024-09-21T02:19:54.899748+0000","last_active":"2024-09-21T02:19:56.032011+0000","last_peered":"2024-09-21T02:19:56.032011+0000","last_clean":"2024-09-21T02:19:56.032011+0000","last_became_active":"2024-09-21T02:19:54.892703+0000","last_became_peered":"2024-09-21T02:19:54.892703+0000","last_unstale":"2024-09-21T02:19:56.032011+0000","last_undegraded":"2024-09-21T02:19:56.032011+0000","last_fullsized":"2024-09-21T02:19:56.032011+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:02.172249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032878+0000","last_change":"2024-09-21T02:19:54.921597+0000","last_active":"2024-09-21T02:19:56.032878+0000","last_peered":"2024-09-21T02:19:56.032878+0000","last_clean":"2024-09-21T02:19:56.032878+0000","last_became_active":"2024-09-21T02:19:54.918592+0000","last_became_peered":"2024-09-21T02:19:54.918592+0000","last_unstale":"2024-09-21T02:19:56.032878+0000","last_undegraded":"2024-09-21T02:19:56.032878+0000","last_fullsized":"2024-09-21T02:19:56.032878+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:33.132875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031542+0000","last_change":"2024-09-21T02:19:54.932937+0000","last_active":"2024-09-21T02:19:56.031542+0000","last_peered":"2024-09-21T02:19:56.031542+0000","last_clean":"2024-09-21T02:19:56.031542+0000","last_became_active":"2024-09-21T02:19:54.932718+0000","last_became_peered":"2024-09-21T02:19:54.932718+0000","last_unstale":"2024-09-21T02:19:56.031542+0000","last_undegraded":"2024-09-21T02:19:56.031542+0000","last_fullsized":"2024-09-21T02:19:56.031542+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:03.639440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884439+0000","last_change":"2024-09-21T02:19:54.902781+0000","last_active":"2024-09-21T02:19:55.884439+0000","last_peered":"2024-09-21T02:19:55.884439+0000","last_clean":"2024-09-21T02:19:55.884439+0000","last_became_active":"2024-09-21T02:19:54.901845+0000","last_became_peered":"2024-09-21T02:19:54.901845+0000","last_unstale":"2024-09-21T02:19:55.884439+0000","last_undegraded":"2024-09-21T02:19:55.884439+0000","last_fullsized":"2024-09-21T02:19:55.884439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:27.966527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884119+0000","last_change":"2024-09-21T02:19:54.888904+0000","last_active":"2024-09-21T02:19:55.884119+0000","last_peered":"2024-09-21T02:19:55.884119+0000","last_clean":"2024-09-21T02:19:55.884119+0000","last_became_active":"2024-09-21T02:19:54.888507+0000","last_became_peered":"2024-09-21T02:19:54.888507+0000","last_unstale":"2024-09-21T02:19:55.884119+0000","last_undegraded":"2024-09-21T02:19:55.884119+0000","last_fullsized":"2024-09-21T02:19:55.884119+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:33.462733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031028+0000","last_change":"2024-09-21T02:19:54.928868+0000","last_active":"2024-09-21T02:19:56.031028+0000","last_peered":"2024-09-21T02:19:56.031028+0000","last_clean":"2024-09-21T02:19:56.031028+0000","last_became_active":"2024-09-21T02:19:54.928261+0000","last_became_peered":"2024-09-21T02:19:54.928261+0000","last_unstale":"2024-09-21T02:19:56.031028+0000","last_undegraded":"2024-09-21T02:19:56.031028+0000","last_fullsized":"2024-09-21T02:19:56.031028+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:47.025583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884322+0000","last_change":"2024-09-21T02:19:54.889702+0000","last_active":"2024-09-21T02:19:55.884322+0000","last_peered":"2024-09-21T02:19:55.884322+0000","last_clean":"2024-09-21T02:19:55.884322+0000","last_became_active":"2024-09-21T02:19:54.888625+0000","last_became_peered":"2024-09-21T02:19:54.888625+0000","last_unstale":"2024-09-21T02:19:55.884322+0000","last_undegraded":"2024-09-21T02:19:55.884322+0000","last_fullsized":"2024-09-21T02:19:55.884322+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:35.038524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.883401+0000","last_change":"2024-09-21T02:19:54.898468+0000","last_active":"2024-09-21T02:19:55.883401+0000","last_peered":"2024-09-21T02:19:55.883401+0000","last_clean":"2024-09-21T02:19:55.883401+0000","last_became_active":"2024-09-21T02:19:54.898141+0000","last_became_peered":"2024-09-21T02:19:54.898141+0000","last_unstale":"2024-09-21T02:19:55.883401+0000","last_undegraded":"2024-09-21T02:19:55.883401+0000","last_fullsized":"2024-09-21T02:19:55.883401+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:05.467442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032260+0000","last_change":"2024-09-21T02:19:54.919794+0000","last_active":"2024-09-21T02:19:56.032260+0000","last_peered":"2024-09-21T02:19:56.032260+0000","last_clean":"2024-09-21T02:19:56.032260+0000","last_became_active":"2024-09-21T02:19:54.919512+0000","last_became_peered":"2024-09-21T02:19:54.919512+0000","last_unstale":"2024-09-21T02:19:56.032260+0000","last_undegraded":"2024-09-21T02:19:56.032260+0000","last_fullsized":"2024-09-21T02:19:56.032260+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:41.510716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033496+0000","last_change":"2024-09-21T02:19:54.888596+0000","last_active":"2024-09-21T02:19:56.033496+0000","last_peered":"2024-09-21T02:19:56.033496+0000","last_clean":"2024-09-21T02:19:56.033496+0000","last_became_active":"2024-09-21T02:19:54.887771+0000","last_became_peered":"2024-09-21T02:19:54.887771+0000","last_unstale":"2024-09-21T02:19:56.033496+0000","last_undegraded":"2024-09-21T02:19:56.033496+0000","last_fullsized":"2024-09-21T02:19:56.033496+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:34.155670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884350+0000","last_change":"2024-09-21T02:19:54.925424+0000","last_active":"2024-09-21T02:19:55.884350+0000","last_peered":"2024-09-21T02:19:55.884350+0000","last_clean":"2024-09-21T02:19:55.884350+0000","last_became_active":"2024-09-21T02:19:54.924595+0000","last_became_peered":"2024-09-21T02:19:54.924595+0000","last_unstale":"2024-09-21T02:19:55.884350+0000","last_undegraded":"2024-09-21T02:19:55.884350+0000","last_fullsized":"2024-09-21T02:19:55.884350+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.290288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033145+0000","last_change":"2024-09-21T02:19:54.903345+0000","last_active":"2024-09-21T02:19:56.033145+0000","last_peered":"2024-09-21T02:19:56.033145+0000","last_clean":"2024-09-21T02:19:56.033145+0000","last_became_active":"2024-09-21T02:19:54.902759+0000","last_became_peered":"2024-09-21T02:19:54.902759+0000","last_unstale":"2024-09-21T02:19:56.033145+0000","last_undegraded":"2024-09-21T02:19:56.033145+0000","last_fullsized":"2024-09-21T02:19:56.033145+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:34.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884556+0000","last_change":"2024-09-21T02:19:54.903660+0000","last_active":"2024-09-21T02:19:55.884556+0000","last_peered":"2024-09-21T02:19:55.884556+0000","last_clean":"2024-09-21T02:19:55.884556+0000","last_became_active":"2024-09-21T02:19:54.902890+0000","last_became_peered":"2024-09-21T02:19:54.902890+0000","last_unstale":"2024-09-21T02:19:55.884556+0000","last_undegraded":"2024-09-21T02:19:55.884556+0000","last_fullsized":"2024-09-21T02:19:55.884556+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:47.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034617+0000","last_change":"2024-09-21T02:19:54.939148+0000","last_active":"2024-09-21T02:19:56.034617+0000","last_peered":"2024-09-21T02:19:56.034617+0000","last_clean":"2024-09-21T02:19:56.034617+0000","last_became_active":"2024-09-21T02:19:54.938430+0000","last_became_peered":"2024-09-21T02:19:54.938430+0000","last_unstale":"2024-09-21T02:19:56.034617+0000","last_undegraded":"2024-09-21T02:19:56.034617+0000","last_fullsized":"2024-09-21T02:19:56.034617+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:13.845095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034942+0000","last_change":"2024-09-21T02:19:54.933757+0000","last_active":"2024-09-21T02:19:56.034942+0000","last_peered":"2024-09-21T02:19:56.034942+0000","last_clean":"2024-09-21T02:19:56.034942+0000","last_became_active":"2024-09-21T02:19:54.932978+0000","last_became_peered":"2024-09-21T02:19:54.932978+0000","last_unstale":"2024-09-21T02:19:56.034942+0000","last_undegraded":"2024-09-21T02:19:56.034942+0000","last_fullsized":"2024-09-21T02:19:56.034942+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:00.044655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031564+0000","last_change":"2024-09-21T02:19:54.910304+0000","last_active":"2024-09-21T02:19:56.031564+0000","last_peered":"2024-09-21T02:19:56.031564+0000","last_clean":"2024-09-21T02:19:56.031564+0000","last_became_active":"2024-09-21T02:19:54.909369+0000","last_became_peered":"2024-09-21T02:19:54.909369+0000","last_unstale":"2024-09-21T02:19:56.031564+0000","last_undegraded":"2024-09-21T02:19:56.031564+0000","last_fullsized":"2024-09-21T02:19:56.031564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:17.077001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032489+0000","last_change":"2024-09-21T02:19:54.927863+0000","last_active":"2024-09-21T02:19:56.032489+0000","last_peered":"2024-09-21T02:19:56.032489+0000","last_clean":"2024-09-21T02:19:56.032489+0000","last_became_active":"2024-09-21T02:19:54.927572+0000","last_became_peered":"2024-09-21T02:19:54.927572+0000","last_unstale":"2024-09-21T02:19:56.032489+0000","last_undegraded":"2024-09-21T02:19:56.032489+0000","last_fullsized":"2024-09-21T02:19:56.032489+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:41.984340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.036001+0000","last_change":"2024-09-21T02:19:54.933858+0000","last_active":"2024-09-21T02:19:56.036001+0000","last_peered":"2024-09-21T02:19:56.036001+0000","last_clean":"2024-09-21T02:19:56.036001+0000","last_became_active":"2024-09-21T02:19:54.932941+0000","last_became_peered":"2024-09-21T02:19:54.932941+0000","last_unstale":"2024-09-21T02:19:56.036001+0000","last_undegraded":"2024-09-21T02:19:56.036001+0000","last_fullsized":"2024-09-21T02:19:56.036001+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:02.929930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031808+0000","last_change":"2024-09-21T02:19:54.919806+0000","last_active":"2024-09-21T02:19:56.031808+0000","last_peered":"2024-09-21T02:19:56.031808+0000","last_clean":"2024-09-21T02:19:56.031808+0000","last_became_active":"2024-09-21T02:19:54.918301+0000","last_became_peered":"2024-09-21T02:19:54.918301+0000","last_unstale":"2024-09-21T02:19:56.031808+0000","last_undegraded":"2024-09-21T02:19:56.031808+0000","last_fullsized":"2024-09-21T02:19:56.031808+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:11.857515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033882+0000","last_change":"2024-09-21T02:19:54.937799+0000","last_active":"2024-09-21T02:19:56.033882+0000","last_peered":"2024-09-21T02:19:56.033882+0000","last_clean":"2024-09-21T02:19:56.033882+0000","last_became_active":"2024-09-21T02:19:54.936900+0000","last_became_peered":"2024-09-21T02:19:54.936900+0000","last_unstale":"2024-09-21T02:19:56.033882+0000","last_undegraded":"2024-09-21T02:19:56.033882+0000","last_fullsized":"2024-09-21T02:19:56.033882+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.498586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031266+0000","last_change":"2024-09-21T02:19:54.890388+0000","last_active":"2024-09-21T02:19:56.031266+0000","last_peered":"2024-09-21T02:19:56.031266+0000","last_clean":"2024-09-21T02:19:56.031266+0000","last_became_active":"2024-09-21T02:19:54.890090+0000","last_became_peered":"2024-09-21T02:19:54.890090+0000","last_unstale":"2024-09-21T02:19:56.031266+0000","last_undegraded":"2024-09-21T02:19:56.031266+0000","last_fullsized":"2024-09-21T02:19:56.031266+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:04.023782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032079+0000","last_change":"2024-09-21T02:19:54.925819+0000","last_active":"2024-09-21T02:19:56.032079+0000","last_peered":"2024-09-21T02:19:56.032079+0000","last_clean":"2024-09-21T02:19:56.032079+0000","last_became_active":"2024-09-21T02:19:54.925092+0000","last_became_peered":"2024-09-21T02:19:54.925092+0000","last_unstale":"2024-09-21T02:19:56.032079+0000","last_undegraded":"2024-09-21T02:19:56.032079+0000","last_fullsized":"2024-09-21T02:19:56.032079+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:02.593728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884576+0000","last_change":"2024-09-21T02:19:54.897666+0000","last_active":"2024-09-21T02:19:55.884576+0000","last_peered":"2024-09-21T02:19:55.884576+0000","last_clean":"2024-09-21T02:19:55.884576+0000","last_became_active":"2024-09-21T02:19:54.897238+0000","last_became_peered":"2024-09-21T02:19:54.897238+0000","last_unstale":"2024-09-21T02:19:55.884576+0000","last_undegraded":"2024-09-21T02:19:55.884576+0000","last_fullsized":"2024-09-21T02:19:55.884576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034834+0000","last_change":"2024-09-21T02:19:54.938002+0000","last_active":"2024-09-21T02:19:56.034834+0000","last_peered":"2024-09-21T02:19:56.034834+0000","last_clean":"2024-09-21T02:19:56.034834+0000","last_became_active":"2024-09-21T02:19:54.933534+0000","last_became_peered":"2024-09-21T02:19:54.933534+0000","last_unstale":"2024-09-21T02:19:56.034834+0000","last_undegraded":"2024-09-21T02:19:56.034834+0000","last_fullsized":"2024-09-21T02:19:56.034834+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:39:09.996644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884648+0000","last_change":"2024-09-21T02:19:54.905204+0000","last_active":"2024-09-21T02:19:55.884648+0000","last_peered":"2024-09-21T02:19:55.884648+0000","last_clean":"2024-09-21T02:19:55.884648+0000","last_became_active":"2024-09-21T02:19:54.901010+0000","last_became_peered":"2024-09-21T02:19:54.901010+0000","last_unstale":"2024-09-21T02:19:55.884648+0000","last_undegraded":"2024-09-21T02:19:55.884648+0000","last_fullsized":"2024-09-21T02:19:55.884648+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:05.022149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034133+0000","last_change":"2024-09-21T02:19:54.922767+0000","last_active":"2024-09-21T02:19:56.034133+0000","last_peered":"2024-09-21T02:19:56.034133+0000","last_clean":"2024-09-21T02:19:56.034133+0000","last_became_active":"2024-09-21T02:19:54.920983+0000","last_became_peered":"2024-09-21T02:19:54.920983+0000","last_unstale":"2024-09-21T02:19:56.034133+0000","last_undegraded":"2024-09-21T02:19:56.034133+0000","last_fullsized":"2024-09-21T02:19:56.034133+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:11.159928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034726+0000","last_change":"2024-09-21T02:19:54.917192+0000","last_active":"2024-09-21T02:19:56.034726+0000","last_peered":"2024-09-21T02:19:56.034726+0000","last_clean":"2024-09-21T02:19:56.034726+0000","last_became_active":"2024-09-21T02:19:54.915272+0000","last_became_peered":"2024-09-21T02:19:54.915272+0000","last_unstale":"2024-09-21T02:19:56.034726+0000","last_undegraded":"2024-09-21T02:19:56.034726+0000","last_fullsized":"2024-09-21T02:19:56.034726+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:03.624103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884969+0000","last_change":"2024-09-21T02:19:54.911900+0000","last_active":"2024-09-21T02:19:55.884969+0000","last_peered":"2024-09-21T02:19:55.884969+0000","last_clean":"2024-09-21T02:19:55.884969+0000","last_became_active":"2024-09-21T02:19:54.911670+0000","last_became_peered":"2024-09-21T02:19:54.911670+0000","last_unstale":"2024-09-21T02:19:55.884969+0000","last_undegraded":"2024-09-21T02:19:55.884969+0000","last_fullsized":"2024-09-21T02:19:55.884969+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:31.053860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031998+0000","last_change":"2024-09-21T02:19:54.916554+0000","last_active":"2024-09-21T02:19:56.031998+0000","last_peered":"2024-09-21T02:19:56.031998+0000","last_clean":"2024-09-21T02:19:56.031998+0000","last_became_active":"2024-09-21T02:19:54.915606+0000","last_became_peered":"2024-09-21T02:19:54.915606+0000","last_unstale":"2024-09-21T02:19:56.031998+0000","last_undegraded":"2024-09-21T02:19:56.031998+0000","last_fullsized":"2024-09-21T02:19:56.031998+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:29.703264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032258+0000","last_change":"2024-09-21T02:19:54.894732+0000","last_active":"2024-09-21T02:19:56.032258+0000","last_peered":"2024-09-21T02:19:56.032258+0000","last_clean":"2024-09-21T02:19:56.032258+0000","last_became_active":"2024-09-21T02:19:54.891804+0000","last_became_peered":"2024-09-21T02:19:54.891804+0000","last_unstale":"2024-09-21T02:19:56.032258+0000","last_undegraded":"2024-09-21T02:19:56.032258+0000","last_fullsized":"2024-09-21T02:19:56.032258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:40.559806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884629+0000","last_change":"2024-09-21T02:19:54.920110+0000","last_active":"2024-09-21T02:19:55.884629+0000","last_peered":"2024-09-21T02:19:55.884629+0000","last_clean":"2024-09-21T02:19:55.884629+0000","last_became_active":"2024-09-21T02:19:54.919775+0000","last_became_peered":"2024-09-21T02:19:54.919775+0000","last_unstale":"2024-09-21T02:19:55.884629+0000","last_undegraded":"2024-09-21T02:19:55.884629+0000","last_fullsized":"2024-09-21T02:19:55.884629+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:38:50.958741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031231+0000","last_change":"2024-09-21T02:19:54.891480+0000","last_active":"2024-09-21T02:19:56.031231+0000","last_peered":"2024-09-21T02:19:56.031231+0000","last_clean":"2024-09-21T02:19:56.031231+0000","last_became_active":"2024-09-21T02:19:54.891121+0000","last_became_peered":"2024-09-21T02:19:54.891121+0000","last_unstale":"2024-09-21T02:19:56.031231+0000","last_undegraded":"2024-09-21T02:19:56.031231+0000","last_fullsized":"2024-09-21T02:19:56.031231+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:16.346281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.035508+0000","last_change":"2024-09-21T02:19:54.926396+0000","last_active":"2024-09-21T02:19:56.035508+0000","last_peered":"2024-09-21T02:19:56.035508+0000","last_clean":"2024-09-21T02:19:56.035508+0000","last_became_active":"2024-09-21T02:19:54.923528+0000","last_became_peered":"2024-09-21T02:19:54.923528+0000","last_unstale":"2024-09-21T02:19:56.035508+0000","last_undegraded":"2024-09-21T02:19:56.035508+0000","last_fullsized":"2024-09-21T02:19:56.035508+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:57.481002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":19,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.885221+0000","last_change":"2024-09-21T02:19:53.899077+0000","last_active":"2024-09-21T02:19:55.885221+0000","last_peered":"2024-09-21T02:19:55.885221+0000","last_clean":"2024-09-21T02:19:55.885221+0000","last_became_active":"2024-09-21T02:19:53.898374+0000","last_became_peered":"2024-09-21T02:19:53.898374+0000","last_unstale":"2024-09-21T02:19:55.885221+0000","last_undegraded":"2024-09-21T02:19:55.885221+0000","last_fullsized":"2024-09-21T02:19:55.885221+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:36.548983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032643+0000","last_change":"2024-09-21T02:19:54.929191+0000","last_active":"2024-09-21T02:19:56.032643+0000","last_peered":"2024-09-21T02:19:56.032643+0000","last_clean":"2024-09-21T02:19:56.032643+0000","last_became_active":"2024-09-21T02:19:54.928094+0000","last_became_peered":"2024-09-21T02:19:54.928094+0000","last_unstale":"2024-09-21T02:19:56.032643+0000","last_undegraded":"2024-09-21T02:19:56.032643+0000","last_fullsized":"2024-09-21T02:19:56.032643+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:03.092577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032144+0000","last_change":"2024-09-21T02:19:53.894376+0000","last_active":"2024-09-21T02:19:56.032144+0000","last_peered":"2024-09-21T02:19:56.032144+0000","last_clean":"2024-09-21T02:19:56.032144+0000","last_became_active":"2024-09-21T02:19:53.893849+0000","last_became_peered":"2024-09-21T02:19:53.893849+0000","last_unstale":"2024-09-21T02:19:56.032144+0000","last_undegraded":"2024-09-21T02:19:56.032144+0000","last_fullsized":"2024-09-21T02:19:56.032144+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:18.822962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031714+0000","last_change":"2024-09-21T02:19:54.901725+0000","last_active":"2024-09-21T02:19:56.031714+0000","last_peered":"2024-09-21T02:19:56.031714+0000","last_clean":"2024-09-21T02:19:56.031714+0000","last_became_active":"2024-09-21T02:19:54.901342+0000","last_became_peered":"2024-09-21T02:19:54.901342+0000","last_unstale":"2024-09-21T02:19:56.031714+0000","last_undegraded":"2024-09-21T02:19:56.031714+0000","last_fullsized":"2024-09-21T02:19:56.031714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:56.028422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.885075+0000","last_change":"2024-09-21T02:19:53.913322+0000","last_active":"2024-09-21T02:19:55.885075+0000","last_peered":"2024-09-21T02:19:55.885075+0000","last_clean":"2024-09-21T02:19:55.885075+0000","last_became_active":"2024-09-21T02:19:53.912089+0000","last_became_peered":"2024-09-21T02:19:53.912089+0000","last_unstale":"2024-09-21T02:19:55.885075+0000","last_undegraded":"2024-09-21T02:19:55.885075+0000","last_fullsized":"2024-09-21T02:19:55.885075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:17.721839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033108+0000","last_change":"2024-09-21T02:19:54.921178+0000","last_active":"2024-09-21T02:19:56.033108+0000","last_peered":"2024-09-21T02:19:56.033108+0000","last_clean":"2024-09-21T02:19:56.033108+0000","last_became_active":"2024-09-21T02:19:54.920867+0000","last_became_peered":"2024-09-21T02:19:54.920867+0000","last_unstale":"2024-09-21T02:19:56.033108+0000","last_undegraded":"2024-09-21T02:19:56.033108+0000","last_fullsized":"2024-09-21T02:19:56.033108+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:18.572331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033058+0000","last_change":"2024-09-21T02:19:53.911987+0000","last_active":"2024-09-21T02:19:56.033058+0000","last_peered":"2024-09-21T02:19:56.033058+0000","last_clean":"2024-09-21T02:19:56.033058+0000","last_became_active":"2024-09-21T02:19:53.911176+0000","last_became_peered":"2024-09-21T02:19:53.911176+0000","last_unstale":"2024-09-21T02:19:56.033058+0000","last_undegraded":"2024-09-21T02:19:56.033058+0000","last_fullsized":"2024-09-21T02:19:56.033058+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:55.290071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034628+0000","last_change":"2024-09-21T02:19:54.938283+0000","last_active":"2024-09-21T02:19:56.034628+0000","last_peered":"2024-09-21T02:19:56.034628+0000","last_clean":"2024-09-21T02:19:56.034628+0000","last_became_active":"2024-09-21T02:19:54.937403+0000","last_became_peered":"2024-09-21T02:19:54.937403+0000","last_unstale":"2024-09-21T02:19:56.034628+0000","last_undegraded":"2024-09-21T02:19:56.034628+0000","last_fullsized":"2024-09-21T02:19:56.034628+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:13.527685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032271+0000","last_change":"2024-09-21T02:19:53.906525+0000","last_active":"2024-09-21T02:19:56.032271+0000","last_peered":"2024-09-21T02:19:56.032271+0000","last_clean":"2024-09-21T02:19:56.032271+0000","last_became_active":"2024-09-21T02:19:53.905607+0000","last_became_peered":"2024-09-21T02:19:53.905607+0000","last_unstale":"2024-09-21T02:19:56.032271+0000","last_undegraded":"2024-09-21T02:19:56.032271+0000","last_fullsized":"2024-09-21T02:19:56.032271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.829774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031895+0000","last_change":"2024-09-21T02:19:54.915684+0000","last_active":"2024-09-21T02:19:56.031895+0000","last_peered":"2024-09-21T02:19:56.031895+0000","last_clean":"2024-09-21T02:19:56.031895+0000","last_became_active":"2024-09-21T02:19:54.914299+0000","last_became_peered":"2024-09-21T02:19:54.914299+0000","last_unstale":"2024-09-21T02:19:56.031895+0000","last_undegraded":"2024-09-21T02:19:56.031895+0000","last_fullsized":"2024-09-21T02:19:56.031895+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:01.615758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.035289+0000","last_change":"2024-09-21T02:19:53.920704+0000","last_active":"2024-09-21T02:19:56.035289+0000","last_peered":"2024-09-21T02:19:56.035289+0000","last_clean":"2024-09-21T02:19:56.035289+0000","last_became_active":"2024-09-21T02:19:53.920013+0000","last_became_peered":"2024-09-21T02:19:53.920013+0000","last_unstale":"2024-09-21T02:19:56.035289+0000","last_undegraded":"2024-09-21T02:19:56.035289+0000","last_fullsized":"2024-09-21T02:19:56.035289+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:59.828152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.030447+0000","last_change":"2024-09-21T02:19:54.897839+0000","last_active":"2024-09-21T02:19:56.030447+0000","last_peered":"2024-09-21T02:19:56.030447+0000","last_clean":"2024-09-21T02:19:56.030447+0000","last_became_active":"2024-09-21T02:19:54.897520+0000","last_became_peered":"2024-09-21T02:19:54.897520+0000","last_unstale":"2024-09-21T02:19:56.030447+0000","last_undegraded":"2024-09-21T02:19:56.030447+0000","last_fullsized":"2024-09-21T02:19:56.030447+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:41.139364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032978+0000","last_change":"2024-09-21T02:19:53.900369+0000","last_active":"2024-09-21T02:19:56.032978+0000","last_peered":"2024-09-21T02:19:56.032978+0000","last_clean":"2024-09-21T02:19:56.032978+0000","last_became_active":"2024-09-21T02:19:53.899961+0000","last_became_peered":"2024-09-21T02:19:53.899961+0000","last_unstale":"2024-09-21T02:19:56.032978+0000","last_undegraded":"2024-09-21T02:19:56.032978+0000","last_fullsized":"2024-09-21T02:19:56.032978+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:02.811318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031535+0000","last_change":"2024-09-21T02:19:54.926183+0000","last_active":"2024-09-21T02:19:56.031535+0000","last_peered":"2024-09-21T02:19:56.031535+0000","last_clean":"2024-09-21T02:19:56.031535+0000","last_became_active":"2024-09-21T02:19:54.925059+0000","last_became_peered":"2024-09-21T02:19:54.925059+0000","last_unstale":"2024-09-21T02:19:56.031535+0000","last_undegraded":"2024-09-21T02:19:56.031535+0000","last_fullsized":"2024-09-21T02:19:56.031535+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:40:03.781380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031581+0000","last_change":"2024-09-21T02:19:53.906244+0000","last_active":"2024-09-21T02:19:56.031581+0000","last_peered":"2024-09-21T02:19:56.031581+0000","last_clean":"2024-09-21T02:19:56.031581+0000","last_became_active":"2024-09-21T02:19:53.905406+0000","last_became_peered":"2024-09-21T02:19:53.905406+0000","last_unstale":"2024-09-21T02:19:56.031581+0000","last_undegraded":"2024-09-21T02:19:56.031581+0000","last_fullsized":"2024-09-21T02:19:56.031581+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:09.984572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.036060+0000","last_change":"2024-09-21T02:19:54.925509+0000","last_active":"2024-09-21T02:19:56.036060+0000","last_peered":"2024-09-21T02:19:56.036060+0000","last_clean":"2024-09-21T02:19:56.036060+0000","last_became_active":"2024-09-21T02:19:54.924801+0000","last_became_peered":"2024-09-21T02:19:54.924801+0000","last_unstale":"2024-09-21T02:19:56.036060+0000","last_undegraded":"2024-09-21T02:19:56.036060+0000","last_fullsized":"2024-09-21T02:19:56.036060+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:36.699842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033416+0000","last_change":"2024-09-21T02:19:53.898084+0000","last_active":"2024-09-21T02:19:56.033416+0000","last_peered":"2024-09-21T02:19:56.033416+0000","last_clean":"2024-09-21T02:19:56.033416+0000","last_became_active":"2024-09-21T02:19:53.897635+0000","last_became_peered":"2024-09-21T02:19:53.897635+0000","last_unstale":"2024-09-21T02:19:56.033416+0000","last_undegraded":"2024-09-21T02:19:56.033416+0000","last_fullsized":"2024-09-21T02:19:56.033416+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:59.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884669+0000","last_change":"2024-09-21T02:19:54.936010+0000","last_active":"2024-09-21T02:19:55.884669+0000","last_peered":"2024-09-21T02:19:55.884669+0000","last_clean":"2024-09-21T02:19:55.884669+0000","last_became_active":"2024-09-21T02:19:54.935627+0000","last_became_peered":"2024-09-21T02:19:54.935627+0000","last_unstale":"2024-09-21T02:19:55.884669+0000","last_undegraded":"2024-09-21T02:19:55.884669+0000","last_fullsized":"2024-09-21T02:19:55.884669+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:19.437383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.035556+0000","last_change":"2024-09-21T02:19:53.914573+0000","last_active":"2024-09-21T02:19:56.035556+0000","last_peered":"2024-09-21T02:19:56.035556+0000","last_clean":"2024-09-21T02:19:56.035556+0000","last_became_active":"2024-09-21T02:19:53.913541+0000","last_became_peered":"2024-09-21T02:19:53.913541+0000","last_unstale":"2024-09-21T02:19:56.035556+0000","last_undegraded":"2024-09-21T02:19:56.035556+0000","last_fullsized":"2024-09-21T02:19:56.035556+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:55.390112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033104+0000","last_change":"2024-09-21T02:19:54.903309+0000","last_active":"2024-09-21T02:19:56.033104+0000","last_peered":"2024-09-21T02:19:56.033104+0000","last_clean":"2024-09-21T02:19:56.033104+0000","last_became_active":"2024-09-21T02:19:54.902326+0000","last_became_peered":"2024-09-21T02:19:54.902326+0000","last_unstale":"2024-09-21T02:19:56.033104+0000","last_undegraded":"2024-09-21T02:19:56.033104+0000","last_fullsized":"2024-09-21T02:19:56.033104+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:18.725779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.883630+0000","last_change":"2024-09-21T02:19:53.899480+0000","last_active":"2024-09-21T02:19:55.883630+0000","last_peered":"2024-09-21T02:19:55.883630+0000","last_clean":"2024-09-21T02:19:55.883630+0000","last_became_active":"2024-09-21T02:19:53.899037+0000","last_became_peered":"2024-09-21T02:19:53.899037+0000","last_unstale":"2024-09-21T02:19:55.883630+0000","last_undegraded":"2024-09-21T02:19:55.883630+0000","last_fullsized":"2024-09-21T02:19:55.883630+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:03.295308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884784+0000","last_change":"2024-09-21T02:19:54.899086+0000","last_active":"2024-09-21T02:19:55.884784+0000","last_peered":"2024-09-21T02:19:55.884784+0000","last_clean":"2024-09-21T02:19:55.884784+0000","last_became_active":"2024-09-21T02:19:54.898248+0000","last_became_peered":"2024-09-21T02:19:54.898248+0000","last_unstale":"2024-09-21T02:19:55.884784+0000","last_undegraded":"2024-09-21T02:19:55.884784+0000","last_fullsized":"2024-09-21T02:19:55.884784+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:53.946581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":21,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.885161+0000","last_change":"2024-09-21T02:19:53.904993+0000","last_active":"2024-09-21T02:19:55.885161+0000","last_peered":"2024-09-21T02:19:55.885161+0000","last_clean":"2024-09-21T02:19:55.885161+0000","last_became_active":"2024-09-21T02:19:53.904509+0000","last_became_peered":"2024-09-21T02:19:53.904509+0000","last_unstale":"2024-09-21T02:19:55.885161+0000","last_undegraded":"2024-09-21T02:19:55.885161+0000","last_fullsized":"2024-09-21T02:19:55.885161+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:20.411897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033675+0000","last_change":"2024-09-21T02:19:54.929018+0000","last_active":"2024-09-21T02:19:56.033675+0000","last_peered":"2024-09-21T02:19:56.033675+0000","last_clean":"2024-09-21T02:19:56.033675+0000","last_became_active":"2024-09-21T02:19:54.928119+0000","last_became_peered":"2024-09-21T02:19:54.928119+0000","last_unstale":"2024-09-21T02:19:56.033675+0000","last_undegraded":"2024-09-21T02:19:56.033675+0000","last_fullsized":"2024-09-21T02:19:56.033675+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:01.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033166+0000","last_change":"2024-09-21T02:19:53.903917+0000","last_active":"2024-09-21T02:19:56.033166+0000","last_peered":"2024-09-21T02:19:56.033166+0000","last_clean":"2024-09-21T02:19:56.033166+0000","last_became_active":"2024-09-21T02:19:53.903089+0000","last_became_peered":"2024-09-21T02:19:53.903089+0000","last_unstale":"2024-09-21T02:19:56.033166+0000","last_undegraded":"2024-09-21T02:19:56.033166+0000","last_fullsized":"2024-09-21T02:19:56.033166+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:17.563024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032261+0000","last_change":"2024-09-21T02:19:54.908363+0000","last_active":"2024-09-21T02:19:56.032261+0000","last_peered":"2024-09-21T02:19:56.032261+0000","last_clean":"2024-09-21T02:19:56.032261+0000","last_became_active":"2024-09-21T02:19:54.907315+0000","last_became_peered":"2024-09-21T02:19:54.907315+0000","last_unstale":"2024-09-21T02:19:56.032261+0000","last_undegraded":"2024-09-21T02:19:56.032261+0000","last_fullsized":"2024-09-21T02:19:56.032261+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:51.791228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.883445+0000","last_change":"2024-09-21T02:19:53.896469+0000","last_active":"2024-09-21T02:19:55.883445+0000","last_peered":"2024-09-21T02:19:55.883445+0000","last_clean":"2024-09-21T02:19:55.883445+0000","last_became_active":"2024-09-21T02:19:53.895529+0000","last_became_peered":"2024-09-21T02:19:53.895529+0000","last_unstale":"2024-09-21T02:19:55.883445+0000","last_undegraded":"2024-09-21T02:19:55.883445+0000","last_fullsized":"2024-09-21T02:19:55.883445+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:30.370234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.030861+0000","last_change":"2024-09-21T02:19:54.898547+0000","last_active":"2024-09-21T02:19:56.030861+0000","last_peered":"2024-09-21T02:19:56.030861+0000","last_clean":"2024-09-21T02:19:56.030861+0000","last_became_active":"2024-09-21T02:19:54.898117+0000","last_became_peered":"2024-09-21T02:19:54.898117+0000","last_unstale":"2024-09-21T02:19:56.030861+0000","last_undegraded":"2024-09-21T02:19:56.030861+0000","last_fullsized":"2024-09-21T02:19:56.030861+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:43.535217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.030606+0000","last_change":"2024-09-21T02:19:53.919682+0000","last_active":"2024-09-21T02:19:56.030606+0000","last_peered":"2024-09-21T02:19:56.030606+0000","last_clean":"2024-09-21T02:19:56.030606+0000","last_became_active":"2024-09-21T02:19:53.919315+0000","last_became_peered":"2024-09-21T02:19:53.919315+0000","last_unstale":"2024-09-21T02:19:56.030606+0000","last_undegraded":"2024-09-21T02:19:56.030606+0000","last_fullsized":"2024-09-21T02:19:56.030606+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:24.141384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.883711+0000","last_change":"2024-09-21T02:19:54.914581+0000","last_active":"2024-09-21T02:19:55.883711+0000","last_peered":"2024-09-21T02:19:55.883711+0000","last_clean":"2024-09-21T02:19:55.883711+0000","last_became_active":"2024-09-21T02:19:54.914244+0000","last_became_peered":"2024-09-21T02:19:54.914244+0000","last_unstale":"2024-09-21T02:19:55.883711+0000","last_undegraded":"2024-09-21T02:19:55.883711+0000","last_fullsized":"2024-09-21T02:19:55.883711+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:14.299993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033475+0000","last_change":"2024-09-21T02:19:53.907585+0000","last_active":"2024-09-21T02:19:56.033475+0000","last_peered":"2024-09-21T02:19:56.033475+0000","last_clean":"2024-09-21T02:19:56.033475+0000","last_became_active":"2024-09-21T02:19:53.906977+0000","last_became_peered":"2024-09-21T02:19:53.906977+0000","last_unstale":"2024-09-21T02:19:56.033475+0000","last_undegraded":"2024-09-21T02:19:56.033475+0000","last_fullsized":"2024-09-21T02:19:56.033475+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:49:37.222021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884990+0000","last_change":"2024-09-21T02:19:54.929584+0000","last_active":"2024-09-21T02:19:55.884990+0000","last_peered":"2024-09-21T02:19:55.884990+0000","last_clean":"2024-09-21T02:19:55.884990+0000","last_became_active":"2024-09-21T02:19:54.928816+0000","last_became_peered":"2024-09-21T02:19:54.928816+0000","last_unstale":"2024-09-21T02:19:55.884990+0000","last_undegraded":"2024-09-21T02:19:55.884990+0000","last_fullsized":"2024-09-21T02:19:55.884990+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:14.378498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884019+0000","last_change":"2024-09-21T02:19:53.904893+0000","last_active":"2024-09-21T02:19:55.884019+0000","last_peered":"2024-09-21T02:19:55.884019+0000","last_clean":"2024-09-21T02:19:55.884019+0000","last_became_active":"2024-09-21T02:19:53.904064+0000","last_became_peered":"2024-09-21T02:19:53.904064+0000","last_unstale":"2024-09-21T02:19:55.884019+0000","last_undegraded":"2024-09-21T02:19:55.884019+0000","last_fullsized":"2024-09-21T02:19:55.884019+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:33.874089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032392+0000","last_change":"2024-09-21T02:19:54.919254+0000","last_active":"2024-09-21T02:19:56.032392+0000","last_peered":"2024-09-21T02:19:56.032392+0000","last_clean":"2024-09-21T02:19:56.032392+0000","last_became_active":"2024-09-21T02:19:54.918380+0000","last_became_peered":"2024-09-21T02:19:54.918380+0000","last_unstale":"2024-09-21T02:19:56.032392+0000","last_undegraded":"2024-09-21T02:19:56.032392+0000","last_fullsized":"2024-09-21T02:19:56.032392+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:54.476348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032030+0000","last_change":"2024-09-21T02:19:53.898390+0000","last_active":"2024-09-21T02:19:56.032030+0000","last_peered":"2024-09-21T02:19:56.032030+0000","last_clean":"2024-09-21T02:19:56.032030+0000","last_became_active":"2024-09-21T02:19:53.897822+0000","last_became_peered":"2024-09-21T02:19:53.897822+0000","last_unstale":"2024-09-21T02:19:56.032030+0000","last_undegraded":"2024-09-21T02:19:56.032030+0000","last_fullsized":"2024-09-21T02:19:56.032030+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:47.055052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032485+0000","last_change":"2024-09-21T02:19:54.932251+0000","last_active":"2024-09-21T02:19:56.032485+0000","last_peered":"2024-09-21T02:19:56.032485+0000","last_clean":"2024-09-21T02:19:56.032485+0000","last_became_active":"2024-09-21T02:19:54.926959+0000","last_became_peered":"2024-09-21T02:19:54.926959+0000","last_unstale":"2024-09-21T02:19:56.032485+0000","last_undegraded":"2024-09-21T02:19:56.032485+0000","last_fullsized":"2024-09-21T02:19:56.032485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:26.385694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884255+0000","last_change":"2024-09-21T02:19:53.889955+0000","last_active":"2024-09-21T02:19:55.884255+0000","last_peered":"2024-09-21T02:19:55.884255+0000","last_clean":"2024-09-21T02:19:55.884255+0000","last_became_active":"2024-09-21T02:19:53.889393+0000","last_became_peered":"2024-09-21T02:19:53.889393+0000","last_unstale":"2024-09-21T02:19:55.884255+0000","last_undegraded":"2024-09-21T02:19:55.884255+0000","last_fullsized":"2024-09-21T02:19:55.884255+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.390167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034869+0000","last_change":"2024-09-21T02:19:54.926269+0000","last_active":"2024-09-21T02:19:56.034869+0000","last_peered":"2024-09-21T02:19:56.034869+0000","last_clean":"2024-09-21T02:19:56.034869+0000","last_became_active":"2024-09-21T02:19:54.923811+0000","last_became_peered":"2024-09-21T02:19:54.923811+0000","last_unstale":"2024-09-21T02:19:56.034869+0000","last_undegraded":"2024-09-21T02:19:56.034869+0000","last_fullsized":"2024-09-21T02:19:56.034869+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:25:21.378164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031455+0000","last_change":"2024-09-21T02:19:53.897209+0000","last_active":"2024-09-21T02:19:56.031455+0000","last_peered":"2024-09-21T02:19:56.031455+0000","last_clean":"2024-09-21T02:19:56.031455+0000","last_became_active":"2024-09-21T02:19:53.896800+0000","last_became_peered":"2024-09-21T02:19:53.896800+0000","last_unstale":"2024-09-21T02:19:56.031455+0000","last_undegraded":"2024-09-21T02:19:56.031455+0000","last_fullsized":"2024-09-21T02:19:56.031455+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:01.983478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884193+0000","last_change":"2024-09-21T02:19:54.897635+0000","last_active":"2024-09-21T02:19:55.884193+0000","last_peered":"2024-09-21T02:19:55.884193+0000","last_clean":"2024-09-21T02:19:55.884193+0000","last_became_active":"2024-09-21T02:19:54.897009+0000","last_became_peered":"2024-09-21T02:19:54.897009+0000","last_unstale":"2024-09-21T02:19:55.884193+0000","last_undegraded":"2024-09-21T02:19:55.884193+0000","last_fullsized":"2024-09-21T02:19:55.884193+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:23.602511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.035650+0000","last_change":"2024-09-21T02:19:53.915141+0000","last_active":"2024-09-21T02:19:56.035650+0000","last_peered":"2024-09-21T02:19:56.035650+0000","last_clean":"2024-09-21T02:19:56.035650+0000","last_became_active":"2024-09-21T02:19:53.914156+0000","last_became_peered":"2024-09-21T02:19:53.914156+0000","last_unstale":"2024-09-21T02:19:56.035650+0000","last_undegraded":"2024-09-21T02:19:56.035650+0000","last_fullsized":"2024-09-21T02:19:56.035650+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:51.674537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033152+0000","last_change":"2024-09-21T02:19:54.941473+0000","last_active":"2024-09-21T02:19:56.033152+0000","last_peered":"2024-09-21T02:19:56.033152+0000","last_clean":"2024-09-21T02:19:56.033152+0000","last_became_active":"2024-09-21T02:19:54.940836+0000","last_became_peered":"2024-09-21T02:19:54.940836+0000","last_unstale":"2024-09-21T02:19:56.033152+0000","last_undegraded":"2024-09-21T02:19:56.033152+0000","last_fullsized":"2024-09-21T02:19:56.033152+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:47.076073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033038+0000","last_change":"2024-09-21T02:19:53.906690+0000","last_active":"2024-09-21T02:19:56.033038+0000","last_peered":"2024-09-21T02:19:56.033038+0000","last_clean":"2024-09-21T02:19:56.033038+0000","last_became_active":"2024-09-21T02:19:53.905289+0000","last_became_peered":"2024-09-21T02:19:53.905289+0000","last_unstale":"2024-09-21T02:19:56.033038+0000","last_undegraded":"2024-09-21T02:19:56.033038+0000","last_fullsized":"2024-09-21T02:19:56.033038+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:24.887668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.030889+0000","last_change":"2024-09-21T02:19:54.931825+0000","last_active":"2024-09-21T02:19:56.030889+0000","last_peered":"2024-09-21T02:19:56.030889+0000","last_clean":"2024-09-21T02:19:56.030889+0000","last_became_active":"2024-09-21T02:19:54.931506+0000","last_became_peered":"2024-09-21T02:19:54.931506+0000","last_unstale":"2024-09-21T02:19:56.030889+0000","last_undegraded":"2024-09-21T02:19:56.030889+0000","last_fullsized":"2024-09-21T02:19:56.030889+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:13.180966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.883422+0000","last_change":"2024-09-21T02:19:53.904489+0000","last_active":"2024-09-21T02:19:55.883422+0000","last_peered":"2024-09-21T02:19:55.883422+0000","last_clean":"2024-09-21T02:19:55.883422+0000","last_became_active":"2024-09-21T02:19:53.903679+0000","last_became_peered":"2024-09-21T02:19:53.903679+0000","last_unstale":"2024-09-21T02:19:55.883422+0000","last_undegraded":"2024-09-21T02:19:55.883422+0000","last_fullsized":"2024-09-21T02:19:55.883422+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:00.611818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.885312+0000","last_change":"2024-09-21T02:19:54.897476+0000","last_active":"2024-09-21T02:19:55.885312+0000","last_peered":"2024-09-21T02:19:55.885312+0000","last_clean":"2024-09-21T02:19:55.885312+0000","last_became_active":"2024-09-21T02:19:54.897190+0000","last_became_peered":"2024-09-21T02:19:54.897190+0000","last_unstale":"2024-09-21T02:19:55.885312+0000","last_undegraded":"2024-09-21T02:19:55.885312+0000","last_fullsized":"2024-09-21T02:19:55.885312+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:26.827414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034628+0000","last_change":"2024-09-21T02:19:53.917476+0000","last_active":"2024-09-21T02:19:56.034628+0000","last_peered":"2024-09-21T02:19:56.034628+0000","last_clean":"2024-09-21T02:19:56.034628+0000","last_became_active":"2024-09-21T02:19:53.916672+0000","last_became_peered":"2024-09-21T02:19:53.916672+0000","last_unstale":"2024-09-21T02:19:56.034628+0000","last_undegraded":"2024-09-21T02:19:56.034628+0000","last_fullsized":"2024-09-21T02:19:56.034628+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:39.091996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033666+0000","last_change":"2024-09-21T02:19:54.918954+0000","last_active":"2024-09-21T02:19:56.033666+0000","last_peered":"2024-09-21T02:19:56.033666+0000","last_clean":"2024-09-21T02:19:56.033666+0000","last_became_active":"2024-09-21T02:19:54.918028+0000","last_became_peered":"2024-09-21T02:19:54.918028+0000","last_unstale":"2024-09-21T02:19:56.033666+0000","last_undegraded":"2024-09-21T02:19:56.033666+0000","last_fullsized":"2024-09-21T02:19:56.033666+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:13.127133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"118'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.884039+0000","last_change":"2024-09-21T02:19:53.900842+0000","last_active":"2024-09-21T02:19:55.884039+0000","last_peered":"2024-09-21T02:19:55.884039+0000","last_clean":"2024-09-21T02:19:55.884039+0000","last_became_active":"2024-09-21T02:19:53.900353+0000","last_became_peered":"2024-09-21T02:19:53.900353+0000","last_unstale":"2024-09-21T02:19:55.884039+0000","last_undegraded":"2024-09-21T02:19:55.884039+0000","last_fullsized":"2024-09-21T02:19:55.884039+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:31.107364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033693+0000","last_change":"2024-09-21T02:19:54.924797+0000","last_active":"2024-09-21T02:19:56.033693+0000","last_peered":"2024-09-21T02:19:56.033693+0000","last_clean":"2024-09-21T02:19:56.033693+0000","last_became_active":"2024-09-21T02:19:54.923556+0000","last_became_peered":"2024-09-21T02:19:54.923556+0000","last_unstale":"2024-09-21T02:19:56.033693+0000","last_undegraded":"2024-09-21T02:19:56.033693+0000","last_fullsized":"2024-09-21T02:19:56.033693+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:26.077608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.034961+0000","last_change":"2024-09-21T02:19:53.916530+0000","last_active":"2024-09-21T02:19:56.034961+0000","last_peered":"2024-09-21T02:19:56.034961+0000","last_clean":"2024-09-21T02:19:56.034961+0000","last_became_active":"2024-09-21T02:19:53.915419+0000","last_became_peered":"2024-09-21T02:19:53.915419+0000","last_unstale":"2024-09-21T02:19:56.034961+0000","last_undegraded":"2024-09-21T02:19:56.034961+0000","last_fullsized":"2024-09-21T02:19:56.034961+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:56.800284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031829+0000","last_change":"2024-09-21T02:19:54.920584+0000","last_active":"2024-09-21T02:19:56.031829+0000","last_peered":"2024-09-21T02:19:56.031829+0000","last_clean":"2024-09-21T02:19:56.031829+0000","last_became_active":"2024-09-21T02:19:54.920198+0000","last_became_peered":"2024-09-21T02:19:54.920198+0000","last_unstale":"2024-09-21T02:19:56.031829+0000","last_undegraded":"2024-09-21T02:19:56.031829+0000","last_fullsized":"2024-09-21T02:19:56.031829+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:31.929067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033891+0000","last_change":"2024-09-21T02:19:53.911432+0000","last_active":"2024-09-21T02:19:56.033891+0000","last_peered":"2024-09-21T02:19:56.033891+0000","last_clean":"2024-09-21T02:19:56.033891+0000","last_became_active":"2024-09-21T02:19:53.910658+0000","last_became_peered":"2024-09-21T02:19:53.910658+0000","last_unstale":"2024-09-21T02:19:56.033891+0000","last_undegraded":"2024-09-21T02:19:56.033891+0000","last_fullsized":"2024-09-21T02:19:56.033891+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:58.045114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031581+0000","last_change":"2024-09-21T02:19:54.892316+0000","last_active":"2024-09-21T02:19:56.031581+0000","last_peered":"2024-09-21T02:19:56.031581+0000","last_clean":"2024-09-21T02:19:56.031581+0000","last_became_active":"2024-09-21T02:19:54.891987+0000","last_became_peered":"2024-09-21T02:19:54.891987+0000","last_unstale":"2024-09-21T02:19:56.031581+0000","last_undegraded":"2024-09-21T02:19:56.031581+0000","last_fullsized":"2024-09-21T02:19:56.031581+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:17.119102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032644+0000","last_change":"2024-09-21T02:19:53.909822+0000","last_active":"2024-09-21T02:19:56.032644+0000","last_peered":"2024-09-21T02:19:56.032644+0000","last_clean":"2024-09-21T02:19:56.032644+0000","last_became_active":"2024-09-21T02:19:53.908919+0000","last_became_peered":"2024-09-21T02:19:53.908919+0000","last_unstale":"2024-09-21T02:19:56.032644+0000","last_undegraded":"2024-09-21T02:19:56.032644+0000","last_fullsized":"2024-09-21T02:19:56.032644+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:10.216241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032366+0000","last_change":"2024-09-21T02:19:54.906640+0000","last_active":"2024-09-21T02:19:56.032366+0000","last_peered":"2024-09-21T02:19:56.032366+0000","last_clean":"2024-09-21T02:19:56.032366+0000","last_became_active":"2024-09-21T02:19:54.906337+0000","last_became_peered":"2024-09-21T02:19:54.906337+0000","last_unstale":"2024-09-21T02:19:56.032366+0000","last_undegraded":"2024-09-21T02:19:56.032366+0000","last_fullsized":"2024-09-21T02:19:56.032366+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:31.374174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031958+0000","last_change":"2024-09-21T02:19:53.915286+0000","last_active":"2024-09-21T02:19:56.031958+0000","last_peered":"2024-09-21T02:19:56.031958+0000","last_clean":"2024-09-21T02:19:56.031958+0000","last_became_active":"2024-09-21T02:19:53.914782+0000","last_became_peered":"2024-09-21T02:19:53.914782+0000","last_unstale":"2024-09-21T02:19:56.031958+0000","last_undegraded":"2024-09-21T02:19:56.031958+0000","last_fullsized":"2024-09-21T02:19:56.031958+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:19.753769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.035717+0000","last_change":"2024-09-21T02:19:54.940804+0000","last_active":"2024-09-21T02:19:56.035717+0000","last_peered":"2024-09-21T02:19:56.035717+0000","last_clean":"2024-09-21T02:19:56.035717+0000","last_became_active":"2024-09-21T02:19:54.938544+0000","last_became_peered":"2024-09-21T02:19:54.938544+0000","last_unstale":"2024-09-21T02:19:56.035717+0000","last_undegraded":"2024-09-21T02:19:56.035717+0000","last_fullsized":"2024-09-21T02:19:56.035717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:08.962262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.883691+0000","last_change":"2024-09-21T02:19:53.896536+0000","last_active":"2024-09-21T02:19:55.883691+0000","last_peered":"2024-09-21T02:19:55.883691+0000","last_clean":"2024-09-21T02:19:55.883691+0000","last_became_active":"2024-09-21T02:19:53.895589+0000","last_became_peered":"2024-09-21T02:19:53.895589+0000","last_unstale":"2024-09-21T02:19:55.883691+0000","last_undegraded":"2024-09-21T02:19:55.883691+0000","last_fullsized":"2024-09-21T02:19:55.883691+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:17.568391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.033377+0000","last_change":"2024-09-21T02:19:54.908653+0000","last_active":"2024-09-21T02:19:56.033377+0000","last_peered":"2024-09-21T02:19:56.033377+0000","last_clean":"2024-09-21T02:19:56.033377+0000","last_became_active":"2024-09-21T02:19:54.907771+0000","last_became_peered":"2024-09-21T02:19:54.907771+0000","last_unstale":"2024-09-21T02:19:56.033377+0000","last_undegraded":"2024-09-21T02:19:56.033377+0000","last_fullsized":"2024-09-21T02:19:56.033377+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:16.505277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.035542+0000","last_change":"2024-09-21T02:19:53.908787+0000","last_active":"2024-09-21T02:19:56.035542+0000","last_peered":"2024-09-21T02:19:56.035542+0000","last_clean":"2024-09-21T02:19:56.035542+0000","last_became_active":"2024-09-21T02:19:53.907954+0000","last_became_peered":"2024-09-21T02:19:53.907954+0000","last_unstale":"2024-09-21T02:19:56.035542+0000","last_undegraded":"2024-09-21T02:19:56.035542+0000","last_fullsized":"2024-09-21T02:19:56.035542+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:25:37.722502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.031876+0000","last_change":"2024-09-21T02:19:54.907933+0000","last_active":"2024-09-21T02:19:56.031876+0000","last_peered":"2024-09-21T02:19:56.031876+0000","last_clean":"2024-09-21T02:19:56.031876+0000","last_became_active":"2024-09-21T02:19:54.907629+0000","last_became_peered":"2024-09-21T02:19:54.907629+0000","last_unstale":"2024-09-21T02:19:56.031876+0000","last_undegraded":"2024-09-21T02:19:56.031876+0000","last_fullsized":"2024-09-21T02:19:56.031876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:53.856540+0000","last_clean_scrub_stamp":"2024-09-21T02:19:53.856540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:11.250600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:56.032455+0000","last_change":"2024-09-21T02:19:53.913344+0000","last_active":"2024-09-21T02:19:56.032455+0000","last_peered":"2024-09-21T02:19:56.032455+0000","last_clean":"2024-09-21T02:19:56.032455+0000","last_became_active":"2024-09-21T02:19:53.912800+0000","last_became_peered":"2024-09-21T02:19:53.912800+0000","last_unstale":"2024-09-21T02:19:56.032455+0000","last_undegraded":"2024-09-21T02:19:56.032455+0000","last_fullsized":"2024-09-21T02:19:56.032455+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:19:52.852600+0000","last_clean_scrub_stamp":"2024-09-21T02:19:52.852600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:44.544830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":312,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T02:19:55.885542+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:19:55.885542+0000","last_peered":"2024-09-21T02:19:55.885542+0000","last_clean":"2024-09-21T02:19:55.885542+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:19:55.885542+0000","last_undegraded":"2024-09-21T02:19:55.885542+0000","last_fullsized":"2024-09-21T02:19:55.885542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1470464,"data_stored":644866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509523,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1466368,"data_stored":644832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509524,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1235942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542229,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1466368,"data_stored":644832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1470464,"data_stored":644902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":1474560,"data_stored":645504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509524,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41536,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330304,"statfs":{"total":96636764160,"available":96594231296,"internally_reserved":0,"allocated":2076672,"data_stored":1236834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1486848,"data_stored":646500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:20:04.043 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:20:04.043 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:20:04.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52ee1c5640 1 -- 172.21.15.170:0/3775297118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 msgr2=0x7f52d0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/3775297118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52d0004c20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c2e0 tx=0x7f52dc01f6c0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52e7fff640 1 -- 172.21.15.170:0/3775297118 reap_dead start 2024-09-21T02:20:04.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52ee1c5640 1 -- 172.21.15.170:0/3775297118 shutdown_connections 2024-09-21T02:20:04.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52ee1c5640 1 -- 172.21.15.170:0/3775297118 >> 172.21.15.170:0/3775297118 conn(0x7f52e80d0170 msgr2=0x7f52e80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:04.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52ee1c5640 1 -- 172.21.15.170:0/3775297118 shutdown_connections 2024-09-21T02:20:04.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.250+0000 7f52ee1c5640 1 -- 172.21.15.170:0/3775297118 wait complete. 2024-09-21T02:20:04.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 Processor -- start 2024-09-21T02:20:04.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 -- start start 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52e80c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52e81f5c20 0x7f52e80c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52e80c4520 0x7f52e80c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80cbb10 con 0x7f52d0004810 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80cb990 con 0x7f52e80c4520 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52ee1c5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f52e80cbc90 con 0x7f52e81f5c20 2024-09-21T02:20:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52e80c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52e80c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59194/0 (socket says 172.21.15.170:59194) 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 -- 172.21.15.170:0/1920474331 learned_addr learned my addr 172.21.15.170:0/1920474331 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e7fff640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52e80c4520 0x7f52e80c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e6ffd640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52e81f5c20 0x7f52e80c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 -- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52e81f5c20 msgr2=0x7f52e80c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52e81f5c20 0x7f52e80c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 -- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52e80c4520 msgr2=0x7f52e80c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52e80c4520 0x7f52e80c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.254+0000 7f52e77fe640 1 -- 172.21.15.170:0/1920474331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80c4e70 con 0x7f52d0004810 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52e77fe640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52e80c6770 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f52d800d670 tx=0x7f52d800db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80060d0 con 0x7f52d0004810 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52d8015020 con 0x7f52d0004810 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800b830 con 0x7f52d0004810 2024-09-21T02:20:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e80c32f0 con 0x7f52d0004810 2024-09-21T02:20:04.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e80b1440 con 0x7f52d0004810 2024-09-21T02:20:04.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e80cc220 con 0x7f52d0004810 2024-09-21T02:20:04.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f52d800b9d0 con 0x7f52d0004810 2024-09-21T02:20:04.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52cffff640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52d4073680 0x7f52d4075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f52d8095330 con 0x7f52d0004810 2024-09-21T02:20:04.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52e6ffd640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52d4073680 0x7f52d4075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.258+0000 7f52e6ffd640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52d4073680 0x7f52d4075b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0b0000 tx=0x7f52dc01c500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:04.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.262+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f52d801f020 con 0x7f52d0004810 2024-09-21T02:20:04.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.494+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f52e806dec0 con 0x7f52d0004810 2024-09-21T02:20:04.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.498+0000 7f52cffff640 1 -- 172.21.15.170:0/1920474331 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) ==== 190+0+147 (secure 0 0 0) 0x7f52e80c4e70 con 0x7f52d0004810 2024-09-21T02:20:04.498 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:20:04.498 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:20:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:20:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:20:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:20:04.499 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52d4073680 msgr2=0x7f52d4075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52d4073680 0x7f52d4075b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0b0000 tx=0x7f52dc01c500 comp rx=0 tx=0).stop 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 msgr2=0x7f52e80c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52e80c6770 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f52d800d670 tx=0x7f52d800db30 comp rx=0 tx=0).stop 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 shutdown_connections 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52d4073680 0x7f52d4075b90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52e81f5c20 0x7f52e80c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52e80c4520 0x7f52e80c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 --2- 172.21.15.170:0/1920474331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52d0004810 0x7f52e80c6770 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 >> 172.21.15.170:0/1920474331 conn(0x7f52e80d0170 msgr2=0x7f52e815ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 shutdown_connections 2024-09-21T02:20:04.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.502+0000 7f52ee1c5640 1 -- 172.21.15.170:0/1920474331 wait complete. 2024-09-21T02:20:04.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 -- 172.21.15.170:0/1564536597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b7ab0 msgr2=0x7ff75015c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 --2- 172.21.15.170:0/1564536597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b7ab0 0x7ff75015c850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff748009900 tx=0x7ff74801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 -- 172.21.15.170:0/1564536597 shutdown_connections 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 --2- 172.21.15.170:0/1564536597 >> v2:172.21.15.191:3300/0 conn(0x7ff7500d0aa0 0x7ff7500d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 --2- 172.21.15.170:0/1564536597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b7ab0 0x7ff75015c850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 -- 172.21.15.170:0/1564536597 >> 172.21.15.170:0/1564536597 conn(0x7ff7500cee30 msgr2=0x7ff7500cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 -- 172.21.15.170:0/1564536597 shutdown_connections 2024-09-21T02:20:04.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.674+0000 7ff758230640 1 -- 172.21.15.170:0/1564536597 wait complete. 2024-09-21T02:20:04.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 Processor -- start 2024-09-21T02:20:04.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 -- start start 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b3e30 0x7ff7501fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 0x7ff75015a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7500d0aa0 0x7ff75015ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff75016a600 con 0x7ff7500b7ab0 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff75016a480 con 0x7ff7500d0aa0 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff75016a780 con 0x7ff7500b3e30 2024-09-21T02:20:04.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 0x7ff75015a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 0x7ff75015a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:59198/0 (socket says 172.21.15.170:59198) 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 -- 172.21.15.170:0/4189515574 learned_addr learned my addr 172.21.15.170:0/4189515574 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7567cd640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7500d0aa0 0x7ff75015ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff755fcc640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b3e30 0x7ff7501fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 -- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b3e30 msgr2=0x7ff7501fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b3e30 0x7ff7501fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 -- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7500d0aa0 msgr2=0x7ff75015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7500d0aa0 0x7ff75015ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 -- 172.21.15.170:0/4189515574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75015ca60 con 0x7ff7500b7ab0 2024-09-21T02:20:04.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7557cb640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 0x7ff75015a640 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c730 tx=0x7ff74c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:04.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c003b80 con 0x7ff7500b7ab0 2024-09-21T02:20:04.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff74c003d40 con 0x7ff7500b7ab0 2024-09-21T02:20:04.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00aab0 con 0x7ff7500b7ab0 2024-09-21T02:20:04.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7501fd200 con 0x7ff7500b7ab0 2024-09-21T02:20:04.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.678+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501fd7c0 con 0x7ff7500b7ab0 2024-09-21T02:20:04.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.682+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7501fcf70 con 0x7ff7500b7ab0 2024-09-21T02:20:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.682+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7ff74c00ac50 con 0x7ff7500b7ab0 2024-09-21T02:20:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.682+0000 7ff7467fc640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff734073680 0x7ff734075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.682+0000 7ff755fcc640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff734073680 0x7ff734075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.682+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7ff74c094660 con 0x7ff7500b7ab0 2024-09-21T02:20:04.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.682+0000 7ff755fcc640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff734073680 0x7ff734075b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7480af000 tx=0x7ff74801b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:04.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.686+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff74c020060 con 0x7ff7500b7ab0 2024-09-21T02:20:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.882+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff75015c850 con 0x7ff7500b7ab0 2024-09-21T02:20:04.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff7467fc640 1 -- 172.21.15.170:0/4189515574 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5228 (secure 0 0 0) 0x7ff74c05ef40 con 0x7ff7500b7ab0 2024-09-21T02:20:04.884 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:04.885 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":81,"btime":"2024-09-21T02:19:57:928053+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:19:54.300110+0000","modified":"2024-09-21T02:19:57.337649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7614":{"gid":7614,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/300226457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":300226457},{"type":"v1","addr":"172.21.15.170:6835","nonce":300226457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7614,"qdb_cluster":[7614]},"id":9}]} 2024-09-21T02:20:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 81 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff734073680 msgr2=0x7ff734075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff734073680 0x7ff734075b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7480af000 tx=0x7ff74801b120 comp rx=0 tx=0).stop 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 msgr2=0x7ff75015a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 0x7ff75015a640 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00c730 tx=0x7ff74c00cbf0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 shutdown_connections 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff734073680 0x7ff734075b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7500d0aa0 0x7ff75015ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7500b7ab0 0x7ff75015a640 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 --2- 172.21.15.170:0/4189515574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7500b3e30 0x7ff7501fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 >> 172.21.15.170:0/4189515574 conn(0x7ff7500cee30 msgr2=0x7ff7500b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 shutdown_connections 2024-09-21T02:20:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:04.886+0000 7ff758230640 1 -- 172.21.15.170:0/4189515574 wait complete. 2024-09-21T02:20:05.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:19:54.300110+0000', 'modified': '2024-09-21T02:19:57.337649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7614': {'gid': 7614, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/300226457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 300226457}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 300226457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7614, 'qdb_cluster': [7614]} 2024-09-21T02:20:05.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:20:05.309 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T02:20:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- 172.21.15.170:0/633667403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64380cd210 msgr2=0x7f64380b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 --2- 172.21.15.170:0/633667403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64380cd210 0x7f64380b3350 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6438071d80 tx=0x7f643001cd80 comp rx=0 tx=0).stop 2024-09-21T02:20:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- 172.21.15.170:0/633667403 shutdown_connections 2024-09-21T02:20:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 --2- 172.21.15.170:0/633667403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64380cd210 0x7f64380b3350 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 --2- 172.21.15.170:0/633667403 >> v2:172.21.15.191:3300/0 conn(0x7f64380b5410 0x7f643806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- 172.21.15.170:0/633667403 >> 172.21.15.170:0/633667403 conn(0x7f64380b1c40 msgr2=0x7f64380b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- 172.21.15.170:0/633667403 shutdown_connections 2024-09-21T02:20:05.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- 172.21.15.170:0/633667403 wait complete. 2024-09-21T02:20:05.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 Processor -- start 2024-09-21T02:20:05.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- start start 2024-09-21T02:20:05.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64380b2f40 0x7f643809d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64380b5410 0x7f64381fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 0x7f64381fc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6438169fd0 con 0x7f6438147960 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6438169e50 con 0x7f64380b5410 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643ee66640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f643816a150 con 0x7f64380b2f40 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643d403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 0x7f64381fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643d403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 0x7f64381fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38250/0 (socket says 172.21.15.170:38250) 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.474+0000 7f643d403640 1 -- 172.21.15.170:0/2512953389 learned_addr learned my addr 172.21.15.170:0/2512953389 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642ffff640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64380b5410 0x7f64381fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643d403640 1 -- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64380b2f40 msgr2=0x7f643809d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643cc02640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64380b2f40 0x7f643809d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643d403640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64380b2f40 0x7f643809d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643d403640 1 -- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64380b5410 msgr2=0x7f64381fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643d403640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64380b5410 0x7f64381fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643d403640 1 -- 172.21.15.170:0/2512953389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381fd010 con 0x7f6438147960 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642ffff640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64380b5410 0x7f64381fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643d403640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 0x7f64381fc760 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f643001b430 tx=0x7f6430003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642d7fa640 1 -- 172.21.15.170:0/2512953389 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300bc070 con 0x7f6438147960 2024-09-21T02:20:05.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642d7fa640 1 -- 172.21.15.170:0/2512953389 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64300042f0 con 0x7f6438147960 2024-09-21T02:20:05.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643815ac60 con 0x7f6438147960 2024-09-21T02:20:05.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642d7fa640 1 -- 172.21.15.170:0/2512953389 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300b2040 con 0x7f6438147960 2024-09-21T02:20:05.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643815b220 con 0x7f6438147960 2024-09-21T02:20:05.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642d7fa640 1 -- 172.21.15.170:0/2512953389 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f64300bf070 con 0x7f6438147960 2024-09-21T02:20:05.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642d7fa640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6424073680 0x7f6424075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643cc02640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6424073680 0x7f6424075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f642d7fa640 1 -- 172.21.15.170:0/2512953389 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f643012eb30 con 0x7f6438147960 2024-09-21T02:20:05.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f643cc02640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6424073680 0x7f6424075b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f643400a790 tx=0x7f6434008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.478+0000 7f6446131000 1 Processor -- start 2024-09-21T02:20:05.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- start start 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 0x55f2271ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f2271abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 0x55f2271ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f227080410 con 0x55f227043ca0 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f226fbdb40 con 0x55f22705a950 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55f226facf70 con 0x55f226c6a460 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 0x55f2271ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 0x55f2271ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38266/0 (socket says 172.21.15.170:38266) 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 -- 172.21.15.170:0/1648190874 learned_addr learned my addr 172.21.15.170:0/1648190874 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 -- 172.21.15.170:0/1648190874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 msgr2=0x55f2271ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 0x55f2271ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 -- 172.21.15.170:0/1648190874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 msgr2=0x55f2271abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f2271abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 -- 172.21.15.170:0/1648190874 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f226f745b0 con 0x55f227043ca0 2024-09-21T02:20:05.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f643d403640 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 0x55f2271ac120 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6430098840 tx=0x7f64300f87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125d8a88acee3972 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f641e7fc640 1 -- 172.21.15.170:0/1648190874 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300f8da0 con 0x55f227043ca0 2024-09-21T02:20:05.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f641e7fc640 1 -- 172.21.15.170:0/1648190874 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64300fd880 con 0x55f227043ca0 2024-09-21T02:20:05.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f641e7fc640 1 -- 172.21.15.170:0/1648190874 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643011e3c0 con 0x55f227043ca0 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- 172.21.15.170:0/1648190874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 msgr2=0x55f2271ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 0x55f2271ac120 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6430098840 tx=0x7f64300f87b0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- 172.21.15.170:0/1648190874 shutdown_connections 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f227043ca0 0x55f2271ac120 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f2271abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 --2- 172.21.15.170:0/1648190874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 0x55f2271ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- 172.21.15.170:0/1648190874 >> 172.21.15.170:0/1648190874 conn(0x55f227130d90 msgr2=0x55f227160690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- 172.21.15.170:0/1648190874 shutdown_connections 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- 172.21.15.170:0/1648190874 wait complete. 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 -- start start 2024-09-21T02:20:05.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.482+0000 7f6446131000 1 Processor -- start 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 0x55f2271b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2271b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 0x55f2271b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f226f475f0 con 0x55f22705a950 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f226fb0d80 con 0x55f227043ca0 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55f226f53520 con 0x55f226c6a460 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 10 client.0 _ll_register_callbacks cb 0x55f226f57830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 0x55f2271b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 0x55f2271b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38282/0 (socket says 172.21.15.170:38282) 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 -- 172.21.15.170:0/1882210944 learned_addr learned my addr 172.21.15.170:0/1882210944 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 -- 172.21.15.170:0/1882210944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 msgr2=0x55f2271b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 0x55f2271b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 -- 172.21.15.170:0/1882210944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 msgr2=0x55f2271b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2271b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 -- 172.21.15.170:0/1882210944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f226f4b190 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643d403640 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 0x55f2271b33a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f64300b0040 tx=0x7f643011e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80485718197cc411 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 1 -- 172.21.15.170:0/1882210944 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300f4c40 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 1 -- 172.21.15.170:0/1882210944 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f64301024b0 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f226f45110 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 1 -- 172.21.15.170:0/1882210944 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f6430102730 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 10 client.7761 fetch_fsmap learned FSMap version 81 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55f227031f10 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 1 -- 172.21.15.170:0/1882210944 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f226f45110 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 1 -- 172.21.15.170:0/1882210944 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f64300b5070 con 0x55f22705a950 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 10 client.7761 fetch_fsmap finished waiting for FSMap version 81 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 10 client.7761 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:20:05.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 20 client.7761 populate_metadata read hostname 'smithi170' 2024-09-21T02:20:05.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c4f7d0 0x55f2271bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 4 client.7761 mds_command: new command op to 7614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:20:05.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 --> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f226fbfa90 con 0x55f226c4f7d0 2024-09-21T02:20:05.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643cc02640 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c4f7d0 0x55f2271bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 10 client.7761 ms_handle_connect on v2:172.21.15.170:6834/300226457 2024-09-21T02:20:05.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f643cc02640 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c4f7d0 0x55f2271bcd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781434118c7dc4f9 server_cookie=2c74f2f332214424 in_seq=0 out_seq=0 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 1 -- 172.21.15.170:0/1882210944 <== mds.0 v2:172.21.15.170:6834/300226457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f226fbfa90 con 0x55f226c4f7d0 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 10 client.7761 handle_command_reply: tid=0 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 client.7761 shutdown 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 msgr2=0x55f2271b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 0x55f2271b33a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f64300b0040 tx=0x7f643011e470 comp rx=0 tx=0).stop 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 shutdown_connections 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c4f7d0 0x55f2271bcd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f22705a950 0x55f2271b33a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2271b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 --2- 172.21.15.170:0/1882210944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f226c6a460 0x55f2271b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 >> 172.21.15.170:0/1882210944 conn(0x55f227130d90 msgr2=0x55f227167440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.486+0000 7f64177fe640 0 client.7761 ms_handle_reset on v2:172.21.15.170:6834/300226457 2024-09-21T02:20:05.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.490+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 shutdown_connections 2024-09-21T02:20:05.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.490+0000 7f6446131000 1 -- 172.21.15.170:0/1882210944 wait complete. 2024-09-21T02:20:05.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.490+0000 7f6446131000 20 client.7761 trim_cache size 0 max 16384 2024-09-21T02:20:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 Processor -- start 2024-09-21T02:20:05.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 -- start start 2024-09-21T02:20:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f22721d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f227043ca0 0x55f22721db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f22721e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64382050a0 con 0x55f226c6a460 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f643809ced0 con 0x55f22705a950 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.506+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6438073910 con 0x55f227043ca0 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f22721d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f22721d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38292/0 (socket says 172.21.15.170:38292) 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 -- 172.21.15.170:0/529366983 learned_addr learned my addr 172.21.15.170:0/529366983 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643d403640 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f22721e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f642ffff640 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f227043ca0 0x55f22721db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 -- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f227043ca0 msgr2=0x55f22721db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f227043ca0 0x55f22721db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 -- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 msgr2=0x55f22721e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f22721e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 -- 172.21.15.170:0/529366983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f227008db0 con 0x55f226c6a460 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f22721d620 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f643400c8d0 tx=0x7f64340046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f0a0566d9cf37d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f641dffb640 1 -- 172.21.15.170:0/529366983 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434014b10 con 0x55f226c6a460 2024-09-21T02:20:05.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f641dffb640 1 -- 172.21.15.170:0/529366983 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f643402a070 con 0x55f226c6a460 2024-09-21T02:20:05.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f641dffb640 1 -- 172.21.15.170:0/529366983 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643401f420 con 0x55f226c6a460 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- 172.21.15.170:0/529366983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 msgr2=0x55f22721d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f22721d620 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f643400c8d0 tx=0x7f64340046d0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- 172.21.15.170:0/529366983 shutdown_connections 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f227043ca0 0x55f22721db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f22705a950 0x55f22721e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- 172.21.15.170:0/529366983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f22721d620 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- 172.21.15.170:0/529366983 >> 172.21.15.170:0/529366983 conn(0x55f227130d90 msgr2=0x55f2271a9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- 172.21.15.170:0/529366983 shutdown_connections 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- 172.21.15.170:0/529366983 wait complete. 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- start start 2024-09-21T02:20:05.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 Processor -- start 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f227221390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2272218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f22705a950 0x55f227221e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f226f53520 con 0x55f226c6a460 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f643809c880 con 0x55f227043ca0 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55f226f475f0 con 0x55f22705a950 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f6446131000 10 client.0 _ll_register_callbacks cb 0x7f64300099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f227221390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f227221390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38300/0 (socket says 172.21.15.170:38300) 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.510+0000 7f643cc02640 1 -- 172.21.15.170:0/3183240164 learned_addr learned my addr 172.21.15.170:0/3183240164 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f642ffff640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2272218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643d403640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f22705a950 0x55f227221e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643cc02640 1 -- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f22705a950 msgr2=0x55f227221e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643cc02640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f22705a950 0x55f227221e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643cc02640 1 -- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 msgr2=0x55f2272218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643cc02640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2272218d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643cc02640 1 -- 172.21.15.170:0/3183240164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f226f73ab0 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643cc02640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f227221390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f643400c8a0 tx=0x7f6434014b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2d2e1da1ac2416 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 1 -- 172.21.15.170:0/3183240164 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643401f620 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 1 -- 172.21.15.170:0/3183240164 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f643401f7c0 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f643402a070 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 1 -- 172.21.15.170:0/3183240164 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f643401f9c0 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 10 client.7767 fetch_fsmap learned FSMap version 81 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55f226f52070 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 1 -- 172.21.15.170:0/3183240164 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f643402a070 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 1 -- 172.21.15.170:0/3183240164 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f643401fb60 con 0x55f226c6a460 2024-09-21T02:20:05.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 10 client.7767 fetch_fsmap finished waiting for FSMap version 81 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 10 client.7767 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 20 client.7767 populate_metadata read hostname 'smithi170' 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c3d830 0x55f2271dd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 4 client.7767 mds_command: new command op to 7614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 --> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] -- command(tid 0: {"prefix": "status"}) -- 0x55f226fbb540 con 0x55f226c3d830 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f642ffff640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c3d830 0x55f2271dd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 10 client.7767 ms_handle_connect on v2:172.21.15.170:6834/300226457 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f642ffff640 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c3d830 0x55f2271dd7f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983fff0bc205452f server_cookie=d5733174a59e549c in_seq=0 out_seq=0 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 1 -- 172.21.15.170:0/3183240164 <== mds.0 v2:172.21.15.170:6834/300226457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f226fbb540 con 0x55f226c3d830 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 10 client.7767 handle_command_reply: tid=0 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 client.7767 shutdown 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 msgr2=0x55f227221390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f227221390 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f643400c8a0 tx=0x7f6434014b40 comp rx=0 tx=0).stop 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 shutdown_connections 2024-09-21T02:20:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:6834/300226457,v1:172.21.15.170:6835/300226457] conn(0x55f226c3d830 0x55f2271dd7f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55f22705a950 0x55f227221e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f227043ca0 0x55f2272218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 --2- 172.21.15.170:0/3183240164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f226c6a460 0x55f227221390 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f64177fe640 0 client.7767 ms_handle_reset on v2:172.21.15.170:6834/300226457 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 >> 172.21.15.170:0/3183240164 conn(0x55f227130d90 msgr2=0x55f2271bc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 shutdown_connections 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 1 -- 172.21.15.170:0/3183240164 wait complete. 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f6446131000 20 client.7767 trim_cache size 0 max 16384 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:20:05.518 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7614, 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.644065632, 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 81, 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 118, 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 118, 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 15.176666641000001 2024-09-21T02:20:05.519 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6424073680 msgr2=0x7f6424075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6424073680 0x7f6424075b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f643400a790 tx=0x7f6434008040 comp rx=0 tx=0).stop 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 msgr2=0x7f64381fc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 0x7f64381fc760 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f643001b430 tx=0x7f6430003f80 comp rx=0 tx=0).stop 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 shutdown_connections 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6424073680 0x7f6424075b90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6438147960 0x7f64381fc760 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64380b5410 0x7f64381fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 --2- 172.21.15.170:0/2512953389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64380b2f40 0x7f643809d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 >> 172.21.15.170:0/2512953389 conn(0x7f64380b1c40 msgr2=0x7f64380cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 shutdown_connections 2024-09-21T02:20:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.514+0000 7f643ee66640 1 -- 172.21.15.170:0/2512953389 wait complete. 2024-09-21T02:20:05.533 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 -- 172.21.15.170:0/3079296031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cf7e0 msgr2=0x7f6f0c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3079296031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cf7e0 0x7f6f0c070340 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c070cc0 tx=0x7f6efc01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 -- 172.21.15.170:0/3079296031 shutdown_connections 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3079296031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3079296031 >> v2:172.21.15.191:3300/0 conn(0x7f6f0c0cb460 0x7f6f0c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3079296031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cf7e0 0x7f6f0c070340 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 -- 172.21.15.170:0/3079296031 >> 172.21.15.170:0/3079296031 conn(0x7f6f0c0b1c40 msgr2=0x7f6f0c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 -- 172.21.15.170:0/3079296031 shutdown_connections 2024-09-21T02:20:05.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 -- 172.21.15.170:0/3079296031 wait complete. 2024-09-21T02:20:05.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 Processor -- start 2024-09-21T02:20:05.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.670+0000 7f6f1114b640 1 -- start start 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f0c0cb460 0x7f6f0c148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f0c0cf7e0 0x7f6f0c148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0ced60 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0cebe0 con 0x7f6f0c0cf7e0 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0ceee0 con 0x7f6f0c0cb460 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38316/0 (socket says 172.21.15.170:38316) 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 -- 172.21.15.170:0/3867632893 learned_addr learned my addr 172.21.15.170:0/3867632893 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0ad9d640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f0c0cb460 0x7f6f0c148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0b59e640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f0c0cf7e0 0x7f6f0c148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 -- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f0c0cb460 msgr2=0x7f6f0c148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f0c0cb460 0x7f6f0c148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 -- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f0c0cf7e0 msgr2=0x7f6f0c148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f0c0cf7e0 0x7f6f0c148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 -- 172.21.15.170:0/3867632893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c0c5760 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f0a59c640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0c98e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6f000078b0 tx=0x7f6f00007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00019020 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f00004020 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004410 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c146510 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c148070 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.674+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c0c54d0 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.678+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f6f0001c070 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.678+0000 7f6ef37fe640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6ef80736b0 0x7f6ef8075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.678+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f6f00096040 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.678+0000 7f6f0ad9d640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6ef80736b0 0x7f6ef8075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.678+0000 7f6f0ad9d640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6ef80736b0 0x7f6ef8075bc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0af000 tx=0x7f6efc01b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.682+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6f000608a0 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.886+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6f0c0cccd0 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.886+0000 7f6ef37fe640 1 -- 172.21.15.170:0/3867632893 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f6f000601a0 con 0x7f6f0c0cc8c0 2024-09-21T02:20:05.887 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6ef80736b0 msgr2=0x7f6ef8075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6ef80736b0 0x7f6ef8075bc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0af000 tx=0x7f6efc01b100 comp rx=0 tx=0).stop 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 msgr2=0x7f6f0c0c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0c98e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6f000078b0 tx=0x7f6f00007d70 comp rx=0 tx=0).stop 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 shutdown_connections 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6ef80736b0 0x7f6ef8075bc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f0c0cf7e0 0x7f6f0c148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f0c0cc8c0 0x7f6f0c0c98e0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 --2- 172.21.15.170:0/3867632893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f0c0cb460 0x7f6f0c148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 >> 172.21.15.170:0/3867632893 conn(0x7f6f0c0b1c40 msgr2=0x7f6f0c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:05.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 shutdown_connections 2024-09-21T02:20:05.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:05.890+0000 7f6f1114b640 1 -- 172.21.15.170:0/3867632893 wait complete. 2024-09-21T02:20:05.908 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:20:05.908 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:20:05.908 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:20:05.908 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:20:05.908 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:05.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:20:05.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:20:05.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:20:05.909 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:20:05.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:05.909 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.919 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42109sec preferred_lft 42109sec 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:20:05.920 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:05.921 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:20:05.922 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:05.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:05.922 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:20:05.981 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:20:05.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:05.983 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:06.035 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:06.036 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:20:06.036 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:20:06.037 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:20:06.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:20:06.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:20:06.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:20:06.037 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:20:06.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:20:06.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:06.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:06.161 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:20:06.166 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:20:06.166 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:20:06.166 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:20:06.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:06.166 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:20:06.218 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:20:06.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:06.218 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:06.264 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:20:06.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:06.264 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:06.316 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:20:06.373 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:06.421 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:20:06.421 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:20:06.421 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:20:06.422 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:06.423 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:06.424 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:06.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:06.424 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:20:06.472 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:20:06.472 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:20:06.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:20:06.531 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:06.576 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:20:06.576 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:20:06.577 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:06.578 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:06.579 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:06.579 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:06.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:06.581 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:20:06.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:20:06.626+0000 7f7f98f8e580 -1 init, newargv = 0x7f7f840ad020 newargc=15 2024-09-21T02:20:06.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[47204]: starting ceph client 2024-09-21T02:20:06.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[47204]: starting fuse 2024-09-21T02:20:07.729 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:20:07.745 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:07.793 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:20:07.793 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:20:07.793 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:20:07.794 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:07.795 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:07.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:07.796 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:20:07.844 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:20:07.844 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:20:07.844 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:20:07.844 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:20:07.844 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:20:07.844 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:20:07.845 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:20:07.846 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:20:07.846 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:20:07.846 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:20:07.846 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:07.846 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:20:07.846 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:20:07.967 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.47204.asok 2024-09-21T02:20:07.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:07.979 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47204.asok 2024-09-21T02:20:07.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:07.979 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47204.asok status 2024-09-21T02:20:08.164 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "47204", 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7782, 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:20:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 7782 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3410403216 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:08.166 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3410403216 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.7782 192.168.0.1:0/3410403216", 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/3410403216", 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 81, 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 118, 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:20:08.167 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:20:08.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.181 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:20:08.181 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:08.187 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:20:08.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:08.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.188 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:08.245 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:20:08.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f585a2e4640 1 -- 172.21.15.170:0/397611740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5840004d50 msgr2=0x7f58400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:08.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f585a2e4640 1 --2- 172.21.15.170:0/397611740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5840004d50 0x7f58400051c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f58540ccaa0 tx=0x7f584c01d2c0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f5858881640 1 -- 172.21.15.170:0/397611740 reap_dead start 2024-09-21T02:20:08.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f585a2e4640 1 -- 172.21.15.170:0/397611740 shutdown_connections 2024-09-21T02:20:08.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f585a2e4640 1 -- 172.21.15.170:0/397611740 >> 172.21.15.170:0/397611740 conn(0x7f58540b1c40 msgr2=0x7f58540b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:08.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f585a2e4640 1 -- 172.21.15.170:0/397611740 shutdown_connections 2024-09-21T02:20:08.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.446+0000 7f585a2e4640 1 -- 172.21.15.170:0/397611740 wait complete. 2024-09-21T02:20:08.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 Processor -- start 2024-09-21T02:20:08.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- start start 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 0x7f585414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58540c9570 0x7f58541f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58541f9700 0x7f585409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58540c39c0 con 0x7f5854074340 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58540cf970 con 0x7f58541f9700 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f585406fa30 con 0x7f58540c9570 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 0x7f585414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 0x7f585414e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38322/0 (socket says 172.21.15.170:38322) 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 -- 172.21.15.170:0/859030264 learned_addr learned my addr 172.21.15.170:0/859030264 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5858881640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58541f9700 0x7f585409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:08.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f584b7fe640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58540c9570 0x7f58541f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 -- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58540c9570 msgr2=0x7f58541f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58540c9570 0x7f58541f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 -- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58541f9700 msgr2=0x7f585409a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58541f9700 0x7f585409a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 -- 172.21.15.170:0/859030264 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58541f9b10 con 0x7f5854074340 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f584b7fe640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58540c9570 0x7f58541f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5858881640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58541f9700 0x7f585409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f5853fff640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 0x7f585414e4d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5844003f60 tx=0x7f5844003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844005220 con 0x7f5854074340 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58440053c0 con 0x7f5854074340 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5854074750 con 0x7f5854074340 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440047b0 con 0x7f5854074340 2024-09-21T02:20:08.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5854073580 con 0x7f5854074340 2024-09-21T02:20:08.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.450+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585406ff30 con 0x7f5854074340 2024-09-21T02:20:08.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.454+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f5844004970 con 0x7f5854074340 2024-09-21T02:20:08.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.454+0000 7f58517fa640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f58340736b0 0x7f5834075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:08.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.454+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f5844095c60 con 0x7f5854074340 2024-09-21T02:20:08.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.454+0000 7f584b7fe640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f58340736b0 0x7f5834075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:08.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.454+0000 7f584b7fe640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f58340736b0 0x7f5834075bc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f583c005fd0 tx=0x7f583c00e4c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:08.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.458+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5844028020 con 0x7f5854074340 2024-09-21T02:20:08.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.666+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58540b46d0 con 0x7f5854074340 2024-09-21T02:20:08.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f58517fa640 1 -- 172.21.15.170:0/859030264 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f58541f9b10 con 0x7f5854074340 2024-09-21T02:20:08.668 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f58340736b0 msgr2=0x7f5834075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f58340736b0 0x7f5834075bc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f583c005fd0 tx=0x7f583c00e4c0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 msgr2=0x7f585414e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 0x7f585414e4d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5844003f60 tx=0x7f5844003f90 comp rx=0 tx=0).stop 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 shutdown_connections 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f58340736b0 0x7f5834075bc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58540c9570 0x7f58541f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58541f9700 0x7f585409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 --2- 172.21.15.170:0/859030264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5854074340 0x7f585414e4d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 >> 172.21.15.170:0/859030264 conn(0x7f58540b1c40 msgr2=0x7f58540c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:08.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 shutdown_connections 2024-09-21T02:20:08.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:08.670+0000 7f585a2e4640 1 -- 172.21.15.170:0/859030264 wait complete. 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:20:08.690 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:20:08.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.690 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:20:08.697 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:20:08.697 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:20:08.697 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:20:08.698 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42100sec preferred_lft 42100sec 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.699 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:08.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.700 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:20:08.750 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:20:08.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.752 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:08.804 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:08.805 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:20:08.805 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:20:08.805 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:20:08.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:20:08.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:20:08.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:20:08.805 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:20:08.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:20:08.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:08.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.928 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:20:08.933 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:20:08.933 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:20:08.933 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:20:08.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:08.934 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:20:08.987 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:20:08.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:08.987 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:09.035 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:20:09.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:09.035 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:09.086 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:20:09.144 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:20:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:20:09.193 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:09.194 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:09.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:09.195 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:20:09.242 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:20:09.243 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:20:09.287 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:20:09.300 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:20:09.347 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:20:09.348 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:09.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:09.350 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:20:09.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:20:09.392+0000 7f5765891580 -1 init, newargv = 0x7f57540ad020 newargc=15 2024-09-21T02:20:09.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35375]: starting ceph client 2024-09-21T02:20:09.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35375]: starting fuse 2024-09-21T02:20:10.494 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:20:10.511 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:10.559 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:20:10.560 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:20:10.561 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:20:10.562 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:20:10.562 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:10.562 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:10.562 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:10.562 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:10.562 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:10.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:10.563 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:20:10.611 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:20:10.611 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:20:10.611 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:20:10.611 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:20:10.611 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:20:10.611 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:20:10.611 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:20:10.612 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:20:10.613 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:20:10.613 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:20:10.613 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:10.613 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:20:10.613 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:20:10.733 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.35375.asok 2024-09-21T02:20:10.737 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:10.746 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35375.asok 2024-09-21T02:20:10.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:10.746 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35375.asok status 2024-09-21T02:20:10.927 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:20:10.927 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "35375", 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6475, 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:20:10.928 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6475 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 889031655 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 889031655 2024-09-21T02:20:10.929 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6475 192.168.0.1:0/889031655", 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/889031655", 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 81, 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 118, 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:20:10.930 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:20:10.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:10.944 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:20:10.944 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:10.950 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:20:10.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:10.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:10.951 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:11.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:20:11.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 -- 172.21.15.170:0/202131392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40b7f20 msgr2=0x7f05f415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:11.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 --2- 172.21.15.170:0/202131392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40b7f20 0x7f05f415c770 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009900 tx=0x7f05e401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 -- 172.21.15.170:0/202131392 shutdown_connections 2024-09-21T02:20:11.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 --2- 172.21.15.170:0/202131392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05f40b4460 0x7f05f40b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 --2- 172.21.15.170:0/202131392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40b7f20 0x7f05f415c770 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 -- 172.21.15.170:0/202131392 >> 172.21.15.170:0/202131392 conn(0x7f05f40ced50 msgr2=0x7f05f40cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:11.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 -- 172.21.15.170:0/202131392 shutdown_connections 2024-09-21T02:20:11.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 -- 172.21.15.170:0/202131392 wait complete. 2024-09-21T02:20:11.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 Processor -- start 2024-09-21T02:20:11.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.182+0000 7f05fafa0640 1 -- start start 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05f40b7f20 0x7f05f406f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 0x7f05f41fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05f415fb60 0x7f05f41496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4169eb0 con 0x7f05f40d0de0 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4169d30 con 0x7f05f40b7f20 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f05f416a030 con 0x7f05f415fb60 2024-09-21T02:20:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 0x7f05f41fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 0x7f05f41fcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38336/0 (socket says 172.21.15.170:38336) 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 -- 172.21.15.170:0/1765844119 learned_addr learned my addr 172.21.15.170:0/1765844119 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f953d640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05f415fb60 0x7f05f41496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f8d3c640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05f40b7f20 0x7f05f406f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 -- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05f415fb60 msgr2=0x7f05f41496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05f415fb60 0x7f05f41496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 -- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05f40b7f20 msgr2=0x7f05f406f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05f40b7f20 0x7f05f406f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 -- 172.21.15.170:0/1765844119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f4149bf0 con 0x7f05f40d0de0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f3fff640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 0x7f05f41fcac0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b3e0 tx=0x7f05e800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8015020 con 0x7f05f40d0de0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e8011940 con 0x7f05f40d0de0 2024-09-21T02:20:11.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8011d90 con 0x7f05f40d0de0 2024-09-21T02:20:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f4166fd0 con 0x7f05f40d0de0 2024-09-21T02:20:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f406dfb0 con 0x7f05f40d0de0 2024-09-21T02:20:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f40b5120 con 0x7f05f40d0de0 2024-09-21T02:20:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.186+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f05e8018020 con 0x7f05f40d0de0 2024-09-21T02:20:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.190+0000 7f05f17fa640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05e0073680 0x7f05e0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.190+0000 7f05f8d3c640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05e0073680 0x7f05e0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.190+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f05e8094f50 con 0x7f05f40d0de0 2024-09-21T02:20:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.190+0000 7f05f8d3c640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05e0073680 0x7f05e0075b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f05e40af000 tx=0x7f05e401b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:11.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.194+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f05e801f020 con 0x7f05f40d0de0 2024-09-21T02:20:11.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.398+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f05f4156020 con 0x7f05f40d0de0 2024-09-21T02:20:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.986+0000 7f05f17fa640 1 -- 172.21.15.170:0/1765844119 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v82) ==== 136+0+0 (secure 0 0 0) 0x7f05e805f930 con 0x7f05f40d0de0 2024-09-21T02:20:11.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05e0073680 msgr2=0x7f05e0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:11.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05e0073680 0x7f05e0075b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f05e40af000 tx=0x7f05e401b120 comp rx=0 tx=0).stop 2024-09-21T02:20:11.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 msgr2=0x7f05f41fcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 0x7f05f41fcac0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b3e0 tx=0x7f05e800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 shutdown_connections 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05e0073680 0x7f05e0075b90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05f415fb60 0x7f05f41496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f40d0de0 0x7f05f41fcac0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 --2- 172.21.15.170:0/1765844119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05f40b7f20 0x7f05f406f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 >> 172.21.15.170:0/1765844119 conn(0x7f05f40ced50 msgr2=0x7f05f4155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 shutdown_connections 2024-09-21T02:20:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:11.990+0000 7f05fafa0640 1 -- 172.21.15.170:0/1765844119 wait complete. 2024-09-21T02:20:12.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 -- 172.21.15.170:0/1158839382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00cf280 msgr2=0x7f39d00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 --2- 172.21.15.170:0/1158839382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00cf280 0x7f39d00cf690 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009900 tx=0x7f39c001c820 comp rx=0 tx=0).stop 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 -- 172.21.15.170:0/1158839382 shutdown_connections 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 --2- 172.21.15.170:0/1158839382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00cf280 0x7f39d00cf690 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 --2- 172.21.15.170:0/1158839382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39d00d1490 0x7f39d00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 -- 172.21.15.170:0/1158839382 >> 172.21.15.170:0/1158839382 conn(0x7f39d009d830 msgr2=0x7f39d008bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 -- 172.21.15.170:0/1158839382 shutdown_connections 2024-09-21T02:20:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.174+0000 7f39d83a9640 1 -- 172.21.15.170:0/1158839382 wait complete. 2024-09-21T02:20:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 Processor -- start 2024-09-21T02:20:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 -- start start 2024-09-21T02:20:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39d00cf280 0x7f39d0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 0x7f39d01606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39d00d3630 0x7f39d01f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39d016bef0 con 0x7f39d00d1490 2024-09-21T02:20:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f39d016bd70 con 0x7f39d00cf280 2024-09-21T02:20:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f39d016c070 con 0x7f39d00d3630 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 0x7f39d01606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 0x7f39d01606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38352/0 (socket says 172.21.15.170:38352) 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 -- 172.21.15.170:0/2334381273 learned_addr learned my addr 172.21.15.170:0/2334381273 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d6145640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39d00cf280 0x7f39d0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d6946640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39d00d3630 0x7f39d01f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 -- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39d00d3630 msgr2=0x7f39d01f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39d00d3630 0x7f39d01f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 -- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39d00cf280 msgr2=0x7f39d0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39d00cf280 0x7f39d0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 -- 172.21.15.170:0/2334381273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d00c8e10 con 0x7f39d00d1490 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d6145640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39d00cf280 0x7f39d0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d5944640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 0x7f39d01606d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f39d009de10 tx=0x7f39c400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4007b80 con 0x7f39d00d1490 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c4007e10 con 0x7f39d00d1490 2024-09-21T02:20:12.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d00c7fc0 con 0x7f39d00d1490 2024-09-21T02:20:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4005280 con 0x7f39d00d1490 2024-09-21T02:20:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.178+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d0159b40 con 0x7f39d00d1490 2024-09-21T02:20:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.182+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d00c8b80 con 0x7f39d00d1490 2024-09-21T02:20:12.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.182+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f39c4005420 con 0x7f39d00d1490 2024-09-21T02:20:12.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.182+0000 7f39ceffd640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f39bc073680 0x7f39bc075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.182+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f39c4094bc0 con 0x7f39d00d1490 2024-09-21T02:20:12.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.182+0000 7f39d6145640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f39bc073680 0x7f39bc075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.182+0000 7f39d6145640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f39bc073680 0x7f39bc075b90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009fc0 tx=0x7f39c8009550 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:12.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.186+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f39c4014030 con 0x7f39d00d1490 2024-09-21T02:20:12.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.382+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f39d00cf690 con 0x7f39bc073680 2024-09-21T02:20:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39ceffd640 1 -- 172.21.15.170:0/2334381273 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f39d00cf690 con 0x7f39bc073680 2024-09-21T02:20:12.385 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:20:12.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:12.385 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:20:12.385 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:12.385 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f39bc073680 msgr2=0x7f39bc075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f39bc073680 0x7f39bc075b90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009fc0 tx=0x7f39c8009550 comp rx=0 tx=0).stop 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 msgr2=0x7f39d01606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 0x7f39d01606d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f39d009de10 tx=0x7f39c400cdd0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 shutdown_connections 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f39bc073680 0x7f39bc075b90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39d00d3630 0x7f39d01f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39d00d1490 0x7f39d01606d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.386+0000 7f39d83a9640 1 --2- 172.21.15.170:0/2334381273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39d00cf280 0x7f39d0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.390+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 >> 172.21.15.170:0/2334381273 conn(0x7f39d009d830 msgr2=0x7f39d00d3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.390+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 shutdown_connections 2024-09-21T02:20:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.390+0000 7f39d83a9640 1 -- 172.21.15.170:0/2334381273 wait complete. 2024-09-21T02:20:12.408 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:20:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 -- 172.21.15.170:0/497488735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 msgr2=0x7f1dd80b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 --2- 172.21.15.170:0/497488735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd80b7720 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009920 tx=0x7f1dc801c830 comp rx=0 tx=0).stop 2024-09-21T02:20:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 -- 172.21.15.170:0/497488735 shutdown_connections 2024-09-21T02:20:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 --2- 172.21.15.170:0/497488735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd80d0c40 0x7f1dd80b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 --2- 172.21.15.170:0/497488735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd80b7720 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 -- 172.21.15.170:0/497488735 >> 172.21.15.170:0/497488735 conn(0x7f1dd80cee00 msgr2=0x7f1dd80cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 -- 172.21.15.170:0/497488735 shutdown_connections 2024-09-21T02:20:12.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.566+0000 7f1ddd209640 1 -- 172.21.15.170:0/497488735 wait complete. 2024-09-21T02:20:12.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 Processor -- start 2024-09-21T02:20:12.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- start start 2024-09-21T02:20:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd81f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd80d0830 0x7f1dd809f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd80d0c40 0x7f1dd815b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80b27f0 con 0x7f1dd80cce90 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80b3750 con 0x7f1dd80d0830 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80b2970 con 0x7f1dd80d0c40 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd81f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd81f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38376/0 (socket says 172.21.15.170:38376) 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 -- 172.21.15.170:0/2405284540 learned_addr learned my addr 172.21.15.170:0/2405284540 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd759e640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd80d0c40 0x7f1dd815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd659c640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd80d0830 0x7f1dd809f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 -- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd80d0c40 msgr2=0x7f1dd815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd80d0c40 0x7f1dd815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 -- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd80d0830 msgr2=0x7f1dd809f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd80d0830 0x7f1dd809f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 -- 172.21.15.170:0/2405284540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd81fa4c0 con 0x7f1dd80cce90 2024-09-21T02:20:12.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd659c640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd80d0830 0x7f1dd809f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dd6d9d640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd81f9ae0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1dc801cd10 tx=0x7f1dc80a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8004040 con 0x7f1dd80cce90 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dc80042d0 con 0x7f1dd80cce90 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8003860 con 0x7f1dd80cce90 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd81fa7a0 con 0x7f1dd80cce90 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd8159a80 con 0x7f1dd80cce90 2024-09-21T02:20:12.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.570+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd81fa9e0 con 0x7f1dd80cce90 2024-09-21T02:20:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.574+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f1dc80c1020 con 0x7f1dd80cce90 2024-09-21T02:20:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.574+0000 7f1dbf7fe640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1dc4073680 0x7f1dc4075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.574+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f1dc812f6f0 con 0x7f1dd80cce90 2024-09-21T02:20:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.574+0000 7f1dd659c640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1dc4073680 0x7f1dc4075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:12.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.574+0000 7f1dd659c640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1dc4073680 0x7f1dc4075b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1dd81fabe0 tx=0x7f1dcc009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:12.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.578+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1dd81fa9e0 con 0x7f1dd80cce90 2024-09-21T02:20:12.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.750+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1dd80b7df0 con 0x7f1dd80cce90 2024-09-21T02:20:12.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.750+0000 7f1dbf7fe640 1 -- 172.21.15.170:0/2405284540 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f1dc80c8020 con 0x7f1dd80cce90 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1dc4073680 msgr2=0x7f1dc4075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1dc4073680 0x7f1dc4075b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1dd81fabe0 tx=0x7f1dcc009040 comp rx=0 tx=0).stop 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 msgr2=0x7f1dd81f9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd81f9ae0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1dc801cd10 tx=0x7f1dc80a6660 comp rx=0 tx=0).stop 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 shutdown_connections 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1dc4073680 0x7f1dc4075b90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dd80d0c40 0x7f1dd815b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dd80d0830 0x7f1dd809f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 --2- 172.21.15.170:0/2405284540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd80cce90 0x7f1dd81f9ae0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:12.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 >> 172.21.15.170:0/2405284540 conn(0x7f1dd80cee00 msgr2=0x7f1dd80ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:12.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 shutdown_connections 2024-09-21T02:20:12.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:12.754+0000 7f1ddd209640 1 -- 172.21.15.170:0/2405284540 wait complete. 2024-09-21T02:20:12.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:12.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol16080/c7d5fc67-0b7f-4b11-bdd7-745c669d4a2c) 2024-09-21T02:20:12.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:12.796 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol16080/c7d5fc67-0b7f-4b11-bdd7-745c669d4a2c/.snap/fake) 2024-09-21T02:20:12.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:12.861 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:20:12.916 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:20:12.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:12.916 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol16080/c7d5fc67-0b7f-4b11-bdd7-745c669d4a2c) 2024-09-21T02:20:12.980 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:12.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol16080/.meta 2024-09-21T02:20:13.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp48950/subvol421/ee59d4a6-a055-4564-a152-93a099d84575) 2024-09-21T02:20:13.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.111 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp48950/subvol421/ee59d4a6-a055-4564-a152-93a099d84575/.snap/fake) 2024-09-21T02:20:13.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.181 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:20:13.236 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:20:13.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp48950/subvol421/ee59d4a6-a055-4564-a152-93a099d84575) 2024-09-21T02:20:13.300 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:13.300 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp48950/subvol421/.meta 2024-09-21T02:20:13.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.361 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol25672/790b725b-c9f6-414c-ac7f-b212c1c9f1a5) 2024-09-21T02:20:13.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.424 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:20:13.479 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:20:13.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:13.479 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol25672/790b725b-c9f6-414c-ac7f-b212c1c9f1a5) 2024-09-21T02:20:13.541 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:13.541 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol25672/.meta 2024-09-21T02:20:13.603 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol16080 2024-09-21T02:20:13.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 -- 172.21.15.170:0/3590306415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd0074270 msgr2=0x7f1fd00cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:13.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 --2- 172.21.15.170:0/3590306415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd0074270 0x7f1fd00cb280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009900 tx=0x7f1fb801c880 comp rx=0 tx=0).stop 2024-09-21T02:20:13.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 -- 172.21.15.170:0/3590306415 shutdown_connections 2024-09-21T02:20:13.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 --2- 172.21.15.170:0/3590306415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd0074680 0x7f1fd00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:13.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 --2- 172.21.15.170:0/3590306415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd0074270 0x7f1fd00cb280 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:13.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 -- 172.21.15.170:0/3590306415 >> 172.21.15.170:0/3590306415 conn(0x7f1fd00d0170 msgr2=0x7f1fd00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:13.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 -- 172.21.15.170:0/3590306415 shutdown_connections 2024-09-21T02:20:13.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.818+0000 7f1fd6920640 1 -- 172.21.15.170:0/3590306415 wait complete. 2024-09-21T02:20:13.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 Processor -- start 2024-09-21T02:20:13.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- start start 2024-09-21T02:20:13.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd0074270 0x7f1fd014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:13.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 0x7f1fd009b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00cf7d0 0x7f1fd009b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0169040 con 0x7f1fd0074680 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0168ec0 con 0x7f1fd00cf7d0 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd01691c0 con 0x7f1fd0074270 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 0x7f1fd009b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 0x7f1fd009b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38384/0 (socket says 172.21.15.170:38384) 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 -- 172.21.15.170:0/897016406 learned_addr learned my addr 172.21.15.170:0/897016406 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fcffff640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd0074270 0x7f1fd014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd4ebd640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00cf7d0 0x7f1fd009b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 -- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd0074270 msgr2=0x7f1fd014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd0074270 0x7f1fd014c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 -- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00cf7d0 msgr2=0x7f1fd009b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00cf7d0 0x7f1fd009b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 -- 172.21.15.170:0/897016406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd014ca70 con 0x7f1fd0074680 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd4ebd640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00cf7d0 0x7f1fd009b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fc77fe640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 0x7f1fd009b2f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009850 tx=0x7f1fb8005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80b2040 con 0x7f1fd0074680 2024-09-21T02:20:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fb8004130 con 0x7f1fd0074680 2024-09-21T02:20:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd00c5090 con 0x7f1fd0074680 2024-09-21T02:20:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8005020 con 0x7f1fd0074680 2024-09-21T02:20:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd009aaf0 con 0x7f1fd0074680 2024-09-21T02:20:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.822+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd00c4de0 con 0x7f1fd0074680 2024-09-21T02:20:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.826+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f1fb80bf030 con 0x7f1fd0074680 2024-09-21T02:20:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.826+0000 7f1fcd7fa640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1fb0073680 0x7f1fb0075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.826+0000 7f1fcffff640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1fb0073680 0x7f1fb0075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.826+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f1fb812ec90 con 0x7f1fd0074680 2024-09-21T02:20:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.826+0000 7f1fcffff640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1fb0073680 0x7f1fb0075b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00c3c70 tx=0x7f1fc0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:13.830+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1fb80c6060 con 0x7f1fd0074680 2024-09-21T02:20:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.034+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol16080", "target": ["mon-mgr", ""]}) -- 0x7f1fd0073b60 con 0x7f1fb0073680 2024-09-21T02:20:14.041 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:14.042+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:14.041 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:14.042+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:14.041 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:14.042+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:14.042 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:14.042+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:14.042 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:14.042+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:14.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.070+0000 7f1fcd7fa640 1 -- 172.21.15.170:0/897016406 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f1fd0073b60 con 0x7f1fb0073680 2024-09-21T02:20:14.071 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol16080/c7d5fc67-0b7f-4b11-bdd7-745c669d4a2c 2024-09-21T02:20:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1fb0073680 msgr2=0x7f1fb0075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1fb0073680 0x7f1fb0075b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00c3c70 tx=0x7f1fc0009040 comp rx=0 tx=0).stop 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 msgr2=0x7f1fd009b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 0x7f1fd009b2f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009850 tx=0x7f1fb8005e90 comp rx=0 tx=0).stop 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 shutdown_connections 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1fb0073680 0x7f1fb0075b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00cf7d0 0x7f1fd009b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fd0074680 0x7f1fd009b2f0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 --2- 172.21.15.170:0/897016406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd0074270 0x7f1fd014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 >> 172.21.15.170:0/897016406 conn(0x7f1fd00d0170 msgr2=0x7f1fd015b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 shutdown_connections 2024-09-21T02:20:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.074+0000 7f1fd6920640 1 -- 172.21.15.170:0/897016406 wait complete. 2024-09-21T02:20:14.093 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol421 subvol_grp48950 2024-09-21T02:20:14.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 -- 172.21.15.170:0/2425937116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380cf8f0 msgr2=0x7faf380d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 --2- 172.21.15.170:0/2425937116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380cf8f0 0x7faf380d1f20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7faf380c8460 tx=0x7faf2c01c7a0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 -- 172.21.15.170:0/2425937116 shutdown_connections 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 --2- 172.21.15.170:0/2425937116 >> v2:172.21.15.191:3300/0 conn(0x7faf380ae0d0 0x7faf380ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 --2- 172.21.15.170:0/2425937116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380cf8f0 0x7faf380d1f20 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 -- 172.21.15.170:0/2425937116 >> 172.21.15.170:0/2425937116 conn(0x7faf38074320 msgr2=0x7faf380d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 -- 172.21.15.170:0/2425937116 shutdown_connections 2024-09-21T02:20:14.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 -- 172.21.15.170:0/2425937116 wait complete. 2024-09-21T02:20:14.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 Processor -- start 2024-09-21T02:20:14.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.242+0000 7faf3db13640 1 -- start start 2024-09-21T02:20:14.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf380ae0d0 0x7faf3815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 0x7faf3806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf3814a5b0 0x7faf3806d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf38168b90 con 0x7faf380d1b10 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faf38168a10 con 0x7faf380ae0d0 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7faf38168d10 con 0x7faf3814a5b0 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 0x7faf3806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 0x7faf3806ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38438/0 (socket says 172.21.15.170:38438) 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 -- 172.21.15.170:0/1870098442 learned_addr learned my addr 172.21.15.170:0/1870098442 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf377fe640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf380ae0d0 0x7faf3815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf37fff640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf3814a5b0 0x7faf3806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 -- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf3814a5b0 msgr2=0x7faf3806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf3814a5b0 0x7faf3806d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 -- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf380ae0d0 msgr2=0x7faf3815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf380ae0d0 0x7faf3815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 -- 172.21.15.170:0/1870098442 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3815ae70 con 0x7faf380d1b10 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf36ffd640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 0x7faf3806ce00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faf2c01aeb0 tx=0x7faf2c0a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0b2040 con 0x7faf380d1b10 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf2c0044b0 con 0x7faf380d1b10 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf380cc440 con 0x7faf380d1b10 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c003860 con 0x7faf380d1b10 2024-09-21T02:20:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf380ccc20 con 0x7faf380d1b10 2024-09-21T02:20:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf380ae4e0 con 0x7faf380d1b10 2024-09-21T02:20:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.246+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7faf2c0bf060 con 0x7faf380d1b10 2024-09-21T02:20:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.250+0000 7faf1ffff640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf24073680 0x7faf24075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.250+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7faf2c12e800 con 0x7faf380d1b10 2024-09-21T02:20:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.250+0000 7faf377fe640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf24073680 0x7faf24075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.250+0000 7faf377fe640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf24073680 0x7faf24075b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf28009f10 tx=0x7faf28009550 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:14.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.254+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faf2c0164b0 con 0x7faf380d1b10 2024-09-21T02:20:14.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.454+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol421", "group_name": "subvol_grp48950", "target": ["mon-mgr", ""]}) -- 0x7faf380cebf0 con 0x7faf24073680 2024-09-21T02:20:14.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.498+0000 7faf1ffff640 1 -- 172.21.15.170:0/1870098442 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7faf380cebf0 con 0x7faf24073680 2024-09-21T02:20:14.497 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp48950/subvol421/ee59d4a6-a055-4564-a152-93a099d84575 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.498+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf24073680 msgr2=0x7faf24075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.498+0000 7faf3db13640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf24073680 0x7faf24075b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf28009f10 tx=0x7faf28009550 comp rx=0 tx=0).stop 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.498+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 msgr2=0x7faf3806ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 0x7faf3806ce00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faf2c01aeb0 tx=0x7faf2c0a5780 comp rx=0 tx=0).stop 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 shutdown_connections 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf24073680 0x7faf24075b90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf3814a5b0 0x7faf3806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf380d1b10 0x7faf3806ce00 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 --2- 172.21.15.170:0/1870098442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf380ae0d0 0x7faf3815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 >> 172.21.15.170:0/1870098442 conn(0x7faf38074320 msgr2=0x7faf380b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 shutdown_connections 2024-09-21T02:20:14.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.502+0000 7faf3db13640 1 -- 172.21.15.170:0/1870098442 wait complete. 2024-09-21T02:20:14.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol25672 2024-09-21T02:20:14.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3544290203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40cf7d0 msgr2=0x7f3aa4073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3544290203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4073b60 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40b61b0 tx=0x7f3a9401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3544290203 shutdown_connections 2024-09-21T02:20:14.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3544290203 >> v2:172.21.15.191:3300/0 conn(0x7f3aa40744b0 0x7f3aa40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3544290203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4073b60 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3544290203 >> 172.21.15.170:0/3544290203 conn(0x7f3aa40d0170 msgr2=0x7f3aa40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:14.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3544290203 shutdown_connections 2024-09-21T02:20:14.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3544290203 wait complete. 2024-09-21T02:20:14.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 Processor -- start 2024-09-21T02:20:14.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.682+0000 7f3aaa59f640 1 -- start start 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 0x7f3aa414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3aa40744b0 0x7f3aa4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa40cee40 con 0x7f3aa40740a0 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa40cecc0 con 0x7f3aa40cf7d0 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa4166ca0 con 0x7f3aa40744b0 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 0x7f3aa414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 0x7f3aa414a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38454/0 (socket says 172.21.15.170:38454) 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 -- 172.21.15.170:0/3272118167 learned_addr learned my addr 172.21.15.170:0/3272118167 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa37fe640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3aa40744b0 0x7f3aa4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 -- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3aa40744b0 msgr2=0x7f3aa4153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa8b3c640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3aa40744b0 0x7f3aa4153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 -- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa40cf7d0 msgr2=0x7f3aa4153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 -- 172.21.15.170:0/3272118167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa4168430 con 0x7f3aa40740a0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa8b3c640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa3fff640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 0x7f3aa414a710 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009850 tx=0x7f3a940a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a940b2040 con 0x7f3aa40740a0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a940bd070 con 0x7f3aa40740a0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41688a0 con 0x7f3aa40740a0 2024-09-21T02:20:14.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94003600 con 0x7f3aa40740a0 2024-09-21T02:20:14.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa40c76a0 con 0x7f3aa40740a0 2024-09-21T02:20:14.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa41671e0 con 0x7f3aa40740a0 2024-09-21T02:20:14.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f3a940037a0 con 0x7f3aa40740a0 2024-09-21T02:20:14.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa0ff9640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a90073680 0x7f3a90075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:14.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f3a941300a0 con 0x7f3aa40740a0 2024-09-21T02:20:14.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.686+0000 7f3aa37fe640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a90073680 0x7f3a90075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:14.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.690+0000 7f3aa37fe640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a90073680 0x7f3a90075b90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3aa415beb0 tx=0x7f3a98009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:14.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.690+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3a940164b0 con 0x7f3aa40740a0 2024-09-21T02:20:14.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.902+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol25672", "target": ["mon-mgr", ""]}) -- 0x7f3aa40b4f40 con 0x7f3a90073680 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.938+0000 7f3aa0ff9640 1 -- 172.21.15.170:0/3272118167 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f3aa40b4f40 con 0x7f3a90073680 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: "status": { 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "pending", 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: "source": { 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: "volume": "fake", 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: "subvolume": "fake", 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: "snapshot": "fake" 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:14.939 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:14.940 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a90073680 msgr2=0x7f3a90075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a90073680 0x7f3a90075b90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3aa415beb0 tx=0x7f3a98009040 comp rx=0 tx=0).stop 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 msgr2=0x7f3aa414a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 0x7f3aa414a710 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009850 tx=0x7f3a940a5710 comp rx=0 tx=0).stop 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 shutdown_connections 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3a90073680 0x7f3a90075b90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3aa40744b0 0x7f3aa4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa40cf7d0 0x7f3aa4153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 --2- 172.21.15.170:0/3272118167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa40740a0 0x7f3aa414a710 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 >> 172.21.15.170:0/3272118167 conn(0x7f3aa40d0170 msgr2=0x7f3aa40b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 shutdown_connections 2024-09-21T02:20:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:14.942+0000 7f3aaa59f640 1 -- 172.21.15.170:0/3272118167 wait complete. 2024-09-21T02:20:14.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol16080 fake 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/1643903901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 msgr2=0x7fd7a80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/1643903901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a80703d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fd798009900 tx=0x7fd79801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/1643903901 shutdown_connections 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/1643903901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a80703d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/1643903901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 0x7fd7a80cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/1643903901 >> 172.21.15.170:0/1643903901 conn(0x7fd7a80b1c40 msgr2=0x7fd7a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/1643903901 shutdown_connections 2024-09-21T02:20:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/1643903901 wait complete. 2024-09-21T02:20:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 Processor -- start 2024-09-21T02:20:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.126+0000 7fd7ad9ab640 1 -- start start 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 0x7fd7a8140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a80cc7e0 0x7fd7a814e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8169210 con 0x7fd7a806ffc0 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8169090 con 0x7fd7a80cc7e0 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a8169390 con 0x7fd7a80ca7b0 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a815e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38474/0 (socket says 172.21.15.170:38474) 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 -- 172.21.15.170:0/2612894441 learned_addr learned my addr 172.21.15.170:0/2612894441 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a67fc640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 0x7fd7a8140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a77fe640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a80cc7e0 0x7fd7a814e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 -- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 msgr2=0x7fd7a8140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 0x7fd7a8140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 -- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a80cc7e0 msgr2=0x7fd7a814e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a80cc7e0 0x7fd7a814e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 -- 172.21.15.170:0/2612894441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a81597f0 con 0x7fd7a806ffc0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a67fc640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 0x7fd7a8140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a77fe640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a80cc7e0 0x7fd7a814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7a6ffd640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a815e690 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c970 tx=0x7fd79c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c007bb0 con 0x7fd7a806ffc0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd79c007e40 con 0x7fd7a806ffc0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a815ebd0 con 0x7fd7a806ffc0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c005270 con 0x7fd7a806ffc0 2024-09-21T02:20:15.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a814f4c0 con 0x7fd7a806ffc0 2024-09-21T02:20:15.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a80703d0 con 0x7fd7a806ffc0 2024-09-21T02:20:15.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.130+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7fd79c005410 con 0x7fd7a806ffc0 2024-09-21T02:20:15.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.134+0000 7fd78ffff640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd794073680 0x7fd794075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.134+0000 7fd7a67fc640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd794073680 0x7fd794075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.134+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fd79c094fe0 con 0x7fd7a806ffc0 2024-09-21T02:20:15.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.134+0000 7fd7a67fc640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd794073680 0x7fd794075b90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7980af000 tx=0x7fd79801b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.138+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd79c014070 con 0x7fd7a806ffc0 2024-09-21T02:20:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.326+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol16080", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7fd7a80c6610 con 0x7fd794073680 2024-09-21T02:20:15.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.366+0000 7fd78ffff640 1 -- 172.21.15.170:0/2612894441 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd7a80c6610 con 0x7fd794073680 2024-09-21T02:20:15.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd794073680 msgr2=0x7fd794075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd794073680 0x7fd794075b90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7980af000 tx=0x7fd79801b120 comp rx=0 tx=0).stop 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 msgr2=0x7fd7a815e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a815e690 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c970 tx=0x7fd79c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 shutdown_connections 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd794073680 0x7fd794075b90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7a80ca7b0 0x7fd7a8140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a80cc7e0 0x7fd7a814e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 --2- 172.21.15.170:0/2612894441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a806ffc0 0x7fd7a815e690 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 >> 172.21.15.170:0/2612894441 conn(0x7fd7a80b1c40 msgr2=0x7fd7a815b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 shutdown_connections 2024-09-21T02:20:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.370+0000 7fd7ad9ab640 1 -- 172.21.15.170:0/2612894441 wait complete. 2024-09-21T02:20:15.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol421 fake subvol_grp48950 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 -- 172.21.15.170:0/2440648060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f687c004010 msgr2=0x7f687c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 --2- 172.21.15.170:0/2440648060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f687c004010 0x7f687c004810 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6888009b60 tx=0x7f688801d290 comp rx=0 tx=0).stop 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f6892d9d640 1 -- 172.21.15.170:0/2440648060 reap_dead start 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 -- 172.21.15.170:0/2440648060 shutdown_connections 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 -- 172.21.15.170:0/2440648060 >> 172.21.15.170:0/2440648060 conn(0x7f689409a9c0 msgr2=0x7f68940d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 -- 172.21.15.170:0/2440648060 shutdown_connections 2024-09-21T02:20:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 -- 172.21.15.170:0/2440648060 wait complete. 2024-09-21T02:20:15.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 Processor -- start 2024-09-21T02:20:15.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.550+0000 7f689889b640 1 -- start start 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 0x7f68940afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68941f96c0 0x7f68941f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894166b90 0x7f6894166fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68941683a0 con 0x7f68941fd5c0 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6894168220 con 0x7f6894166b90 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6894168520 con 0x7f68941f96c0 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 0x7f68940afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 0x7f68940afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46060/0 (socket says 172.21.15.170:46060) 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 -- 172.21.15.170:0/2912064608 learned_addr learned my addr 172.21.15.170:0/2912064608 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:15.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f6891d9b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68941f96c0 0x7f68941f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f6892d9d640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894166b90 0x7f6894166fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 -- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68941f96c0 msgr2=0x7f68941f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68941f96c0 0x7f68941f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 -- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894166b90 msgr2=0x7f6894166fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894166b90 0x7f6894166fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 -- 172.21.15.170:0/2912064608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68941674e0 con 0x7f68941fd5c0 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689259c640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 0x7f68940afa20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f688400db20 tx=0x7f68840020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884016020 con 0x7f68941fd5c0 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f688400b990 con 0x7f68941fd5c0 2024-09-21T02:20:15.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400bda0 con 0x7f68941fd5c0 2024-09-21T02:20:15.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68941fa180 con 0x7f68941fd5c0 2024-09-21T02:20:15.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68940af220 con 0x7f68941fd5c0 2024-09-21T02:20:15.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.554+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68940b0b60 con 0x7f68941fd5c0 2024-09-21T02:20:15.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.558+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) ==== 97642+0+0 (secure 0 0 0) 0x7f6884019020 con 0x7f68941fd5c0 2024-09-21T02:20:15.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.558+0000 7f6882ffd640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6874073680 0x7f6874075b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:15.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.558+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f6884094600 con 0x7f68941fd5c0 2024-09-21T02:20:15.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.558+0000 7f6891d9b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6874073680 0x7f6874075b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:15.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.558+0000 7f6891d9b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6874073680 0x7f6874075b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68880ae000 tx=0x7f688801a030 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:15.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.562+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6884020070 con 0x7f68941fd5c0 2024-09-21T02:20:15.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.766+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol421", "snap_name": "fake", "group_name": "subvol_grp48950", "target": ["mon-mgr", ""]}) -- 0x7f68941fa6a0 con 0x7f6874073680 2024-09-21T02:20:15.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.826+0000 7f6882ffd640 1 -- 172.21.15.170:0/2912064608 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68880052b0 con 0x7f6874073680 2024-09-21T02:20:15.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6874073680 msgr2=0x7f6874075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6874073680 0x7f6874075b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f68880ae000 tx=0x7f688801a030 comp rx=0 tx=0).stop 2024-09-21T02:20:15.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 msgr2=0x7f68940afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 0x7f68940afa20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f688400db20 tx=0x7f68840020c0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 shutdown_connections 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6874073680 0x7f6874075b90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68941f96c0 0x7f68941f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894166b90 0x7f6894166fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 --2- 172.21.15.170:0/2912064608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68941fd5c0 0x7f68940afa20 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 >> 172.21.15.170:0/2912064608 conn(0x7f689409a9c0 msgr2=0x7f68940b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 shutdown_connections 2024-09-21T02:20:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:15.830+0000 7f689889b640 1 -- 172.21.15.170:0/2912064608 wait complete. 2024-09-21T02:20:15.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:15.849 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol16080/.meta) 2024-09-21T02:20:15.865 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:20:15.865 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:20:15.865 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:20:15.866 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol16080/c7d5fc67-0b7f-4b11-bdd7-745c669d4a2c 2024-09-21T02:20:15.866 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:20:15.866 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:15.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:15.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp48950/subvol421/.meta) 2024-09-21T02:20:15.926 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:20:15.927 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:20:15.927 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:20:15.927 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/subvol_grp48950/subvol421/ee59d4a6-a055-4564-a152-93a099d84575 2024-09-21T02:20:15.927 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:20:15.927 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:15.928 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol16080 2024-09-21T02:20:16.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c55adf640 1 -- 172.21.15.170:0/4011423885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 msgr2=0x7f6c3c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c55adf640 1 --2- 172.21.15.170:0/4011423885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c3c004810 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6c38006fd0 tx=0x7f6c3801f6a0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c4ffff640 1 -- 172.21.15.170:0/4011423885 reap_dead start 2024-09-21T02:20:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c55adf640 1 -- 172.21.15.170:0/4011423885 shutdown_connections 2024-09-21T02:20:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c55adf640 1 -- 172.21.15.170:0/4011423885 >> 172.21.15.170:0/4011423885 conn(0x7f6c500d0170 msgr2=0x7f6c500d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c55adf640 1 -- 172.21.15.170:0/4011423885 shutdown_connections 2024-09-21T02:20:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.130+0000 7f6c55adf640 1 -- 172.21.15.170:0/4011423885 wait complete. 2024-09-21T02:20:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 Processor -- start 2024-09-21T02:20:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 -- start start 2024-09-21T02:20:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c500b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c500b25b0 0x7f6c501e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c501e9800 0x7f6c500acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500b3550 con 0x7f6c3c0040c0 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500b33d0 con 0x7f6c501e9800 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500cf060 con 0x7f6c500b25b0 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c500b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c500b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46078/0 (socket says 172.21.15.170:46078) 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 -- 172.21.15.170:0/2188313871 learned_addr learned my addr 172.21.15.170:0/2188313871 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4ffff640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c501e9800 0x7f6c500acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 -- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c500b25b0 msgr2=0x7f6c501e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4effd640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c500b25b0 0x7f6c501e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c500b25b0 0x7f6c501e92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 -- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c501e9800 msgr2=0x7f6c500acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c501e9800 0x7f6c500acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 -- 172.21.15.170:0/2188313871 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c501e9c10 con 0x7f6c3c0040c0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c4f7fe640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c500b0440 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6c480018a0 tx=0x7f6c48001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48015070 con 0x7f6c3c0040c0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c48010980 con 0x7f6c3c0040c0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c500affd0 con 0x7f6c3c0040c0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48010db0 con 0x7f6c3c0040c0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.134+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501f6480 con 0x7f6c3c0040c0 2024-09-21T02:20:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.138+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c500ca2f0 con 0x7f6c3c0040c0 2024-09-21T02:20:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.138+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 97693+0+0 (secure 0 0 0) 0x7f6c48018070 con 0x7f6c3c0040c0 2024-09-21T02:20:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.138+0000 7f6c47fff640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6c30073740 0x7f6c30075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.138+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f6c48095300 con 0x7f6c3c0040c0 2024-09-21T02:20:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.138+0000 7f6c4effd640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6c30073740 0x7f6c30075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.138+0000 7f6c4effd640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6c30073740 0x7f6c30075c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b0000 tx=0x7f6c3801c4e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:16.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.142+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6c501e9c10 con 0x7f6c3c0040c0 2024-09-21T02:20:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.322+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol16080", "target": ["mon-mgr", ""]}) -- 0x7f6c500ca010 con 0x7f6c30073740 2024-09-21T02:20:16.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.398+0000 7f6c47fff640 1 -- 172.21.15.170:0/2188313871 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c500ca010 con 0x7f6c30073740 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6c30073740 msgr2=0x7f6c30075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6c30073740 0x7f6c30075c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b0000 tx=0x7f6c3801c4e0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 msgr2=0x7f6c500b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c500b0440 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6c480018a0 tx=0x7f6c48001d60 comp rx=0 tx=0).stop 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 shutdown_connections 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6c30073740 0x7f6c30075c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c500b25b0 0x7f6c501e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c501e9800 0x7f6c500acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 --2- 172.21.15.170:0/2188313871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c3c0040c0 0x7f6c500b0440 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 >> 172.21.15.170:0/2188313871 conn(0x7f6c500d0170 msgr2=0x7f6c5015e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 shutdown_connections 2024-09-21T02:20:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.402+0000 7f6c55adf640 1 -- 172.21.15.170:0/2188313871 wait complete. 2024-09-21T02:20:16.405 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.406+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.405 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.406+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.405 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.406+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.405 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.406+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.405 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.406+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.421 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.422+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.421 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.422+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.421 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.422+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.421 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.422+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.421 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:16.422+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:16.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol421 subvol_grp48950 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 -- 172.21.15.170:0/1611374601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44380ca720 msgr2=0x7f44380cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 --2- 172.21.15.170:0/1611374601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44380ca720 0x7f44380cab30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4420009900 tx=0x7f442001c700 comp rx=0 tx=0).stop 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 -- 172.21.15.170:0/1611374601 shutdown_connections 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 --2- 172.21.15.170:0/1611374601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f443806f2d0 0x7f443806f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 --2- 172.21.15.170:0/1611374601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44380ca720 0x7f44380cab30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 -- 172.21.15.170:0/1611374601 >> 172.21.15.170:0/1611374601 conn(0x7f44380b1c40 msgr2=0x7f44380b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 -- 172.21.15.170:0/1611374601 shutdown_connections 2024-09-21T02:20:16.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.581+0000 7f443de69640 1 -- 172.21.15.170:0/1611374601 wait complete. 2024-09-21T02:20:16.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 Processor -- start 2024-09-21T02:20:16.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- start start 2024-09-21T02:20:16.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 0x7f4438145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f443809dc00 0x7f44381477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44380ca720 0x7f4438147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44381688f0 con 0x7f443806f2d0 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4438168770 con 0x7f443809dc00 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4438168a70 con 0x7f44380ca720 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 0x7f4438145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 0x7f4438145060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46126/0 (socket says 172.21.15.170:46126) 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 -- 172.21.15.170:0/20282749 learned_addr learned my addr 172.21.15.170:0/20282749 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f4436ffd640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f443809dc00 0x7f44381477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f4437fff640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44380ca720 0x7f4438147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 -- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44380ca720 msgr2=0x7f4438147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44380ca720 0x7f4438147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 -- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f443809dc00 msgr2=0x7f44381477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f443809dc00 0x7f44381477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 -- 172.21.15.170:0/20282749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4438148610 con 0x7f443806f2d0 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f4436ffd640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f443809dc00 0x7f44381477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:16.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f44377fe640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 0x7f4438145060 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f442800d9c0 tx=0x7f442800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442800ed40 con 0x7f443806f2d0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4428005930 con 0x7f443806f2d0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443815ba70 con 0x7f443806f2d0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428005d80 con 0x7f443806f2d0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443815fd30 con 0x7f443806f2d0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.585+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443806f6e0 con 0x7f443806f2d0 2024-09-21T02:20:16.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.589+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 97693+0+0 (secure 0 0 0) 0x7f442801d020 con 0x7f443806f2d0 2024-09-21T02:20:16.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.589+0000 7f442ffff640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4418073710 0x7f4418075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:16.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.589+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f4428096bb0 con 0x7f443806f2d0 2024-09-21T02:20:16.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.589+0000 7f4436ffd640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4418073710 0x7f4418075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:16.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.589+0000 7f4436ffd640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4418073710 0x7f4418075c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44200af000 tx=0x7f442001af40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:16.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.593+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4428052290 con 0x7f443806f2d0 2024-09-21T02:20:16.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.789+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol421", "group_name": "subvol_grp48950", "target": ["mon-mgr", ""]}) -- 0x7f443815c0c0 con 0x7f4418073710 2024-09-21T02:20:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.861+0000 7f442ffff640 1 -- 172.21.15.170:0/20282749 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f443815c0c0 con 0x7f4418073710 2024-09-21T02:20:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4418073710 msgr2=0x7f4418075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4418073710 0x7f4418075c20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44200af000 tx=0x7f442001af40 comp rx=0 tx=0).stop 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 msgr2=0x7f4438145060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 0x7f4438145060 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f442800d9c0 tx=0x7f442800de80 comp rx=0 tx=0).stop 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 shutdown_connections 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4418073710 0x7f4418075c20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44380ca720 0x7f4438147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f443809dc00 0x7f44381477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 --2- 172.21.15.170:0/20282749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443806f2d0 0x7f4438145060 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 >> 172.21.15.170:0/20282749 conn(0x7f44380b1c40 msgr2=0x7f44380b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 shutdown_connections 2024-09-21T02:20:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:16.865+0000 7f443de69640 1 -- 172.21.15.170:0/20282749 wait complete. 2024-09-21T02:20:16.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol25672 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363fd04640 1 -- 172.21.15.170:0/1233982803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624004d50 msgr2=0x7f3624005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363fd04640 1 --2- 172.21.15.170:0/1233982803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624004d50 0x7f3624005200 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f363000c260 tx=0x7f363001f6c0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363e2a1640 1 -- 172.21.15.170:0/1233982803 reap_dead start 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363fd04640 1 -- 172.21.15.170:0/1233982803 shutdown_connections 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363fd04640 1 -- 172.21.15.170:0/1233982803 >> 172.21.15.170:0/1233982803 conn(0x7f36380d0170 msgr2=0x7f36380d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363fd04640 1 -- 172.21.15.170:0/1233982803 shutdown_connections 2024-09-21T02:20:17.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.053+0000 7f363fd04640 1 -- 172.21.15.170:0/1233982803 wait complete. 2024-09-21T02:20:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 Processor -- start 2024-09-21T02:20:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 -- start start 2024-09-21T02:20:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3624004d50 0x7f36381f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36380b4180 0x7f36380b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36380c9a20 0x7f36380c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f363806f650 con 0x7f36380c9a20 2024-09-21T02:20:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f363806f4d0 con 0x7f3624004d50 2024-09-21T02:20:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f36380ce070 con 0x7f36380b4180 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36380c9a20 0x7f36380c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36380c9a20 0x7f36380c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46144/0 (socket says 172.21.15.170:46144) 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 -- 172.21.15.170:0/1938964216 learned_addr learned my addr 172.21.15.170:0/1938964216 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363daa0640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3624004d50 0x7f36381f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363d29f640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36380b4180 0x7f36380b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 -- 172.21.15.170:0/1938964216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36380b4180 msgr2=0x7f36380b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36380b4180 0x7f36380b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 -- 172.21.15.170:0/1938964216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3624004d50 msgr2=0x7f36381f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3624004d50 0x7f36381f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 -- 172.21.15.170:0/1938964216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36380b4ad0 con 0x7f36380c9a20 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363e2a1640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36380c9a20 0x7f36380c9e30 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f3630000c10 tx=0x7f3630001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300b3070 con 0x7f36380c9a20 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3630006180 con 0x7f36380c9a20 2024-09-21T02:20:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 -- 172.21.15.170:0/1938964216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363809c9d0 con 0x7f36380c9a20 2024-09-21T02:20:17.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630006570 con 0x7f36380c9a20 2024-09-21T02:20:17.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.057+0000 7f363fd04640 1 -- 172.21.15.170:0/1938964216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363809cf70 con 0x7f36380c9a20 2024-09-21T02:20:17.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.061+0000 7f363fd04640 1 -- 172.21.15.170:0/1938964216 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36380ce5d0 con 0x7f36380c9a20 2024-09-21T02:20:17.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.061+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 97693+0+0 (secure 0 0 0) 0x7f36300c1060 con 0x7f36380c9a20 2024-09-21T02:20:17.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.061+0000 7f362e7fc640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f361c073710 0x7f361c075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.061+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f3630131660 con 0x7f36380c9a20 2024-09-21T02:20:17.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.061+0000 7f363daa0640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f361c073710 0x7f361c075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.061+0000 7f363daa0640 1 --2- 172.21.15.170:0/1938964216 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f361c073710 0x7f361c075c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3634009f10 tx=0x7f3634009500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.065+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f36380b4ad0 con 0x7f36380c9a20 2024-09-21T02:20:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.265+0000 7f363fd04640 1 -- 172.21.15.170:0/1938964216 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol25672", "target": ["mon-mgr", ""]}) -- 0x7f36380b5c80 con 0x7f361c073710 2024-09-21T02:20:17.277 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:17.277+0000 7efb9f2c4640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol25672' is not ready for operation rm (use --force to override) 2024-09-21T02:20:17.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.277+0000 7f362e7fc640 1 -- 172.21.15.170:0/1938964216 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol25672' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7f36380b5c80 con 0x7f361c073710 2024-09-21T02:20:17.278 INFO:teuthology.orchestra.run.smithi170.stderr:Error EAGAIN: subvolume 'subvol25672' is not ready for operation rm (use --force to override) 2024-09-21T02:20:17.289 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-21T02:20:17.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:17.290 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol25672/.meta) 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:type = clone 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol25672/790b725b-c9f6-414c-ac7f-b212c1c9f1a5 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:state = pending 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:[source] 2024-09-21T02:20:17.309 INFO:teuthology.orchestra.run.smithi170.stdout:volume = fake 2024-09-21T02:20:17.310 INFO:teuthology.orchestra.run.smithi170.stdout:subvolume = fake 2024-09-21T02:20:17.310 INFO:teuthology.orchestra.run.smithi170.stdout:snapshot = fake 2024-09-21T02:20:17.311 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol25672 --force 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.517+0000 7f292c9e8640 1 -- 172.21.15.170:0/1123846403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 msgr2=0x7f2928073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.517+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1123846403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f2928073ce0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2920009bc0 tx=0x7f292001cb40 comp rx=0 tx=0).stop 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- 172.21.15.170:0/1123846403 shutdown_connections 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1123846403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f2928073ce0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- 172.21.15.170:0/1123846403 >> 172.21.15.170:0/1123846403 conn(0x7f29280b1c40 msgr2=0x7f29280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- 172.21.15.170:0/1123846403 shutdown_connections 2024-09-21T02:20:17.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- 172.21.15.170:0/1123846403 wait complete. 2024-09-21T02:20:17.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 Processor -- start 2024-09-21T02:20:17.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- start start 2024-09-21T02:20:17.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29280ca7b0 0x7f29281f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f29281fa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29281fa940 0x7f292806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2928169c50 con 0x7f29280cabc0 2024-09-21T02:20:17.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2928169ad0 con 0x7f29280ca7b0 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292c9e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2928169dd0 con 0x7f29281fa940 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f29281fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f29281fa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46148/0 (socket says 172.21.15.170:46148) 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 -- 172.21.15.170:0/1842690540 learned_addr learned my addr 172.21.15.170:0/1842690540 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f292659c640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29280ca7b0 0x7f29281f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2926d9d640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29281fa940 0x7f292806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 -- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29281fa940 msgr2=0x7f292806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29281fa940 0x7f292806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 -- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29280ca7b0 msgr2=0x7f29281f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29280ca7b0 0x7f29281f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.521+0000 7f2925d9b640 1 -- 172.21.15.170:0/1842690540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29281f70a0 con 0x7f29280cabc0 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f2925d9b640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f29281fa400 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f291800c3c0 tx=0x7f291800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:17.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918011790 con 0x7f29280cabc0 2024-09-21T02:20:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2918017070 con 0x7f29280cabc0 2024-09-21T02:20:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29281f7380 con 0x7f29280cabc0 2024-09-21T02:20:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918011d40 con 0x7f29280cabc0 2024-09-21T02:20:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29281f7940 con 0x7f29280cabc0 2024-09-21T02:20:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292809da60 con 0x7f29280cabc0 2024-09-21T02:20:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 97693+0+0 (secure 0 0 0) 0x7f291801c460 con 0x7f29280cabc0 2024-09-21T02:20:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f291effd640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2908073710 0x7f2908075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f292659c640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2908073710 0x7f2908075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f2918094a60 con 0x7f29280cabc0 2024-09-21T02:20:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.525+0000 7f292659c640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2908073710 0x7f2908075c20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2910006fd0 tx=0x7f2910008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.529+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f291805f1e0 con 0x7f29280cabc0 2024-09-21T02:20:17.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.725+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol25672", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f29280afef0 con 0x7f2908073710 2024-09-21T02:20:17.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.745+0000 7f291effd640 1 -- 172.21.15.170:0/1842690540 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29280afef0 con 0x7f2908073710 2024-09-21T02:20:17.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2908073710 msgr2=0x7f2908075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2908073710 0x7f2908075c20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2910006fd0 tx=0x7f2910008040 comp rx=0 tx=0).stop 2024-09-21T02:20:17.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 msgr2=0x7f29281fa400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f29281fa400 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f291800c3c0 tx=0x7f291800c880 comp rx=0 tx=0).stop 2024-09-21T02:20:17.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 shutdown_connections 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2908073710 0x7f2908075c20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29281fa940 0x7f292806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29280cabc0 0x7f29281fa400 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 --2- 172.21.15.170:0/1842690540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29280ca7b0 0x7f29281f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 >> 172.21.15.170:0/1842690540 conn(0x7f29280b1c40 msgr2=0x7f29280adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 shutdown_connections 2024-09-21T02:20:17.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.749+0000 7f292c9e8640 1 -- 172.21.15.170:0/1842690540 wait complete. 2024-09-21T02:20:17.770 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-21T02:20:17.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50ead22640 1 -- 172.21.15.170:0/1399351345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d4004e50 msgr2=0x7f50d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50ead22640 1 --2- 172.21.15.170:0/1399351345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50d4004e50 0x7f50d400b9d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f50e40d0050 tx=0x7f50dc01d280 comp rx=0 tx=0).stop 2024-09-21T02:20:17.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50e92bf640 1 -- 172.21.15.170:0/1399351345 reap_dead start 2024-09-21T02:20:17.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50ead22640 1 -- 172.21.15.170:0/1399351345 shutdown_connections 2024-09-21T02:20:17.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50ead22640 1 -- 172.21.15.170:0/1399351345 >> 172.21.15.170:0/1399351345 conn(0x7f50e40d0170 msgr2=0x7f50e40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:17.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50ead22640 1 -- 172.21.15.170:0/1399351345 shutdown_connections 2024-09-21T02:20:17.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.929+0000 7f50ead22640 1 -- 172.21.15.170:0/1399351345 wait complete. 2024-09-21T02:20:17.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 Processor -- start 2024-09-21T02:20:17.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- start start 2024-09-21T02:20:17.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50d4004e50 0x7f50e40c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 0x7f50e40c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e40c9ad0 0x7f50e41f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f50e40cb400 con 0x7f50e40b2790 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f50e40cb280 con 0x7f50e40c9ad0 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f50e40cb580 con 0x7f50d4004e50 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 0x7f50e40c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 0x7f50e40c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46178/0 (socket says 172.21.15.170:46178) 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 -- 172.21.15.170:0/696436665 learned_addr learned my addr 172.21.15.170:0/696436665 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e92bf640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e40c9ad0 0x7f50e41f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 -- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50d4004e50 msgr2=0x7f50e40c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50d4004e50 0x7f50e40c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 -- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e40c9ad0 msgr2=0x7f50e41f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e40c9ad0 0x7f50e41f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 -- 172.21.15.170:0/696436665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e40cfe40 con 0x7f50e40b2790 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e3fff640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 0x7f50e40c9590 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f50d000e760 tx=0x7f50d000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:17.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d000cc60 con 0x7f50e40b2790 2024-09-21T02:20:17.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50d000ee70 con 0x7f50e40b2790 2024-09-21T02:20:17.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e41f3a50 con 0x7f50e40b2790 2024-09-21T02:20:17.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0011690 con 0x7f50e40b2790 2024-09-21T02:20:17.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e41f3d10 con 0x7f50e40b2790 2024-09-21T02:20:17.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.933+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e40cc250 con 0x7f50e40b2790 2024-09-21T02:20:17.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.937+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 97693+0+0 (secure 0 0 0) 0x7f50d0011830 con 0x7f50e40b2790 2024-09-21T02:20:17.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.937+0000 7f50e17fa640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f50c8073710 0x7f50c8075c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:17.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.937+0000 7f50e8abe640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f50c8073710 0x7f50c8075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:17.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.937+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f50d0095670 con 0x7f50e40b2790 2024-09-21T02:20:17.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.937+0000 7f50e8abe640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f50c8073710 0x7f50c8075c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f50d80064b0 tx=0x7f50d8006ab0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:17.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:17.941+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f50d0016030 con 0x7f50e40b2790 2024-09-21T02:20:18.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.057+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f50d005bba0 con 0x7f50e40b2790 2024-09-21T02:20:18.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.145+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f50e409db20 con 0x7f50c8073710 2024-09-21T02:20:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.153+0000 7f50e17fa640 1 -- 172.21.15.170:0/696436665 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f50e409db20 con 0x7f50c8073710 2024-09-21T02:20:18.154 INFO:teuthology.orchestra.run.smithi170.stdout:[] 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f50c8073710 msgr2=0x7f50c8075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f50c8073710 0x7f50c8075c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f50d80064b0 tx=0x7f50d8006ab0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 msgr2=0x7f50e40c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 0x7f50e40c9590 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f50d000e760 tx=0x7f50d000ec20 comp rx=0 tx=0).stop 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 shutdown_connections 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f50c8073710 0x7f50c8075c20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e40c9ad0 0x7f50e41f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50e40b2790 0x7f50e40c9590 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 --2- 172.21.15.170:0/696436665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50d4004e50 0x7f50e40c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 >> 172.21.15.170:0/696436665 conn(0x7f50e40d0170 msgr2=0x7f50e4075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 shutdown_connections 2024-09-21T02:20:18.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.157+0000 7f50ead22640 1 -- 172.21.15.170:0/696436665 wait complete. 2024-09-21T02:20:18.177 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:20:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:20:18.226 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:20:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:18.230 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:20:18.230 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:20:18.333 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:20:18.412 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:20:18.412 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:18.412 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:18.412 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:18.418 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:18.418 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:18.465 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:20:18.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 -- 172.21.15.170:0/3095711202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 msgr2=0x7f9764160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 --2- 172.21.15.170:0/3095711202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f9764160390 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f9754009900 tx=0x7f975401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 -- 172.21.15.170:0/3095711202 shutdown_connections 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 --2- 172.21.15.170:0/3095711202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f9764160390 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 --2- 172.21.15.170:0/3095711202 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f976406d510 0x7f97640b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 -- 172.21.15.170:0/3095711202 >> 172.21.15.170:0/3095711202 conn(0x7f976409a9c0 msgr2=0x7f97640d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 -- 172.21.15.170:0/3095711202 shutdown_connections 2024-09-21T02:20:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.673+0000 7f9769177640 1 -- 172.21.15.170:0/3095711202 wait complete. 2024-09-21T02:20:18.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 Processor -- start 2024-09-21T02:20:18.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 -- start start 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97640b22b0 0x7f97640cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f976415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97640c6ad0 0x7f97641fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f976415c6f0 con 0x7f97640b6930 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f976415c570 con 0x7f97640c6ad0 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f976415c870 con 0x7f97640b22b0 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f976415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f976415cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46206/0 (socket says 172.21.15.170:46206) 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 -- 172.21.15.170:0/2025437059 learned_addr learned my addr 172.21.15.170:0/2025437059 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9762d9d640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97640b22b0 0x7f97640cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976359e640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97640c6ad0 0x7f97641fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 -- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97640b22b0 msgr2=0x7f97640cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97640b22b0 0x7f97640cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 -- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97640c6ad0 msgr2=0x7f97641fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97640c6ad0 0x7f97641fa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 -- 172.21.15.170:0/2025437059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97640c5d70 con 0x7f97640b6930 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f976259c640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f976415cd80 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c9a0 tx=0x7f974c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c007bb0 con 0x7f97640b6930 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f974c007e40 con 0x7f97640b6930 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97640cb120 con 0x7f97640b6930 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c005280 con 0x7f97640b6930 2024-09-21T02:20:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.677+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97640c5ff0 con 0x7f97640b6930 2024-09-21T02:20:18.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.681+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97640b1730 con 0x7f97640b6930 2024-09-21T02:20:18.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.681+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f974c005420 con 0x7f97640b6930 2024-09-21T02:20:18.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.681+0000 7f975b7fe640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9744073830 0x7f9744075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:18.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.681+0000 7f9762d9d640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9744073830 0x7f9744075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:18.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.681+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f974c095460 con 0x7f97640b6930 2024-09-21T02:20:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.681+0000 7f9762d9d640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9744073830 0x7f9744075d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f975c009f20 tx=0x7f975c0094d0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:18.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.685+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f97640cb120 con 0x7f97640b6930 2024-09-21T02:20:18.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.869+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f97640707c0 con 0x7f97640b6930 2024-09-21T02:20:18.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f975b7fe640 1 -- 172.21.15.170:0/2025437059 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7f97640c5d70 con 0x7f97640b6930 2024-09-21T02:20:18.872 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:20:18.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9744073830 msgr2=0x7f9744075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9744073830 0x7f9744075d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f975c009f20 tx=0x7f975c0094d0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 msgr2=0x7f976415cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f976415cd80 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c9a0 tx=0x7f974c00ce60 comp rx=0 tx=0).stop 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 shutdown_connections 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9744073830 0x7f9744075d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97640c6ad0 0x7f97641fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97640b6930 0x7f976415cd80 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 --2- 172.21.15.170:0/2025437059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f97640b22b0 0x7f97640cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 >> 172.21.15.170:0/2025437059 conn(0x7f976409a9c0 msgr2=0x7f97640b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 shutdown_connections 2024-09-21T02:20:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:18.873+0000 7f9769177640 1 -- 172.21.15.170:0/2025437059 wait complete. 2024-09-21T02:20:18.897 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:20:18.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:18.897 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:18.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[47204]: fuse finished with error 0 and tester_r 0 2024-09-21T02:20:18.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:20:18.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:18.910 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:20:18.999 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:19.000 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:19.000 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:19.008 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:19.008 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:19.055 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 -- 172.21.15.170:0/774970028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d8005260 msgr2=0x7fa6d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 --2- 172.21.15.170:0/774970028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d8005260 0x7fa6d800b9d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00cbfa0 tx=0x7fa6e801caf0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 -- 172.21.15.170:0/774970028 shutdown_connections 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 --2- 172.21.15.170:0/774970028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d8005260 0x7fa6d800b9d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 -- 172.21.15.170:0/774970028 >> 172.21.15.170:0/774970028 conn(0x7fa6f00b1c40 msgr2=0x7fa6f00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 -- 172.21.15.170:0/774970028 shutdown_connections 2024-09-21T02:20:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 -- 172.21.15.170:0/774970028 wait complete. 2024-09-21T02:20:19.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 Processor -- start 2024-09-21T02:20:19.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.213+0000 7fa6f5734640 1 -- start start 2024-09-21T02:20:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6d8005260 0x7fa6f0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 0x7fa6f00c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00b41d0 0x7fa6f00b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00cbc00 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00cba80 con 0x7fa6f00b41d0 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f00cbd80 con 0x7fa6d8005260 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 0x7fa6f00c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 0x7fa6f00c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46232/0 (socket says 172.21.15.170:46232) 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 -- 172.21.15.170:0/2024709001 learned_addr learned my addr 172.21.15.170:0/2024709001 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ef7fe640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00b41d0 0x7fa6f00b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6eeffd640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6d8005260 0x7fa6f0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 -- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6d8005260 msgr2=0x7fa6f0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6d8005260 0x7fa6f0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 -- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00b41d0 msgr2=0x7fa6f00b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00b41d0 0x7fa6f00b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 -- 172.21.15.170:0/2024709001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f00b4b20 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6eeffd640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6d8005260 0x7fa6f0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ef7fe640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00b41d0 0x7fa6f00b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6ee7fc640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 0x7fa6f00c8d90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000d920 tx=0x7fa6e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000b830 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e000bac0 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f01f9770 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0003600 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f01f9d30 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.217+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f01f94e0 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.221+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fa6e00037a0 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.221+0000 7fa6d77fe640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6dc073860 0x7fa6dc075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.221+0000 7fa6eeffd640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6dc073860 0x7fa6dc075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.221+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e0095e60 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.221+0000 7fa6eeffd640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6dc073860 0x7fa6dc075d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009a80 tx=0x7fa6e40091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.225+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa6f00b4b20 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.409+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa6f00706c0 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.409+0000 7fa6d77fe640 1 -- 172.21.15.170:0/2024709001 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7fa6e00605c0 con 0x7fa6f01f8fd0 2024-09-21T02:20:19.410 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:20:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6dc073860 msgr2=0x7fa6dc075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6dc073860 0x7fa6dc075d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009a80 tx=0x7fa6e40091e0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 msgr2=0x7fa6f00c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 0x7fa6f00c8d90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000d920 tx=0x7fa6e000dde0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 shutdown_connections 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa6dc073860 0x7fa6dc075d70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00b41d0 0x7fa6f00b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6f01f8fd0 0x7fa6f00c8d90 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 --2- 172.21.15.170:0/2024709001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6d8005260 0x7fa6f0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 >> 172.21.15.170:0/2024709001 conn(0x7fa6f00b1c40 msgr2=0x7fa6f00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 shutdown_connections 2024-09-21T02:20:19.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.413+0000 7fa6f5734640 1 -- 172.21.15.170:0/2024709001 wait complete. 2024-09-21T02:20:19.434 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:20:19.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:19.434 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:19.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35375]: fuse finished with error 0 and tester_r 0 2024-09-21T02:20:19.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:20:19.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:19.449 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:20:19.575 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:19.575 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:19.575 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:19.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:20:19.581 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:19.581 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:19.582 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:19.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:20:19.587 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 -- 172.21.15.170:0/1366178107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 msgr2=0x7f5ac816fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/1366178107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac816fc90 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009840 tx=0x7f5ab801c820 comp rx=0 tx=0).stop 2024-09-21T02:20:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 -- 172.21.15.170:0/1366178107 shutdown_connections 2024-09-21T02:20:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/1366178107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac816fc90 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/1366178107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ac80d0c10 0x7f5ac80cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 -- 172.21.15.170:0/1366178107 >> 172.21.15.170:0/1366178107 conn(0x7f5ac80d00f0 msgr2=0x7f5ac80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 -- 172.21.15.170:0/1366178107 shutdown_connections 2024-09-21T02:20:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 -- 172.21.15.170:0/1366178107 wait complete. 2024-09-21T02:20:19.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 Processor -- start 2024-09-21T02:20:19.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.805+0000 7f5acfb6b640 1 -- start start 2024-09-21T02:20:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac820b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ac80d0c10 0x7f5ac816e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ac81701d0 0x7f5ac80c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81794d0 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8179350 con 0x7f5ac81701d0 2024-09-21T02:20:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac8179650 con 0x7f5ac80d0c10 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac820b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac820b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46242/0 (socket says 172.21.15.170:46242) 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 -- 172.21.15.170:0/2489969403 learned_addr learned my addr 172.21.15.170:0/2489969403 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5ace108640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ac81701d0 0x7f5ac80c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd106640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ac80d0c10 0x7f5ac816e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 -- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ac80d0c10 msgr2=0x7f5ac816e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ac80d0c10 0x7f5ac816e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 -- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ac81701d0 msgr2=0x7f5ac80c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ac81701d0 0x7f5ac80c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 -- 172.21.15.170:0/2489969403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac809a9c0 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acd907640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac820b5e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400c940 tx=0x7f5ac400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4004220 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ac40079b0 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac809bec0 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40044b0 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac809c480 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.809+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac816fc90 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.813+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f5ac4003600 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.813+0000 7f5abe7fc640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5aac073830 0x7f5aac075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.813+0000 7f5acd106640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5aac073830 0x7f5aac075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.813+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f5ac4094f20 con 0x7f5ac80cb1c0 2024-09-21T02:20:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.813+0000 7f5acd106640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5aac073830 0x7f5aac075d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80af000 tx=0x7f5ab801b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:19.817+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5ac4014030 con 0x7f5ac80cb1c0 2024-09-21T02:20:20.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.033+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ac80af4b0 con 0x7f5ac80cb1c0 2024-09-21T02:20:20.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.033+0000 7f5abe7fc640 1 -- 172.21.15.170:0/2489969403 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5228 (secure 0 0 0) 0x7f5ac405f580 con 0x7f5ac80cb1c0 2024-09-21T02:20:20.033 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:20.034 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":82,"btime":"2024-09-21T02:20:11:986307+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:19:54.300110+0000","modified":"2024-09-21T02:19:57.337649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7614":{"gid":7614,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/300226457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":300226457},{"type":"v1","addr":"172.21.15.170:6835","nonce":300226457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7614,"qdb_cluster":[7614]},"id":9}]} 2024-09-21T02:20:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 82 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.033+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5aac073830 msgr2=0x7f5aac075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.033+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5aac073830 0x7f5aac075d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80af000 tx=0x7f5ab801b060 comp rx=0 tx=0).stop 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.033+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 msgr2=0x7f5ac820b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.033+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac820b5e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400c940 tx=0x7f5ac400ce00 comp rx=0 tx=0).stop 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 shutdown_connections 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5aac073830 0x7f5aac075d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ac80d0c10 0x7f5ac816e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ac81701d0 0x7f5ac80c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 --2- 172.21.15.170:0/2489969403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac80cb1c0 0x7f5ac820b5e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 >> 172.21.15.170:0/2489969403 conn(0x7f5ac80d00f0 msgr2=0x7f5ac80b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 shutdown_connections 2024-09-21T02:20:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.037+0000 7f5acfb6b640 1 -- 172.21.15.170:0/2489969403 wait complete. 2024-09-21T02:20:20.057 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:20.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 -- 172.21.15.170:0/3962126707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 msgr2=0x7fe938070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3962126707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe938070030 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe930009900 tx=0x7fe93001db10 comp rx=0 tx=0).stop 2024-09-21T02:20:20.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 -- 172.21.15.170:0/3962126707 shutdown_connections 2024-09-21T02:20:20.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3962126707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe938070030 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 -- 172.21.15.170:0/3962126707 >> 172.21.15.170:0/3962126707 conn(0x7fe9380b1c40 msgr2=0x7fe9380b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:20.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 -- 172.21.15.170:0/3962126707 shutdown_connections 2024-09-21T02:20:20.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 -- 172.21.15.170:0/3962126707 wait complete. 2024-09-21T02:20:20.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 Processor -- start 2024-09-21T02:20:20.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.201+0000 7fe93fb48640 1 -- start start 2024-09-21T02:20:20.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe938070570 0x7fe9381fae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9380cadf0 0x7fe9380d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe9380d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe938169390 con 0x7fe93815f990 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe938169210 con 0x7fe938070570 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe938169510 con 0x7fe9380cadf0 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe9380d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe9380d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46250/0 (socket says 172.21.15.170:46250) 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 -- 172.21.15.170:0/3717388171 learned_addr learned my addr 172.21.15.170:0/3717388171 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93d8e4640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe938070570 0x7fe9381fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 -- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9380cadf0 msgr2=0x7fe9380d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93d0e3640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9380cadf0 0x7fe9380d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9380cadf0 0x7fe9380d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 -- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe938070570 msgr2=0x7fe9381fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe938070570 0x7fe9381fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 -- 172.21.15.170:0/3717388171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9381f7190 con 0x7fe93815f990 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93d8e4640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe938070570 0x7fe9381fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93d0e3640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9380cadf0 0x7fe9380d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93e0e5640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe9380d06a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b410 tx=0x7fe92800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928015020 con 0x7fe93815f990 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe928011920 con 0x7fe93815f990 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9381f7470 con 0x7fe93815f990 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928011d70 con 0x7fe93815f990 2024-09-21T02:20:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9381f7a30 con 0x7fe93815f990 2024-09-21T02:20:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938070030 con 0x7fe93815f990 2024-09-21T02:20:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fe928018020 con 0x7fe93815f990 2024-09-21T02:20:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe9367fc640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe924073830 0x7fe924075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fe928094d00 con 0x7fe93815f990 2024-09-21T02:20:20.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.205+0000 7fe93d8e4640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe924073830 0x7fe924075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.209+0000 7fe93d8e4640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe924073830 0x7fe924075d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9300b0000 tx=0x7fe93001c350 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:20.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.213+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe92801f020 con 0x7fe93815f990 2024-09-21T02:20:20.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.425+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9380cd8a0 con 0x7fe93815f990 2024-09-21T02:20:20.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.425+0000 7fe9367fc640 1 -- 172.21.15.170:0/3717388171 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5228 (secure 0 0 0) 0x7fe92805f360 con 0x7fe93815f990 2024-09-21T02:20:20.425 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:20.425 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":82,"btime":"2024-09-21T02:20:11:986307+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:19:54.300110+0000","modified":"2024-09-21T02:19:57.337649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7614":{"gid":7614,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/300226457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":300226457},{"type":"v1","addr":"172.21.15.170:6835","nonce":300226457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7614,"qdb_cluster":[7614]},"id":9}]} 2024-09-21T02:20:20.426 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 82 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.425+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe924073830 msgr2=0x7fe924075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.425+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe924073830 0x7fe924075d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe9300b0000 tx=0x7fe93001c350 comp rx=0 tx=0).stop 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.425+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 msgr2=0x7fe9380d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.425+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe9380d06a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b410 tx=0x7fe92800b8d0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 shutdown_connections 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe924073830 0x7fe924075d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9380cadf0 0x7fe9380d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe93815f990 0x7fe9380d06a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 --2- 172.21.15.170:0/3717388171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe938070570 0x7fe9381fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 >> 172.21.15.170:0/3717388171 conn(0x7fe9380b1c40 msgr2=0x7fe9380b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:20.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 shutdown_connections 2024-09-21T02:20:20.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.429+0000 7fe93fb48640 1 -- 172.21.15.170:0/3717388171 wait complete. 2024-09-21T02:20:20.447 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 -- 172.21.15.170:0/3728963381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 msgr2=0x7f78e015ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 --2- 172.21.15.170:0/3728963381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e015ff00 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f78e00d2370 tx=0x7f78d401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 -- 172.21.15.170:0/3728963381 shutdown_connections 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 --2- 172.21.15.170:0/3728963381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e015ff00 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 --2- 172.21.15.170:0/3728963381 >> v2:172.21.15.191:3300/0 conn(0x7f78e00b25f0 0x7f78e00cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 -- 172.21.15.170:0/3728963381 >> 172.21.15.170:0/3728963381 conn(0x7f78e009a9c0 msgr2=0x7f78e00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 -- 172.21.15.170:0/3728963381 shutdown_connections 2024-09-21T02:20:20.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.585+0000 7f78e707b640 1 -- 172.21.15.170:0/3728963381 wait complete. 2024-09-21T02:20:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 Processor -- start 2024-09-21T02:20:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 -- start start 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78e00b25f0 0x7f78e0150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78e0160440 0x7f78e0157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0169400 con 0x7f78e015faf0 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0169280 con 0x7f78e00b25f0 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f78e0169580 con 0x7f78e0160440 2024-09-21T02:20:20.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e014c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46258/0 (socket says 172.21.15.170:46258) 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 -- 172.21.15.170:0/4051703742 learned_addr learned my addr 172.21.15.170:0/4051703742 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e4e17640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78e00b25f0 0x7f78e0150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e5618640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78e0160440 0x7f78e0157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 -- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78e0160440 msgr2=0x7f78e0157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78e0160440 0x7f78e0157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 -- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78e00b25f0 msgr2=0x7f78e0150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78e00b25f0 0x7f78e0150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 -- 172.21.15.170:0/4051703742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e015f670 con 0x7f78e015faf0 2024-09-21T02:20:20.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dffff640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e014c1d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f78d401af70 tx=0x7f78d4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40b2040 con 0x7f78e015faf0 2024-09-21T02:20:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78d4005320 con 0x7f78e015faf0 2024-09-21T02:20:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e006cfa0 con 0x7f78e015faf0 2024-09-21T02:20:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40046d0 con 0x7f78e015faf0 2024-09-21T02:20:20.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.589+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e014c710 con 0x7f78e015faf0 2024-09-21T02:20:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.593+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e015f3e0 con 0x7f78e015faf0 2024-09-21T02:20:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.593+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f78d4004890 con 0x7f78e015faf0 2024-09-21T02:20:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.593+0000 7f78dd7fa640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78cc073830 0x7f78cc075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.593+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f78d412f150 con 0x7f78e015faf0 2024-09-21T02:20:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.593+0000 7f78e4e17640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78cc073830 0x7f78cc075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.593+0000 7f78e4e17640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78cc073830 0x7f78cc075d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009e00 tx=0x7f78d8009390 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:20.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.597+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f78d40be060 con 0x7f78e015faf0 2024-09-21T02:20:20.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.769+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78e00cf950 con 0x7f78e015faf0 2024-09-21T02:20:20.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78dd7fa640 1 -- 172.21.15.170:0/4051703742 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16735 (secure 0 0 0) 0x7f78d40164b0 con 0x7f78e015faf0 2024-09-21T02:20:20.772 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:20.772 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":118,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:19:55.871538+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:19:51.966032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T02:19:53.233775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78cc073830 msgr2=0x7f78cc075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78cc073830 0x7f78cc075d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009e00 tx=0x7f78d8009390 comp rx=0 tx=0).stop 2024-09-21T02:20:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 msgr2=0x7f78e014c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e014c1d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f78d401af70 tx=0x7f78d4005e00 comp rx=0 tx=0).stop 2024-09-21T02:20:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 shutdown_connections 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78cc073830 0x7f78cc075d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78e0160440 0x7f78e0157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e015faf0 0x7f78e014c1d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 --2- 172.21.15.170:0/4051703742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78e00b25f0 0x7f78e0150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 >> 172.21.15.170:0/4051703742 conn(0x7f78e009a9c0 msgr2=0x7f78e0070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 shutdown_connections 2024-09-21T02:20:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.773+0000 7f78e707b640 1 -- 172.21.15.170:0/4051703742 wait complete. 2024-09-21T02:20:20.794 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:20:20.795 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:20.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 -- 172.21.15.170:0/4142171223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cb460 msgr2=0x7f8dd00cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 --2- 172.21.15.170:0/4142171223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cb460 0x7f8dd00cb870 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009900 tx=0x7f8dc401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 -- 172.21.15.170:0/4142171223 shutdown_connections 2024-09-21T02:20:20.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 --2- 172.21.15.170:0/4142171223 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00ce1c0 0x7f8dd00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 --2- 172.21.15.170:0/4142171223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cb460 0x7f8dd00cb870 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 -- 172.21.15.170:0/4142171223 >> 172.21.15.170:0/4142171223 conn(0x7f8dd00b1c40 msgr2=0x7f8dd00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:20.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 -- 172.21.15.170:0/4142171223 shutdown_connections 2024-09-21T02:20:20.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.953+0000 7f8dd6424640 1 -- 172.21.15.170:0/4142171223 wait complete. 2024-09-21T02:20:20.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 Processor -- start 2024-09-21T02:20:20.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- start start 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00cb460 0x7f8dd00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd00ce1c0 0x7f8dd0158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 0x7f8dd00af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd01690d0 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd0168f50 con 0x7f8dd00ce1c0 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd0169250 con 0x7f8dd00cb460 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 0x7f8dd00af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 0x7f8dd00af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46276/0 (socket says 172.21.15.170:46276) 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 -- 172.21.15.170:0/590759572 learned_addr learned my addr 172.21.15.170:0/590759572 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dcf7fe640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd00ce1c0 0x7f8dd0158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dcffff640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00cb460 0x7f8dd00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 -- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00cb460 msgr2=0x7f8dd00c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00cb460 0x7f8dd00c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 -- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd00ce1c0 msgr2=0x7f8dd0158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd00ce1c0 0x7f8dd0158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 -- 172.21.15.170:0/590759572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd0158b90 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dcffff640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00cb460 0x7f8dd00c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd49c1640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 0x7f8dd00af3b0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e910 tx=0x7f8dc800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8004d30 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc800cb20 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd0159370 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8005020 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd0159930 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.957+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd01590e0 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.961+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f8dc80051c0 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.961+0000 7f8dccff9640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc073830 0x7f8dbc075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:20.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.961+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f8dc8095920 con 0x7f8dd00cf7e0 2024-09-21T02:20:20.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.961+0000 7f8dcffff640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc073830 0x7f8dbc075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:20.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.961+0000 7f8dcffff640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc073830 0x7f8dbc075d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0000f70 tx=0x7f8dc0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:20.965+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8dd0158b90 con 0x7f8dd00cf7e0 2024-09-21T02:20:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dd00cb870 con 0x7f8dd00cf7e0 2024-09-21T02:20:21.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dccff9640 1 -- 172.21.15.170:0/590759572 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5228 (secure 0 0 0) 0x7f8dc8016030 con 0x7f8dd00cf7e0 2024-09-21T02:20:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:21.182 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":82,"btime":"2024-09-21T02:20:11:986307+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:19:54.300110+0000","modified":"2024-09-21T02:19:57.337649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7614":{"gid":7614,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/300226457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":300226457},{"type":"v1","addr":"172.21.15.170:6835","nonce":300226457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7614,"qdb_cluster":[7614]},"id":9}]} 2024-09-21T02:20:21.182 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 82 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc073830 msgr2=0x7f8dbc075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc073830 0x7f8dbc075d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0000f70 tx=0x7f8dc0009040 comp rx=0 tx=0).stop 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 msgr2=0x7f8dd00af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 0x7f8dd00af3b0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e910 tx=0x7f8dc800edd0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 shutdown_connections 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.181+0000 7f8dd6424640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8dbc073830 0x7f8dbc075d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.185+0000 7f8dd6424640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dd00cf7e0 0x7f8dd00af3b0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.185+0000 7f8dd6424640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8dd00ce1c0 0x7f8dd0158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.185+0000 7f8dd6424640 1 --2- 172.21.15.170:0/590759572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8dd00cb460 0x7f8dd00c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.185+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 >> 172.21.15.170:0/590759572 conn(0x7f8dd00b1c40 msgr2=0x7f8dd00b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.185+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 shutdown_connections 2024-09-21T02:20:21.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.185+0000 7f8dd6424640 1 -- 172.21.15.170:0/590759572 wait complete. 2024-09-21T02:20:21.203 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe5605f8640 1 -- 172.21.15.170:0/3990321116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe544004810 msgr2=0x7fe544004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe5605f8640 1 --2- 172.21.15.170:0/3990321116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe544004810 0x7fe544004c20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe554009b40 tx=0x7fe55401d270 comp rx=0 tx=0).stop 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe55eb95640 1 -- 172.21.15.170:0/3990321116 reap_dead start 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe5605f8640 1 -- 172.21.15.170:0/3990321116 shutdown_connections 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe5605f8640 1 -- 172.21.15.170:0/3990321116 >> 172.21.15.170:0/3990321116 conn(0x7fe5580cee30 msgr2=0x7fe5580cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe5605f8640 1 -- 172.21.15.170:0/3990321116 shutdown_connections 2024-09-21T02:20:21.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.353+0000 7fe5605f8640 1 -- 172.21.15.170:0/3990321116 wait complete. 2024-09-21T02:20:21.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 Processor -- start 2024-09-21T02:20:21.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 -- start start 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe558151730 0x7fe558154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe558142670 0x7fe5581525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 0x7fe5580cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe55815c9d0 con 0x7fe558152b30 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe55815c850 con 0x7fe558151730 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe55815cb50 con 0x7fe558142670 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 0x7fe5580cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 0x7fe5580cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46290/0 (socket says 172.21.15.170:46290) 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 -- 172.21.15.170:0/917274894 learned_addr learned my addr 172.21.15.170:0/917274894 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55db93640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe558142670 0x7fe5581525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 -- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe558142670 msgr2=0x7fe5581525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55e394640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe558151730 0x7fe558154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe558142670 0x7fe5581525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 -- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe558151730 msgr2=0x7fe558154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe558151730 0x7fe558154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 -- 172.21.15.170:0/917274894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5580b7810 con 0x7fe558152b30 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe55eb95640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 0x7fe5580cb130 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe5580750a0 tx=0x7fe550010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550017070 con 0x7fe558152b30 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe550015070 con 0x7fe558152b30 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550004d90 con 0x7fe558152b30 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.357+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55815db00 con 0x7fe558152b30 2024-09-21T02:20:21.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5580ca650 con 0x7fe558152b30 2024-09-21T02:20:21.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5580ca440 con 0x7fe558152b30 2024-09-21T02:20:21.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fe550011440 con 0x7fe558152b30 2024-09-21T02:20:21.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe54affd640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe53c073830 0x7fe53c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7fe5500952d0 con 0x7fe558152b30 2024-09-21T02:20:21.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe55e394640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe53c073830 0x7fe53c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.361+0000 7fe55e394640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe53c073830 0x7fe53c075d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe54c001000 tx=0x7fe54c0067b0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:21.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.365+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe550022030 con 0x7fe558152b30 2024-09-21T02:20:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.581+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5580d0eb0 con 0x7fe558152b30 2024-09-21T02:20:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.585+0000 7fe54affd640 1 -- 172.21.15.170:0/917274894 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5228 (secure 0 0 0) 0x7fe55005f980 con 0x7fe558152b30 2024-09-21T02:20:21.585 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:21.585 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":82,"btime":"2024-09-21T02:20:11:986307+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:19:54.300110+0000","modified":"2024-09-21T02:19:57.337649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7614":{"gid":7614,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/300226457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":300226457},{"type":"v1","addr":"172.21.15.170:6835","nonce":300226457}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7614,"qdb_cluster":[7614]},"id":9}]} 2024-09-21T02:20:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 82 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe53c073830 msgr2=0x7fe53c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe53c073830 0x7fe53c075d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe54c001000 tx=0x7fe54c0067b0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 msgr2=0x7fe5580cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 0x7fe5580cb130 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe5580750a0 tx=0x7fe550010e80 comp rx=0 tx=0).stop 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 shutdown_connections 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe53c073830 0x7fe53c075d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe558142670 0x7fe5581525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe558152b30 0x7fe5580cb130 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 --2- 172.21.15.170:0/917274894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe558151730 0x7fe558154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 >> 172.21.15.170:0/917274894 conn(0x7fe5580cee30 msgr2=0x7fe5580c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 shutdown_connections 2024-09-21T02:20:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.589+0000 7fe5605f8640 1 -- 172.21.15.170:0/917274894 wait complete. 2024-09-21T02:20:21.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 -- 172.21.15.170:0/500921158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80ce1c0 msgr2=0x7f13e80ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 --2- 172.21.15.170:0/500921158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80ce1c0 0x7f13e80ce5d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f13e8070cc0 tx=0x7f13e001cd80 comp rx=0 tx=0).stop 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 -- 172.21.15.170:0/500921158 shutdown_connections 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 --2- 172.21.15.170:0/500921158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80ce1c0 0x7f13e80ce5d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 --2- 172.21.15.170:0/500921158 >> v2:172.21.15.191:3300/0 conn(0x7f13e80cb460 0x7f13e80cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 -- 172.21.15.170:0/500921158 >> 172.21.15.170:0/500921158 conn(0x7f13e80b1c40 msgr2=0x7f13e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 -- 172.21.15.170:0/500921158 shutdown_connections 2024-09-21T02:20:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 -- 172.21.15.170:0/500921158 wait complete. 2024-09-21T02:20:21.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 Processor -- start 2024-09-21T02:20:21.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.765+0000 7f13ec908640 1 -- start start 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13e80cb460 0x7f13e809b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e80ce1c0 0x7f13e81f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 0x7f13e806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8168f50 con 0x7f13e80cf7e0 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f13e80cf140 con 0x7f13e80ce1c0 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f13e81690d0 con 0x7f13e80cb460 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 0x7f13e806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 0x7f13e806eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46314/0 (socket says 172.21.15.170:46314) 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 -- 172.21.15.170:0/1493922548 learned_addr learned my addr 172.21.15.170:0/1493922548 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e659c640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13e80cb460 0x7f13e809b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13dffff640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e80ce1c0 0x7f13e81f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 -- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13e80cb460 msgr2=0x7f13e809b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13e80cb460 0x7f13e809b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 -- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e80ce1c0 msgr2=0x7f13e81f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e80ce1c0 0x7f13e81f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 -- 172.21.15.170:0/1493922548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e809d090 con 0x7f13e80cf7e0 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13e6d9d640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 0x7f13e806eeb0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009bd0 tx=0x7f13e00a5c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00bb070 con 0x7f13e80cf7e0 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13e0004e00 con 0x7f13e80cf7e0 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e809d310 con 0x7f13e80cf7e0 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00b2040 con 0x7f13e80cf7e0 2024-09-21T02:20:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e809d8d0 con 0x7f13e80cf7e0 2024-09-21T02:20:21.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e80cb870 con 0x7f13e80cf7e0 2024-09-21T02:20:21.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.769+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f13e00be070 con 0x7f13e80cf7e0 2024-09-21T02:20:21.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.773+0000 7f13deffd640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f13c8073860 0x7f13c8075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:21.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.773+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7f13e012ed40 con 0x7f13e80cf7e0 2024-09-21T02:20:21.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.773+0000 7f13e659c640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f13c8073860 0x7f13c8075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:21.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.773+0000 7f13e659c640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f13c8073860 0x7f13c8075d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13d800b810 tx=0x7f13d8008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:21.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.777+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f13e809d090 con 0x7f13e80cf7e0 2024-09-21T02:20:21.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.961+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13e809b9b0 con 0x7f13e80cf7e0 2024-09-21T02:20:21.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13deffd640 1 -- 172.21.15.170:0/1493922548 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16846 (secure 0 0 0) 0x7f13e0016970 con 0x7f13e80cf7e0 2024-09-21T02:20:21.964 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:21.964 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":119,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:20:21.069680+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:19:51.966032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T02:19:53.233775+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f13c8073860 msgr2=0x7f13c8075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f13c8073860 0x7f13c8075d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13d800b810 tx=0x7f13d8008040 comp rx=0 tx=0).stop 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 msgr2=0x7f13e806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 0x7f13e806eeb0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009bd0 tx=0x7f13e00a5c10 comp rx=0 tx=0).stop 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 shutdown_connections 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f13c8073860 0x7f13c8075d70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13e80cf7e0 0x7f13e806eeb0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e80ce1c0 0x7f13e81f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 --2- 172.21.15.170:0/1493922548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13e80cb460 0x7f13e809b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 >> 172.21.15.170:0/1493922548 conn(0x7f13e80b1c40 msgr2=0x7f13e80cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 shutdown_connections 2024-09-21T02:20:21.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:21.965+0000 7f13ec908640 1 -- 172.21.15.170:0/1493922548 wait complete. 2024-09-21T02:20:21.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:20:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 -- 172.21.15.170:0/153505909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0ce1c0 msgr2=0x7efd2c0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 --2- 172.21.15.170:0/153505909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0ce1c0 0x7efd2c0ce5d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7efd24009900 tx=0x7efd2401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 -- 172.21.15.170:0/153505909 shutdown_connections 2024-09-21T02:20:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 --2- 172.21.15.170:0/153505909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0ce1c0 0x7efd2c0ce5d0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 --2- 172.21.15.170:0/153505909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd2c0cf7e0 0x7efd2c070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 -- 172.21.15.170:0/153505909 >> 172.21.15.170:0/153505909 conn(0x7efd2c0b1c40 msgr2=0x7efd2c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 -- 172.21.15.170:0/153505909 shutdown_connections 2024-09-21T02:20:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 -- 172.21.15.170:0/153505909 wait complete. 2024-09-21T02:20:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 Processor -- start 2024-09-21T02:20:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.145+0000 7efd342f0640 1 -- start start 2024-09-21T02:20:22.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd2c0cb460 0x7efd2c0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c0ce1c0 0x7efd2c0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 0x7efd2c1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1690d0 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c168f50 con 0x7efd2c0ce1c0 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efd2c169250 con 0x7efd2c0cb460 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 0x7efd2c1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 0x7efd2c1466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46324/0 (socket says 172.21.15.170:46324) 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 -- 172.21.15.170:0/12707233 learned_addr learned my addr 172.21.15.170:0/12707233 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 -- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd2c0cb460 msgr2=0x7efd2c0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3208c640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd2c0cb460 0x7efd2c0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3188b640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c0ce1c0 0x7efd2c0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd2c0cb460 0x7efd2c0c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 -- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c0ce1c0 msgr2=0x7efd2c0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c0ce1c0 0x7efd2c0c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 -- 172.21.15.170:0/12707233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c0c9ec0 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd3288d640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 0x7efd2c1466a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7efd240b0040 tx=0x7efd240a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:22.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd240b2040 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd24003970 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24003da0 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c0c92b0 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c0c9870 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2c0c9020 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7efd240bf060 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.149+0000 7efd22ffd640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd10073830 0x7efd10075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.153+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5765+0+0 (secure 0 0 0) 0x7efd2412f360 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.153+0000 7efd3208c640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd10073830 0x7efd10075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.153+0000 7efd3208c640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd10073830 0x7efd10075d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd28005c50 tx=0x7efd28005f50 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:22.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.157+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efd240164d0 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.361+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efd2c0ce5d0 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.364 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:20:22.365+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:20:22.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.365+0000 7efd22ffd640 1 -- 172.21.15.170:0/12707233 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7efd240f9980 con 0x7efd2c0cf7e0 2024-09-21T02:20:22.367 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd10073830 msgr2=0x7efd10075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd10073830 0x7efd10075d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd28005c50 tx=0x7efd28005f50 comp rx=0 tx=0).stop 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 msgr2=0x7efd2c1466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 0x7efd2c1466a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7efd240b0040 tx=0x7efd240a5740 comp rx=0 tx=0).stop 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 shutdown_connections 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd10073830 0x7efd10075d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd2c0cf7e0 0x7efd2c1466a0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd2c0ce1c0 0x7efd2c0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 --2- 172.21.15.170:0/12707233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd2c0cb460 0x7efd2c0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 >> 172.21.15.170:0/12707233 conn(0x7efd2c0b1c40 msgr2=0x7efd2c0acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:22.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 shutdown_connections 2024-09-21T02:20:22.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.373+0000 7efd342f0640 1 -- 172.21.15.170:0/12707233 wait complete. 2024-09-21T02:20:22.400 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:20:22.453 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:20:22.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17e67e640 1 -- 172.21.15.170:0/2402212207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd164003a10 msgr2=0x7fd164003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17e67e640 1 --2- 172.21.15.170:0/2402212207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd164003a10 0x7fd164003e80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd170009eb0 tx=0x7fd17001d5c0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17cc1b640 1 -- 172.21.15.170:0/2402212207 reap_dead start 2024-09-21T02:20:22.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17e67e640 1 -- 172.21.15.170:0/2402212207 shutdown_connections 2024-09-21T02:20:22.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17e67e640 1 -- 172.21.15.170:0/2402212207 >> 172.21.15.170:0/2402212207 conn(0x7fd1780d2a30 msgr2=0x7fd1780cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:22.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17e67e640 1 -- 172.21.15.170:0/2402212207 shutdown_connections 2024-09-21T02:20:22.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.553+0000 7fd17e67e640 1 -- 172.21.15.170:0/2402212207 wait complete. 2024-09-21T02:20:22.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 Processor -- start 2024-09-21T02:20:22.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- start start 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd164003a10 0x7fd178165f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1781fa6f0 0x7fd1780b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 0x7fd1781f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd178160e80 con 0x7fd1781f9b50 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd178160d00 con 0x7fd164003a10 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd17815d7f0 con 0x7fd1781fa6f0 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 0x7fd1781f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 0x7fd1781f9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46364/0 (socket says 172.21.15.170:46364) 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 -- 172.21.15.170:0/4023573773 learned_addr learned my addr 172.21.15.170:0/4023573773 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd1777fe640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1781fa6f0 0x7fd1780b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd177fff640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd164003a10 0x7fd178165f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 -- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1781fa6f0 msgr2=0x7fd1780b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1781fa6f0 0x7fd1780b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 -- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd164003a10 msgr2=0x7fd178165f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd164003a10 0x7fd178165f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 -- 172.21.15.170:0/4023573773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1781fa4a0 con 0x7fd1781f9b50 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17cc1b640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 0x7fd1781f9f60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd1680046e0 tx=0x7fd16800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168017070 con 0x7fd1781f9b50 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd168005d90 con 0x7fd1781f9b50 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1781664b0 con 0x7fd1781f9b50 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680072d0 con 0x7fd1781f9b50 2024-09-21T02:20:22.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1780b79d0 con 0x7fd1781f9b50 2024-09-21T02:20:22.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.557+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1780d0b10 con 0x7fd1781f9b50 2024-09-21T02:20:22.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.561+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fd16801a070 con 0x7fd1781f9b50 2024-09-21T02:20:22.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.561+0000 7fd174ff9640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd158073830 0x7fd158075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:22.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.561+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5823+0+0 (secure 0 0 0) 0x7fd1680940f0 con 0x7fd1781f9b50 2024-09-21T02:20:22.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.561+0000 7fd177fff640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd158073830 0x7fd158075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:22.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.561+0000 7fd177fff640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd158073830 0x7fd158075d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd170009eb0 tx=0x7fd1700ab040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:22.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.565+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd1781fa4a0 con 0x7fd1781f9b50 2024-09-21T02:20:22.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:22.777+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd178071320 con 0x7fd1781f9b50 2024-09-21T02:20:23.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.369+0000 7fd174ff9640 1 -- 172.21.15.170:0/4023573773 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) ==== 104+0+0 (secure 0 0 0) 0x7fd16805e6e0 con 0x7fd1781f9b50 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd158073830 msgr2=0x7fd158075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd158073830 0x7fd158075d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd170009eb0 tx=0x7fd1700ab040 comp rx=0 tx=0).stop 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 msgr2=0x7fd1781f9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 0x7fd1781f9f60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd1680046e0 tx=0x7fd16800c960 comp rx=0 tx=0).stop 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 shutdown_connections 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd158073830 0x7fd158075d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1781fa6f0 0x7fd1780b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1781f9b50 0x7fd1781f9f60 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 --2- 172.21.15.170:0/4023573773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd164003a10 0x7fd178165f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 >> 172.21.15.170:0/4023573773 conn(0x7fd1780d2a30 msgr2=0x7fd17815c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:23.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 shutdown_connections 2024-09-21T02:20:23.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.373+0000 7fd17e67e640 1 -- 172.21.15.170:0/4023573773 wait complete. 2024-09-21T02:20:23.403 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:20:23.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 -- 172.21.15.170:0/2271014361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3570003a60 msgr2=0x7f3570003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:23.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 --2- 172.21.15.170:0/2271014361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3570003a60 0x7f3570003ed0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3584072c50 tx=0x7f356c01cec0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 -- 172.21.15.170:0/2271014361 shutdown_connections 2024-09-21T02:20:23.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 --2- 172.21.15.170:0/2271014361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3570003a60 0x7f3570003ed0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 -- 172.21.15.170:0/2271014361 >> 172.21.15.170:0/2271014361 conn(0x7f35840b1c40 msgr2=0x7f35840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:23.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 -- 172.21.15.170:0/2271014361 shutdown_connections 2024-09-21T02:20:23.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 -- 172.21.15.170:0/2271014361 wait complete. 2024-09-21T02:20:23.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 Processor -- start 2024-09-21T02:20:23.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 -- start start 2024-09-21T02:20:23.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3570003a60 0x7f358415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:23.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.569+0000 7f358a92f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 0x7f358409d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:23.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f358409d960 0x7f358415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:23.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3584070460 con 0x7f358415e690 2024-09-21T02:20:23.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f35840702e0 con 0x7f3570003a60 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f35840705e0 con 0x7f358409d960 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 0x7f358409d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 0x7f358409d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46390/0 (socket says 172.21.15.170:46390) 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 -- 172.21.15.170:0/1473466995 learned_addr learned my addr 172.21.15.170:0/1473466995 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3588ecc640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f358409d960 0x7f358415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3583fff640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3570003a60 0x7f358415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 -- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f358409d960 msgr2=0x7f358415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f358409d960 0x7f358415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 -- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3570003a60 msgr2=0x7f358415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3570003a60 0x7f358415e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 -- 172.21.15.170:0/1473466995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35841fb7d0 con 0x7f358415e690 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f35837fe640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 0x7f358409d420 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f357400d7b0 tx=0x7f357400dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:23.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574016020 con 0x7f358415e690 2024-09-21T02:20:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f357400b6c0 con 0x7f358415e690 2024-09-21T02:20:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35841fbab0 con 0x7f358415e690 2024-09-21T02:20:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400baf0 con 0x7f358415e690 2024-09-21T02:20:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35841fc070 con 0x7f358415e690 2024-09-21T02:20:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358409df30 con 0x7f358415e690 2024-09-21T02:20:23.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f3574019020 con 0x7f358415e690 2024-09-21T02:20:23.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3580ff9640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3564073830 0x7f3564075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:23.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5823+0+0 (secure 0 0 0) 0x7f3574095240 con 0x7f358415e690 2024-09-21T02:20:23.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.573+0000 7f3583fff640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3564073830 0x7f3564075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:23.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.577+0000 7f3583fff640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3564073830 0x7f3564075d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f356c0a9000 tx=0x7f356c019d00 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:23.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.577+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3574020020 con 0x7f358415e690 2024-09-21T02:20:23.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:23.753+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35840b0850 con 0x7f358415e690 2024-09-21T02:20:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.381+0000 7f3580ff9640 1 -- 172.21.15.170:0/1473466995 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) ==== 181+0+0 (secure 0 0 0) 0x7f357405f8a0 con 0x7f358415e690 2024-09-21T02:20:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3564073830 msgr2=0x7f3564075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3564073830 0x7f3564075d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f356c0a9000 tx=0x7f356c019d00 comp rx=0 tx=0).stop 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 msgr2=0x7f358409d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 0x7f358409d420 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f357400d7b0 tx=0x7f357400dc70 comp rx=0 tx=0).stop 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 shutdown_connections 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3564073830 0x7f3564075d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f358409d960 0x7f358415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f358415e690 0x7f358409d420 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 --2- 172.21.15.170:0/1473466995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3570003a60 0x7f358415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 >> 172.21.15.170:0/1473466995 conn(0x7f35840b1c40 msgr2=0x7f358415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:24.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 shutdown_connections 2024-09-21T02:20:24.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.385+0000 7f358a92f640 1 -- 172.21.15.170:0/1473466995 wait complete. 2024-09-21T02:20:24.412 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 -- 172.21.15.170:0/1137417134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e8005360 msgr2=0x7f24e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 --2- 172.21.15.170:0/1137417134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e8005360 0x7f24e800b9d0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f2500074a90 tx=0x7f24f801cf20 comp rx=0 tx=0).stop 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 -- 172.21.15.170:0/1137417134 shutdown_connections 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 --2- 172.21.15.170:0/1137417134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e8005360 0x7f24e800b9d0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 -- 172.21.15.170:0/1137417134 >> 172.21.15.170:0/1137417134 conn(0x7f25000d0940 msgr2=0x7f25000d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 -- 172.21.15.170:0/1137417134 shutdown_connections 2024-09-21T02:20:24.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.577+0000 7f2505042640 1 -- 172.21.15.170:0/1137417134 wait complete. 2024-09-21T02:20:24.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 Processor -- start 2024-09-21T02:20:24.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 -- start start 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24e8005360 0x7f250014ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25000c64a0 0x7f25001f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 0x7f25000c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f25000cb320 con 0x7f25000c82a0 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25000cb1a0 con 0x7f25000c64a0 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f25000cb4a0 con 0x7f24e8005360 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 0x7f25000c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:24.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 0x7f25000c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46414/0 (socket says 172.21.15.170:46414) 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 -- 172.21.15.170:0/1014877913 learned_addr learned my addr 172.21.15.170:0/1014877913 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24fe59c640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25000c64a0 0x7f25001f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24fed9d640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24e8005360 0x7f250014ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 -- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24e8005360 msgr2=0x7f250014ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24e8005360 0x7f250014ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 -- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25000c64a0 msgr2=0x7f25001f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25000c64a0 0x7f25001f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 -- 172.21.15.170:0/1014877913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25000c7200 con 0x7f25000c82a0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24fe59c640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25000c64a0 0x7f25001f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24ff59e640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 0x7f25000c86b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f24f801d7c0 tx=0x7f24f8006cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f80060f0 con 0x7f25000c82a0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24f8001400 con 0x7f25000c82a0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25000c74e0 con 0x7f25000c82a0 2024-09-21T02:20:24.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8003600 con 0x7f25000c82a0 2024-09-21T02:20:24.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.581+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25000c7aa0 con 0x7f25000c82a0 2024-09-21T02:20:24.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.585+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000cba00 con 0x7f25000c82a0 2024-09-21T02:20:24.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.585+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f24f80037a0 con 0x7f25000c82a0 2024-09-21T02:20:24.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.585+0000 7f24e77fe640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f24ec073830 0x7f24ec075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:24.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.585+0000 7f24fed9d640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f24ec073830 0x7f24ec075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:24.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.585+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5385+0+0 (secure 0 0 0) 0x7f24f812eae0 con 0x7f25000c82a0 2024-09-21T02:20:24.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.585+0000 7f24fed9d640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f24ec073830 0x7f24ec075d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009db0 tx=0x7f24f0009380 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:24.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.589+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f24f8016cd0 con 0x7f25000c82a0 2024-09-21T02:20:24.746 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:20:24.749 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:20:24.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:24.765+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25000b4bc0 con 0x7f25000c82a0 2024-09-21T02:20:25.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.389+0000 7f24e77fe640 1 -- 172.21.15.170:0/1014877913 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) ==== 169+0+0 (secure 0 0 0) 0x7f24f80f92e0 con 0x7f25000c82a0 2024-09-21T02:20:25.391 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f24ec073830 msgr2=0x7f24ec075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f24ec073830 0x7f24ec075d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009db0 tx=0x7f24f0009380 comp rx=0 tx=0).stop 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 msgr2=0x7f25000c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 0x7f25000c86b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f24f801d7c0 tx=0x7f24f8006cd0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 shutdown_connections 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f24ec073830 0x7f24ec075d40 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009db0 tx=0x7f24f0009380 comp rx=0 tx=0).stop 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25000c82a0 0x7f25000c86b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25000c64a0 0x7f25001f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 --2- 172.21.15.170:0/1014877913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24e8005360 0x7f250014ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.393+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 >> 172.21.15.170:0/1014877913 conn(0x7f25000d0940 msgr2=0x7f25000b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:25.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.397+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 shutdown_connections 2024-09-21T02:20:25.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.397+0000 7f2505042640 1 -- 172.21.15.170:0/1014877913 wait complete. 2024-09-21T02:20:25.422 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:20:25.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 -- 172.21.15.170:0/1363214708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 msgr2=0x7f91b00cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 --2- 172.21.15.170:0/1363214708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b00cca40 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f91b00b4ef0 tx=0x7f91a001c890 comp rx=0 tx=0).stop 2024-09-21T02:20:25.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 -- 172.21.15.170:0/1363214708 shutdown_connections 2024-09-21T02:20:25.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 --2- 172.21.15.170:0/1363214708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b00ccf80 0x7f91b00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 --2- 172.21.15.170:0/1363214708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b00cca40 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 -- 172.21.15.170:0/1363214708 >> 172.21.15.170:0/1363214708 conn(0x7f91b00711c0 msgr2=0x7f91b00715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:25.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 -- 172.21.15.170:0/1363214708 shutdown_connections 2024-09-21T02:20:25.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 -- 172.21.15.170:0/1363214708 wait complete. 2024-09-21T02:20:25.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 Processor -- start 2024-09-21T02:20:25.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.577+0000 7f91b5979640 1 -- start start 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b00ccf80 0x7f91b0161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b01422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b0160f60 0x7f91b0154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0169f10 con 0x7f91b0160b50 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0169d90 con 0x7f91b00ccf80 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f91b016a090 con 0x7f91b0160f60 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b01422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b01422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56462/0 (socket says 172.21.15.170:56462) 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 -- 172.21.15.170:0/221824149 learned_addr learned my addr 172.21.15.170:0/221824149 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91af7fe640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b0160f60 0x7f91b0154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91aeffd640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b00ccf80 0x7f91b0161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 -- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b0160f60 msgr2=0x7f91b0154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b0160f60 0x7f91b0154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 -- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b00ccf80 msgr2=0x7f91b0161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b00ccf80 0x7f91b0161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 -- 172.21.15.170:0/221824149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b0151fa0 con 0x7f91b0160b50 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91af7fe640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b0160f60 0x7f91b0154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91ae7fc640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b01422d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f91a400c940 tx=0x7f91a400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004220 con 0x7f91b0160b50 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91a40079b0 con 0x7f91b0160b50 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a40044b0 con 0x7f91b0160b50 2024-09-21T02:20:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b0150f00 con 0x7f91b0160b50 2024-09-21T02:20:25.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b01514c0 con 0x7f91b0160b50 2024-09-21T02:20:25.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b00cec70 con 0x7f91b0160b50 2024-09-21T02:20:25.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.581+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f91a4003600 con 0x7f91b0160b50 2024-09-21T02:20:25.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.585+0000 7f9197fff640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f919c073830 0x7f919c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.585+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7f91a40953a0 con 0x7f91b0160b50 2024-09-21T02:20:25.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.585+0000 7f91aeffd640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f919c073830 0x7f919c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.585+0000 7f91aeffd640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f919c073830 0x7f919c075d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91a00af000 tx=0x7f91a001b0d0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:25.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.589+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f91a4014030 con 0x7f91b0160b50 2024-09-21T02:20:25.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.769+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f91b00b5dd0 con 0x7f91b0160b50 2024-09-21T02:20:25.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.773+0000 7f9197fff640 1 -- 172.21.15.170:0/221824149 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7f91a405fd50 con 0x7f91b0160b50 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f919c073830 msgr2=0x7f919c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f919c073830 0x7f919c075d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f91a00af000 tx=0x7f91a001b0d0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 msgr2=0x7f91b01422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b01422d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f91a400c940 tx=0x7f91a400ce00 comp rx=0 tx=0).stop 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 shutdown_connections 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f919c073830 0x7f919c075d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b0160f60 0x7f91b0154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b0160b50 0x7f91b01422d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 --2- 172.21.15.170:0/221824149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b00ccf80 0x7f91b0161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 >> 172.21.15.170:0/221824149 conn(0x7f91b00711c0 msgr2=0x7f91b00b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:25.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 shutdown_connections 2024-09-21T02:20:25.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.777+0000 7f91b5979640 1 -- 172.21.15.170:0/221824149 wait complete. 2024-09-21T02:20:25.797 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-21T02:20:25.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 -- 172.21.15.170:0/207089850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 msgr2=0x7fee7c0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 --2- 172.21.15.170:0/207089850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c0b3350 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0b3a10 tx=0x7fee7001c820 comp rx=0 tx=0).stop 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 -- 172.21.15.170:0/207089850 shutdown_connections 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 --2- 172.21.15.170:0/207089850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c0b3350 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 --2- 172.21.15.170:0/207089850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0b5410 0x7fee7c06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 -- 172.21.15.170:0/207089850 >> 172.21.15.170:0/207089850 conn(0x7fee7c0b1c40 msgr2=0x7fee7c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 -- 172.21.15.170:0/207089850 shutdown_connections 2024-09-21T02:20:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 -- 172.21.15.170:0/207089850 wait complete. 2024-09-21T02:20:25.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 Processor -- start 2024-09-21T02:20:25.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.941+0000 7fee82d38640 1 -- start start 2024-09-21T02:20:25.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c0b5410 0x7fee7c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0ccb00 0x7fee7c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c16a060 con 0x7fee7c0b2f40 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c0701b0 con 0x7fee7c0b5410 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c16a1e0 con 0x7fee7c0ccb00 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56474/0 (socket says 172.21.15.170:56474) 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 -- 172.21.15.170:0/68984056 learned_addr learned my addr 172.21.15.170:0/68984056 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee812d5640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0ccb00 0x7fee7c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee7bfff640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c0b5410 0x7fee7c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 -- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0ccb00 msgr2=0x7fee7c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0ccb00 0x7fee7c0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 -- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c0b5410 msgr2=0x7fee7c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c0b5410 0x7fee7c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 -- 172.21.15.170:0/68984056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c1575f0 con 0x7fee7c0b2f40 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee812d5640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0ccb00 0x7fee7c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee80ad4640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c156de0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00b440 tx=0x7fee6c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c015020 con 0x7fee7c0b2f40 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee6c0119a0 con 0x7fee7c0b2f40 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c157320 con 0x7fee7c0b2f40 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c011df0 con 0x7fee7c0b2f40 2024-09-21T02:20:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c0c8940 con 0x7fee7c0b2f40 2024-09-21T02:20:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c0b3350 con 0x7fee7c0b2f40 2024-09-21T02:20:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fee6c018020 con 0x7fee7c0b2f40 2024-09-21T02:20:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.945+0000 7fee797fa640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee68073830 0x7fee68075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.949+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7fee6c095070 con 0x7fee7c0b2f40 2024-09-21T02:20:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.949+0000 7fee7bfff640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee68073830 0x7fee68075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.949+0000 7fee7bfff640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee68073830 0x7fee68075d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee700af000 tx=0x7fee7001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:25.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:25.949+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fee6c01f020 con 0x7fee7c0b2f40 2024-09-21T02:20:26.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.189+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fee7c0c6920 con 0x7fee7c0b2f40 2024-09-21T02:20:26.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.393+0000 7fee797fa640 1 -- 172.21.15.170:0/68984056 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v371) ==== 148+0+0 (secure 0 0 0) 0x7fee7c1575f0 con 0x7fee7c0b2f40 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee68073830 msgr2=0x7fee68075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee68073830 0x7fee68075d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fee700af000 tx=0x7fee7001b060 comp rx=0 tx=0).stop 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 msgr2=0x7fee7c156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c156de0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00b440 tx=0x7fee6c00b900 comp rx=0 tx=0).stop 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 shutdown_connections 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fee68073830 0x7fee68075d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fee7c0ccb00 0x7fee7c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee7c0b5410 0x7fee7c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 --2- 172.21.15.170:0/68984056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee7c0b2f40 0x7fee7c156de0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 >> 172.21.15.170:0/68984056 conn(0x7fee7c0b1c40 msgr2=0x7fee7c0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:26.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 shutdown_connections 2024-09-21T02:20:26.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.397+0000 7fee82d38640 1 -- 172.21.15.170:0/68984056 wait complete. 2024-09-21T02:20:26.421 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-21T02:20:26.421 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:20:26.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 -- 172.21.15.170:0/1332431228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22680b5410 msgr2=0x7f226806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 --2- 172.21.15.170:0/1332431228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22680b5410 0x7f226806fa30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f22680b3a10 tx=0x7f226401c820 comp rx=0 tx=0).stop 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 -- 172.21.15.170:0/1332431228 shutdown_connections 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 --2- 172.21.15.170:0/1332431228 >> v2:172.21.15.191:3300/0 conn(0x7f22680cd210 0x7f22680b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 --2- 172.21.15.170:0/1332431228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22680b5410 0x7f226806fa30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 -- 172.21.15.170:0/1332431228 >> 172.21.15.170:0/1332431228 conn(0x7f22680b1c40 msgr2=0x7f22680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 -- 172.21.15.170:0/1332431228 shutdown_connections 2024-09-21T02:20:26.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.581+0000 7f226f35d640 1 -- 172.21.15.170:0/1332431228 wait complete. 2024-09-21T02:20:26.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 Processor -- start 2024-09-21T02:20:26.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 -- start start 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 0x7f2268159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f226809d520 0x7f22680c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22680c92b0 0x7f226815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2268169fd0 con 0x7f22680b2f40 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2268169e50 con 0x7f22680c92b0 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f226816a150 con 0x7f226809d520 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 0x7f2268159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 0x7f2268159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56480/0 (socket says 172.21.15.170:56480) 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 -- 172.21.15.170:0/1770499625 learned_addr learned my addr 172.21.15.170:0/1770499625 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d8fa640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22680c92b0 0x7f226815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226c8f8640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f226809d520 0x7f22680c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 -- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f226809d520 msgr2=0x7f22680c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f226809d520 0x7f22680c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 -- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22680c92b0 msgr2=0x7f226815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22680c92b0 0x7f226815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 -- 172.21.15.170:0/1770499625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22680c96c0 con 0x7f22680b2f40 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226c8f8640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f226809d520 0x7f22680c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226d0f9640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 0x7f2268159690 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c10 tx=0x7f22640a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640bc070 con 0x7f22680b2f40 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22681f67a0 con 0x7f22680b2f40 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22640a5c50 con 0x7f22680b2f40 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640b2040 con 0x7f22680b2f40 2024-09-21T02:20:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.585+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681f6d60 con 0x7f22680b2f40 2024-09-21T02:20:26.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.589+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226806fa30 con 0x7f22680b2f40 2024-09-21T02:20:26.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.589+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f2264003600 con 0x7f22680b2f40 2024-09-21T02:20:26.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.589+0000 7f225dffb640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f224c073830 0x7f224c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.589+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7f226412e5a0 con 0x7f22680b2f40 2024-09-21T02:20:26.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.589+0000 7f226c8f8640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f224c073830 0x7f224c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.589+0000 7f226c8f8640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f224c073830 0x7f224c075d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2258000f70 tx=0x7f2258009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:26.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.593+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f22680c96c0 con 0x7f22680b2f40 2024-09-21T02:20:26.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.753+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f226809b450 con 0x7f22680b2f40 2024-09-21T02:20:26.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.753+0000 7f225dffb640 1 -- 172.21.15.170:0/1770499625 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+13134 (secure 0 0 0) 0x7f2264016410 con 0x7f22680b2f40 2024-09-21T02:20:26.755 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:26.755 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":122,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:20:25.383994+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6835/300226457":"2024-09-22T02:20:22.367109+0000","172.21.15.170:6834/300226457":"2024-09-22T02:20:22.367109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:26.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f224c073830 msgr2=0x7f224c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f224c073830 0x7f224c075d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2258000f70 tx=0x7f2258009040 comp rx=0 tx=0).stop 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 msgr2=0x7f2268159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 0x7f2268159690 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c10 tx=0x7f22640a55f0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 shutdown_connections 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f224c073830 0x7f224c075d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f226809d520 0x7f22680c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22680c92b0 0x7f226815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 --2- 172.21.15.170:0/1770499625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22680b2f40 0x7f2268159690 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 >> 172.21.15.170:0/1770499625 conn(0x7f22680b1c40 msgr2=0x7f22680c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 shutdown_connections 2024-09-21T02:20:26.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.757+0000 7f226f35d640 1 -- 172.21.15.170:0/1770499625 wait complete. 2024-09-21T02:20:26.777 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:20:26.777 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:20:26.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 -- 172.21.15.170:0/402665670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 msgr2=0x7f6f200ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 --2- 172.21.15.170:0/402665670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f200ce070 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800ad90 tx=0x7f6f1801dc20 comp rx=0 tx=0).stop 2024-09-21T02:20:26.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 -- 172.21.15.170:0/402665670 shutdown_connections 2024-09-21T02:20:26.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 --2- 172.21.15.170:0/402665670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f200ce070 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 -- 172.21.15.170:0/402665670 >> 172.21.15.170:0/402665670 conn(0x7f6f200d0170 msgr2=0x7f6f200d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:26.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 -- 172.21.15.170:0/402665670 shutdown_connections 2024-09-21T02:20:26.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.929+0000 7f6f26484640 1 -- 172.21.15.170:0/402665670 wait complete. 2024-09-21T02:20:26.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 Processor -- start 2024-09-21T02:20:26.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 -- start start 2024-09-21T02:20:26.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f200700b0 0x7f6f201629f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f201542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f200cb790 0x7f6f201fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20074560 con 0x7f6f200cb380 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6f200743e0 con 0x7f6f200cb790 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f26484640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6f200746e0 con 0x7f6f200700b0 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f1f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f201542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f1f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f201542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56506/0 (socket says 172.21.15.170:56506) 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.933+0000 7f6f1f7fe640 1 -- 172.21.15.170:0/1586463027 learned_addr learned my addr 172.21.15.170:0/1586463027 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f24a21640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f200cb790 0x7f6f201fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1ffff640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f200700b0 0x7f6f201629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1f7fe640 1 -- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f200700b0 msgr2=0x7f6f201629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1f7fe640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f200700b0 0x7f6f201629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1f7fe640 1 -- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f200cb790 msgr2=0x7f6f201fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1f7fe640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f200cb790 0x7f6f201fa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1f7fe640 1 -- 172.21.15.170:0/1586463027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f201fb220 con 0x7f6f200cb380 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1f7fe640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f201542f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c3c0 tx=0x7f6f0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08017070 con 0x7f6f200cb380 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f080053c0 con 0x7f6f200cb380 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2009b3e0 con 0x7f6f200cb380 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08003860 con 0x7f6f200cb380 2024-09-21T02:20:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f20154980 con 0x7f6f200cb380 2024-09-21T02:20:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f201faf90 con 0x7f6f200cb380 2024-09-21T02:20:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f6f0801a070 con 0x7f6f200cb380 2024-09-21T02:20:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1cff9640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6f00073860 0x7f6f00075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7f6f080941e0 con 0x7f6f200cb380 2024-09-21T02:20:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1ffff640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6f00073860 0x7f6f00075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.937+0000 7f6f1ffff640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6f00073860 0x7f6f00075d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f180b0000 tx=0x7f6f1801c460 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:26.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:26.941+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6f08021020 con 0x7f6f200cb380 2024-09-21T02:20:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.109+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f20154ca0 con 0x7f6f200cb380 2024-09-21T02:20:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f1cff9640 1 -- 172.21.15.170:0/1586463027 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) ==== 93+0+10 (secure 0 0 0) 0x7f6f0805ec20 con 0x7f6f200cb380 2024-09-21T02:20:27.112 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6f00073860 msgr2=0x7f6f00075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6f00073860 0x7f6f00075d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f180b0000 tx=0x7f6f1801c460 comp rx=0 tx=0).stop 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 msgr2=0x7f6f201542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f201542f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c3c0 tx=0x7f6f0800c880 comp rx=0 tx=0).stop 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 shutdown_connections 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6f00073860 0x7f6f00075d70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f200cb790 0x7f6f201fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f200cb380 0x7f6f201542f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 --2- 172.21.15.170:0/1586463027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f200700b0 0x7f6f201629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 >> 172.21.15.170:0/1586463027 conn(0x7f6f200d0170 msgr2=0x7f6f200ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 shutdown_connections 2024-09-21T02:20:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.113+0000 7f6f26484640 1 -- 172.21.15.170:0/1586463027 wait complete. 2024-09-21T02:20:27.134 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-21T02:20:27.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 -- 172.21.15.170:0/2025227299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 msgr2=0x7fa9a8070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2025227299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8070070 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa998009840 tx=0x7fa99801c790 comp rx=0 tx=0).stop 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 -- 172.21.15.170:0/2025227299 shutdown_connections 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2025227299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8070070 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2025227299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 0x7fa9a80cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 -- 172.21.15.170:0/2025227299 >> 172.21.15.170:0/2025227299 conn(0x7fa9a80b25c0 msgr2=0x7fa9a80b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 -- 172.21.15.170:0/2025227299 shutdown_connections 2024-09-21T02:20:27.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.285+0000 7fa9ada46640 1 -- 172.21.15.170:0/2025227299 wait complete. 2024-09-21T02:20:27.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 Processor -- start 2024-09-21T02:20:27.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 -- start start 2024-09-21T02:20:27.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:27.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a80cb830 0x7fa9a8154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:27.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 0x7fa9a81f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8169f10 con 0x7fa9a806fc60 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80707a0 con 0x7fa9a80cb830 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a816a090 con 0x7fa9a80cd190 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56514/0 (socket says 172.21.15.170:56514) 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 -- 172.21.15.170:0/2638034759 learned_addr learned my addr 172.21.15.170:0/2638034759 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a67fc640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a80cb830 0x7fa9a8154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a77fe640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 0x7fa9a81f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 -- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 msgr2=0x7fa9a81f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 0x7fa9a81f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 -- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a80cb830 msgr2=0x7fa9a8154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:27.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a80cb830 0x7fa9a8154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:27.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 -- 172.21.15.170:0/2638034759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a81558a0 con 0x7fa9a806fc60 2024-09-21T02:20:27.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a77fe640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 0x7fa9a81f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:27.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9a6ffd640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8154fa0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000cbf0 tx=0x7fa9a000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:27.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0016020 con 0x7fa9a806fc60 2024-09-21T02:20:27.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9a000ab40 con 0x7fa9a806fc60 2024-09-21T02:20:27.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0012420 con 0x7fa9a806fc60 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a8153a50 con 0x7fa9a806fc60 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.289+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a8154010 con 0x7fa9a806fc60 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.293+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a8070070 con 0x7fa9a806fc60 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.293+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fa9a0019020 con 0x7fa9a806fc60 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.293+0000 7fa98ffff640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa994073830 0x7fa994075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.293+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7fa9a00948f0 con 0x7fa9a806fc60 2024-09-21T02:20:27.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.293+0000 7fa9a67fc640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa994073830 0x7fa994075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:27.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.293+0000 7fa9a67fc640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa994073830 0x7fa994075d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0058d0 tx=0x7fa99c005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:27.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.297+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9a0020070 con 0x7fa9a806fc60 2024-09-21T02:20:27.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:27.533+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fa9a80c40c0 con 0x7fa9a806fc60 2024-09-21T02:20:28.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.405+0000 7fa98ffff640 1 -- 172.21.15.170:0/2638034759 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v373) ==== 154+0+0 (secure 0 0 0) 0x7fa9a005f1a0 con 0x7fa9a806fc60 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa994073830 msgr2=0x7fa994075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa994073830 0x7fa994075d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0058d0 tx=0x7fa99c005860 comp rx=0 tx=0).stop 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 msgr2=0x7fa9a8154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8154fa0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000cbf0 tx=0x7fa9a000cc20 comp rx=0 tx=0).stop 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 shutdown_connections 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa994073830 0x7fa994075d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9a80cd190 0x7fa9a81f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9a80cb830 0x7fa9a8154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 --2- 172.21.15.170:0/2638034759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9a806fc60 0x7fa9a8154fa0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 >> 172.21.15.170:0/2638034759 conn(0x7fa9a80b25c0 msgr2=0x7fa9a80b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 shutdown_connections 2024-09-21T02:20:28.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.409+0000 7fa9ada46640 1 -- 172.21.15.170:0/2638034759 wait complete. 2024-09-21T02:20:28.435 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:20:28.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 -- 172.21.15.170:0/3293531191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 msgr2=0x7f2c000cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 --2- 172.21.15.170:0/3293531191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c000cb870 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009900 tx=0x7f2bf001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 -- 172.21.15.170:0/3293531191 shutdown_connections 2024-09-21T02:20:28.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 --2- 172.21.15.170:0/3293531191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c000cb870 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 --2- 172.21.15.170:0/3293531191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c000cf7e0 0x7f2c00070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 -- 172.21.15.170:0/3293531191 >> 172.21.15.170:0/3293531191 conn(0x7f2c000b1c40 msgr2=0x7f2c000b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 -- 172.21.15.170:0/3293531191 shutdown_connections 2024-09-21T02:20:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.593+0000 7f2c084d4640 1 -- 172.21.15.170:0/3293531191 wait complete. 2024-09-21T02:20:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 Processor -- start 2024-09-21T02:20:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- start start 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c001529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c000cc8c0 0x7f2c0015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c000cf7e0 0x7f2c0015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00169180 con 0x7f2c000cb460 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00169000 con 0x7f2c000cc8c0 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2c00169300 con 0x7f2c000cf7e0 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c001529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c001529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56540/0 (socket says 172.21.15.170:56540) 2024-09-21T02:20:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 -- 172.21.15.170:0/1779861303 learned_addr learned my addr 172.21.15.170:0/1779861303 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c05a6f640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c000cc8c0 0x7f2c0015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06a71640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c000cf7e0 0x7f2c0015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 -- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c000cf7e0 msgr2=0x7f2c0015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c000cf7e0 0x7f2c0015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 -- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c000cc8c0 msgr2=0x7f2c0015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c000cc8c0 0x7f2c0015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 -- 172.21.15.170:0/1779861303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c00152f10 con 0x7f2c000cb460 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c06270640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c001529d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00c970 tx=0x7f2bfc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc004220 con 0x7f2c000cb460 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c00156d00 con 0x7f2c000cb460 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bfc007950 con 0x7f2c000cb460 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0044b0 con 0x7f2c000cb460 2024-09-21T02:20:28.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001572c0 con 0x7f2c000cb460 2024-09-21T02:20:28.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.597+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c000cb870 con 0x7f2c000cb460 2024-09-21T02:20:28.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.601+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f2bfc005020 con 0x7f2c000cb460 2024-09-21T02:20:28.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.601+0000 7f2bf6ffd640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2be4073860 0x7f2be4075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.601+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7f2bfc095110 con 0x7f2c000cb460 2024-09-21T02:20:28.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.601+0000 7f2c05a6f640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2be4073860 0x7f2be4075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.601+0000 7f2c05a6f640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2be4073860 0x7f2be4075d70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00af000 tx=0x7f2bf001b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:28.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.605+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2bfc014030 con 0x7f2c000cb460 2024-09-21T02:20:28.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.761+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2c00070640 con 0x7f2c000cb460 2024-09-21T02:20:28.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2bf6ffd640 1 -- 172.21.15.170:0/1779861303 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v45) ==== 59+0+2294 (secure 0 0 0) 0x7f2bfc05f9c0 con 0x7f2c000cb460 2024-09-21T02:20:28.764 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:20:28.764 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:20:28.765 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:20:28.766 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:20:28.767 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2be4073860 msgr2=0x7f2be4075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2be4073860 0x7f2be4075d70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00af000 tx=0x7f2bf001b120 comp rx=0 tx=0).stop 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 msgr2=0x7f2c001529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c001529d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00c970 tx=0x7f2bfc00ce30 comp rx=0 tx=0).stop 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 shutdown_connections 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2be4073860 0x7f2be4075d70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c000cf7e0 0x7f2c0015ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c000cc8c0 0x7f2c0015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 --2- 172.21.15.170:0/1779861303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c000cb460 0x7f2c001529d0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 >> 172.21.15.170:0/1779861303 conn(0x7f2c000b1c40 msgr2=0x7f2c000b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 shutdown_connections 2024-09-21T02:20:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.765+0000 7f2c084d4640 1 -- 172.21.15.170:0/1779861303 wait complete. 2024-09-21T02:20:28.785 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 -- 172.21.15.170:0/2242499841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cccb0 msgr2=0x7fdb5c15fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 --2- 172.21.15.170:0/2242499841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cccb0 0x7fdb5c15fb80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c073a30 tx=0x7fdb5401ccf0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 -- 172.21.15.170:0/2242499841 shutdown_connections 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 --2- 172.21.15.170:0/2242499841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cccb0 0x7fdb5c15fb80 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 --2- 172.21.15.170:0/2242499841 >> v2:172.21.15.191:3300/0 conn(0x7fdb5c0b04e0 0x7fdb5c0b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 -- 172.21.15.170:0/2242499841 >> 172.21.15.170:0/2242499841 conn(0x7fdb5c09a9c0 msgr2=0x7fdb5c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 -- 172.21.15.170:0/2242499841 shutdown_connections 2024-09-21T02:20:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 -- 172.21.15.170:0/2242499841 wait complete. 2024-09-21T02:20:28.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 Processor -- start 2024-09-21T02:20:28.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.945+0000 7fdb61817640 1 -- start start 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c0b04e0 0x7fdb5c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 0x7fdb5c1f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c0cccb0 0x7fdb5c154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c168fc0 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c0d1d60 con 0x7fdb5c0cccb0 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c169140 con 0x7fdb5c0b04e0 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 0x7fdb5c1f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 0x7fdb5c1f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56564/0 (socket says 172.21.15.170:56564) 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 -- 172.21.15.170:0/3370383767 learned_addr learned my addr 172.21.15.170:0/3370383767 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 -- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c0b04e0 msgr2=0x7fdb5c15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5affd640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c0b04e0 0x7fdb5c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5b7fe640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c0cccb0 0x7fdb5c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c0b04e0 0x7fdb5c15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 -- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c0cccb0 msgr2=0x7fdb5c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c0cccb0 0x7fdb5c154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 -- 172.21.15.170:0/3370383767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c15c480 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb5a7fc640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 0x7fdb5c1f6ee0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdb50001ec0 tx=0x7fdb5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50017070 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb50003600 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50003a50 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c1f7740 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.949+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1f7d00 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.953+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c0b0b60 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.953+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fdb5001a070 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.953+0000 7fdb43fff640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdb48073830 0x7fdb48075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.953+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7fdb50093ed0 con 0x7fdb5c0cc8a0 2024-09-21T02:20:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.953+0000 7fdb5affd640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdb48073830 0x7fdb48075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.953+0000 7fdb5affd640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdb48073830 0x7fdb48075d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b9a0 tx=0x7fdb4c005c20 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:28.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:28.957+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdb5005e780 con 0x7fdb5c0cc8a0 2024-09-21T02:20:29.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.133+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdb5c0afea0 con 0x7fdb5c0cc8a0 2024-09-21T02:20:29.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.137+0000 7fdb43fff640 1 -- 172.21.15.170:0/3370383767 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fdb5005e080 con 0x7fdb5c0cc8a0 2024-09-21T02:20:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdb48073830 msgr2=0x7fdb48075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdb48073830 0x7fdb48075d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00b9a0 tx=0x7fdb4c005c20 comp rx=0 tx=0).stop 2024-09-21T02:20:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 msgr2=0x7fdb5c1f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 0x7fdb5c1f6ee0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdb50001ec0 tx=0x7fdb5000c890 comp rx=0 tx=0).stop 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 shutdown_connections 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdb48073830 0x7fdb48075d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb5c0cccb0 0x7fdb5c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb5c0cc8a0 0x7fdb5c1f6ee0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 --2- 172.21.15.170:0/3370383767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb5c0b04e0 0x7fdb5c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 >> 172.21.15.170:0/3370383767 conn(0x7fdb5c09a9c0 msgr2=0x7fdb5c0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 shutdown_connections 2024-09-21T02:20:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.141+0000 7fdb61817640 1 -- 172.21.15.170:0/3370383767 wait complete. 2024-09-21T02:20:29.161 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:20:29.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:20:29.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:20:29.332 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:20:29.332 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:29.332 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:29.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:20:29.338 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:29.338 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:29.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:20:29.344 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:29.344 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:29.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:20:29.385 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:29.385 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:29.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:20:29.395 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:20:29.395 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 -- 172.21.15.170:0/2520119963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 msgr2=0x7f28200cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 --2- 172.21.15.170:0/2520119963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f28200cb200 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f28200b2d00 tx=0x7f281001d9e0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 -- 172.21.15.170:0/2520119963 shutdown_connections 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 --2- 172.21.15.170:0/2520119963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f28200cb200 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 --2- 172.21.15.170:0/2520119963 >> v2:172.21.15.191:3300/0 conn(0x7f282015f990 0x7f2820070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 -- 172.21.15.170:0/2520119963 >> 172.21.15.170:0/2520119963 conn(0x7f28200b1c40 msgr2=0x7f28200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 -- 172.21.15.170:0/2520119963 shutdown_connections 2024-09-21T02:20:29.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.593+0000 7f2826dbe640 1 -- 172.21.15.170:0/2520119963 wait complete. 2024-09-21T02:20:29.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 Processor -- start 2024-09-21T02:20:29.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 -- start start 2024-09-21T02:20:29.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2820070570 0x7f28200c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:29.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f282009d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:29.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282015f990 0x7f28201f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:29.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2820167ff0 con 0x7f28200cadf0 2024-09-21T02:20:29.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f28200cede0 con 0x7f282015f990 2024-09-21T02:20:29.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2820168170 con 0x7f2820070570 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f282009d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f282009d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56582/0 (socket says 172.21.15.170:56582) 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 -- 172.21.15.170:0/1123273674 learned_addr learned my addr 172.21.15.170:0/1123273674 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f282535b640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282015f990 0x7f28201f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 -- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2820070570 msgr2=0x7f28200c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2824b5a640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2820070570 0x7f28200c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2820070570 0x7f28200c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 -- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282015f990 msgr2=0x7f28201f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282015f990 0x7f28201f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 -- 172.21.15.170:0/1123273674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28201596b0 con 0x7f28200cadf0 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f282535b640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282015f990 0x7f28201f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281ffff640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f282009d250 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f281800bab0 tx=0x7f281800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818019070 con 0x7f28200cadf0 2024-09-21T02:20:29.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f281800cd80 con 0x7f28200cadf0 2024-09-21T02:20:29.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28180146a0 con 0x7f28200cadf0 2024-09-21T02:20:29.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2820159990 con 0x7f28200cadf0 2024-09-21T02:20:29.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.597+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282015d0f0 con 0x7f28200cadf0 2024-09-21T02:20:29.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.601+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200cb200 con 0x7f28200cadf0 2024-09-21T02:20:29.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.601+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f281801c070 con 0x7f28200cadf0 2024-09-21T02:20:29.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.601+0000 7f281d7fa640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2800073830 0x7f2800075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:29.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.601+0000 7f2824b5a640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2800073830 0x7f2800075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:29.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.601+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7f2818094970 con 0x7f28200cadf0 2024-09-21T02:20:29.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.601+0000 7f2824b5a640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2800073830 0x7f2800075d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28080058d0 tx=0x7f2808005bd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.605+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f281805f270 con 0x7f28200cadf0 2024-09-21T02:20:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28200ce2a0 con 0x7f28200cadf0 2024-09-21T02:20:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f281d7fa640 1 -- 172.21.15.170:0/1123273674 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+3917 (secure 0 0 0) 0x7f281805eb70 con 0x7f28200cadf0 2024-09-21T02:20:29.809 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:29.809 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":84,"btime":"2024-09-21T02:20:23:369719+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/722595187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":722595187},{"type":"v1","addr":"172.21.15.191:6835","nonce":722595187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6376,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1527764480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1527764480},{"type":"v1","addr":"172.21.15.191:6834","nonce":1527764480}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7611,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/990793464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":990793464},{"type":"v1","addr":"172.21.15.170:6833","nonce":990793464}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7959,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2960692994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2960692994},{"type":"v1","addr":"172.21.15.170:6835","nonce":2960692994}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[]} 2024-09-21T02:20:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 84 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2800073830 msgr2=0x7f2800075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2800073830 0x7f2800075d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28080058d0 tx=0x7f2808005bd0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 msgr2=0x7f282009d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f282009d250 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f281800bab0 tx=0x7f281800bf70 comp rx=0 tx=0).stop 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 shutdown_connections 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2800073830 0x7f2800075d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f282015f990 0x7f28201f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28200cadf0 0x7f282009d250 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 --2- 172.21.15.170:0/1123273674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2820070570 0x7f28200c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.809+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 >> 172.21.15.170:0/1123273674 conn(0x7f28200b1c40 msgr2=0x7f28200b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.813+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 shutdown_connections 2024-09-21T02:20:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:29.813+0000 7f2826dbe640 1 -- 172.21.15.170:0/1123273674 wait complete. 2024-09-21T02:20:29.830 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:20:29.830 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:20:29.831 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:20:29.831 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:20:29.831 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:20:29.831 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:20:29.831 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:20:29.831 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:20:29.831 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:20:29.832 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:20:29.832 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:20:29.832 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:20:29.832 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:20:29.832 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:20:29.832 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:20:29.832 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:20:29.845 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:20:29.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:20:29.846 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:20:29.846 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:20:29.854 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:20:29.855 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:20:29.856 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:20:29.856 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:20:29.889 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:20:29.889 DEBUG:teuthology.parallel:result is None 2024-09-21T02:20:29.890 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:20:29.890 DEBUG:teuthology.parallel:result is None 2024-09-21T02:20:29.899 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:20:29.899 DEBUG:teuthology.parallel:result is None 2024-09-21T02:20:29.900 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:20:29.900 DEBUG:teuthology.parallel:result is None 2024-09-21T02:20:29.900 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:20:29.977 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:20:29.987 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:20:30.000 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:20:30.000 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:20:30.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 -- 172.21.15.170:0/1936296764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 msgr2=0x7f5d5c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 --2- 172.21.15.170:0/1936296764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d5c003e80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c2e0 tx=0x7f5d6c01cfa0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d76a6d640 1 -- 172.21.15.170:0/1936296764 reap_dead start 2024-09-21T02:20:30.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 -- 172.21.15.170:0/1936296764 shutdown_connections 2024-09-21T02:20:30.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 -- 172.21.15.170:0/1936296764 >> 172.21.15.170:0/1936296764 conn(0x7f5d7008bf50 msgr2=0x7f5d7009d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:30.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 -- 172.21.15.170:0/1936296764 shutdown_connections 2024-09-21T02:20:30.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 -- 172.21.15.170:0/1936296764 wait complete. 2024-09-21T02:20:30.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 Processor -- start 2024-09-21T02:20:30.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.045+0000 7f5d784d0640 1 -- start start 2024-09-21T02:20:30.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d70154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d70155010 0x7f5d701676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d700b63b0 0x7f5d70167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70169e40 con 0x7f5d5c003a10 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70169cc0 con 0x7f5d700b63b0 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5d70169fc0 con 0x7f5d70155010 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d70154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d70154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56632/0 (socket says 172.21.15.170:56632) 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 -- 172.21.15.170:0/3371118912 learned_addr learned my addr 172.21.15.170:0/3371118912 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d75a6b640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d70155010 0x7f5d701676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d76a6d640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d700b63b0 0x7f5d70167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 -- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d70155010 msgr2=0x7f5d701676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d70155010 0x7f5d701676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 -- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d700b63b0 msgr2=0x7f5d70167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d700b63b0 0x7f5d70167c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 -- 172.21.15.170:0/3371118912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d701fc520 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d7626c640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d70154750 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c01b6a0 tx=0x7f5d6c001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c001540 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d6c001700 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d701fc800 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0064e0 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d701fcdc0 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d7006e080 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f5d6c003600 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d66ffd640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d54073860 0x7f5d54075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 4971+0+0 (secure 0 0 0) 0x7f5d6c12e2c0 con 0x7f5d5c003a10 2024-09-21T02:20:30.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.049+0000 7f5d75a6b640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d54073860 0x7f5d54075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.053+0000 7f5d75a6b640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d54073860 0x7f5d54075d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d700ce990 tx=0x7f5d600091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:30.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.053+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5d6c0be070 con 0x7f5d5c003a10 2024-09-21T02:20:30.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.225+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5d700b1640 con 0x7f5d5c003a10 2024-09-21T02:20:30.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.437+0000 7f5d66ffd640 1 -- 172.21.15.170:0/3371118912 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v124) ==== 97+0+0 (secure 0 0 0) 0x7f5d6c016d50 con 0x7f5d5c003a10 2024-09-21T02:20:30.438 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:20:30.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d54073860 msgr2=0x7f5d54075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d54073860 0x7f5d54075d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d700ce990 tx=0x7f5d600091e0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 msgr2=0x7f5d70154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d70154750 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c01b6a0 tx=0x7f5d6c001d10 comp rx=0 tx=0).stop 2024-09-21T02:20:30.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 shutdown_connections 2024-09-21T02:20:30.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d54073860 0x7f5d54075d70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d70155010 0x7f5d701676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d700b63b0 0x7f5d70167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 --2- 172.21.15.170:0/3371118912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d5c003a10 0x7f5d70154750 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 >> 172.21.15.170:0/3371118912 conn(0x7f5d7008bf50 msgr2=0x7f5d7006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:30.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 shutdown_connections 2024-09-21T02:20:30.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.441+0000 7f5d784d0640 1 -- 172.21.15.170:0/3371118912 wait complete. 2024-09-21T02:20:30.464 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:20:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 -- 172.21.15.170:0/2357558750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0ae500 msgr2=0x7f09dc169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 --2- 172.21.15.170:0/2357558750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0ae500 0x7f09dc169870 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f09dc073f30 tx=0x7f09d401ccf0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 -- 172.21.15.170:0/2357558750 shutdown_connections 2024-09-21T02:20:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 --2- 172.21.15.170:0/2357558750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0ae500 0x7f09dc169870 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 --2- 172.21.15.170:0/2357558750 >> v2:172.21.15.191:3300/0 conn(0x7f09dc06d510 0x7f09dc0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 -- 172.21.15.170:0/2357558750 >> 172.21.15.170:0/2357558750 conn(0x7f09dc09a9c0 msgr2=0x7f09dc0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 -- 172.21.15.170:0/2357558750 shutdown_connections 2024-09-21T02:20:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.609+0000 7f09e4087640 1 -- 172.21.15.170:0/2357558750 wait complete. 2024-09-21T02:20:30.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 Processor -- start 2024-09-21T02:20:30.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- start start 2024-09-21T02:20:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0ae500 0x7f09dc1f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 0x7f09dc1f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc09b140 0x7f09dc161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc16c750 con 0x7f09dc162e80 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc16c5d0 con 0x7f09dc09b140 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f09dc16c8d0 con 0x7f09dc0ae500 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 0x7f09dc1f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 0x7f09dc1f9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56650/0 (socket says 172.21.15.170:56650) 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 -- 172.21.15.170:0/888945043 learned_addr learned my addr 172.21.15.170:0/888945043 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1e23640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0ae500 0x7f09dc1f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e2624640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc09b140 0x7f09dc161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 -- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0ae500 msgr2=0x7f09dc1f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0ae500 0x7f09dc1f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 -- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc09b140 msgr2=0x7f09dc161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc09b140 0x7f09dc161380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 -- 172.21.15.170:0/888945043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc152fb0 con 0x7f09dc162e80 2024-09-21T02:20:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e2624640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc09b140 0x7f09dc161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e1622640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 0x7f09dc1f9970 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f09d800e7a0 tx=0x7f09d800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800ccc0 con 0x7f09dc162e80 2024-09-21T02:20:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09d8010430 con 0x7f09dc162e80 2024-09-21T02:20:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8010860 con 0x7f09dc162e80 2024-09-21T02:20:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc153290 con 0x7f09dc162e80 2024-09-21T02:20:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc153850 con 0x7f09dc162e80 2024-09-21T02:20:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.613+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc09b6c0 con 0x7f09dc162e80 2024-09-21T02:20:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.617+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f09d8010a00 con 0x7f09dc162e80 2024-09-21T02:20:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.617+0000 7f09d27fc640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09c0073830 0x7f09c0075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.617+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4885+0+0 (secure 0 0 0) 0x7f09d8095570 con 0x7f09dc162e80 2024-09-21T02:20:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.617+0000 7f09e1e23640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09c0073830 0x7f09c0075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.617+0000 7f09e1e23640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09c0073830 0x7f09c0075d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00a750 tx=0x7f09cc008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:30.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.621+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f09d805ff00 con 0x7f09dc162e80 2024-09-21T02:20:30.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.837+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f09dc0adfc0 con 0x7f09dc162e80 2024-09-21T02:20:30.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.837+0000 7f09d27fc640 1 -- 172.21.15.170:0/888945043 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) ==== 80+0+5304 (secure 0 0 0) 0x7f09d805f800 con 0x7f09dc162e80 2024-09-21T02:20:30.839 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:30.839 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:20:30.839 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:20:30.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.839 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.840 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.842 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.844 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.845 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:20:30.846 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:20:30.847 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.849 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:20:30.851 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:20:30.852 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:20:30.852 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:20:30.852 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.852 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:30.852 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:20:30.852 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09c0073830 msgr2=0x7f09c0075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09c0073830 0x7f09c0075d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00a750 tx=0x7f09cc008040 comp rx=0 tx=0).stop 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 msgr2=0x7f09dc1f9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 0x7f09dc1f9970 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f09d800e7a0 tx=0x7f09d800ec60 comp rx=0 tx=0).stop 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 shutdown_connections 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09c0073830 0x7f09c0075d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc09b140 0x7f09dc161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc162e80 0x7f09dc1f9970 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 --2- 172.21.15.170:0/888945043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0ae500 0x7f09dc1f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 >> 172.21.15.170:0/888945043 conn(0x7f09dc09a9c0 msgr2=0x7f09dc0b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:30.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 shutdown_connections 2024-09-21T02:20:30.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:30.841+0000 7f09e4087640 1 -- 172.21.15.170:0/888945043 wait complete. 2024-09-21T02:20:30.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:20:31.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 -- 172.21.15.170:0/1562705477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 msgr2=0x7f3328070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 --2- 172.21.15.170:0/1562705477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f3328070340 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f331c009900 tx=0x7f331c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 -- 172.21.15.170:0/1562705477 shutdown_connections 2024-09-21T02:20:31.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 --2- 172.21.15.170:0/1562705477 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33280cb460 0x7f33280cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 --2- 172.21.15.170:0/1562705477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f3328070340 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 -- 172.21.15.170:0/1562705477 >> 172.21.15.170:0/1562705477 conn(0x7f33280b1c40 msgr2=0x7f33280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:31.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 -- 172.21.15.170:0/1562705477 shutdown_connections 2024-09-21T02:20:31.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 -- 172.21.15.170:0/1562705477 wait complete. 2024-09-21T02:20:31.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 Processor -- start 2024-09-21T02:20:31.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.013+0000 7f332cbf8640 1 -- start start 2024-09-21T02:20:31.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33280cb460 0x7f332815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33280cc8c0 0x7f332815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f332815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33281690d0 con 0x7f33280cf7e0 2024-09-21T02:20:31.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3328168f50 con 0x7f33280cc8c0 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3328169250 con 0x7f33280cb460 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f332815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f332815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56670/0 (socket says 172.21.15.170:56670) 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 -- 172.21.15.170:0/2529117524 learned_addr learned my addr 172.21.15.170:0/2529117524 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332659c640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33280cb460 0x7f332815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3325d9b640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33280cc8c0 0x7f332815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 -- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33280cb460 msgr2=0x7f332815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33280cb460 0x7f332815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 -- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33280cc8c0 msgr2=0x7f332815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33280cc8c0 0x7f332815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 -- 172.21.15.170:0/2529117524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33281f9010 con 0x7f33280cf7e0 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3325d9b640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33280cc8c0 0x7f332815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3326d9d640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f332815b7e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f332000c730 tx=0x7f332000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:31.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320003b80 con 0x7f33280cf7e0 2024-09-21T02:20:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3320003d40 con 0x7f33280cf7e0 2024-09-21T02:20:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33281f92f0 con 0x7f33280cf7e0 2024-09-21T02:20:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200052d0 con 0x7f33280cf7e0 2024-09-21T02:20:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281f98b0 con 0x7f33280cf7e0 2024-09-21T02:20:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328070340 con 0x7f33280cf7e0 2024-09-21T02:20:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f3320005680 con 0x7f33280cf7e0 2024-09-21T02:20:31.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.017+0000 7f3316ffd640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3308073830 0x7f3308075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.021+0000 7f332659c640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3308073830 0x7f3308075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.021+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4885+0+0 (secure 0 0 0) 0x7f33200954c0 con 0x7f33280cf7e0 2024-09-21T02:20:31.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.021+0000 7f332659c640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3308073830 0x7f3308075d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f331c0af000 tx=0x7f331c01b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:31.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.021+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f332005ffe0 con 0x7f33280cf7e0 2024-09-21T02:20:31.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.197+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f33280adfe0 con 0x7f33280cf7e0 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.197+0000 7f3316ffd640 1 -- 172.21.15.170:0/2529117524 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) ==== 84+0+65 (secure 0 0 0) 0x7f332005f8e0 con 0x7f33280cf7e0 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:31.198 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3308073830 msgr2=0x7f3308075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3308073830 0x7f3308075d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f331c0af000 tx=0x7f331c01b120 comp rx=0 tx=0).stop 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 msgr2=0x7f332815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f332815b7e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f332000c730 tx=0x7f332000cbf0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 shutdown_connections 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3308073830 0x7f3308075d40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33280cf7e0 0x7f332815b7e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33280cc8c0 0x7f332815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 --2- 172.21.15.170:0/2529117524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33280cb460 0x7f332815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 >> 172.21.15.170:0/2529117524 conn(0x7f33280b1c40 msgr2=0x7f33280ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:31.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 shutdown_connections 2024-09-21T02:20:31.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.201+0000 7f332cbf8640 1 -- 172.21.15.170:0/2529117524 wait complete. 2024-09-21T02:20:31.222 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:20:31.222 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 -- 172.21.15.170:0/2486925184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680d25e0 msgr2=0x7fba680d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 --2- 172.21.15.170:0/2486925184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680d25e0 0x7fba680d29f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fba680d1ae0 tx=0x7fba6001ccc0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 -- 172.21.15.170:0/2486925184 shutdown_connections 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 --2- 172.21.15.170:0/2486925184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680d25e0 0x7fba680d29f0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 --2- 172.21.15.170:0/2486925184 >> v2:172.21.15.191:3300/0 conn(0x7fba680cf860 0x7fba680cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 -- 172.21.15.170:0/2486925184 >> 172.21.15.170:0/2486925184 conn(0x7fba6809d830 msgr2=0x7fba6808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 -- 172.21.15.170:0/2486925184 shutdown_connections 2024-09-21T02:20:31.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.377+0000 7fba7064a640 1 -- 172.21.15.170:0/2486925184 wait complete. 2024-09-21T02:20:31.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 Processor -- start 2024-09-21T02:20:31.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- start start 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba680b95f0 0x7fba68159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 0x7fba68155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba680d25e0 0x7fba6815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fba6816bd70 con 0x7fba680cf860 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fba680b5260 con 0x7fba680b95f0 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fba6816bef0 con 0x7fba680d25e0 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 0x7fba68155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 0x7fba68155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56698/0 (socket says 172.21.15.170:56698) 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 -- 172.21.15.170:0/757605998 learned_addr learned my addr 172.21.15.170:0/757605998 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6ebe7640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba680d25e0 0x7fba6815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6e3e6640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba680b95f0 0x7fba68159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 -- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba680d25e0 msgr2=0x7fba6815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba680d25e0 0x7fba6815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 -- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba680b95f0 msgr2=0x7fba68159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba680b95f0 0x7fba68159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 -- 172.21.15.170:0/757605998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6815abc0 con 0x7fba680cf860 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba6dbe5640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 0x7fba68155790 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fba5800c7a0 tx=0x7fba5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58004220 con 0x7fba680cf860 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba580079b0 con 0x7fba680cf860 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba68154a10 con 0x7fba680cf860 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580044d0 con 0x7fba680cf860 2024-09-21T02:20:31.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6815a600 con 0x7fba680cf860 2024-09-21T02:20:31.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.381+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba680cfc70 con 0x7fba680cf860 2024-09-21T02:20:31.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.385+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fba58005020 con 0x7fba680cf860 2024-09-21T02:20:31.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.385+0000 7fba5effd640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fba4c073830 0x7fba4c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:31.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.385+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4885+0+0 (secure 0 0 0) 0x7fba58095060 con 0x7fba680cf860 2024-09-21T02:20:31.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.385+0000 7fba6e3e6640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fba4c073830 0x7fba4c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:31.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.385+0000 7fba6e3e6640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fba4c073830 0x7fba4c075d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba6400a790 tx=0x7fba64008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:31.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.389+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fba58014030 con 0x7fba680cf860 2024-09-21T02:20:31.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:31.569+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fba6815f2a0 con 0x7fba680cf860 2024-09-21T02:20:32.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.457+0000 7fba5effd640 1 -- 172.21.15.170:0/757605998 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) ==== 138+0+0 (secure 0 0 0) 0x7fba5805fa60 con 0x7fba680cf860 2024-09-21T02:20:32.460 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:20:32.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fba4c073830 msgr2=0x7fba4c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fba4c073830 0x7fba4c075d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba6400a790 tx=0x7fba64008040 comp rx=0 tx=0).stop 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 msgr2=0x7fba68155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 0x7fba68155790 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fba5800c7a0 tx=0x7fba5800cc60 comp rx=0 tx=0).stop 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 shutdown_connections 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fba4c073830 0x7fba4c075d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba680d25e0 0x7fba6815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba680cf860 0x7fba68155790 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 --2- 172.21.15.170:0/757605998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba680b95f0 0x7fba68159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.461+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 >> 172.21.15.170:0/757605998 conn(0x7fba6809d830 msgr2=0x7fba680b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.465+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 shutdown_connections 2024-09-21T02:20:32.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.465+0000 7fba7064a640 1 -- 172.21.15.170:0/757605998 wait complete. 2024-09-21T02:20:32.492 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:20:32.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 -- 172.21.15.170:0/2143402269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 msgr2=0x7fd7c80cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:32.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 --2- 172.21.15.170:0/2143402269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c80cb990 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009bc0 tx=0x7fd7c001cac0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 -- 172.21.15.170:0/2143402269 shutdown_connections 2024-09-21T02:20:32.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 --2- 172.21.15.170:0/2143402269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c80cb990 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 -- 172.21.15.170:0/2143402269 >> 172.21.15.170:0/2143402269 conn(0x7fd7c80d0170 msgr2=0x7fd7c80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:32.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 -- 172.21.15.170:0/2143402269 shutdown_connections 2024-09-21T02:20:32.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 -- 172.21.15.170:0/2143402269 wait complete. 2024-09-21T02:20:32.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 Processor -- start 2024-09-21T02:20:32.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.661+0000 7fd7cff19640 1 -- start start 2024-09-21T02:20:32.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c8074270 0x7fd7c81fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:32.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c8153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:32.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80cf7d0 0x7fd7c8159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:32.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c816a260 con 0x7fd7c8074680 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c816a0e0 con 0x7fd7c8074270 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c816a3e0 con 0x7fd7c80cf7d0 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c8153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c8153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56724/0 (socket says 172.21.15.170:56724) 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 -- 172.21.15.170:0/3232514919 learned_addr learned my addr 172.21.15.170:0/3232514919 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cdcb5640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c8074270 0x7fd7c81fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7ce4b6640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80cf7d0 0x7fd7c8159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 -- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80cf7d0 msgr2=0x7fd7c8159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80cf7d0 0x7fd7c8159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 -- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c8074270 msgr2=0x7fd7c81fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c8074270 0x7fd7c81fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 -- 172.21.15.170:0/3232514919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c81fd690 con 0x7fd7c8074680 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cd4b4640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c8153510 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c450 tx=0x7fd7b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:32.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80052d0 con 0x7fd7c8074680 2024-09-21T02:20:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7b8005470 con 0x7fd7c8074680 2024-09-21T02:20:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c809d470 con 0x7fd7c8074680 2024-09-21T02:20:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8012870 con 0x7fd7c8074680 2024-09-21T02:20:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c809b020 con 0x7fd7c8074680 2024-09-21T02:20:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c8073b60 con 0x7fd7c8074680 2024-09-21T02:20:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.665+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fd7b8003600 con 0x7fd7c8074680 2024-09-21T02:20:32.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.669+0000 7fd7be7fc640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7ac073830 0x7fd7ac075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:32.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.669+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5255+0+0 (secure 0 0 0) 0x7fd7b8094890 con 0x7fd7c8074680 2024-09-21T02:20:32.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.669+0000 7fd7cdcb5640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7ac073830 0x7fd7ac075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:32.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.669+0000 7fd7cdcb5640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7ac073830 0x7fd7ac075d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4001230 tx=0x7fd7c4009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:32.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.673+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd7b805f030 con 0x7fd7c8074680 2024-09-21T02:20:32.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:32.849+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd7c80aeb10 con 0x7fd7c8074680 2024-09-21T02:20:33.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.473+0000 7fd7be7fc640 1 -- 172.21.15.170:0/3232514919 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) ==== 170+0+0 (secure 0 0 0) 0x7fd7b805e930 con 0x7fd7c8074680 2024-09-21T02:20:33.473 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7ac073830 msgr2=0x7fd7ac075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7ac073830 0x7fd7ac075d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4001230 tx=0x7fd7c4009040 comp rx=0 tx=0).stop 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 msgr2=0x7fd7c8153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c8153510 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c450 tx=0x7fd7b800c910 comp rx=0 tx=0).stop 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 shutdown_connections 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7ac073830 0x7fd7ac075d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7c80cf7d0 0x7fd7c8159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7c8074680 0x7fd7c8153510 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 --2- 172.21.15.170:0/3232514919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7c8074270 0x7fd7c81fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 >> 172.21.15.170:0/3232514919 conn(0x7fd7c80d0170 msgr2=0x7fd7c80b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 shutdown_connections 2024-09-21T02:20:33.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.477+0000 7fd7cff19640 1 -- 172.21.15.170:0/3232514919 wait complete. 2024-09-21T02:20:33.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 -- 172.21.15.170:0/1510014970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 msgr2=0x7f01ec0b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 --2- 172.21.15.170:0/1510014970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0b5af0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c80 tx=0x7f01e401ccd0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 -- 172.21.15.170:0/1510014970 shutdown_connections 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 --2- 172.21.15.170:0/1510014970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0b5af0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 --2- 172.21.15.170:0/1510014970 >> v2:172.21.15.191:3300/0 conn(0x7f01ec0b4d90 0x7f01ec0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 -- 172.21.15.170:0/1510014970 >> 172.21.15.170:0/1510014970 conn(0x7f01ec0d1280 msgr2=0x7f01ec0d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 -- 172.21.15.170:0/1510014970 shutdown_connections 2024-09-21T02:20:33.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.673+0000 7f01f15c0640 1 -- 172.21.15.170:0/1510014970 wait complete. 2024-09-21T02:20:33.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 Processor -- start 2024-09-21T02:20:33.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- start start 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4d90 0x7f01ec1f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cf640 0x7f01ec160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec168bb0 con 0x7f01ec0b56e0 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec168a30 con 0x7f01ec0b4d90 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec168d30 con 0x7f01ec0cf640 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56740/0 (socket says 172.21.15.170:56740) 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 -- 172.21.15.170:0/2485669313 learned_addr learned my addr 172.21.15.170:0/2485669313 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:33.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01eb7fe640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cf640 0x7f01ec160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01eaffd640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4d90 0x7f01ec1f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 -- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cf640 msgr2=0x7f01ec160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cf640 0x7f01ec160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 -- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4d90 msgr2=0x7f01ec1f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4d90 0x7f01ec1f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 -- 172.21.15.170:0/2485669313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec0c8010 con 0x7f01ec0b56e0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01eaffd640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4d90 0x7f01ec1f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01ea7fc640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0c6200 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d720 tx=0x7f01e000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0017070 con 0x7f01ec0b56e0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01e0005310 con 0x7f01ec0b56e0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec0c6f70 con 0x7f01ec0b56e0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00097c0 con 0x7f01ec0b56e0 2024-09-21T02:20:33.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec0c7530 con 0x7f01ec0b56e0 2024-09-21T02:20:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.677+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec0c6ce0 con 0x7f01ec0b56e0 2024-09-21T02:20:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.681+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f01e0009960 con 0x7f01ec0b56e0 2024-09-21T02:20:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.681+0000 7f01d37fe640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f01d8073830 0x7f01d8075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.681+0000 7f01eaffd640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f01d8073830 0x7f01d8075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:33.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.681+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5637+0+0 (secure 0 0 0) 0x7f01e0095230 con 0x7f01ec0b56e0 2024-09-21T02:20:33.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.681+0000 7f01eaffd640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f01d8073830 0x7f01d8075d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00a810 tx=0x7f01dc008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.685+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f01e005f9d0 con 0x7f01ec0b56e0 2024-09-21T02:20:33.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:33.893+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f01ec0b51a0 con 0x7f01ec0b56e0 2024-09-21T02:20:34.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.529+0000 7f01d37fe640 1 -- 172.21.15.170:0/2485669313 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v87) ==== 174+0+0 (secure 0 0 0) 0x7f01e005f2d0 con 0x7f01ec0b56e0 2024-09-21T02:20:34.533 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 20 and data pool 21 2024-09-21T02:20:34.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f01d8073830 msgr2=0x7f01d8075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f01d8073830 0x7f01d8075d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00a810 tx=0x7f01dc008040 comp rx=0 tx=0).stop 2024-09-21T02:20:34.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 msgr2=0x7f01ec0c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0c6200 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d720 tx=0x7f01e000dbe0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 shutdown_connections 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f01d8073830 0x7f01d8075d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01ec0cf640 0x7f01ec160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01ec0b56e0 0x7f01ec0c6200 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 --2- 172.21.15.170:0/2485669313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01ec0b4d90 0x7f01ec1f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 >> 172.21.15.170:0/2485669313 conn(0x7f01ec0d1280 msgr2=0x7f01ec0d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 shutdown_connections 2024-09-21T02:20:34.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.537+0000 7f01f15c0640 1 -- 172.21.15.170:0/2485669313 wait complete. 2024-09-21T02:20:34.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:34.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3042090600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0ccce0 msgr2=0x7f259c0cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3042090600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0ccce0 0x7f259c0cd0f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2590009900 tx=0x7f259001c850 comp rx=0 tx=0).stop 2024-09-21T02:20:34.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3042090600 shutdown_connections 2024-09-21T02:20:34.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3042090600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 0x7f259c0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3042090600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0ccce0 0x7f259c0cd0f0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3042090600 >> 172.21.15.170:0/3042090600 conn(0x7f259c09a9c0 msgr2=0x7f259c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:34.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3042090600 shutdown_connections 2024-09-21T02:20:34.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.721+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3042090600 wait complete. 2024-09-21T02:20:34.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 Processor -- start 2024-09-21T02:20:34.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 -- start start 2024-09-21T02:20:34.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 0x7f259c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:34.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 0x7f259c1fada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f259c0ccce0 0x7f259c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f259c0b1910 con 0x7f259c0b21a0 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f259c0b1790 con 0x7f259c0ccce0 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f25a0e4a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f259c16a4f0 con 0x7f259c0b60e0 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f259a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 0x7f259c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f259a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 0x7f259c151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56764/0 (socket says 172.21.15.170:56764) 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f259a59c640 1 -- 172.21.15.170:0/3538673822 learned_addr learned my addr 172.21.15.170:0/3538673822 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.725+0000 7f2599d9b640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 0x7f259c1fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259a59c640 1 -- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 msgr2=0x7f259c1fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259ad9d640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f259c0ccce0 0x7f259c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259a59c640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 0x7f259c1fada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259a59c640 1 -- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f259c0ccce0 msgr2=0x7f259c15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259a59c640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f259c0ccce0 0x7f259c15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259a59c640 1 -- 172.21.15.170:0/3538673822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c0b06a0 con 0x7f259c0b21a0 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f2599d9b640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 0x7f259c1fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f259a59c640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 0x7f259c151220 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c3f0 tx=0x7f258c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c017070 con 0x7f259c0b21a0 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f258c003600 con 0x7f259c0b21a0 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c15f9d0 con 0x7f259c0b21a0 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c003a10 con 0x7f259c0b21a0 2024-09-21T02:20:34.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c15ff90 con 0x7f259c0b21a0 2024-09-21T02:20:34.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c0b64f0 con 0x7f259c0b21a0 2024-09-21T02:20:34.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f258c01a070 con 0x7f259c0b21a0 2024-09-21T02:20:34.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f258affd640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f257c073830 0x7f257c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:34.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f2599d9b640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f257c073830 0x7f257c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:34.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7f258c094250 con 0x7f259c0b21a0 2024-09-21T02:20:34.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.729+0000 7f2599d9b640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f257c073830 0x7f257c075d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25900ae000 tx=0x7f259001b090 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:34.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.733+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f259c0d1cb0 con 0x7f259c0b21a0 2024-09-21T02:20:34.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.913+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f259c15a420 con 0x7f259c0b21a0 2024-09-21T02:20:34.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.913+0000 7f258affd640 1 -- 172.21.15.170:0/3538673822 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16844 (secure 0 0 0) 0x7f258c05e740 con 0x7f259c0b21a0 2024-09-21T02:20:34.916 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:34.916 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":129,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:20:33.897832+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:20:31.581917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T02:20:32.865501+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:34.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f257c073830 msgr2=0x7f257c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f257c073830 0x7f257c075d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25900ae000 tx=0x7f259001b090 comp rx=0 tx=0).stop 2024-09-21T02:20:34.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 msgr2=0x7f259c151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:34.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 0x7f259c151220 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c3f0 tx=0x7f258c00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 shutdown_connections 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f257c073830 0x7f257c075d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f259c0b60e0 0x7f259c1fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f259c0ccce0 0x7f259c15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 --2- 172.21.15.170:0/3538673822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f259c0b21a0 0x7f259c151220 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 >> 172.21.15.170:0/3538673822 conn(0x7f259c09a9c0 msgr2=0x7f259c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 shutdown_connections 2024-09-21T02:20:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:34.917+0000 7f25a0e4a640 1 -- 172.21.15.170:0/3538673822 wait complete. 2024-09-21T02:20:34.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 -- 172.21.15.170:0/809128855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc140cea00 msgr2=0x7efc140cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 --2- 172.21.15.170:0/809128855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc140cea00 0x7efc140cadf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efc140b2d00 tx=0x7efc0c01cd50 comp rx=0 tx=0).stop 2024-09-21T02:20:35.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 -- 172.21.15.170:0/809128855 shutdown_connections 2024-09-21T02:20:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 --2- 172.21.15.170:0/809128855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc140cea00 0x7efc140cadf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 --2- 172.21.15.170:0/809128855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc14070570 0x7efc140ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 --2- 172.21.15.170:0/809128855 >> v2:172.21.15.191:3300/0 conn(0x7efc1415f990 0x7efc14070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 -- 172.21.15.170:0/809128855 >> 172.21.15.170:0/809128855 conn(0x7efc140b1c40 msgr2=0x7efc140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 -- 172.21.15.170:0/809128855 shutdown_connections 2024-09-21T02:20:35.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.097+0000 7efc19dba640 1 -- 172.21.15.170:0/809128855 wait complete. 2024-09-21T02:20:35.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 Processor -- start 2024-09-21T02:20:35.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 -- start start 2024-09-21T02:20:35.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 0x7efc14154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc140cea00 0x7efc1409c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc1415f990 0x7efc1409c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efc140732d0 con 0x7efc14070570 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efc14073150 con 0x7efc140cea00 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efc14073450 con 0x7efc1415f990 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 0x7efc14154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 0x7efc14154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56778/0 (socket says 172.21.15.170:56778) 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 -- 172.21.15.170:0/862315002 learned_addr learned my addr 172.21.15.170:0/862315002 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc13fff640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc1415f990 0x7efc1409c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc12ffd640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc140cea00 0x7efc1409c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 -- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc1415f990 msgr2=0x7efc1409c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc1415f990 0x7efc1409c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 -- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc140cea00 msgr2=0x7efc1409c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc140cea00 0x7efc1409c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 -- 172.21.15.170:0/862315002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1409bc20 con 0x7efc14070570 2024-09-21T02:20:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc137fe640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 0x7efc14154a60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efc080058d0 tx=0x7efc0800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08017070 con 0x7efc14070570 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc08009d20 con 0x7efc14070570 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1415d060 con 0x7efc14070570 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc080106c0 con 0x7efc14070570 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.101+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc14157160 con 0x7efc14070570 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.105+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc140cadf0 con 0x7efc14070570 2024-09-21T02:20:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.105+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7efc0801a070 con 0x7efc14070570 2024-09-21T02:20:35.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.105+0000 7efbfbfff640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc00073860 0x7efc00075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.105+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7efc08095300 con 0x7efc14070570 2024-09-21T02:20:35.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.105+0000 7efc12ffd640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc00073860 0x7efc00075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.105+0000 7efc12ffd640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc00073860 0x7efc00075d70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc04005fd0 tx=0x7efc04005d80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.109+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efc0805fad0 con 0x7efc14070570 2024-09-21T02:20:35.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.281+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc140ce2a0 con 0x7efc14070570 2024-09-21T02:20:35.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.285+0000 7efbfbfff640 1 -- 172.21.15.170:0/862315002 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16844 (secure 0 0 0) 0x7efc0805f3d0 con 0x7efc14070570 2024-09-21T02:20:35.286 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:35.286 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":129,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:20:33.897832+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:20:31.581917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T02:20:32.865501+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:35.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc00073860 msgr2=0x7efc00075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc00073860 0x7efc00075d70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc04005fd0 tx=0x7efc04005d80 comp rx=0 tx=0).stop 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 msgr2=0x7efc14154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 0x7efc14154a60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efc080058d0 tx=0x7efc0800dde0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 shutdown_connections 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efc00073860 0x7efc00075d70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc1415f990 0x7efc1409c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc140cea00 0x7efc1409c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 --2- 172.21.15.170:0/862315002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc14070570 0x7efc14154a60 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 >> 172.21.15.170:0/862315002 conn(0x7efc140b1c40 msgr2=0x7efc140b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 shutdown_connections 2024-09-21T02:20:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.289+0000 7efc19dba640 1 -- 172.21.15.170:0/862315002 wait complete. 2024-09-21T02:20:35.310 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 -- 172.21.15.170:0/712206817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8281606e0 msgr2=0x7fa828160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 --2- 172.21.15.170:0/712206817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8281606e0 0x7fa828160af0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa8280ce420 tx=0x7fa82001caa0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 -- 172.21.15.170:0/712206817 shutdown_connections 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 --2- 172.21.15.170:0/712206817 >> v2:172.21.15.191:3300/0 conn(0x7fa8280d0700 0x7fa8280d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 --2- 172.21.15.170:0/712206817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8281606e0 0x7fa828160af0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 -- 172.21.15.170:0/712206817 >> 172.21.15.170:0/712206817 conn(0x7fa8280d2a30 msgr2=0x7fa8280cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 -- 172.21.15.170:0/712206817 shutdown_connections 2024-09-21T02:20:35.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 -- 172.21.15.170:0/712206817 wait complete. 2024-09-21T02:20:35.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.465+0000 7fa82ca69640 1 Processor -- start 2024-09-21T02:20:35.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- start start 2024-09-21T02:20:35.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8280d0700 0x7fa8281534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 0x7fa8280b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8281606e0 0x7fa8280b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa82815dc00 con 0x7fa82815d3e0 2024-09-21T02:20:35.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa82815da80 con 0x7fa8281606e0 2024-09-21T02:20:35.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa828169cb0 con 0x7fa8280d0700 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 0x7fa8280b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 0x7fa8280b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50580/0 (socket says 172.21.15.170:50580) 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 -- 172.21.15.170:0/1874175227 learned_addr learned my addr 172.21.15.170:0/1874175227 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa826d9d640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8281606e0 0x7fa8280b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82659c640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8280d0700 0x7fa8281534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 -- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8280d0700 msgr2=0x7fa8281534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8280d0700 0x7fa8281534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 -- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8281606e0 msgr2=0x7fa8280b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8281606e0 0x7fa8280b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 -- 172.21.15.170:0/1874175227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa828076910 con 0x7fa82815d3e0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa826d9d640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8281606e0 0x7fa8280b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa825d9b640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 0x7fa8280b2ef0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00d720 tx=0x7fa81c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c017070 con 0x7fa82815d3e0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa81c004310 con 0x7fa82815d3e0 2024-09-21T02:20:35.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c009a80 con 0x7fa82815d3e0 2024-09-21T02:20:35.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8281f9c00 con 0x7fa82815d3e0 2024-09-21T02:20:35.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8281fa100 con 0x7fa82815d3e0 2024-09-21T02:20:35.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.469+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8281f9e70 con 0x7fa82815d3e0 2024-09-21T02:20:35.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.473+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fa81c009c20 con 0x7fa82815d3e0 2024-09-21T02:20:35.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.473+0000 7fa816ffd640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa808073830 0x7fa808075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:35.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.473+0000 7fa82659c640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa808073830 0x7fa808075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:35.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.473+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5737+0+0 (secure 0 0 0) 0x7fa81c094f90 con 0x7fa82815d3e0 2024-09-21T02:20:35.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.473+0000 7fa82659c640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa808073830 0x7fa808075d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8200af000 tx=0x7fa82001b2e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:35.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.477+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa81c05f590 con 0x7fa82815d3e0 2024-09-21T02:20:35.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:35.701+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa8280d0b10 con 0x7fa82815d3e0 2024-09-21T02:20:36.452 INFO:tasks.ceph.mds.d.smithi191.stderr:2024-09-21T02:20:36.452+0000 7f66d3c75640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:20:36.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.505+0000 7fa816ffd640 1 -- 172.21.15.170:0/1874175227 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) ==== 118+0+0 (secure 0 0 0) 0x7fa81c05ee90 con 0x7fa82815d3e0 2024-09-21T02:20:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa808073830 msgr2=0x7fa808075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa808073830 0x7fa808075d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8200af000 tx=0x7fa82001b2e0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 msgr2=0x7fa8280b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 0x7fa8280b2ef0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00d720 tx=0x7fa81c00dbe0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 shutdown_connections 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa808073830 0x7fa808075d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8281606e0 0x7fa8280b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa82815d3e0 0x7fa8280b2ef0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 --2- 172.21.15.170:0/1874175227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8280d0700 0x7fa8281534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 >> 172.21.15.170:0/1874175227 conn(0x7fa8280d2a30 msgr2=0x7fa82815b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 shutdown_connections 2024-09-21T02:20:36.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.509+0000 7fa82ca69640 1 -- 172.21.15.170:0/1874175227 wait complete. 2024-09-21T02:20:36.543 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:20:36.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:20:36.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 -- 172.21.15.170:0/633208871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340700b0 msgr2=0x7fb334074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:36.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 --2- 172.21.15.170:0/633208871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340700b0 0x7fb334074390 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fb324009bc0 tx=0x7fb32401cb10 comp rx=0 tx=0).stop 2024-09-21T02:20:36.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 -- 172.21.15.170:0/633208871 shutdown_connections 2024-09-21T02:20:36.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 --2- 172.21.15.170:0/633208871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340700b0 0x7fb334074390 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 -- 172.21.15.170:0/633208871 >> 172.21.15.170:0/633208871 conn(0x7fb3340d0170 msgr2=0x7fb3340d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:36.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 -- 172.21.15.170:0/633208871 shutdown_connections 2024-09-21T02:20:36.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.705+0000 7fb339b68640 1 -- 172.21.15.170:0/633208871 wait complete. 2024-09-21T02:20:36.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 Processor -- start 2024-09-21T02:20:36.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 -- start start 2024-09-21T02:20:36.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3340700b0 0x7fb3341560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:36.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 0x7fb334152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:36.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3340ce780 0x7fb334073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:36.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3341695e0 con 0x7fb3340cb380 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb334169460 con 0x7fb3340700b0 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb334169760 con 0x7fb3340ce780 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 0x7fb334152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 0x7fb334152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50608/0 (socket says 172.21.15.170:50608) 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 -- 172.21.15.170:0/2551769641 learned_addr learned my addr 172.21.15.170:0/2551769641 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb3337fe640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3340700b0 0x7fb3341560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb333fff640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3340ce780 0x7fb334073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 -- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3340ce780 msgr2=0x7fb334073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3340ce780 0x7fb334073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 -- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3340700b0 msgr2=0x7fb3341560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3340700b0 0x7fb3341560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 -- 172.21.15.170:0/2551769641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3341fc5a0 con 0x7fb3340cb380 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb332ffd640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 0x7fb334152d00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00b3e0 tx=0x7fb31c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:36.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c015020 con 0x7fb3340cb380 2024-09-21T02:20:36.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb31c011920 con 0x7fb3340cb380 2024-09-21T02:20:36.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3341fc880 con 0x7fb3340cb380 2024-09-21T02:20:36.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c011d70 con 0x7fb3340cb380 2024-09-21T02:20:36.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.709+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341fce40 con 0x7fb3340cb380 2024-09-21T02:20:36.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.713+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3340ce070 con 0x7fb3340cb380 2024-09-21T02:20:36.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.713+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fb31c018020 con 0x7fb3340cb380 2024-09-21T02:20:36.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.713+0000 7fb32bfff640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb314073830 0x7fb314075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:36.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.713+0000 7fb3337fe640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb314073830 0x7fb314075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:36.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.713+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fb31c094f30 con 0x7fb3340cb380 2024-09-21T02:20:36.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.713+0000 7fb3337fe640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb314073830 0x7fb314075d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0015f0 tx=0x7fb32c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:36.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.717+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb31c01f030 con 0x7fb3340cb380 2024-09-21T02:20:36.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:36.925+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb334073f10 con 0x7fb3340cb380 2024-09-21T02:20:37.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.505+0000 7fb32bfff640 1 -- 172.21.15.170:0/2551769641 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) ==== 122+0+0 (secure 0 0 0) 0x7fb31c05f5e0 con 0x7fb3340cb380 2024-09-21T02:20:37.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.509+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb314073830 msgr2=0x7fb314075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.509+0000 7fb339b68640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb314073830 0x7fb314075d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb32c0015f0 tx=0x7fb32c009040 comp rx=0 tx=0).stop 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.509+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 msgr2=0x7fb334152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.509+0000 7fb339b68640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 0x7fb334152d00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00b3e0 tx=0x7fb31c00b8a0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 shutdown_connections 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb314073830 0x7fb314075d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3340ce780 0x7fb334073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3340cb380 0x7fb334152d00 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 --2- 172.21.15.170:0/2551769641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3340700b0 0x7fb3341560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 >> 172.21.15.170:0/2551769641 conn(0x7fb3340d0170 msgr2=0x7fb33406ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 shutdown_connections 2024-09-21T02:20:37.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.513+0000 7fb339b68640 1 -- 172.21.15.170:0/2551769641 wait complete. 2024-09-21T02:20:37.540 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:20:37.541 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:37.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 -- 172.21.15.170:0/900381089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0ccb00 msgr2=0x7f7b4c0ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 --2- 172.21.15.170:0/900381089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ccf10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009bc0 tx=0x7f7b4401cba0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 -- 172.21.15.170:0/900381089 shutdown_connections 2024-09-21T02:20:37.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 --2- 172.21.15.170:0/900381089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ccf10 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 --2- 172.21.15.170:0/900381089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b4c0b2f40 0x7f7b4c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 -- 172.21.15.170:0/900381089 >> 172.21.15.170:0/900381089 conn(0x7f7b4c0b1c40 msgr2=0x7f7b4c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:37.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 -- 172.21.15.170:0/900381089 shutdown_connections 2024-09-21T02:20:37.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 -- 172.21.15.170:0/900381089 wait complete. 2024-09-21T02:20:37.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 Processor -- start 2024-09-21T02:20:37.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.701+0000 7f7b53c70640 1 -- start start 2024-09-21T02:20:37.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 0x7f7b4c15e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:37.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b4c0b5410 0x7f7b4c14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:37.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:37.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c169e50 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0cf970 con 0x7f7b4c0ccb00 2024-09-21T02:20:37.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c169fd0 con 0x7f7b4c0b5410 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 0x7f7b4c15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 0x7f7b4c15e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50630/0 (socket says 172.21.15.170:50630) 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 -- 172.21.15.170:0/635884237 learned_addr learned my addr 172.21.15.170:0/635884237 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b5120b640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b4c0b5410 0x7f7b4c14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b5220d640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 -- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b4c0b5410 msgr2=0x7f7b4c14b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b4c0b5410 0x7f7b4c14b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 -- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b4c0ccb00 msgr2=0x7f7b4c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 -- 172.21.15.170:0/635884237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c15e680 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b5220d640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b51a0c640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 0x7f7b4c15e140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c730 tx=0x7f7b4800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48016020 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b4800a950 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c09b890 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4800ada0 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1597d0 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c0b3350 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f7b48019020 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b427fc640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b30073830 0x7f7b30075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:37.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f7b48094d60 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.705+0000 7f7b5120b640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b30073830 0x7f7b30075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:37.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.709+0000 7f7b5120b640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b30073830 0x7f7b30075d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0058d0 tx=0x7f7b3c005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.713+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7b48020020 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.909+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b4c06fe90 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.909+0000 7f7b427fc640 1 -- 172.21.15.170:0/635884237 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5236 (secure 0 0 0) 0x7f7b4c0cd450 con 0x7f7b4c0b2f40 2024-09-21T02:20:37.911 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:37.911 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":90,"btime":"2024-09-21T02:20:37:507464+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:20:33.897736+0000","modified":"2024-09-21T02:20:36.929189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/169289834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":169289834},{"type":"v1","addr":"172.21.15.191:6835","nonce":169289834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8061,"qdb_cluster":[8061]},"id":10}]} 2024-09-21T02:20:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 90 2024-09-21T02:20:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b30073830 msgr2=0x7f7b30075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b30073830 0x7f7b30075d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0058d0 tx=0x7f7b3c005860 comp rx=0 tx=0).stop 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 msgr2=0x7f7b4c15e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 0x7f7b4c15e140 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c730 tx=0x7f7b4800cbf0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 shutdown_connections 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b30073830 0x7f7b30075d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b4c0b5410 0x7f7b4c14b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b4c0ccb00 0x7f7b4c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 --2- 172.21.15.170:0/635884237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b4c0b2f40 0x7f7b4c15e140 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 >> 172.21.15.170:0/635884237 conn(0x7f7b4c0b1c40 msgr2=0x7f7b4c0c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 shutdown_connections 2024-09-21T02:20:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:37.913+0000 7f7b53c70640 1 -- 172.21.15.170:0/635884237 wait complete. 2024-09-21T02:20:37.933 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4336562640 1 -- 172.21.15.170:0/2268029253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4318004810 msgr2=0x7f4318004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4336562640 1 --2- 172.21.15.170:0/2268029253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4318004810 0x7f4318004c20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4324009b60 tx=0x7f432401d2b0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4334aff640 1 -- 172.21.15.170:0/2268029253 reap_dead start 2024-09-21T02:20:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4336562640 1 -- 172.21.15.170:0/2268029253 shutdown_connections 2024-09-21T02:20:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4336562640 1 -- 172.21.15.170:0/2268029253 >> 172.21.15.170:0/2268029253 conn(0x7f43300b1c40 msgr2=0x7f43300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4336562640 1 -- 172.21.15.170:0/2268029253 shutdown_connections 2024-09-21T02:20:38.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.089+0000 7f4336562640 1 -- 172.21.15.170:0/2268029253 wait complete. 2024-09-21T02:20:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 Processor -- start 2024-09-21T02:20:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 -- start start 2024-09-21T02:20:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318004810 0x7f43301f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 0x7f433009c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300b0100 0x7f43300b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43300cd770 con 0x7f43301e8590 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f43300cd5f0 con 0x7f43300b0100 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4330070340 con 0x7f4318004810 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 0x7f433009c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 0x7f433009c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50646/0 (socket says 172.21.15.170:50646) 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 -- 172.21.15.170:0/2564116023 learned_addr learned my addr 172.21.15.170:0/2564116023 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432ffff640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318004810 0x7f43301f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4334aff640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300b0100 0x7f43300b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 -- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318004810 msgr2=0x7f43301f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318004810 0x7f43301f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 -- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300b0100 msgr2=0x7f43300b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300b0100 0x7f43300b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 -- 172.21.15.170:0/2564116023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300b0a50 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4334aff640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300b0100 0x7f43300b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432f7fe640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 0x7f433009c450 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4324009a80 tx=0x7f43240055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240bd070 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43240b3040 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240b2070 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43301e8c70 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.093+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43301e9230 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.097+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43300708d0 con 0x7f43301e8590 2024-09-21T02:20:38.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.097+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f43240c0070 con 0x7f43301e8590 2024-09-21T02:20:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.097+0000 7f432cff9640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f431c073830 0x7f431c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.097+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f432412f3f0 con 0x7f43301e8590 2024-09-21T02:20:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.097+0000 7f432ffff640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f431c073830 0x7f431c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.097+0000 7f432ffff640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f431c073830 0x7f431c075d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4320005ec0 tx=0x7f43200091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.101+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f43300b0a50 con 0x7f43301e8590 2024-09-21T02:20:38.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.277+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43301e8a00 con 0x7f43301e8590 2024-09-21T02:20:38.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.281+0000 7f432cff9640 1 -- 172.21.15.170:0/2564116023 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16831 (secure 0 0 0) 0x7f4324016fc0 con 0x7f43301e8590 2024-09-21T02:20:38.282 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:38.282 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":130,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:20:35.480836+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:20:31.581917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T02:20:32.865501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f431c073830 msgr2=0x7f431c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f431c073830 0x7f431c075d40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4320005ec0 tx=0x7f43200091e0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 msgr2=0x7f433009c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 0x7f433009c450 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f4324009a80 tx=0x7f43240055d0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 shutdown_connections 2024-09-21T02:20:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f431c073830 0x7f431c075d40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300b0100 0x7f43300b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43301e8590 0x7f433009c450 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 --2- 172.21.15.170:0/2564116023 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4318004810 0x7f43301f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 >> 172.21.15.170:0/2564116023 conn(0x7f43300b1c40 msgr2=0x7f433006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 shutdown_connections 2024-09-21T02:20:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.285+0000 7f4336562640 1 -- 172.21.15.170:0/2564116023 wait complete. 2024-09-21T02:20:38.305 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:20:38.305 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:20:38.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 -- 172.21.15.170:0/328130141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 msgr2=0x7fb88406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 --2- 172.21.15.170:0/328130141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb88406fa30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009840 tx=0x7fb87c01c820 comp rx=0 tx=0).stop 2024-09-21T02:20:38.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 -- 172.21.15.170:0/328130141 shutdown_connections 2024-09-21T02:20:38.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 --2- 172.21.15.170:0/328130141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8840cd210 0x7fb8840b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 --2- 172.21.15.170:0/328130141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb88406fa30 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 -- 172.21.15.170:0/328130141 >> 172.21.15.170:0/328130141 conn(0x7fb8840b1c40 msgr2=0x7fb8840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:38.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 -- 172.21.15.170:0/328130141 shutdown_connections 2024-09-21T02:20:38.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 -- 172.21.15.170:0/328130141 wait complete. 2024-09-21T02:20:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.461+0000 7fb889a47640 1 Processor -- start 2024-09-21T02:20:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- start start 2024-09-21T02:20:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8840b2f40 0x7fb884158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb884156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8840c6480 0x7fb8841573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb884169fd0 con 0x7fb8840b5410 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb884169e50 con 0x7fb8840b2f40 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb88416a150 con 0x7fb8840c6480 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb884156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb884156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50658/0 (socket says 172.21.15.170:50658) 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 -- 172.21.15.170:0/812983084 learned_addr learned my addr 172.21.15.170:0/812983084 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8837fe640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8840c6480 0x7fb8841573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb882ffd640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8840b2f40 0x7fb884158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 -- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8840c6480 msgr2=0x7fb8841573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8840c6480 0x7fb8841573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 -- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8840b2f40 msgr2=0x7fb884158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8840b2f40 0x7fb884158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 -- 172.21.15.170:0/812983084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8841589d0 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb882ffd640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8840b2f40 0x7fb884158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb8827fc640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb884156ea0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00cbf0 tx=0x7fb86c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c016020 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb86c00ab40 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb884074c80 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c012420 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8840c5c80 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.465+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8840749f0 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.469+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fb86c019020 con 0x7fb8840b5410 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.469+0000 7fb87bfff640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb864073860 0x7fb864075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.469+0000 7fb882ffd640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb864073860 0x7fb864075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.469+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fb86c095150 con 0x7fb8840b5410 2024-09-21T02:20:38.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.469+0000 7fb882ffd640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb864073860 0x7fb864075d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0af000 tx=0x7fb87c01b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:38.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.473+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb86c020020 con 0x7fb8840b5410 2024-09-21T02:20:38.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.645+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8840c3a10 con 0x7fb864073860 2024-09-21T02:20:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.657+0000 7fb87bfff640 1 -- 172.21.15.170:0/812983084 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7fb8840c3a10 con 0x7fb864073860 2024-09-21T02:20:38.661 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:38.662 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:20:38.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb864073860 msgr2=0x7fb864075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb864073860 0x7fb864075d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0af000 tx=0x7fb87c01b060 comp rx=0 tx=0).stop 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 msgr2=0x7fb884156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb884156ea0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00cbf0 tx=0x7fb86c00cce0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 shutdown_connections 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb864073860 0x7fb864075d70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8840c6480 0x7fb8841573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8840b5410 0x7fb884156ea0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 --2- 172.21.15.170:0/812983084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8840b2f40 0x7fb884158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 >> 172.21.15.170:0/812983084 conn(0x7fb8840b1c40 msgr2=0x7fb88415c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 shutdown_connections 2024-09-21T02:20:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.661+0000 7fb889a47640 1 -- 172.21.15.170:0/812983084 wait complete. 2024-09-21T02:20:38.683 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-21T02:20:36.908376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288156,"kb_used_data":13668,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754686564,"statfs":{"total":773094113280,"available":772799041536,"internally_reserved":0,"allocated":13996032,"data_stored":6807108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":280874545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.444818"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538984+0000","last_change":"2024-09-21T02:20:34.535065+0000","last_active":"2024-09-21T02:20:34.538984+0000","last_peered":"2024-09-21T02:20:34.538984+0000","last_clean":"2024-09-21T02:20:34.538984+0000","last_became_active":"2024-09-21T02:20:34.534089+0000","last_became_peered":"2024-09-21T02:20:34.534089+0000","last_unstale":"2024-09-21T02:20:34.538984+0000","last_undegraded":"2024-09-21T02:20:34.538984+0000","last_fullsized":"2024-09-21T02:20:34.538984+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:50.290788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528806+0000","last_change":"2024-09-21T02:20:34.522400+0000","last_active":"2024-09-21T02:20:34.528806+0000","last_peered":"2024-09-21T02:20:34.528806+0000","last_clean":"2024-09-21T02:20:34.528806+0000","last_became_active":"2024-09-21T02:20:34.517969+0000","last_became_peered":"2024-09-21T02:20:34.517969+0000","last_unstale":"2024-09-21T02:20:34.528806+0000","last_undegraded":"2024-09-21T02:20:34.528806+0000","last_fullsized":"2024-09-21T02:20:34.528806+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:52.787462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544910+0000","last_change":"2024-09-21T02:20:34.543954+0000","last_active":"2024-09-21T02:20:34.544910+0000","last_peered":"2024-09-21T02:20:34.544910+0000","last_clean":"2024-09-21T02:20:34.544910+0000","last_became_active":"2024-09-21T02:20:34.543638+0000","last_became_peered":"2024-09-21T02:20:34.543638+0000","last_unstale":"2024-09-21T02:20:34.544910+0000","last_undegraded":"2024-09-21T02:20:34.544910+0000","last_fullsized":"2024-09-21T02:20:34.544910+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:27.676880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535340+0000","last_change":"2024-09-21T02:20:34.527002+0000","last_active":"2024-09-21T02:20:34.535340+0000","last_peered":"2024-09-21T02:20:34.535340+0000","last_clean":"2024-09-21T02:20:34.535340+0000","last_became_active":"2024-09-21T02:20:34.524614+0000","last_became_peered":"2024-09-21T02:20:34.524614+0000","last_unstale":"2024-09-21T02:20:34.535340+0000","last_undegraded":"2024-09-21T02:20:34.535340+0000","last_fullsized":"2024-09-21T02:20:34.535340+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:58.332275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.512426+0000","last_change":"2024-09-21T02:20:34.499734+0000","last_active":"2024-09-21T02:20:34.512426+0000","last_peered":"2024-09-21T02:20:34.512426+0000","last_clean":"2024-09-21T02:20:34.512426+0000","last_became_active":"2024-09-21T02:20:34.497863+0000","last_became_peered":"2024-09-21T02:20:34.497863+0000","last_unstale":"2024-09-21T02:20:34.512426+0000","last_undegraded":"2024-09-21T02:20:34.512426+0000","last_fullsized":"2024-09-21T02:20:34.512426+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:47.006213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542687+0000","last_change":"2024-09-21T02:20:34.540900+0000","last_active":"2024-09-21T02:20:34.542687+0000","last_peered":"2024-09-21T02:20:34.542687+0000","last_clean":"2024-09-21T02:20:34.542687+0000","last_became_active":"2024-09-21T02:20:34.531895+0000","last_became_peered":"2024-09-21T02:20:34.531895+0000","last_unstale":"2024-09-21T02:20:34.542687+0000","last_undegraded":"2024-09-21T02:20:34.542687+0000","last_fullsized":"2024-09-21T02:20:34.542687+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:52.537505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.504405+0000","last_change":"2024-09-21T02:20:34.496183+0000","last_active":"2024-09-21T02:20:34.504405+0000","last_peered":"2024-09-21T02:20:34.504405+0000","last_clean":"2024-09-21T02:20:34.504405+0000","last_became_active":"2024-09-21T02:20:34.495285+0000","last_became_peered":"2024-09-21T02:20:34.495285+0000","last_unstale":"2024-09-21T02:20:34.504405+0000","last_undegraded":"2024-09-21T02:20:34.504405+0000","last_fullsized":"2024-09-21T02:20:34.504405+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:27.665488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541001+0000","last_change":"2024-09-21T02:20:34.539212+0000","last_active":"2024-09-21T02:20:34.541001+0000","last_peered":"2024-09-21T02:20:34.541001+0000","last_clean":"2024-09-21T02:20:34.541001+0000","last_became_active":"2024-09-21T02:20:34.536871+0000","last_became_peered":"2024-09-21T02:20:34.536871+0000","last_unstale":"2024-09-21T02:20:34.541001+0000","last_undegraded":"2024-09-21T02:20:34.541001+0000","last_fullsized":"2024-09-21T02:20:34.541001+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:58.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541967+0000","last_change":"2024-09-21T02:20:34.530830+0000","last_active":"2024-09-21T02:20:34.541967+0000","last_peered":"2024-09-21T02:20:34.541967+0000","last_clean":"2024-09-21T02:20:34.541967+0000","last_became_active":"2024-09-21T02:20:34.519606+0000","last_became_peered":"2024-09-21T02:20:34.519606+0000","last_unstale":"2024-09-21T02:20:34.541967+0000","last_undegraded":"2024-09-21T02:20:34.541967+0000","last_fullsized":"2024-09-21T02:20:34.541967+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:03.634018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.522869+0000","last_change":"2024-09-21T02:20:34.520833+0000","last_active":"2024-09-21T02:20:34.522869+0000","last_peered":"2024-09-21T02:20:34.522869+0000","last_clean":"2024-09-21T02:20:34.522869+0000","last_became_active":"2024-09-21T02:20:34.519981+0000","last_became_peered":"2024-09-21T02:20:34.519981+0000","last_unstale":"2024-09-21T02:20:34.522869+0000","last_undegraded":"2024-09-21T02:20:34.522869+0000","last_fullsized":"2024-09-21T02:20:34.522869+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:48.915339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542732+0000","last_change":"2024-09-21T02:20:34.535099+0000","last_active":"2024-09-21T02:20:34.542732+0000","last_peered":"2024-09-21T02:20:34.542732+0000","last_clean":"2024-09-21T02:20:34.542732+0000","last_became_active":"2024-09-21T02:20:34.533790+0000","last_became_peered":"2024-09-21T02:20:34.533790+0000","last_unstale":"2024-09-21T02:20:34.542732+0000","last_undegraded":"2024-09-21T02:20:34.542732+0000","last_fullsized":"2024-09-21T02:20:34.542732+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:36.277141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542039+0000","last_change":"2024-09-21T02:20:34.532103+0000","last_active":"2024-09-21T02:20:34.542039+0000","last_peered":"2024-09-21T02:20:34.542039+0000","last_clean":"2024-09-21T02:20:34.542039+0000","last_became_active":"2024-09-21T02:20:34.531266+0000","last_became_peered":"2024-09-21T02:20:34.531266+0000","last_unstale":"2024-09-21T02:20:34.542039+0000","last_undegraded":"2024-09-21T02:20:34.542039+0000","last_fullsized":"2024-09-21T02:20:34.542039+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:41.281457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.512501+0000","last_change":"2024-09-21T02:20:34.500672+0000","last_active":"2024-09-21T02:20:34.512501+0000","last_peered":"2024-09-21T02:20:34.512501+0000","last_clean":"2024-09-21T02:20:34.512501+0000","last_became_active":"2024-09-21T02:20:34.498928+0000","last_became_peered":"2024-09-21T02:20:34.498928+0000","last_unstale":"2024-09-21T02:20:34.512501+0000","last_undegraded":"2024-09-21T02:20:34.512501+0000","last_fullsized":"2024-09-21T02:20:34.512501+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.305564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542266+0000","last_change":"2024-09-21T02:20:34.536443+0000","last_active":"2024-09-21T02:20:34.542266+0000","last_peered":"2024-09-21T02:20:34.542266+0000","last_clean":"2024-09-21T02:20:34.542266+0000","last_became_active":"2024-09-21T02:20:34.534148+0000","last_became_peered":"2024-09-21T02:20:34.534148+0000","last_unstale":"2024-09-21T02:20:34.542266+0000","last_undegraded":"2024-09-21T02:20:34.542266+0000","last_fullsized":"2024-09-21T02:20:34.542266+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:44.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.522368+0000","last_change":"2024-09-21T02:20:34.515963+0000","last_active":"2024-09-21T02:20:34.522368+0000","last_peered":"2024-09-21T02:20:34.522368+0000","last_clean":"2024-09-21T02:20:34.522368+0000","last_became_active":"2024-09-21T02:20:34.515002+0000","last_became_peered":"2024-09-21T02:20:34.515002+0000","last_unstale":"2024-09-21T02:20:34.522368+0000","last_undegraded":"2024-09-21T02:20:34.522368+0000","last_fullsized":"2024-09-21T02:20:34.522368+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:42.482933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.516527+0000","last_change":"2024-09-21T02:20:34.511843+0000","last_active":"2024-09-21T02:20:34.516527+0000","last_peered":"2024-09-21T02:20:34.516527+0000","last_clean":"2024-09-21T02:20:34.516527+0000","last_became_active":"2024-09-21T02:20:34.510663+0000","last_became_peered":"2024-09-21T02:20:34.510663+0000","last_unstale":"2024-09-21T02:20:34.516527+0000","last_undegraded":"2024-09-21T02:20:34.516527+0000","last_fullsized":"2024-09-21T02:20:34.516527+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:59.433398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538096+0000","last_change":"2024-09-21T02:20:34.536434+0000","last_active":"2024-09-21T02:20:34.538096+0000","last_peered":"2024-09-21T02:20:34.538096+0000","last_clean":"2024-09-21T02:20:34.538096+0000","last_became_active":"2024-09-21T02:20:34.534129+0000","last_became_peered":"2024-09-21T02:20:34.534129+0000","last_unstale":"2024-09-21T02:20:34.538096+0000","last_undegraded":"2024-09-21T02:20:34.538096+0000","last_fullsized":"2024-09-21T02:20:34.538096+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:16.346709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541103+0000","last_change":"2024-09-21T02:20:34.535012+0000","last_active":"2024-09-21T02:20:34.541103+0000","last_peered":"2024-09-21T02:20:34.541103+0000","last_clean":"2024-09-21T02:20:34.541103+0000","last_became_active":"2024-09-21T02:20:34.534322+0000","last_became_peered":"2024-09-21T02:20:34.534322+0000","last_unstale":"2024-09-21T02:20:34.541103+0000","last_undegraded":"2024-09-21T02:20:34.541103+0000","last_fullsized":"2024-09-21T02:20:34.541103+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:00.344661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542884+0000","last_change":"2024-09-21T02:20:34.542161+0000","last_active":"2024-09-21T02:20:34.542884+0000","last_peered":"2024-09-21T02:20:34.542884+0000","last_clean":"2024-09-21T02:20:34.542884+0000","last_became_active":"2024-09-21T02:20:34.541774+0000","last_became_peered":"2024-09-21T02:20:34.541774+0000","last_unstale":"2024-09-21T02:20:34.542884+0000","last_undegraded":"2024-09-21T02:20:34.542884+0000","last_fullsized":"2024-09-21T02:20:34.542884+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:44.732555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528392+0000","last_change":"2024-09-21T02:20:34.522809+0000","last_active":"2024-09-21T02:20:34.528392+0000","last_peered":"2024-09-21T02:20:34.528392+0000","last_clean":"2024-09-21T02:20:34.528392+0000","last_became_active":"2024-09-21T02:20:34.522000+0000","last_became_peered":"2024-09-21T02:20:34.522000+0000","last_unstale":"2024-09-21T02:20:34.528392+0000","last_undegraded":"2024-09-21T02:20:34.528392+0000","last_fullsized":"2024-09-21T02:20:34.528392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:31.419902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538891+0000","last_change":"2024-09-21T02:20:34.536391+0000","last_active":"2024-09-21T02:20:34.538891+0000","last_peered":"2024-09-21T02:20:34.538891+0000","last_clean":"2024-09-21T02:20:34.538891+0000","last_became_active":"2024-09-21T02:20:34.535614+0000","last_became_peered":"2024-09-21T02:20:34.535614+0000","last_unstale":"2024-09-21T02:20:34.538891+0000","last_undegraded":"2024-09-21T02:20:34.538891+0000","last_fullsized":"2024-09-21T02:20:34.538891+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:15.577194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542187+0000","last_change":"2024-09-21T02:20:34.527995+0000","last_active":"2024-09-21T02:20:34.542187+0000","last_peered":"2024-09-21T02:20:34.542187+0000","last_clean":"2024-09-21T02:20:34.542187+0000","last_became_active":"2024-09-21T02:20:34.526274+0000","last_became_peered":"2024-09-21T02:20:34.526274+0000","last_unstale":"2024-09-21T02:20:34.542187+0000","last_undegraded":"2024-09-21T02:20:34.542187+0000","last_fullsized":"2024-09-21T02:20:34.542187+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538229+0000","last_change":"2024-09-21T02:20:34.536767+0000","last_active":"2024-09-21T02:20:34.538229+0000","last_peered":"2024-09-21T02:20:34.538229+0000","last_clean":"2024-09-21T02:20:34.538229+0000","last_became_active":"2024-09-21T02:20:34.536063+0000","last_became_peered":"2024-09-21T02:20:34.536063+0000","last_unstale":"2024-09-21T02:20:34.538229+0000","last_undegraded":"2024-09-21T02:20:34.538229+0000","last_fullsized":"2024-09-21T02:20:34.538229+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:25.693616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.510370+0000","last_change":"2024-09-21T02:20:34.507236+0000","last_active":"2024-09-21T02:20:34.510370+0000","last_peered":"2024-09-21T02:20:34.510370+0000","last_clean":"2024-09-21T02:20:34.510370+0000","last_became_active":"2024-09-21T02:20:34.506382+0000","last_became_peered":"2024-09-21T02:20:34.506382+0000","last_unstale":"2024-09-21T02:20:34.510370+0000","last_undegraded":"2024-09-21T02:20:34.510370+0000","last_fullsized":"2024-09-21T02:20:34.510370+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:25.895929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535179+0000","last_change":"2024-09-21T02:20:34.527564+0000","last_active":"2024-09-21T02:20:34.535179+0000","last_peered":"2024-09-21T02:20:34.535179+0000","last_clean":"2024-09-21T02:20:34.535179+0000","last_became_active":"2024-09-21T02:20:34.525977+0000","last_became_peered":"2024-09-21T02:20:34.525977+0000","last_unstale":"2024-09-21T02:20:34.535179+0000","last_undegraded":"2024-09-21T02:20:34.535179+0000","last_fullsized":"2024-09-21T02:20:34.535179+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:56.725599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.521337+0000","last_change":"2024-09-21T02:20:34.513932+0000","last_active":"2024-09-21T02:20:34.521337+0000","last_peered":"2024-09-21T02:20:34.521337+0000","last_clean":"2024-09-21T02:20:34.521337+0000","last_became_active":"2024-09-21T02:20:34.513037+0000","last_became_peered":"2024-09-21T02:20:34.513037+0000","last_unstale":"2024-09-21T02:20:34.521337+0000","last_undegraded":"2024-09-21T02:20:34.521337+0000","last_fullsized":"2024-09-21T02:20:34.521337+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:47.843405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.529367+0000","last_change":"2024-09-21T02:20:34.522766+0000","last_active":"2024-09-21T02:20:34.529367+0000","last_peered":"2024-09-21T02:20:34.529367+0000","last_clean":"2024-09-21T02:20:34.529367+0000","last_became_active":"2024-09-21T02:20:34.521882+0000","last_became_peered":"2024-09-21T02:20:34.521882+0000","last_unstale":"2024-09-21T02:20:34.529367+0000","last_undegraded":"2024-09-21T02:20:34.529367+0000","last_fullsized":"2024-09-21T02:20:34.529367+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:23.141714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.534433+0000","last_change":"2024-09-21T02:20:34.523927+0000","last_active":"2024-09-21T02:20:34.534433+0000","last_peered":"2024-09-21T02:20:34.534433+0000","last_clean":"2024-09-21T02:20:34.534433+0000","last_became_active":"2024-09-21T02:20:34.523087+0000","last_became_peered":"2024-09-21T02:20:34.523087+0000","last_unstale":"2024-09-21T02:20:34.534433+0000","last_undegraded":"2024-09-21T02:20:34.534433+0000","last_fullsized":"2024-09-21T02:20:34.534433+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:26.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541267+0000","last_change":"2024-09-21T02:20:34.540061+0000","last_active":"2024-09-21T02:20:34.541267+0000","last_peered":"2024-09-21T02:20:34.541267+0000","last_clean":"2024-09-21T02:20:34.541267+0000","last_became_active":"2024-09-21T02:20:34.533434+0000","last_became_peered":"2024-09-21T02:20:34.533434+0000","last_unstale":"2024-09-21T02:20:34.541267+0000","last_undegraded":"2024-09-21T02:20:34.541267+0000","last_fullsized":"2024-09-21T02:20:34.541267+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:57.569357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544173+0000","last_change":"2024-09-21T02:20:34.541320+0000","last_active":"2024-09-21T02:20:34.544173+0000","last_peered":"2024-09-21T02:20:34.544173+0000","last_clean":"2024-09-21T02:20:34.544173+0000","last_became_active":"2024-09-21T02:20:34.533413+0000","last_became_peered":"2024-09-21T02:20:34.533413+0000","last_unstale":"2024-09-21T02:20:34.544173+0000","last_undegraded":"2024-09-21T02:20:34.544173+0000","last_fullsized":"2024-09-21T02:20:34.544173+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:12.370349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528719+0000","last_change":"2024-09-21T02:20:34.522545+0000","last_active":"2024-09-21T02:20:34.528719+0000","last_peered":"2024-09-21T02:20:34.528719+0000","last_clean":"2024-09-21T02:20:34.528719+0000","last_became_active":"2024-09-21T02:20:34.519913+0000","last_became_peered":"2024-09-21T02:20:34.519913+0000","last_unstale":"2024-09-21T02:20:34.528719+0000","last_undegraded":"2024-09-21T02:20:34.528719+0000","last_fullsized":"2024-09-21T02:20:34.528719+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:20.745861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540353+0000","last_change":"2024-09-21T02:20:34.536997+0000","last_active":"2024-09-21T02:20:34.540353+0000","last_peered":"2024-09-21T02:20:34.540353+0000","last_clean":"2024-09-21T02:20:34.540353+0000","last_became_active":"2024-09-21T02:20:34.536171+0000","last_became_peered":"2024-09-21T02:20:34.536171+0000","last_unstale":"2024-09-21T02:20:34.540353+0000","last_undegraded":"2024-09-21T02:20:34.540353+0000","last_fullsized":"2024-09-21T02:20:34.540353+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:58.178828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.504059+0000","last_change":"2024-09-21T02:20:34.497138+0000","last_active":"2024-09-21T02:20:34.504059+0000","last_peered":"2024-09-21T02:20:34.504059+0000","last_clean":"2024-09-21T02:20:34.504059+0000","last_became_active":"2024-09-21T02:20:34.496270+0000","last_became_peered":"2024-09-21T02:20:34.496270+0000","last_unstale":"2024-09-21T02:20:34.504059+0000","last_undegraded":"2024-09-21T02:20:34.504059+0000","last_fullsized":"2024-09-21T02:20:34.504059+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:06.844713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"129'4","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.556472+0000","last_change":"2024-09-21T02:20:33.529111+0000","last_active":"2024-09-21T02:20:34.556472+0000","last_peered":"2024-09-21T02:20:34.556472+0000","last_clean":"2024-09-21T02:20:34.556472+0000","last_became_active":"2024-09-21T02:20:33.528609+0000","last_became_peered":"2024-09-21T02:20:33.528609+0000","last_unstale":"2024-09-21T02:20:34.556472+0000","last_undegraded":"2024-09-21T02:20:34.556472+0000","last_fullsized":"2024-09-21T02:20:34.556472+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:17.603041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.520421+0000","last_change":"2024-09-21T02:20:34.516689+0000","last_active":"2024-09-21T02:20:34.520421+0000","last_peered":"2024-09-21T02:20:34.520421+0000","last_clean":"2024-09-21T02:20:34.520421+0000","last_became_active":"2024-09-21T02:20:34.515826+0000","last_became_peered":"2024-09-21T02:20:34.515826+0000","last_unstale":"2024-09-21T02:20:34.520421+0000","last_undegraded":"2024-09-21T02:20:34.520421+0000","last_fullsized":"2024-09-21T02:20:34.520421+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:17.486614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552581+0000","last_change":"2024-09-21T02:20:33.506795+0000","last_active":"2024-09-21T02:20:34.552581+0000","last_peered":"2024-09-21T02:20:34.552581+0000","last_clean":"2024-09-21T02:20:34.552581+0000","last_became_active":"2024-09-21T02:20:33.505984+0000","last_became_peered":"2024-09-21T02:20:33.505984+0000","last_unstale":"2024-09-21T02:20:34.552581+0000","last_undegraded":"2024-09-21T02:20:34.552581+0000","last_fullsized":"2024-09-21T02:20:34.552581+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:21.892699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541996+0000","last_change":"2024-09-21T02:20:34.530853+0000","last_active":"2024-09-21T02:20:34.541996+0000","last_peered":"2024-09-21T02:20:34.541996+0000","last_clean":"2024-09-21T02:20:34.541996+0000","last_became_active":"2024-09-21T02:20:34.519747+0000","last_became_peered":"2024-09-21T02:20:34.519747+0000","last_unstale":"2024-09-21T02:20:34.541996+0000","last_undegraded":"2024-09-21T02:20:34.541996+0000","last_fullsized":"2024-09-21T02:20:34.541996+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:48.807205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.551289+0000","last_change":"2024-09-21T02:20:33.517342+0000","last_active":"2024-09-21T02:20:34.551289+0000","last_peered":"2024-09-21T02:20:34.551289+0000","last_clean":"2024-09-21T02:20:34.551289+0000","last_became_active":"2024-09-21T02:20:33.517029+0000","last_became_peered":"2024-09-21T02:20:33.517029+0000","last_unstale":"2024-09-21T02:20:34.551289+0000","last_undegraded":"2024-09-21T02:20:34.551289+0000","last_fullsized":"2024-09-21T02:20:34.551289+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:19.048319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535540+0000","last_change":"2024-09-21T02:20:34.528802+0000","last_active":"2024-09-21T02:20:34.535540+0000","last_peered":"2024-09-21T02:20:34.535540+0000","last_clean":"2024-09-21T02:20:34.535540+0000","last_became_active":"2024-09-21T02:20:34.527960+0000","last_became_peered":"2024-09-21T02:20:34.527960+0000","last_unstale":"2024-09-21T02:20:34.535540+0000","last_undegraded":"2024-09-21T02:20:34.535540+0000","last_fullsized":"2024-09-21T02:20:34.535540+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:05.932488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.554266+0000","last_change":"2024-09-21T02:20:33.516427+0000","last_active":"2024-09-21T02:20:34.554266+0000","last_peered":"2024-09-21T02:20:34.554266+0000","last_clean":"2024-09-21T02:20:34.554266+0000","last_became_active":"2024-09-21T02:20:33.515615+0000","last_became_peered":"2024-09-21T02:20:33.515615+0000","last_unstale":"2024-09-21T02:20:34.554266+0000","last_undegraded":"2024-09-21T02:20:34.554266+0000","last_fullsized":"2024-09-21T02:20:34.554266+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:57.979056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538816+0000","last_change":"2024-09-21T02:20:34.531195+0000","last_active":"2024-09-21T02:20:34.538816+0000","last_peered":"2024-09-21T02:20:34.538816+0000","last_clean":"2024-09-21T02:20:34.538816+0000","last_became_active":"2024-09-21T02:20:34.530312+0000","last_became_peered":"2024-09-21T02:20:34.530312+0000","last_unstale":"2024-09-21T02:20:34.538816+0000","last_undegraded":"2024-09-21T02:20:34.538816+0000","last_fullsized":"2024-09-21T02:20:34.538816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:41.222255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552657+0000","last_change":"2024-09-21T02:20:33.512665+0000","last_active":"2024-09-21T02:20:34.552657+0000","last_peered":"2024-09-21T02:20:34.552657+0000","last_clean":"2024-09-21T02:20:34.552657+0000","last_became_active":"2024-09-21T02:20:33.511943+0000","last_became_peered":"2024-09-21T02:20:33.511943+0000","last_unstale":"2024-09-21T02:20:34.552657+0000","last_undegraded":"2024-09-21T02:20:34.552657+0000","last_fullsized":"2024-09-21T02:20:34.552657+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:46.390258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535017+0000","last_change":"2024-09-21T02:20:34.527064+0000","last_active":"2024-09-21T02:20:34.535017+0000","last_peered":"2024-09-21T02:20:34.535017+0000","last_clean":"2024-09-21T02:20:34.535017+0000","last_became_active":"2024-09-21T02:20:34.525286+0000","last_became_peered":"2024-09-21T02:20:34.525286+0000","last_unstale":"2024-09-21T02:20:34.535017+0000","last_undegraded":"2024-09-21T02:20:34.535017+0000","last_fullsized":"2024-09-21T02:20:34.535017+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.484525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552751+0000","last_change":"2024-09-21T02:20:33.498751+0000","last_active":"2024-09-21T02:20:34.552751+0000","last_peered":"2024-09-21T02:20:34.552751+0000","last_clean":"2024-09-21T02:20:34.552751+0000","last_became_active":"2024-09-21T02:20:33.498223+0000","last_became_peered":"2024-09-21T02:20:33.498223+0000","last_unstale":"2024-09-21T02:20:34.552751+0000","last_undegraded":"2024-09-21T02:20:34.552751+0000","last_fullsized":"2024-09-21T02:20:34.552751+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:12.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540321+0000","last_change":"2024-09-21T02:20:34.536805+0000","last_active":"2024-09-21T02:20:34.540321+0000","last_peered":"2024-09-21T02:20:34.540321+0000","last_clean":"2024-09-21T02:20:34.540321+0000","last_became_active":"2024-09-21T02:20:34.536024+0000","last_became_peered":"2024-09-21T02:20:34.536024+0000","last_unstale":"2024-09-21T02:20:34.540321+0000","last_undegraded":"2024-09-21T02:20:34.540321+0000","last_fullsized":"2024-09-21T02:20:34.540321+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.980670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.515981+0000","last_change":"2024-09-21T02:20:33.518261+0000","last_active":"2024-09-21T02:20:34.515981+0000","last_peered":"2024-09-21T02:20:34.515981+0000","last_clean":"2024-09-21T02:20:34.515981+0000","last_became_active":"2024-09-21T02:20:33.517593+0000","last_became_peered":"2024-09-21T02:20:33.517593+0000","last_unstale":"2024-09-21T02:20:34.515981+0000","last_undegraded":"2024-09-21T02:20:34.515981+0000","last_fullsized":"2024-09-21T02:20:34.515981+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:36.813034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.532694+0000","last_change":"2024-09-21T02:20:34.524314+0000","last_active":"2024-09-21T02:20:34.532694+0000","last_peered":"2024-09-21T02:20:34.532694+0000","last_clean":"2024-09-21T02:20:34.532694+0000","last_became_active":"2024-09-21T02:20:34.523286+0000","last_became_peered":"2024-09-21T02:20:34.523286+0000","last_unstale":"2024-09-21T02:20:34.532694+0000","last_undegraded":"2024-09-21T02:20:34.532694+0000","last_fullsized":"2024-09-21T02:20:34.532694+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:41.222255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.513043+0000","last_change":"2024-09-21T02:20:33.502584+0000","last_active":"2024-09-21T02:20:34.513043+0000","last_peered":"2024-09-21T02:20:34.513043+0000","last_clean":"2024-09-21T02:20:34.513043+0000","last_became_active":"2024-09-21T02:20:33.502021+0000","last_became_peered":"2024-09-21T02:20:33.502021+0000","last_unstale":"2024-09-21T02:20:34.513043+0000","last_undegraded":"2024-09-21T02:20:34.513043+0000","last_fullsized":"2024-09-21T02:20:34.513043+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:23.689333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544811+0000","last_change":"2024-09-21T02:20:34.543988+0000","last_active":"2024-09-21T02:20:34.544811+0000","last_peered":"2024-09-21T02:20:34.544811+0000","last_clean":"2024-09-21T02:20:34.544811+0000","last_became_active":"2024-09-21T02:20:34.543398+0000","last_became_peered":"2024-09-21T02:20:34.543398+0000","last_unstale":"2024-09-21T02:20:34.544811+0000","last_undegraded":"2024-09-21T02:20:34.544811+0000","last_fullsized":"2024-09-21T02:20:34.544811+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:07.573024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.515203+0000","last_change":"2024-09-21T02:20:33.529086+0000","last_active":"2024-09-21T02:20:34.515203+0000","last_peered":"2024-09-21T02:20:34.515203+0000","last_clean":"2024-09-21T02:20:34.515203+0000","last_became_active":"2024-09-21T02:20:33.528259+0000","last_became_peered":"2024-09-21T02:20:33.528259+0000","last_unstale":"2024-09-21T02:20:34.515203+0000","last_undegraded":"2024-09-21T02:20:34.515203+0000","last_fullsized":"2024-09-21T02:20:34.515203+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:28.432138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540057+0000","last_change":"2024-09-21T02:20:34.524233+0000","last_active":"2024-09-21T02:20:34.540057+0000","last_peered":"2024-09-21T02:20:34.540057+0000","last_clean":"2024-09-21T02:20:34.540057+0000","last_became_active":"2024-09-21T02:20:34.523459+0000","last_became_peered":"2024-09-21T02:20:34.523459+0000","last_unstale":"2024-09-21T02:20:34.540057+0000","last_undegraded":"2024-09-21T02:20:34.540057+0000","last_fullsized":"2024-09-21T02:20:34.540057+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:31.943339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.488623+0000","last_change":"2024-09-21T02:20:33.499112+0000","last_active":"2024-09-21T02:20:34.488623+0000","last_peered":"2024-09-21T02:20:34.488623+0000","last_clean":"2024-09-21T02:20:34.488623+0000","last_became_active":"2024-09-21T02:20:33.498733+0000","last_became_peered":"2024-09-21T02:20:33.498733+0000","last_unstale":"2024-09-21T02:20:34.488623+0000","last_undegraded":"2024-09-21T02:20:34.488623+0000","last_fullsized":"2024-09-21T02:20:34.488623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:06.556724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.531506+0000","last_change":"2024-09-21T02:20:34.527696+0000","last_active":"2024-09-21T02:20:34.531506+0000","last_peered":"2024-09-21T02:20:34.531506+0000","last_clean":"2024-09-21T02:20:34.531506+0000","last_became_active":"2024-09-21T02:20:34.525873+0000","last_became_peered":"2024-09-21T02:20:34.525873+0000","last_unstale":"2024-09-21T02:20:34.531506+0000","last_undegraded":"2024-09-21T02:20:34.531506+0000","last_fullsized":"2024-09-21T02:20:34.531506+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:41.095762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"129'6","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.555828+0000","last_change":"2024-09-21T02:20:33.527173+0000","last_active":"2024-09-21T02:20:34.555828+0000","last_peered":"2024-09-21T02:20:34.555828+0000","last_clean":"2024-09-21T02:20:34.555828+0000","last_became_active":"2024-09-21T02:20:33.526195+0000","last_became_peered":"2024-09-21T02:20:33.526195+0000","last_unstale":"2024-09-21T02:20:34.555828+0000","last_undegraded":"2024-09-21T02:20:34.555828+0000","last_fullsized":"2024-09-21T02:20:34.555828+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:30.456111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544582+0000","last_change":"2024-09-21T02:20:34.543396+0000","last_active":"2024-09-21T02:20:34.544582+0000","last_peered":"2024-09-21T02:20:34.544582+0000","last_clean":"2024-09-21T02:20:34.544582+0000","last_became_active":"2024-09-21T02:20:34.542993+0000","last_became_peered":"2024-09-21T02:20:34.542993+0000","last_unstale":"2024-09-21T02:20:34.544582+0000","last_undegraded":"2024-09-21T02:20:34.544582+0000","last_fullsized":"2024-09-21T02:20:34.544582+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:02.896784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.553153+0000","last_change":"2024-09-21T02:20:33.517386+0000","last_active":"2024-09-21T02:20:34.553153+0000","last_peered":"2024-09-21T02:20:34.553153+0000","last_clean":"2024-09-21T02:20:34.553153+0000","last_became_active":"2024-09-21T02:20:33.517013+0000","last_became_peered":"2024-09-21T02:20:33.517013+0000","last_unstale":"2024-09-21T02:20:34.553153+0000","last_undegraded":"2024-09-21T02:20:34.553153+0000","last_fullsized":"2024-09-21T02:20:34.553153+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:17.283423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540556+0000","last_change":"2024-09-21T02:20:34.535522+0000","last_active":"2024-09-21T02:20:34.540556+0000","last_peered":"2024-09-21T02:20:34.540556+0000","last_clean":"2024-09-21T02:20:34.540556+0000","last_became_active":"2024-09-21T02:20:34.534557+0000","last_became_peered":"2024-09-21T02:20:34.534557+0000","last_unstale":"2024-09-21T02:20:34.540556+0000","last_undegraded":"2024-09-21T02:20:34.540556+0000","last_fullsized":"2024-09-21T02:20:34.540556+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:46.071286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.520882+0000","last_change":"2024-09-21T02:20:33.500292+0000","last_active":"2024-09-21T02:20:34.520882+0000","last_peered":"2024-09-21T02:20:34.520882+0000","last_clean":"2024-09-21T02:20:34.520882+0000","last_became_active":"2024-09-21T02:20:33.499871+0000","last_became_peered":"2024-09-21T02:20:33.499871+0000","last_unstale":"2024-09-21T02:20:34.520882+0000","last_undegraded":"2024-09-21T02:20:34.520882+0000","last_fullsized":"2024-09-21T02:20:34.520882+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:38.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.530514+0000","last_change":"2024-09-21T02:20:34.512961+0000","last_active":"2024-09-21T02:20:34.530514+0000","last_peered":"2024-09-21T02:20:34.530514+0000","last_clean":"2024-09-21T02:20:34.530514+0000","last_became_active":"2024-09-21T02:20:34.512138+0000","last_became_peered":"2024-09-21T02:20:34.512138+0000","last_unstale":"2024-09-21T02:20:34.530514+0000","last_undegraded":"2024-09-21T02:20:34.530514+0000","last_fullsized":"2024-09-21T02:20:34.530514+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:56.725599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.509152+0000","last_change":"2024-09-21T02:20:33.519947+0000","last_active":"2024-09-21T02:20:34.509152+0000","last_peered":"2024-09-21T02:20:34.509152+0000","last_clean":"2024-09-21T02:20:34.509152+0000","last_became_active":"2024-09-21T02:20:33.518916+0000","last_became_peered":"2024-09-21T02:20:33.518916+0000","last_unstale":"2024-09-21T02:20:34.509152+0000","last_undegraded":"2024-09-21T02:20:34.509152+0000","last_fullsized":"2024-09-21T02:20:34.509152+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:55.998249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.537231+0000","last_change":"2024-09-21T02:20:34.535034+0000","last_active":"2024-09-21T02:20:34.537231+0000","last_peered":"2024-09-21T02:20:34.537231+0000","last_clean":"2024-09-21T02:20:34.537231+0000","last_became_active":"2024-09-21T02:20:34.532471+0000","last_became_peered":"2024-09-21T02:20:34.532471+0000","last_unstale":"2024-09-21T02:20:34.537231+0000","last_undegraded":"2024-09-21T02:20:34.537231+0000","last_fullsized":"2024-09-21T02:20:34.537231+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:17.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"129'8","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.555694+0000","last_change":"2024-09-21T02:20:33.522068+0000","last_active":"2024-09-21T02:20:34.555694+0000","last_peered":"2024-09-21T02:20:34.555694+0000","last_clean":"2024-09-21T02:20:34.555694+0000","last_became_active":"2024-09-21T02:20:33.516617+0000","last_became_peered":"2024-09-21T02:20:33.516617+0000","last_unstale":"2024-09-21T02:20:34.555694+0000","last_undegraded":"2024-09-21T02:20:34.555694+0000","last_fullsized":"2024-09-21T02:20:34.555694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:28.917161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541032+0000","last_change":"2024-09-21T02:20:34.533927+0000","last_active":"2024-09-21T02:20:34.541032+0000","last_peered":"2024-09-21T02:20:34.541032+0000","last_clean":"2024-09-21T02:20:34.541032+0000","last_became_active":"2024-09-21T02:20:34.533141+0000","last_became_peered":"2024-09-21T02:20:34.533141+0000","last_unstale":"2024-09-21T02:20:34.541032+0000","last_undegraded":"2024-09-21T02:20:34.541032+0000","last_fullsized":"2024-09-21T02:20:34.541032+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:51.749702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.554699+0000","last_change":"2024-09-21T02:20:33.526887+0000","last_active":"2024-09-21T02:20:34.554699+0000","last_peered":"2024-09-21T02:20:34.554699+0000","last_clean":"2024-09-21T02:20:34.554699+0000","last_became_active":"2024-09-21T02:20:33.526541+0000","last_became_peered":"2024-09-21T02:20:33.526541+0000","last_unstale":"2024-09-21T02:20:34.554699+0000","last_undegraded":"2024-09-21T02:20:34.554699+0000","last_fullsized":"2024-09-21T02:20:34.554699+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:51.783547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.530062+0000","last_change":"2024-09-21T02:20:34.526735+0000","last_active":"2024-09-21T02:20:34.530062+0000","last_peered":"2024-09-21T02:20:34.530062+0000","last_clean":"2024-09-21T02:20:34.530062+0000","last_became_active":"2024-09-21T02:20:34.524993+0000","last_became_peered":"2024-09-21T02:20:34.524993+0000","last_unstale":"2024-09-21T02:20:34.530062+0000","last_undegraded":"2024-09-21T02:20:34.530062+0000","last_fullsized":"2024-09-21T02:20:34.530062+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:43.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552963+0000","last_change":"2024-09-21T02:20:33.525324+0000","last_active":"2024-09-21T02:20:34.552963+0000","last_peered":"2024-09-21T02:20:34.552963+0000","last_clean":"2024-09-21T02:20:34.552963+0000","last_became_active":"2024-09-21T02:20:33.524426+0000","last_became_peered":"2024-09-21T02:20:33.524426+0000","last_unstale":"2024-09-21T02:20:34.552963+0000","last_undegraded":"2024-09-21T02:20:34.552963+0000","last_fullsized":"2024-09-21T02:20:34.552963+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:09.549686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.518718+0000","last_change":"2024-09-21T02:20:34.511770+0000","last_active":"2024-09-21T02:20:34.518718+0000","last_peered":"2024-09-21T02:20:34.518718+0000","last_clean":"2024-09-21T02:20:34.518718+0000","last_became_active":"2024-09-21T02:20:34.510962+0000","last_became_peered":"2024-09-21T02:20:34.510962+0000","last_unstale":"2024-09-21T02:20:34.518718+0000","last_undegraded":"2024-09-21T02:20:34.518718+0000","last_fullsized":"2024-09-21T02:20:34.518718+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:41:51.265058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552507+0000","last_change":"2024-09-21T02:20:33.508423+0000","last_active":"2024-09-21T02:20:34.552507+0000","last_peered":"2024-09-21T02:20:34.552507+0000","last_clean":"2024-09-21T02:20:34.552507+0000","last_became_active":"2024-09-21T02:20:33.503663+0000","last_became_peered":"2024-09-21T02:20:33.503663+0000","last_unstale":"2024-09-21T02:20:34.552507+0000","last_undegraded":"2024-09-21T02:20:34.552507+0000","last_fullsized":"2024-09-21T02:20:34.552507+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:42.417840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542119+0000","last_change":"2024-09-21T02:20:34.534065+0000","last_active":"2024-09-21T02:20:34.542119+0000","last_peered":"2024-09-21T02:20:34.542119+0000","last_clean":"2024-09-21T02:20:34.542119+0000","last_became_active":"2024-09-21T02:20:34.531250+0000","last_became_peered":"2024-09-21T02:20:34.531250+0000","last_unstale":"2024-09-21T02:20:34.542119+0000","last_undegraded":"2024-09-21T02:20:34.542119+0000","last_fullsized":"2024-09-21T02:20:34.542119+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.439603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.511051+0000","last_change":"2024-09-21T02:20:33.527734+0000","last_active":"2024-09-21T02:20:34.511051+0000","last_peered":"2024-09-21T02:20:34.511051+0000","last_clean":"2024-09-21T02:20:34.511051+0000","last_became_active":"2024-09-21T02:20:33.526851+0000","last_became_peered":"2024-09-21T02:20:33.526851+0000","last_unstale":"2024-09-21T02:20:34.511051+0000","last_undegraded":"2024-09-21T02:20:34.511051+0000","last_fullsized":"2024-09-21T02:20:34.511051+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:14.041756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542111+0000","last_change":"2024-09-21T02:20:34.525263+0000","last_active":"2024-09-21T02:20:34.542111+0000","last_peered":"2024-09-21T02:20:34.542111+0000","last_clean":"2024-09-21T02:20:34.542111+0000","last_became_active":"2024-09-21T02:20:34.524346+0000","last_became_peered":"2024-09-21T02:20:34.524346+0000","last_unstale":"2024-09-21T02:20:34.542111+0000","last_undegraded":"2024-09-21T02:20:34.542111+0000","last_fullsized":"2024-09-21T02:20:34.542111+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:24.742898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552878+0000","last_change":"2024-09-21T02:20:33.511973+0000","last_active":"2024-09-21T02:20:34.552878+0000","last_peered":"2024-09-21T02:20:34.552878+0000","last_clean":"2024-09-21T02:20:34.552878+0000","last_became_active":"2024-09-21T02:20:33.511142+0000","last_became_peered":"2024-09-21T02:20:33.511142+0000","last_unstale":"2024-09-21T02:20:34.552878+0000","last_undegraded":"2024-09-21T02:20:34.552878+0000","last_fullsized":"2024-09-21T02:20:34.552878+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:28.907611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528430+0000","last_change":"2024-09-21T02:20:34.520389+0000","last_active":"2024-09-21T02:20:34.528430+0000","last_peered":"2024-09-21T02:20:34.528430+0000","last_clean":"2024-09-21T02:20:34.528430+0000","last_became_active":"2024-09-21T02:20:34.519547+0000","last_became_peered":"2024-09-21T02:20:34.519547+0000","last_unstale":"2024-09-21T02:20:34.528430+0000","last_undegraded":"2024-09-21T02:20:34.528430+0000","last_fullsized":"2024-09-21T02:20:34.528430+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:16.300803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.485199+0000","last_change":"2024-09-21T02:20:33.504251+0000","last_active":"2024-09-21T02:20:34.485199+0000","last_peered":"2024-09-21T02:20:34.485199+0000","last_clean":"2024-09-21T02:20:34.485199+0000","last_became_active":"2024-09-21T02:20:33.503581+0000","last_became_peered":"2024-09-21T02:20:33.503581+0000","last_unstale":"2024-09-21T02:20:34.485199+0000","last_undegraded":"2024-09-21T02:20:34.485199+0000","last_fullsized":"2024-09-21T02:20:34.485199+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:19.741946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542201+0000","last_change":"2024-09-21T02:20:34.537434+0000","last_active":"2024-09-21T02:20:34.542201+0000","last_peered":"2024-09-21T02:20:34.542201+0000","last_clean":"2024-09-21T02:20:34.542201+0000","last_became_active":"2024-09-21T02:20:34.536722+0000","last_became_peered":"2024-09-21T02:20:34.536722+0000","last_unstale":"2024-09-21T02:20:34.542201+0000","last_undegraded":"2024-09-21T02:20:34.542201+0000","last_fullsized":"2024-09-21T02:20:34.542201+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:07.829961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.486162+0000","last_change":"2024-09-21T02:20:33.512859+0000","last_active":"2024-09-21T02:20:34.486162+0000","last_peered":"2024-09-21T02:20:34.486162+0000","last_clean":"2024-09-21T02:20:34.486162+0000","last_became_active":"2024-09-21T02:20:33.512305+0000","last_became_peered":"2024-09-21T02:20:33.512305+0000","last_unstale":"2024-09-21T02:20:34.486162+0000","last_undegraded":"2024-09-21T02:20:34.486162+0000","last_fullsized":"2024-09-21T02:20:34.486162+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:22.137799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528639+0000","last_change":"2024-09-21T02:20:34.520589+0000","last_active":"2024-09-21T02:20:34.528639+0000","last_peered":"2024-09-21T02:20:34.528639+0000","last_clean":"2024-09-21T02:20:34.528639+0000","last_became_active":"2024-09-21T02:20:34.519701+0000","last_became_peered":"2024-09-21T02:20:34.519701+0000","last_unstale":"2024-09-21T02:20:34.528639+0000","last_undegraded":"2024-09-21T02:20:34.528639+0000","last_fullsized":"2024-09-21T02:20:34.528639+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:25.286772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.485155+0000","last_change":"2024-09-21T02:20:33.504212+0000","last_active":"2024-09-21T02:20:34.485155+0000","last_peered":"2024-09-21T02:20:34.485155+0000","last_clean":"2024-09-21T02:20:34.485155+0000","last_became_active":"2024-09-21T02:20:33.503829+0000","last_became_peered":"2024-09-21T02:20:33.503829+0000","last_unstale":"2024-09-21T02:20:34.485155+0000","last_undegraded":"2024-09-21T02:20:34.485155+0000","last_fullsized":"2024-09-21T02:20:34.485155+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:24.282858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.534778+0000","last_change":"2024-09-21T02:20:34.512804+0000","last_active":"2024-09-21T02:20:34.534778+0000","last_peered":"2024-09-21T02:20:34.534778+0000","last_clean":"2024-09-21T02:20:34.534778+0000","last_became_active":"2024-09-21T02:20:34.511929+0000","last_became_peered":"2024-09-21T02:20:34.511929+0000","last_unstale":"2024-09-21T02:20:34.534778+0000","last_undegraded":"2024-09-21T02:20:34.534778+0000","last_fullsized":"2024-09-21T02:20:34.534778+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:58.178828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.553374+0000","last_change":"2024-09-21T02:20:33.489359+0000","last_active":"2024-09-21T02:20:34.553374+0000","last_peered":"2024-09-21T02:20:34.553374+0000","last_clean":"2024-09-21T02:20:34.553374+0000","last_became_active":"2024-09-21T02:20:33.488188+0000","last_became_peered":"2024-09-21T02:20:33.488188+0000","last_unstale":"2024-09-21T02:20:34.553374+0000","last_undegraded":"2024-09-21T02:20:34.553374+0000","last_fullsized":"2024-09-21T02:20:34.553374+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:25.628164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.537014+0000","last_change":"2024-09-21T02:20:34.527281+0000","last_active":"2024-09-21T02:20:34.537014+0000","last_peered":"2024-09-21T02:20:34.537014+0000","last_clean":"2024-09-21T02:20:34.537014+0000","last_became_active":"2024-09-21T02:20:34.525893+0000","last_became_peered":"2024-09-21T02:20:34.525893+0000","last_unstale":"2024-09-21T02:20:34.537014+0000","last_undegraded":"2024-09-21T02:20:34.537014+0000","last_fullsized":"2024-09-21T02:20:34.537014+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:21.526454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.509559+0000","last_change":"2024-09-21T02:20:33.521521+0000","last_active":"2024-09-21T02:20:34.509559+0000","last_peered":"2024-09-21T02:20:34.509559+0000","last_clean":"2024-09-21T02:20:34.509559+0000","last_became_active":"2024-09-21T02:20:33.520458+0000","last_became_peered":"2024-09-21T02:20:33.520458+0000","last_unstale":"2024-09-21T02:20:34.509559+0000","last_undegraded":"2024-09-21T02:20:34.509559+0000","last_fullsized":"2024-09-21T02:20:34.509559+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:40.434366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.534154+0000","last_change":"2024-09-21T02:20:34.525131+0000","last_active":"2024-09-21T02:20:34.534154+0000","last_peered":"2024-09-21T02:20:34.534154+0000","last_clean":"2024-09-21T02:20:34.534154+0000","last_became_active":"2024-09-21T02:20:34.524211+0000","last_became_peered":"2024-09-21T02:20:34.524211+0000","last_unstale":"2024-09-21T02:20:34.534154+0000","last_undegraded":"2024-09-21T02:20:34.534154+0000","last_fullsized":"2024-09-21T02:20:34.534154+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:01.486134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.487497+0000","last_change":"2024-09-21T02:20:33.528538+0000","last_active":"2024-09-21T02:20:34.487497+0000","last_peered":"2024-09-21T02:20:34.487497+0000","last_clean":"2024-09-21T02:20:34.487497+0000","last_became_active":"2024-09-21T02:20:33.525796+0000","last_became_peered":"2024-09-21T02:20:33.525796+0000","last_unstale":"2024-09-21T02:20:34.487497+0000","last_undegraded":"2024-09-21T02:20:34.487497+0000","last_fullsized":"2024-09-21T02:20:34.487497+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:41.590000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538878+0000","last_change":"2024-09-21T02:20:34.534785+0000","last_active":"2024-09-21T02:20:34.538878+0000","last_peered":"2024-09-21T02:20:34.538878+0000","last_clean":"2024-09-21T02:20:34.538878+0000","last_became_active":"2024-09-21T02:20:34.533856+0000","last_became_peered":"2024-09-21T02:20:34.533856+0000","last_unstale":"2024-09-21T02:20:34.538878+0000","last_undegraded":"2024-09-21T02:20:34.538878+0000","last_fullsized":"2024-09-21T02:20:34.538878+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.255788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.487094+0000","last_change":"2024-09-21T02:20:33.523952+0000","last_active":"2024-09-21T02:20:34.487094+0000","last_peered":"2024-09-21T02:20:34.487094+0000","last_clean":"2024-09-21T02:20:34.487094+0000","last_became_active":"2024-09-21T02:20:33.523051+0000","last_became_peered":"2024-09-21T02:20:33.523051+0000","last_unstale":"2024-09-21T02:20:34.487094+0000","last_undegraded":"2024-09-21T02:20:34.487094+0000","last_fullsized":"2024-09-21T02:20:34.487094+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:44.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.539799+0000","last_change":"2024-09-21T02:20:34.537007+0000","last_active":"2024-09-21T02:20:34.539799+0000","last_peered":"2024-09-21T02:20:34.539799+0000","last_clean":"2024-09-21T02:20:34.539799+0000","last_became_active":"2024-09-21T02:20:34.536308+0000","last_became_peered":"2024-09-21T02:20:34.536308+0000","last_unstale":"2024-09-21T02:20:34.539799+0000","last_undegraded":"2024-09-21T02:20:34.539799+0000","last_fullsized":"2024-09-21T02:20:34.539799+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:48.275554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.508659+0000","last_change":"2024-09-21T02:20:33.521787+0000","last_active":"2024-09-21T02:20:34.508659+0000","last_peered":"2024-09-21T02:20:34.508659+0000","last_clean":"2024-09-21T02:20:34.508659+0000","last_became_active":"2024-09-21T02:20:33.520331+0000","last_became_peered":"2024-09-21T02:20:33.520331+0000","last_unstale":"2024-09-21T02:20:34.508659+0000","last_undegraded":"2024-09-21T02:20:34.508659+0000","last_fullsized":"2024-09-21T02:20:34.508659+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:31.342503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.507653+0000","last_change":"2024-09-21T02:20:34.497424+0000","last_active":"2024-09-21T02:20:34.507653+0000","last_peered":"2024-09-21T02:20:34.507653+0000","last_clean":"2024-09-21T02:20:34.507653+0000","last_became_active":"2024-09-21T02:20:34.496404+0000","last_became_peered":"2024-09-21T02:20:34.496404+0000","last_unstale":"2024-09-21T02:20:34.507653+0000","last_undegraded":"2024-09-21T02:20:34.507653+0000","last_fullsized":"2024-09-21T02:20:34.507653+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:43.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.515279+0000","last_change":"2024-09-21T02:20:33.501283+0000","last_active":"2024-09-21T02:20:34.515279+0000","last_peered":"2024-09-21T02:20:34.515279+0000","last_clean":"2024-09-21T02:20:34.515279+0000","last_became_active":"2024-09-21T02:20:33.500838+0000","last_became_peered":"2024-09-21T02:20:33.500838+0000","last_unstale":"2024-09-21T02:20:34.515279+0000","last_undegraded":"2024-09-21T02:20:34.515279+0000","last_fullsized":"2024-09-21T02:20:34.515279+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:41.785432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.532754+0000","last_change":"2024-09-21T02:20:34.524303+0000","last_active":"2024-09-21T02:20:34.532754+0000","last_peered":"2024-09-21T02:20:34.532754+0000","last_clean":"2024-09-21T02:20:34.532754+0000","last_became_active":"2024-09-21T02:20:34.514729+0000","last_became_peered":"2024-09-21T02:20:34.514729+0000","last_unstale":"2024-09-21T02:20:34.532754+0000","last_undegraded":"2024-09-21T02:20:34.532754+0000","last_fullsized":"2024-09-21T02:20:34.532754+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:47.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.501778+0000","last_change":"2024-09-21T02:20:33.516222+0000","last_active":"2024-09-21T02:20:34.501778+0000","last_peered":"2024-09-21T02:20:34.501778+0000","last_clean":"2024-09-21T02:20:34.501778+0000","last_became_active":"2024-09-21T02:20:33.515760+0000","last_became_peered":"2024-09-21T02:20:33.515760+0000","last_unstale":"2024-09-21T02:20:34.501778+0000","last_undegraded":"2024-09-21T02:20:34.501778+0000","last_fullsized":"2024-09-21T02:20:34.501778+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:11.343705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.522956+0000","last_change":"2024-09-21T02:20:34.520753+0000","last_active":"2024-09-21T02:20:34.522956+0000","last_peered":"2024-09-21T02:20:34.522956+0000","last_clean":"2024-09-21T02:20:34.522956+0000","last_became_active":"2024-09-21T02:20:34.519925+0000","last_became_peered":"2024-09-21T02:20:34.519925+0000","last_unstale":"2024-09-21T02:20:34.522956+0000","last_undegraded":"2024-09-21T02:20:34.522956+0000","last_fullsized":"2024-09-21T02:20:34.522956+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:07.716367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.500820+0000","last_change":"2024-09-21T02:20:33.514975+0000","last_active":"2024-09-21T02:20:34.500820+0000","last_peered":"2024-09-21T02:20:34.500820+0000","last_clean":"2024-09-21T02:20:34.500820+0000","last_became_active":"2024-09-21T02:20:33.514083+0000","last_became_peered":"2024-09-21T02:20:33.514083+0000","last_unstale":"2024-09-21T02:20:34.500820+0000","last_undegraded":"2024-09-21T02:20:34.500820+0000","last_fullsized":"2024-09-21T02:20:34.500820+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:25.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.527052+0000","last_change":"2024-09-21T02:20:34.523512+0000","last_active":"2024-09-21T02:20:34.527052+0000","last_peered":"2024-09-21T02:20:34.527052+0000","last_clean":"2024-09-21T02:20:34.527052+0000","last_became_active":"2024-09-21T02:20:34.522658+0000","last_became_peered":"2024-09-21T02:20:34.522658+0000","last_unstale":"2024-09-21T02:20:34.527052+0000","last_undegraded":"2024-09-21T02:20:34.527052+0000","last_fullsized":"2024-09-21T02:20:34.527052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:25.761361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"129'3","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552526+0000","last_change":"2024-09-21T02:20:33.518340+0000","last_active":"2024-09-21T02:20:34.552526+0000","last_peered":"2024-09-21T02:20:34.552526+0000","last_clean":"2024-09-21T02:20:34.552526+0000","last_became_active":"2024-09-21T02:20:33.517987+0000","last_became_peered":"2024-09-21T02:20:33.517987+0000","last_unstale":"2024-09-21T02:20:34.552526+0000","last_undegraded":"2024-09-21T02:20:34.552526+0000","last_fullsized":"2024-09-21T02:20:34.552526+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:50.644522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":331,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.485229+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:20:34.485229+0000","last_peered":"2024-09-21T02:20:34.485229+0000","last_clean":"2024-09-21T02:20:34.485229+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:20:34.485229+0000","last_undegraded":"2024-09-21T02:20:34.485229+0000","last_fullsized":"2024-09-21T02:20:34.485229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1609728,"data_stored":708396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2191360,"data_stored":1291961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509532,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1613824,"data_stored":708418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41652,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330188,"statfs":{"total":96636764160,"available":96594112512,"internally_reserved":0,"allocated":2191360,"data_stored":1291961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:20:38.687 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 -- 172.21.15.170:0/2001084861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda780cadf0 msgr2=0x7fda780cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2001084861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda780cadf0 0x7fda780cb200 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fda70009c80 tx=0x7fda7001cd70 comp rx=0 tx=0).stop 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 -- 172.21.15.170:0/2001084861 shutdown_connections 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2001084861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda780cadf0 0x7fda780cb200 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2001084861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda7815f990 0x7fda78070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 -- 172.21.15.170:0/2001084861 >> 172.21.15.170:0/2001084861 conn(0x7fda780b1c40 msgr2=0x7fda780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 -- 172.21.15.170:0/2001084861 shutdown_connections 2024-09-21T02:20:38.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 -- 172.21.15.170:0/2001084861 wait complete. 2024-09-21T02:20:38.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 Processor -- start 2024-09-21T02:20:38.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.885+0000 7fda7eb76640 1 -- start start 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda78070570 0x7fda78151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda780cadf0 0x7fda781551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 0x7fda7815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda78169100 con 0x7fda7815f990 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fda780cede0 con 0x7fda780cadf0 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fda78169280 con 0x7fda78070570 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 0x7fda7815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 0x7fda7815ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50666/0 (socket says 172.21.15.170:50666) 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 -- 172.21.15.170:0/2182048727 learned_addr learned my addr 172.21.15.170:0/2182048727 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7c912640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda78070570 0x7fda78151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda77fff640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda780cadf0 0x7fda781551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 -- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda78070570 msgr2=0x7fda78151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda78070570 0x7fda78151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 -- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda780cadf0 msgr2=0x7fda781551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda780cadf0 0x7fda781551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 -- 172.21.15.170:0/2182048727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda78155730 con 0x7fda7815f990 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7c912640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda78070570 0x7fda78151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7d113640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 0x7fda7815ed40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fda7001a9a0 tx=0x7fda700092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700bc070 con 0x7fda7815f990 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda70004350 con 0x7fda7815f990 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7815f280 con 0x7fda7815f990 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda700b2040 con 0x7fda7815f990 2024-09-21T02:20:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7809bf70 con 0x7fda7815f990 2024-09-21T02:20:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda780cb200 con 0x7fda7815f990 2024-09-21T02:20:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.889+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fda700bf070 con 0x7fda7815f990 2024-09-21T02:20:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.893+0000 7fda757fa640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda64073860 0x7fda64075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.893+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fda7012ef00 con 0x7fda7815f990 2024-09-21T02:20:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.893+0000 7fda7c912640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda64073860 0x7fda64075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.893+0000 7fda7c912640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda64073860 0x7fda64075d70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda68005fd0 tx=0x7fda68005dc0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:38.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:38.897+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fda70016990 con 0x7fda7815f990 2024-09-21T02:20:39.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.061+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda78151a40 con 0x7fda64073860 2024-09-21T02:20:39.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.065+0000 7fda757fa640 1 -- 172.21.15.170:0/2182048727 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7fda78151a40 con 0x7fda64073860 2024-09-21T02:20:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:39.070 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda64073860 msgr2=0x7fda64075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda64073860 0x7fda64075d70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda68005fd0 tx=0x7fda68005dc0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 msgr2=0x7fda7815ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 0x7fda7815ed40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fda7001a9a0 tx=0x7fda700092c0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 shutdown_connections 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda64073860 0x7fda64075d70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda7815f990 0x7fda7815ed40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda780cadf0 0x7fda781551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 --2- 172.21.15.170:0/2182048727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda78070570 0x7fda78151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 >> 172.21.15.170:0/2182048727 conn(0x7fda780b1c40 msgr2=0x7fda780ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:39.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 shutdown_connections 2024-09-21T02:20:39.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.069+0000 7fda7eb76640 1 -- 172.21.15.170:0/2182048727 wait complete. 2024-09-21T02:20:39.089 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-21T02:20:38.908942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":288156,"kb_used_data":13668,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754686564,"statfs":{"total":773094113280,"available":772799041536,"internally_reserved":0,"allocated":13996032,"data_stored":6807108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":280874545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001921"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538984+0000","last_change":"2024-09-21T02:20:34.535065+0000","last_active":"2024-09-21T02:20:34.538984+0000","last_peered":"2024-09-21T02:20:34.538984+0000","last_clean":"2024-09-21T02:20:34.538984+0000","last_became_active":"2024-09-21T02:20:34.534089+0000","last_became_peered":"2024-09-21T02:20:34.534089+0000","last_unstale":"2024-09-21T02:20:34.538984+0000","last_undegraded":"2024-09-21T02:20:34.538984+0000","last_fullsized":"2024-09-21T02:20:34.538984+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:39:50.290788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528806+0000","last_change":"2024-09-21T02:20:34.522400+0000","last_active":"2024-09-21T02:20:34.528806+0000","last_peered":"2024-09-21T02:20:34.528806+0000","last_clean":"2024-09-21T02:20:34.528806+0000","last_became_active":"2024-09-21T02:20:34.517969+0000","last_became_peered":"2024-09-21T02:20:34.517969+0000","last_unstale":"2024-09-21T02:20:34.528806+0000","last_undegraded":"2024-09-21T02:20:34.528806+0000","last_fullsized":"2024-09-21T02:20:34.528806+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:52.787462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544910+0000","last_change":"2024-09-21T02:20:34.543954+0000","last_active":"2024-09-21T02:20:34.544910+0000","last_peered":"2024-09-21T02:20:34.544910+0000","last_clean":"2024-09-21T02:20:34.544910+0000","last_became_active":"2024-09-21T02:20:34.543638+0000","last_became_peered":"2024-09-21T02:20:34.543638+0000","last_unstale":"2024-09-21T02:20:34.544910+0000","last_undegraded":"2024-09-21T02:20:34.544910+0000","last_fullsized":"2024-09-21T02:20:34.544910+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:27.676880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535340+0000","last_change":"2024-09-21T02:20:34.527002+0000","last_active":"2024-09-21T02:20:34.535340+0000","last_peered":"2024-09-21T02:20:34.535340+0000","last_clean":"2024-09-21T02:20:34.535340+0000","last_became_active":"2024-09-21T02:20:34.524614+0000","last_became_peered":"2024-09-21T02:20:34.524614+0000","last_unstale":"2024-09-21T02:20:34.535340+0000","last_undegraded":"2024-09-21T02:20:34.535340+0000","last_fullsized":"2024-09-21T02:20:34.535340+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:58.332275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.512426+0000","last_change":"2024-09-21T02:20:34.499734+0000","last_active":"2024-09-21T02:20:34.512426+0000","last_peered":"2024-09-21T02:20:34.512426+0000","last_clean":"2024-09-21T02:20:34.512426+0000","last_became_active":"2024-09-21T02:20:34.497863+0000","last_became_peered":"2024-09-21T02:20:34.497863+0000","last_unstale":"2024-09-21T02:20:34.512426+0000","last_undegraded":"2024-09-21T02:20:34.512426+0000","last_fullsized":"2024-09-21T02:20:34.512426+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:47.006213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542687+0000","last_change":"2024-09-21T02:20:34.540900+0000","last_active":"2024-09-21T02:20:34.542687+0000","last_peered":"2024-09-21T02:20:34.542687+0000","last_clean":"2024-09-21T02:20:34.542687+0000","last_became_active":"2024-09-21T02:20:34.531895+0000","last_became_peered":"2024-09-21T02:20:34.531895+0000","last_unstale":"2024-09-21T02:20:34.542687+0000","last_undegraded":"2024-09-21T02:20:34.542687+0000","last_fullsized":"2024-09-21T02:20:34.542687+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:52.537505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.504405+0000","last_change":"2024-09-21T02:20:34.496183+0000","last_active":"2024-09-21T02:20:34.504405+0000","last_peered":"2024-09-21T02:20:34.504405+0000","last_clean":"2024-09-21T02:20:34.504405+0000","last_became_active":"2024-09-21T02:20:34.495285+0000","last_became_peered":"2024-09-21T02:20:34.495285+0000","last_unstale":"2024-09-21T02:20:34.504405+0000","last_undegraded":"2024-09-21T02:20:34.504405+0000","last_fullsized":"2024-09-21T02:20:34.504405+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:27.665488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541001+0000","last_change":"2024-09-21T02:20:34.539212+0000","last_active":"2024-09-21T02:20:34.541001+0000","last_peered":"2024-09-21T02:20:34.541001+0000","last_clean":"2024-09-21T02:20:34.541001+0000","last_became_active":"2024-09-21T02:20:34.536871+0000","last_became_peered":"2024-09-21T02:20:34.536871+0000","last_unstale":"2024-09-21T02:20:34.541001+0000","last_undegraded":"2024-09-21T02:20:34.541001+0000","last_fullsized":"2024-09-21T02:20:34.541001+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:58.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541967+0000","last_change":"2024-09-21T02:20:34.530830+0000","last_active":"2024-09-21T02:20:34.541967+0000","last_peered":"2024-09-21T02:20:34.541967+0000","last_clean":"2024-09-21T02:20:34.541967+0000","last_became_active":"2024-09-21T02:20:34.519606+0000","last_became_peered":"2024-09-21T02:20:34.519606+0000","last_unstale":"2024-09-21T02:20:34.541967+0000","last_undegraded":"2024-09-21T02:20:34.541967+0000","last_fullsized":"2024-09-21T02:20:34.541967+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:03.634018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.522869+0000","last_change":"2024-09-21T02:20:34.520833+0000","last_active":"2024-09-21T02:20:34.522869+0000","last_peered":"2024-09-21T02:20:34.522869+0000","last_clean":"2024-09-21T02:20:34.522869+0000","last_became_active":"2024-09-21T02:20:34.519981+0000","last_became_peered":"2024-09-21T02:20:34.519981+0000","last_unstale":"2024-09-21T02:20:34.522869+0000","last_undegraded":"2024-09-21T02:20:34.522869+0000","last_fullsized":"2024-09-21T02:20:34.522869+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:48.915339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542732+0000","last_change":"2024-09-21T02:20:34.535099+0000","last_active":"2024-09-21T02:20:34.542732+0000","last_peered":"2024-09-21T02:20:34.542732+0000","last_clean":"2024-09-21T02:20:34.542732+0000","last_became_active":"2024-09-21T02:20:34.533790+0000","last_became_peered":"2024-09-21T02:20:34.533790+0000","last_unstale":"2024-09-21T02:20:34.542732+0000","last_undegraded":"2024-09-21T02:20:34.542732+0000","last_fullsized":"2024-09-21T02:20:34.542732+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:36.277141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542039+0000","last_change":"2024-09-21T02:20:34.532103+0000","last_active":"2024-09-21T02:20:34.542039+0000","last_peered":"2024-09-21T02:20:34.542039+0000","last_clean":"2024-09-21T02:20:34.542039+0000","last_became_active":"2024-09-21T02:20:34.531266+0000","last_became_peered":"2024-09-21T02:20:34.531266+0000","last_unstale":"2024-09-21T02:20:34.542039+0000","last_undegraded":"2024-09-21T02:20:34.542039+0000","last_fullsized":"2024-09-21T02:20:34.542039+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:41.281457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.512501+0000","last_change":"2024-09-21T02:20:34.500672+0000","last_active":"2024-09-21T02:20:34.512501+0000","last_peered":"2024-09-21T02:20:34.512501+0000","last_clean":"2024-09-21T02:20:34.512501+0000","last_became_active":"2024-09-21T02:20:34.498928+0000","last_became_peered":"2024-09-21T02:20:34.498928+0000","last_unstale":"2024-09-21T02:20:34.512501+0000","last_undegraded":"2024-09-21T02:20:34.512501+0000","last_fullsized":"2024-09-21T02:20:34.512501+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:41:43.305564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542266+0000","last_change":"2024-09-21T02:20:34.536443+0000","last_active":"2024-09-21T02:20:34.542266+0000","last_peered":"2024-09-21T02:20:34.542266+0000","last_clean":"2024-09-21T02:20:34.542266+0000","last_became_active":"2024-09-21T02:20:34.534148+0000","last_became_peered":"2024-09-21T02:20:34.534148+0000","last_unstale":"2024-09-21T02:20:34.542266+0000","last_undegraded":"2024-09-21T02:20:34.542266+0000","last_fullsized":"2024-09-21T02:20:34.542266+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:44.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.522368+0000","last_change":"2024-09-21T02:20:34.515963+0000","last_active":"2024-09-21T02:20:34.522368+0000","last_peered":"2024-09-21T02:20:34.522368+0000","last_clean":"2024-09-21T02:20:34.522368+0000","last_became_active":"2024-09-21T02:20:34.515002+0000","last_became_peered":"2024-09-21T02:20:34.515002+0000","last_unstale":"2024-09-21T02:20:34.522368+0000","last_undegraded":"2024-09-21T02:20:34.522368+0000","last_fullsized":"2024-09-21T02:20:34.522368+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:42.482933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.516527+0000","last_change":"2024-09-21T02:20:34.511843+0000","last_active":"2024-09-21T02:20:34.516527+0000","last_peered":"2024-09-21T02:20:34.516527+0000","last_clean":"2024-09-21T02:20:34.516527+0000","last_became_active":"2024-09-21T02:20:34.510663+0000","last_became_peered":"2024-09-21T02:20:34.510663+0000","last_unstale":"2024-09-21T02:20:34.516527+0000","last_undegraded":"2024-09-21T02:20:34.516527+0000","last_fullsized":"2024-09-21T02:20:34.516527+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:59.433398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538096+0000","last_change":"2024-09-21T02:20:34.536434+0000","last_active":"2024-09-21T02:20:34.538096+0000","last_peered":"2024-09-21T02:20:34.538096+0000","last_clean":"2024-09-21T02:20:34.538096+0000","last_became_active":"2024-09-21T02:20:34.534129+0000","last_became_peered":"2024-09-21T02:20:34.534129+0000","last_unstale":"2024-09-21T02:20:34.538096+0000","last_undegraded":"2024-09-21T02:20:34.538096+0000","last_fullsized":"2024-09-21T02:20:34.538096+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:16.346709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541103+0000","last_change":"2024-09-21T02:20:34.535012+0000","last_active":"2024-09-21T02:20:34.541103+0000","last_peered":"2024-09-21T02:20:34.541103+0000","last_clean":"2024-09-21T02:20:34.541103+0000","last_became_active":"2024-09-21T02:20:34.534322+0000","last_became_peered":"2024-09-21T02:20:34.534322+0000","last_unstale":"2024-09-21T02:20:34.541103+0000","last_undegraded":"2024-09-21T02:20:34.541103+0000","last_fullsized":"2024-09-21T02:20:34.541103+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:00.344661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542884+0000","last_change":"2024-09-21T02:20:34.542161+0000","last_active":"2024-09-21T02:20:34.542884+0000","last_peered":"2024-09-21T02:20:34.542884+0000","last_clean":"2024-09-21T02:20:34.542884+0000","last_became_active":"2024-09-21T02:20:34.541774+0000","last_became_peered":"2024-09-21T02:20:34.541774+0000","last_unstale":"2024-09-21T02:20:34.542884+0000","last_undegraded":"2024-09-21T02:20:34.542884+0000","last_fullsized":"2024-09-21T02:20:34.542884+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:44.732555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528392+0000","last_change":"2024-09-21T02:20:34.522809+0000","last_active":"2024-09-21T02:20:34.528392+0000","last_peered":"2024-09-21T02:20:34.528392+0000","last_clean":"2024-09-21T02:20:34.528392+0000","last_became_active":"2024-09-21T02:20:34.522000+0000","last_became_peered":"2024-09-21T02:20:34.522000+0000","last_unstale":"2024-09-21T02:20:34.528392+0000","last_undegraded":"2024-09-21T02:20:34.528392+0000","last_fullsized":"2024-09-21T02:20:34.528392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:31.419902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538891+0000","last_change":"2024-09-21T02:20:34.536391+0000","last_active":"2024-09-21T02:20:34.538891+0000","last_peered":"2024-09-21T02:20:34.538891+0000","last_clean":"2024-09-21T02:20:34.538891+0000","last_became_active":"2024-09-21T02:20:34.535614+0000","last_became_peered":"2024-09-21T02:20:34.535614+0000","last_unstale":"2024-09-21T02:20:34.538891+0000","last_undegraded":"2024-09-21T02:20:34.538891+0000","last_fullsized":"2024-09-21T02:20:34.538891+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:15.577194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542187+0000","last_change":"2024-09-21T02:20:34.527995+0000","last_active":"2024-09-21T02:20:34.542187+0000","last_peered":"2024-09-21T02:20:34.542187+0000","last_clean":"2024-09-21T02:20:34.542187+0000","last_became_active":"2024-09-21T02:20:34.526274+0000","last_became_peered":"2024-09-21T02:20:34.526274+0000","last_unstale":"2024-09-21T02:20:34.542187+0000","last_undegraded":"2024-09-21T02:20:34.542187+0000","last_fullsized":"2024-09-21T02:20:34.542187+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:17.547968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538229+0000","last_change":"2024-09-21T02:20:34.536767+0000","last_active":"2024-09-21T02:20:34.538229+0000","last_peered":"2024-09-21T02:20:34.538229+0000","last_clean":"2024-09-21T02:20:34.538229+0000","last_became_active":"2024-09-21T02:20:34.536063+0000","last_became_peered":"2024-09-21T02:20:34.536063+0000","last_unstale":"2024-09-21T02:20:34.538229+0000","last_undegraded":"2024-09-21T02:20:34.538229+0000","last_fullsized":"2024-09-21T02:20:34.538229+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:25.693616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.510370+0000","last_change":"2024-09-21T02:20:34.507236+0000","last_active":"2024-09-21T02:20:34.510370+0000","last_peered":"2024-09-21T02:20:34.510370+0000","last_clean":"2024-09-21T02:20:34.510370+0000","last_became_active":"2024-09-21T02:20:34.506382+0000","last_became_peered":"2024-09-21T02:20:34.506382+0000","last_unstale":"2024-09-21T02:20:34.510370+0000","last_undegraded":"2024-09-21T02:20:34.510370+0000","last_fullsized":"2024-09-21T02:20:34.510370+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:48:25.895929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535179+0000","last_change":"2024-09-21T02:20:34.527564+0000","last_active":"2024-09-21T02:20:34.535179+0000","last_peered":"2024-09-21T02:20:34.535179+0000","last_clean":"2024-09-21T02:20:34.535179+0000","last_became_active":"2024-09-21T02:20:34.525977+0000","last_became_peered":"2024-09-21T02:20:34.525977+0000","last_unstale":"2024-09-21T02:20:34.535179+0000","last_undegraded":"2024-09-21T02:20:34.535179+0000","last_fullsized":"2024-09-21T02:20:34.535179+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:56.725599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.521337+0000","last_change":"2024-09-21T02:20:34.513932+0000","last_active":"2024-09-21T02:20:34.521337+0000","last_peered":"2024-09-21T02:20:34.521337+0000","last_clean":"2024-09-21T02:20:34.521337+0000","last_became_active":"2024-09-21T02:20:34.513037+0000","last_became_peered":"2024-09-21T02:20:34.513037+0000","last_unstale":"2024-09-21T02:20:34.521337+0000","last_undegraded":"2024-09-21T02:20:34.521337+0000","last_fullsized":"2024-09-21T02:20:34.521337+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:47.843405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.529367+0000","last_change":"2024-09-21T02:20:34.522766+0000","last_active":"2024-09-21T02:20:34.529367+0000","last_peered":"2024-09-21T02:20:34.529367+0000","last_clean":"2024-09-21T02:20:34.529367+0000","last_became_active":"2024-09-21T02:20:34.521882+0000","last_became_peered":"2024-09-21T02:20:34.521882+0000","last_unstale":"2024-09-21T02:20:34.529367+0000","last_undegraded":"2024-09-21T02:20:34.529367+0000","last_fullsized":"2024-09-21T02:20:34.529367+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:23.141714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.534433+0000","last_change":"2024-09-21T02:20:34.523927+0000","last_active":"2024-09-21T02:20:34.534433+0000","last_peered":"2024-09-21T02:20:34.534433+0000","last_clean":"2024-09-21T02:20:34.534433+0000","last_became_active":"2024-09-21T02:20:34.523087+0000","last_became_peered":"2024-09-21T02:20:34.523087+0000","last_unstale":"2024-09-21T02:20:34.534433+0000","last_undegraded":"2024-09-21T02:20:34.534433+0000","last_fullsized":"2024-09-21T02:20:34.534433+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:26.632079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541267+0000","last_change":"2024-09-21T02:20:34.540061+0000","last_active":"2024-09-21T02:20:34.541267+0000","last_peered":"2024-09-21T02:20:34.541267+0000","last_clean":"2024-09-21T02:20:34.541267+0000","last_became_active":"2024-09-21T02:20:34.533434+0000","last_became_peered":"2024-09-21T02:20:34.533434+0000","last_unstale":"2024-09-21T02:20:34.541267+0000","last_undegraded":"2024-09-21T02:20:34.541267+0000","last_fullsized":"2024-09-21T02:20:34.541267+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:57.569357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544173+0000","last_change":"2024-09-21T02:20:34.541320+0000","last_active":"2024-09-21T02:20:34.544173+0000","last_peered":"2024-09-21T02:20:34.544173+0000","last_clean":"2024-09-21T02:20:34.544173+0000","last_became_active":"2024-09-21T02:20:34.533413+0000","last_became_peered":"2024-09-21T02:20:34.533413+0000","last_unstale":"2024-09-21T02:20:34.544173+0000","last_undegraded":"2024-09-21T02:20:34.544173+0000","last_fullsized":"2024-09-21T02:20:34.544173+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:12.370349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528719+0000","last_change":"2024-09-21T02:20:34.522545+0000","last_active":"2024-09-21T02:20:34.528719+0000","last_peered":"2024-09-21T02:20:34.528719+0000","last_clean":"2024-09-21T02:20:34.528719+0000","last_became_active":"2024-09-21T02:20:34.519913+0000","last_became_peered":"2024-09-21T02:20:34.519913+0000","last_unstale":"2024-09-21T02:20:34.528719+0000","last_undegraded":"2024-09-21T02:20:34.528719+0000","last_fullsized":"2024-09-21T02:20:34.528719+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:20.745861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540353+0000","last_change":"2024-09-21T02:20:34.536997+0000","last_active":"2024-09-21T02:20:34.540353+0000","last_peered":"2024-09-21T02:20:34.540353+0000","last_clean":"2024-09-21T02:20:34.540353+0000","last_became_active":"2024-09-21T02:20:34.536171+0000","last_became_peered":"2024-09-21T02:20:34.536171+0000","last_unstale":"2024-09-21T02:20:34.540353+0000","last_undegraded":"2024-09-21T02:20:34.540353+0000","last_fullsized":"2024-09-21T02:20:34.540353+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:58.178828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.504059+0000","last_change":"2024-09-21T02:20:34.497138+0000","last_active":"2024-09-21T02:20:34.504059+0000","last_peered":"2024-09-21T02:20:34.504059+0000","last_clean":"2024-09-21T02:20:34.504059+0000","last_became_active":"2024-09-21T02:20:34.496270+0000","last_became_peered":"2024-09-21T02:20:34.496270+0000","last_unstale":"2024-09-21T02:20:34.504059+0000","last_undegraded":"2024-09-21T02:20:34.504059+0000","last_fullsized":"2024-09-21T02:20:34.504059+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:06.844713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"129'4","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.556472+0000","last_change":"2024-09-21T02:20:33.529111+0000","last_active":"2024-09-21T02:20:34.556472+0000","last_peered":"2024-09-21T02:20:34.556472+0000","last_clean":"2024-09-21T02:20:34.556472+0000","last_became_active":"2024-09-21T02:20:33.528609+0000","last_became_peered":"2024-09-21T02:20:33.528609+0000","last_unstale":"2024-09-21T02:20:34.556472+0000","last_undegraded":"2024-09-21T02:20:34.556472+0000","last_fullsized":"2024-09-21T02:20:34.556472+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:17.603041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.520421+0000","last_change":"2024-09-21T02:20:34.516689+0000","last_active":"2024-09-21T02:20:34.520421+0000","last_peered":"2024-09-21T02:20:34.520421+0000","last_clean":"2024-09-21T02:20:34.520421+0000","last_became_active":"2024-09-21T02:20:34.515826+0000","last_became_peered":"2024-09-21T02:20:34.515826+0000","last_unstale":"2024-09-21T02:20:34.520421+0000","last_undegraded":"2024-09-21T02:20:34.520421+0000","last_fullsized":"2024-09-21T02:20:34.520421+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:17.486614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552581+0000","last_change":"2024-09-21T02:20:33.506795+0000","last_active":"2024-09-21T02:20:34.552581+0000","last_peered":"2024-09-21T02:20:34.552581+0000","last_clean":"2024-09-21T02:20:34.552581+0000","last_became_active":"2024-09-21T02:20:33.505984+0000","last_became_peered":"2024-09-21T02:20:33.505984+0000","last_unstale":"2024-09-21T02:20:34.552581+0000","last_undegraded":"2024-09-21T02:20:34.552581+0000","last_fullsized":"2024-09-21T02:20:34.552581+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:21.892699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541996+0000","last_change":"2024-09-21T02:20:34.530853+0000","last_active":"2024-09-21T02:20:34.541996+0000","last_peered":"2024-09-21T02:20:34.541996+0000","last_clean":"2024-09-21T02:20:34.541996+0000","last_became_active":"2024-09-21T02:20:34.519747+0000","last_became_peered":"2024-09-21T02:20:34.519747+0000","last_unstale":"2024-09-21T02:20:34.541996+0000","last_undegraded":"2024-09-21T02:20:34.541996+0000","last_fullsized":"2024-09-21T02:20:34.541996+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:48.807205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.551289+0000","last_change":"2024-09-21T02:20:33.517342+0000","last_active":"2024-09-21T02:20:34.551289+0000","last_peered":"2024-09-21T02:20:34.551289+0000","last_clean":"2024-09-21T02:20:34.551289+0000","last_became_active":"2024-09-21T02:20:33.517029+0000","last_became_peered":"2024-09-21T02:20:33.517029+0000","last_unstale":"2024-09-21T02:20:34.551289+0000","last_undegraded":"2024-09-21T02:20:34.551289+0000","last_fullsized":"2024-09-21T02:20:34.551289+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:19.048319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535540+0000","last_change":"2024-09-21T02:20:34.528802+0000","last_active":"2024-09-21T02:20:34.535540+0000","last_peered":"2024-09-21T02:20:34.535540+0000","last_clean":"2024-09-21T02:20:34.535540+0000","last_became_active":"2024-09-21T02:20:34.527960+0000","last_became_peered":"2024-09-21T02:20:34.527960+0000","last_unstale":"2024-09-21T02:20:34.535540+0000","last_undegraded":"2024-09-21T02:20:34.535540+0000","last_fullsized":"2024-09-21T02:20:34.535540+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:05.932488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.554266+0000","last_change":"2024-09-21T02:20:33.516427+0000","last_active":"2024-09-21T02:20:34.554266+0000","last_peered":"2024-09-21T02:20:34.554266+0000","last_clean":"2024-09-21T02:20:34.554266+0000","last_became_active":"2024-09-21T02:20:33.515615+0000","last_became_peered":"2024-09-21T02:20:33.515615+0000","last_unstale":"2024-09-21T02:20:34.554266+0000","last_undegraded":"2024-09-21T02:20:34.554266+0000","last_fullsized":"2024-09-21T02:20:34.554266+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:57.979056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538816+0000","last_change":"2024-09-21T02:20:34.531195+0000","last_active":"2024-09-21T02:20:34.538816+0000","last_peered":"2024-09-21T02:20:34.538816+0000","last_clean":"2024-09-21T02:20:34.538816+0000","last_became_active":"2024-09-21T02:20:34.530312+0000","last_became_peered":"2024-09-21T02:20:34.530312+0000","last_unstale":"2024-09-21T02:20:34.538816+0000","last_undegraded":"2024-09-21T02:20:34.538816+0000","last_fullsized":"2024-09-21T02:20:34.538816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:41.222255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552657+0000","last_change":"2024-09-21T02:20:33.512665+0000","last_active":"2024-09-21T02:20:34.552657+0000","last_peered":"2024-09-21T02:20:34.552657+0000","last_clean":"2024-09-21T02:20:34.552657+0000","last_became_active":"2024-09-21T02:20:33.511943+0000","last_became_peered":"2024-09-21T02:20:33.511943+0000","last_unstale":"2024-09-21T02:20:34.552657+0000","last_undegraded":"2024-09-21T02:20:34.552657+0000","last_fullsized":"2024-09-21T02:20:34.552657+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:46.390258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.535017+0000","last_change":"2024-09-21T02:20:34.527064+0000","last_active":"2024-09-21T02:20:34.535017+0000","last_peered":"2024-09-21T02:20:34.535017+0000","last_clean":"2024-09-21T02:20:34.535017+0000","last_became_active":"2024-09-21T02:20:34.525286+0000","last_became_peered":"2024-09-21T02:20:34.525286+0000","last_unstale":"2024-09-21T02:20:34.535017+0000","last_undegraded":"2024-09-21T02:20:34.535017+0000","last_fullsized":"2024-09-21T02:20:34.535017+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:14.484525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552751+0000","last_change":"2024-09-21T02:20:33.498751+0000","last_active":"2024-09-21T02:20:34.552751+0000","last_peered":"2024-09-21T02:20:34.552751+0000","last_clean":"2024-09-21T02:20:34.552751+0000","last_became_active":"2024-09-21T02:20:33.498223+0000","last_became_peered":"2024-09-21T02:20:33.498223+0000","last_unstale":"2024-09-21T02:20:34.552751+0000","last_undegraded":"2024-09-21T02:20:34.552751+0000","last_fullsized":"2024-09-21T02:20:34.552751+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:15:12.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540321+0000","last_change":"2024-09-21T02:20:34.536805+0000","last_active":"2024-09-21T02:20:34.540321+0000","last_peered":"2024-09-21T02:20:34.540321+0000","last_clean":"2024-09-21T02:20:34.540321+0000","last_became_active":"2024-09-21T02:20:34.536024+0000","last_became_peered":"2024-09-21T02:20:34.536024+0000","last_unstale":"2024-09-21T02:20:34.540321+0000","last_undegraded":"2024-09-21T02:20:34.540321+0000","last_fullsized":"2024-09-21T02:20:34.540321+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.980670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.515981+0000","last_change":"2024-09-21T02:20:33.518261+0000","last_active":"2024-09-21T02:20:34.515981+0000","last_peered":"2024-09-21T02:20:34.515981+0000","last_clean":"2024-09-21T02:20:34.515981+0000","last_became_active":"2024-09-21T02:20:33.517593+0000","last_became_peered":"2024-09-21T02:20:33.517593+0000","last_unstale":"2024-09-21T02:20:34.515981+0000","last_undegraded":"2024-09-21T02:20:34.515981+0000","last_fullsized":"2024-09-21T02:20:34.515981+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:36.813034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.532694+0000","last_change":"2024-09-21T02:20:34.524314+0000","last_active":"2024-09-21T02:20:34.532694+0000","last_peered":"2024-09-21T02:20:34.532694+0000","last_clean":"2024-09-21T02:20:34.532694+0000","last_became_active":"2024-09-21T02:20:34.523286+0000","last_became_peered":"2024-09-21T02:20:34.523286+0000","last_unstale":"2024-09-21T02:20:34.532694+0000","last_undegraded":"2024-09-21T02:20:34.532694+0000","last_fullsized":"2024-09-21T02:20:34.532694+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:41.222255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.513043+0000","last_change":"2024-09-21T02:20:33.502584+0000","last_active":"2024-09-21T02:20:34.513043+0000","last_peered":"2024-09-21T02:20:34.513043+0000","last_clean":"2024-09-21T02:20:34.513043+0000","last_became_active":"2024-09-21T02:20:33.502021+0000","last_became_peered":"2024-09-21T02:20:33.502021+0000","last_unstale":"2024-09-21T02:20:34.513043+0000","last_undegraded":"2024-09-21T02:20:34.513043+0000","last_fullsized":"2024-09-21T02:20:34.513043+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:23.689333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544811+0000","last_change":"2024-09-21T02:20:34.543988+0000","last_active":"2024-09-21T02:20:34.544811+0000","last_peered":"2024-09-21T02:20:34.544811+0000","last_clean":"2024-09-21T02:20:34.544811+0000","last_became_active":"2024-09-21T02:20:34.543398+0000","last_became_peered":"2024-09-21T02:20:34.543398+0000","last_unstale":"2024-09-21T02:20:34.544811+0000","last_undegraded":"2024-09-21T02:20:34.544811+0000","last_fullsized":"2024-09-21T02:20:34.544811+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:07.573024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.515203+0000","last_change":"2024-09-21T02:20:33.529086+0000","last_active":"2024-09-21T02:20:34.515203+0000","last_peered":"2024-09-21T02:20:34.515203+0000","last_clean":"2024-09-21T02:20:34.515203+0000","last_became_active":"2024-09-21T02:20:33.528259+0000","last_became_peered":"2024-09-21T02:20:33.528259+0000","last_unstale":"2024-09-21T02:20:34.515203+0000","last_undegraded":"2024-09-21T02:20:34.515203+0000","last_fullsized":"2024-09-21T02:20:34.515203+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:28.432138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540057+0000","last_change":"2024-09-21T02:20:34.524233+0000","last_active":"2024-09-21T02:20:34.540057+0000","last_peered":"2024-09-21T02:20:34.540057+0000","last_clean":"2024-09-21T02:20:34.540057+0000","last_became_active":"2024-09-21T02:20:34.523459+0000","last_became_peered":"2024-09-21T02:20:34.523459+0000","last_unstale":"2024-09-21T02:20:34.540057+0000","last_undegraded":"2024-09-21T02:20:34.540057+0000","last_fullsized":"2024-09-21T02:20:34.540057+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:31.943339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.488623+0000","last_change":"2024-09-21T02:20:33.499112+0000","last_active":"2024-09-21T02:20:34.488623+0000","last_peered":"2024-09-21T02:20:34.488623+0000","last_clean":"2024-09-21T02:20:34.488623+0000","last_became_active":"2024-09-21T02:20:33.498733+0000","last_became_peered":"2024-09-21T02:20:33.498733+0000","last_unstale":"2024-09-21T02:20:34.488623+0000","last_undegraded":"2024-09-21T02:20:34.488623+0000","last_fullsized":"2024-09-21T02:20:34.488623+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:06.556724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.531506+0000","last_change":"2024-09-21T02:20:34.527696+0000","last_active":"2024-09-21T02:20:34.531506+0000","last_peered":"2024-09-21T02:20:34.531506+0000","last_clean":"2024-09-21T02:20:34.531506+0000","last_became_active":"2024-09-21T02:20:34.525873+0000","last_became_peered":"2024-09-21T02:20:34.525873+0000","last_unstale":"2024-09-21T02:20:34.531506+0000","last_undegraded":"2024-09-21T02:20:34.531506+0000","last_fullsized":"2024-09-21T02:20:34.531506+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:41.095762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"129'6","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.555828+0000","last_change":"2024-09-21T02:20:33.527173+0000","last_active":"2024-09-21T02:20:34.555828+0000","last_peered":"2024-09-21T02:20:34.555828+0000","last_clean":"2024-09-21T02:20:34.555828+0000","last_became_active":"2024-09-21T02:20:33.526195+0000","last_became_peered":"2024-09-21T02:20:33.526195+0000","last_unstale":"2024-09-21T02:20:34.555828+0000","last_undegraded":"2024-09-21T02:20:34.555828+0000","last_fullsized":"2024-09-21T02:20:34.555828+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:30.456111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.544582+0000","last_change":"2024-09-21T02:20:34.543396+0000","last_active":"2024-09-21T02:20:34.544582+0000","last_peered":"2024-09-21T02:20:34.544582+0000","last_clean":"2024-09-21T02:20:34.544582+0000","last_became_active":"2024-09-21T02:20:34.542993+0000","last_became_peered":"2024-09-21T02:20:34.542993+0000","last_unstale":"2024-09-21T02:20:34.544582+0000","last_undegraded":"2024-09-21T02:20:34.544582+0000","last_fullsized":"2024-09-21T02:20:34.544582+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:02.896784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.553153+0000","last_change":"2024-09-21T02:20:33.517386+0000","last_active":"2024-09-21T02:20:34.553153+0000","last_peered":"2024-09-21T02:20:34.553153+0000","last_clean":"2024-09-21T02:20:34.553153+0000","last_became_active":"2024-09-21T02:20:33.517013+0000","last_became_peered":"2024-09-21T02:20:33.517013+0000","last_unstale":"2024-09-21T02:20:34.553153+0000","last_undegraded":"2024-09-21T02:20:34.553153+0000","last_fullsized":"2024-09-21T02:20:34.553153+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:17.283423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.540556+0000","last_change":"2024-09-21T02:20:34.535522+0000","last_active":"2024-09-21T02:20:34.540556+0000","last_peered":"2024-09-21T02:20:34.540556+0000","last_clean":"2024-09-21T02:20:34.540556+0000","last_became_active":"2024-09-21T02:20:34.534557+0000","last_became_peered":"2024-09-21T02:20:34.534557+0000","last_unstale":"2024-09-21T02:20:34.540556+0000","last_undegraded":"2024-09-21T02:20:34.540556+0000","last_fullsized":"2024-09-21T02:20:34.540556+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:46.071286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.520882+0000","last_change":"2024-09-21T02:20:33.500292+0000","last_active":"2024-09-21T02:20:34.520882+0000","last_peered":"2024-09-21T02:20:34.520882+0000","last_clean":"2024-09-21T02:20:34.520882+0000","last_became_active":"2024-09-21T02:20:33.499871+0000","last_became_peered":"2024-09-21T02:20:33.499871+0000","last_unstale":"2024-09-21T02:20:34.520882+0000","last_undegraded":"2024-09-21T02:20:34.520882+0000","last_fullsized":"2024-09-21T02:20:34.520882+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:38.083189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.530514+0000","last_change":"2024-09-21T02:20:34.512961+0000","last_active":"2024-09-21T02:20:34.530514+0000","last_peered":"2024-09-21T02:20:34.530514+0000","last_clean":"2024-09-21T02:20:34.530514+0000","last_became_active":"2024-09-21T02:20:34.512138+0000","last_became_peered":"2024-09-21T02:20:34.512138+0000","last_unstale":"2024-09-21T02:20:34.530514+0000","last_undegraded":"2024-09-21T02:20:34.530514+0000","last_fullsized":"2024-09-21T02:20:34.530514+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:55:56.725599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.509152+0000","last_change":"2024-09-21T02:20:33.519947+0000","last_active":"2024-09-21T02:20:34.509152+0000","last_peered":"2024-09-21T02:20:34.509152+0000","last_clean":"2024-09-21T02:20:34.509152+0000","last_became_active":"2024-09-21T02:20:33.518916+0000","last_became_peered":"2024-09-21T02:20:33.518916+0000","last_unstale":"2024-09-21T02:20:34.509152+0000","last_undegraded":"2024-09-21T02:20:34.509152+0000","last_fullsized":"2024-09-21T02:20:34.509152+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:34:55.998249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.537231+0000","last_change":"2024-09-21T02:20:34.535034+0000","last_active":"2024-09-21T02:20:34.537231+0000","last_peered":"2024-09-21T02:20:34.537231+0000","last_clean":"2024-09-21T02:20:34.537231+0000","last_became_active":"2024-09-21T02:20:34.532471+0000","last_became_peered":"2024-09-21T02:20:34.532471+0000","last_unstale":"2024-09-21T02:20:34.537231+0000","last_undegraded":"2024-09-21T02:20:34.537231+0000","last_fullsized":"2024-09-21T02:20:34.537231+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:17.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"129'8","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.555694+0000","last_change":"2024-09-21T02:20:33.522068+0000","last_active":"2024-09-21T02:20:34.555694+0000","last_peered":"2024-09-21T02:20:34.555694+0000","last_clean":"2024-09-21T02:20:34.555694+0000","last_became_active":"2024-09-21T02:20:33.516617+0000","last_became_peered":"2024-09-21T02:20:33.516617+0000","last_unstale":"2024-09-21T02:20:34.555694+0000","last_undegraded":"2024-09-21T02:20:34.555694+0000","last_fullsized":"2024-09-21T02:20:34.555694+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:18:28.917161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.541032+0000","last_change":"2024-09-21T02:20:34.533927+0000","last_active":"2024-09-21T02:20:34.541032+0000","last_peered":"2024-09-21T02:20:34.541032+0000","last_clean":"2024-09-21T02:20:34.541032+0000","last_became_active":"2024-09-21T02:20:34.533141+0000","last_became_peered":"2024-09-21T02:20:34.533141+0000","last_unstale":"2024-09-21T02:20:34.541032+0000","last_undegraded":"2024-09-21T02:20:34.541032+0000","last_fullsized":"2024-09-21T02:20:34.541032+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:51.749702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.554699+0000","last_change":"2024-09-21T02:20:33.526887+0000","last_active":"2024-09-21T02:20:34.554699+0000","last_peered":"2024-09-21T02:20:34.554699+0000","last_clean":"2024-09-21T02:20:34.554699+0000","last_became_active":"2024-09-21T02:20:33.526541+0000","last_became_peered":"2024-09-21T02:20:33.526541+0000","last_unstale":"2024-09-21T02:20:34.554699+0000","last_undegraded":"2024-09-21T02:20:34.554699+0000","last_fullsized":"2024-09-21T02:20:34.554699+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:51.783547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.530062+0000","last_change":"2024-09-21T02:20:34.526735+0000","last_active":"2024-09-21T02:20:34.530062+0000","last_peered":"2024-09-21T02:20:34.530062+0000","last_clean":"2024-09-21T02:20:34.530062+0000","last_became_active":"2024-09-21T02:20:34.524993+0000","last_became_peered":"2024-09-21T02:20:34.524993+0000","last_unstale":"2024-09-21T02:20:34.530062+0000","last_undegraded":"2024-09-21T02:20:34.530062+0000","last_fullsized":"2024-09-21T02:20:34.530062+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:43.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552963+0000","last_change":"2024-09-21T02:20:33.525324+0000","last_active":"2024-09-21T02:20:34.552963+0000","last_peered":"2024-09-21T02:20:34.552963+0000","last_clean":"2024-09-21T02:20:34.552963+0000","last_became_active":"2024-09-21T02:20:33.524426+0000","last_became_peered":"2024-09-21T02:20:33.524426+0000","last_unstale":"2024-09-21T02:20:34.552963+0000","last_undegraded":"2024-09-21T02:20:34.552963+0000","last_fullsized":"2024-09-21T02:20:34.552963+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:09.549686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.518718+0000","last_change":"2024-09-21T02:20:34.511770+0000","last_active":"2024-09-21T02:20:34.518718+0000","last_peered":"2024-09-21T02:20:34.518718+0000","last_clean":"2024-09-21T02:20:34.518718+0000","last_became_active":"2024-09-21T02:20:34.510962+0000","last_became_peered":"2024-09-21T02:20:34.510962+0000","last_unstale":"2024-09-21T02:20:34.518718+0000","last_undegraded":"2024-09-21T02:20:34.518718+0000","last_fullsized":"2024-09-21T02:20:34.518718+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:41:51.265058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"129'1","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552507+0000","last_change":"2024-09-21T02:20:33.508423+0000","last_active":"2024-09-21T02:20:34.552507+0000","last_peered":"2024-09-21T02:20:34.552507+0000","last_clean":"2024-09-21T02:20:34.552507+0000","last_became_active":"2024-09-21T02:20:33.503663+0000","last_became_peered":"2024-09-21T02:20:33.503663+0000","last_unstale":"2024-09-21T02:20:34.552507+0000","last_undegraded":"2024-09-21T02:20:34.552507+0000","last_fullsized":"2024-09-21T02:20:34.552507+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:42.417840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542119+0000","last_change":"2024-09-21T02:20:34.534065+0000","last_active":"2024-09-21T02:20:34.542119+0000","last_peered":"2024-09-21T02:20:34.542119+0000","last_clean":"2024-09-21T02:20:34.542119+0000","last_became_active":"2024-09-21T02:20:34.531250+0000","last_became_peered":"2024-09-21T02:20:34.531250+0000","last_unstale":"2024-09-21T02:20:34.542119+0000","last_undegraded":"2024-09-21T02:20:34.542119+0000","last_fullsized":"2024-09-21T02:20:34.542119+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:46:49.439603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.511051+0000","last_change":"2024-09-21T02:20:33.527734+0000","last_active":"2024-09-21T02:20:34.511051+0000","last_peered":"2024-09-21T02:20:34.511051+0000","last_clean":"2024-09-21T02:20:34.511051+0000","last_became_active":"2024-09-21T02:20:33.526851+0000","last_became_peered":"2024-09-21T02:20:33.526851+0000","last_unstale":"2024-09-21T02:20:34.511051+0000","last_undegraded":"2024-09-21T02:20:34.511051+0000","last_fullsized":"2024-09-21T02:20:34.511051+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:14.041756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542111+0000","last_change":"2024-09-21T02:20:34.525263+0000","last_active":"2024-09-21T02:20:34.542111+0000","last_peered":"2024-09-21T02:20:34.542111+0000","last_clean":"2024-09-21T02:20:34.542111+0000","last_became_active":"2024-09-21T02:20:34.524346+0000","last_became_peered":"2024-09-21T02:20:34.524346+0000","last_unstale":"2024-09-21T02:20:34.542111+0000","last_undegraded":"2024-09-21T02:20:34.542111+0000","last_fullsized":"2024-09-21T02:20:34.542111+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:24.742898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552878+0000","last_change":"2024-09-21T02:20:33.511973+0000","last_active":"2024-09-21T02:20:34.552878+0000","last_peered":"2024-09-21T02:20:34.552878+0000","last_clean":"2024-09-21T02:20:34.552878+0000","last_became_active":"2024-09-21T02:20:33.511142+0000","last_became_peered":"2024-09-21T02:20:33.511142+0000","last_unstale":"2024-09-21T02:20:34.552878+0000","last_undegraded":"2024-09-21T02:20:34.552878+0000","last_fullsized":"2024-09-21T02:20:34.552878+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:28.907611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528430+0000","last_change":"2024-09-21T02:20:34.520389+0000","last_active":"2024-09-21T02:20:34.528430+0000","last_peered":"2024-09-21T02:20:34.528430+0000","last_clean":"2024-09-21T02:20:34.528430+0000","last_became_active":"2024-09-21T02:20:34.519547+0000","last_became_peered":"2024-09-21T02:20:34.519547+0000","last_unstale":"2024-09-21T02:20:34.528430+0000","last_undegraded":"2024-09-21T02:20:34.528430+0000","last_fullsized":"2024-09-21T02:20:34.528430+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:16.300803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.485199+0000","last_change":"2024-09-21T02:20:33.504251+0000","last_active":"2024-09-21T02:20:34.485199+0000","last_peered":"2024-09-21T02:20:34.485199+0000","last_clean":"2024-09-21T02:20:34.485199+0000","last_became_active":"2024-09-21T02:20:33.503581+0000","last_became_peered":"2024-09-21T02:20:33.503581+0000","last_unstale":"2024-09-21T02:20:34.485199+0000","last_undegraded":"2024-09-21T02:20:34.485199+0000","last_fullsized":"2024-09-21T02:20:34.485199+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:19.741946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.542201+0000","last_change":"2024-09-21T02:20:34.537434+0000","last_active":"2024-09-21T02:20:34.542201+0000","last_peered":"2024-09-21T02:20:34.542201+0000","last_clean":"2024-09-21T02:20:34.542201+0000","last_became_active":"2024-09-21T02:20:34.536722+0000","last_became_peered":"2024-09-21T02:20:34.536722+0000","last_unstale":"2024-09-21T02:20:34.542201+0000","last_undegraded":"2024-09-21T02:20:34.542201+0000","last_fullsized":"2024-09-21T02:20:34.542201+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:05:07.829961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.486162+0000","last_change":"2024-09-21T02:20:33.512859+0000","last_active":"2024-09-21T02:20:34.486162+0000","last_peered":"2024-09-21T02:20:34.486162+0000","last_clean":"2024-09-21T02:20:34.486162+0000","last_became_active":"2024-09-21T02:20:33.512305+0000","last_became_peered":"2024-09-21T02:20:33.512305+0000","last_unstale":"2024-09-21T02:20:34.486162+0000","last_undegraded":"2024-09-21T02:20:34.486162+0000","last_fullsized":"2024-09-21T02:20:34.486162+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:22.137799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.528639+0000","last_change":"2024-09-21T02:20:34.520589+0000","last_active":"2024-09-21T02:20:34.528639+0000","last_peered":"2024-09-21T02:20:34.528639+0000","last_clean":"2024-09-21T02:20:34.528639+0000","last_became_active":"2024-09-21T02:20:34.519701+0000","last_became_peered":"2024-09-21T02:20:34.519701+0000","last_unstale":"2024-09-21T02:20:34.528639+0000","last_undegraded":"2024-09-21T02:20:34.528639+0000","last_fullsized":"2024-09-21T02:20:34.528639+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:25.286772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.485155+0000","last_change":"2024-09-21T02:20:33.504212+0000","last_active":"2024-09-21T02:20:34.485155+0000","last_peered":"2024-09-21T02:20:34.485155+0000","last_clean":"2024-09-21T02:20:34.485155+0000","last_became_active":"2024-09-21T02:20:33.503829+0000","last_became_peered":"2024-09-21T02:20:33.503829+0000","last_unstale":"2024-09-21T02:20:34.485155+0000","last_undegraded":"2024-09-21T02:20:34.485155+0000","last_fullsized":"2024-09-21T02:20:34.485155+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:07:24.282858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.534778+0000","last_change":"2024-09-21T02:20:34.512804+0000","last_active":"2024-09-21T02:20:34.534778+0000","last_peered":"2024-09-21T02:20:34.534778+0000","last_clean":"2024-09-21T02:20:34.534778+0000","last_became_active":"2024-09-21T02:20:34.511929+0000","last_became_peered":"2024-09-21T02:20:34.511929+0000","last_unstale":"2024-09-21T02:20:34.534778+0000","last_undegraded":"2024-09-21T02:20:34.534778+0000","last_fullsized":"2024-09-21T02:20:34.534778+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:04:58.178828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"129'2","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.553374+0000","last_change":"2024-09-21T02:20:33.489359+0000","last_active":"2024-09-21T02:20:34.553374+0000","last_peered":"2024-09-21T02:20:34.553374+0000","last_clean":"2024-09-21T02:20:34.553374+0000","last_became_active":"2024-09-21T02:20:33.488188+0000","last_became_peered":"2024-09-21T02:20:33.488188+0000","last_unstale":"2024-09-21T02:20:34.553374+0000","last_undegraded":"2024-09-21T02:20:34.553374+0000","last_fullsized":"2024-09-21T02:20:34.553374+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:25.628164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.537014+0000","last_change":"2024-09-21T02:20:34.527281+0000","last_active":"2024-09-21T02:20:34.537014+0000","last_peered":"2024-09-21T02:20:34.537014+0000","last_clean":"2024-09-21T02:20:34.537014+0000","last_became_active":"2024-09-21T02:20:34.525893+0000","last_became_peered":"2024-09-21T02:20:34.525893+0000","last_unstale":"2024-09-21T02:20:34.537014+0000","last_undegraded":"2024-09-21T02:20:34.537014+0000","last_fullsized":"2024-09-21T02:20:34.537014+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:21.526454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.509559+0000","last_change":"2024-09-21T02:20:33.521521+0000","last_active":"2024-09-21T02:20:34.509559+0000","last_peered":"2024-09-21T02:20:34.509559+0000","last_clean":"2024-09-21T02:20:34.509559+0000","last_became_active":"2024-09-21T02:20:33.520458+0000","last_became_peered":"2024-09-21T02:20:33.520458+0000","last_unstale":"2024-09-21T02:20:34.509559+0000","last_undegraded":"2024-09-21T02:20:34.509559+0000","last_fullsized":"2024-09-21T02:20:34.509559+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:40.434366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.534154+0000","last_change":"2024-09-21T02:20:34.525131+0000","last_active":"2024-09-21T02:20:34.534154+0000","last_peered":"2024-09-21T02:20:34.534154+0000","last_clean":"2024-09-21T02:20:34.534154+0000","last_became_active":"2024-09-21T02:20:34.524211+0000","last_became_peered":"2024-09-21T02:20:34.524211+0000","last_unstale":"2024-09-21T02:20:34.534154+0000","last_undegraded":"2024-09-21T02:20:34.534154+0000","last_fullsized":"2024-09-21T02:20:34.534154+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:01.486134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.487497+0000","last_change":"2024-09-21T02:20:33.528538+0000","last_active":"2024-09-21T02:20:34.487497+0000","last_peered":"2024-09-21T02:20:34.487497+0000","last_clean":"2024-09-21T02:20:34.487497+0000","last_became_active":"2024-09-21T02:20:33.525796+0000","last_became_peered":"2024-09-21T02:20:33.525796+0000","last_unstale":"2024-09-21T02:20:34.487497+0000","last_undegraded":"2024-09-21T02:20:34.487497+0000","last_fullsized":"2024-09-21T02:20:34.487497+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:41.590000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.538878+0000","last_change":"2024-09-21T02:20:34.534785+0000","last_active":"2024-09-21T02:20:34.538878+0000","last_peered":"2024-09-21T02:20:34.538878+0000","last_clean":"2024-09-21T02:20:34.538878+0000","last_became_active":"2024-09-21T02:20:34.533856+0000","last_became_peered":"2024-09-21T02:20:34.533856+0000","last_unstale":"2024-09-21T02:20:34.538878+0000","last_undegraded":"2024-09-21T02:20:34.538878+0000","last_fullsized":"2024-09-21T02:20:34.538878+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.255788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.487094+0000","last_change":"2024-09-21T02:20:33.523952+0000","last_active":"2024-09-21T02:20:34.487094+0000","last_peered":"2024-09-21T02:20:34.487094+0000","last_clean":"2024-09-21T02:20:34.487094+0000","last_became_active":"2024-09-21T02:20:33.523051+0000","last_became_peered":"2024-09-21T02:20:33.523051+0000","last_unstale":"2024-09-21T02:20:34.487094+0000","last_undegraded":"2024-09-21T02:20:34.487094+0000","last_fullsized":"2024-09-21T02:20:34.487094+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:44.078935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.539799+0000","last_change":"2024-09-21T02:20:34.537007+0000","last_active":"2024-09-21T02:20:34.539799+0000","last_peered":"2024-09-21T02:20:34.539799+0000","last_clean":"2024-09-21T02:20:34.539799+0000","last_became_active":"2024-09-21T02:20:34.536308+0000","last_became_peered":"2024-09-21T02:20:34.536308+0000","last_unstale":"2024-09-21T02:20:34.539799+0000","last_undegraded":"2024-09-21T02:20:34.539799+0000","last_fullsized":"2024-09-21T02:20:34.539799+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:48.275554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.508659+0000","last_change":"2024-09-21T02:20:33.521787+0000","last_active":"2024-09-21T02:20:34.508659+0000","last_peered":"2024-09-21T02:20:34.508659+0000","last_clean":"2024-09-21T02:20:34.508659+0000","last_became_active":"2024-09-21T02:20:33.520331+0000","last_became_peered":"2024-09-21T02:20:33.520331+0000","last_unstale":"2024-09-21T02:20:34.508659+0000","last_undegraded":"2024-09-21T02:20:34.508659+0000","last_fullsized":"2024-09-21T02:20:34.508659+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:31.342503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.507653+0000","last_change":"2024-09-21T02:20:34.497424+0000","last_active":"2024-09-21T02:20:34.507653+0000","last_peered":"2024-09-21T02:20:34.507653+0000","last_clean":"2024-09-21T02:20:34.507653+0000","last_became_active":"2024-09-21T02:20:34.496404+0000","last_became_peered":"2024-09-21T02:20:34.496404+0000","last_unstale":"2024-09-21T02:20:34.507653+0000","last_undegraded":"2024-09-21T02:20:34.507653+0000","last_fullsized":"2024-09-21T02:20:34.507653+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:43:43.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.515279+0000","last_change":"2024-09-21T02:20:33.501283+0000","last_active":"2024-09-21T02:20:34.515279+0000","last_peered":"2024-09-21T02:20:34.515279+0000","last_clean":"2024-09-21T02:20:34.515279+0000","last_became_active":"2024-09-21T02:20:33.500838+0000","last_became_peered":"2024-09-21T02:20:33.500838+0000","last_unstale":"2024-09-21T02:20:34.515279+0000","last_undegraded":"2024-09-21T02:20:34.515279+0000","last_fullsized":"2024-09-21T02:20:34.515279+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:41.785432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.532754+0000","last_change":"2024-09-21T02:20:34.524303+0000","last_active":"2024-09-21T02:20:34.532754+0000","last_peered":"2024-09-21T02:20:34.532754+0000","last_clean":"2024-09-21T02:20:34.532754+0000","last_became_active":"2024-09-21T02:20:34.514729+0000","last_became_peered":"2024-09-21T02:20:34.514729+0000","last_unstale":"2024-09-21T02:20:34.532754+0000","last_undegraded":"2024-09-21T02:20:34.532754+0000","last_fullsized":"2024-09-21T02:20:34.532754+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:47.230487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.501778+0000","last_change":"2024-09-21T02:20:33.516222+0000","last_active":"2024-09-21T02:20:34.501778+0000","last_peered":"2024-09-21T02:20:34.501778+0000","last_clean":"2024-09-21T02:20:34.501778+0000","last_became_active":"2024-09-21T02:20:33.515760+0000","last_became_peered":"2024-09-21T02:20:33.515760+0000","last_unstale":"2024-09-21T02:20:34.501778+0000","last_undegraded":"2024-09-21T02:20:34.501778+0000","last_fullsized":"2024-09-21T02:20:34.501778+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:11.343705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.522956+0000","last_change":"2024-09-21T02:20:34.520753+0000","last_active":"2024-09-21T02:20:34.522956+0000","last_peered":"2024-09-21T02:20:34.522956+0000","last_clean":"2024-09-21T02:20:34.522956+0000","last_became_active":"2024-09-21T02:20:34.519925+0000","last_became_peered":"2024-09-21T02:20:34.519925+0000","last_unstale":"2024-09-21T02:20:34.522956+0000","last_undegraded":"2024-09-21T02:20:34.522956+0000","last_fullsized":"2024-09-21T02:20:34.522956+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:07.716367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.500820+0000","last_change":"2024-09-21T02:20:33.514975+0000","last_active":"2024-09-21T02:20:34.500820+0000","last_peered":"2024-09-21T02:20:34.500820+0000","last_clean":"2024-09-21T02:20:34.500820+0000","last_became_active":"2024-09-21T02:20:33.514083+0000","last_became_peered":"2024-09-21T02:20:33.514083+0000","last_unstale":"2024-09-21T02:20:34.500820+0000","last_undegraded":"2024-09-21T02:20:34.500820+0000","last_fullsized":"2024-09-21T02:20:34.500820+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:25.541742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.527052+0000","last_change":"2024-09-21T02:20:34.523512+0000","last_active":"2024-09-21T02:20:34.527052+0000","last_peered":"2024-09-21T02:20:34.527052+0000","last_clean":"2024-09-21T02:20:34.527052+0000","last_became_active":"2024-09-21T02:20:34.522658+0000","last_became_peered":"2024-09-21T02:20:34.522658+0000","last_unstale":"2024-09-21T02:20:34.527052+0000","last_undegraded":"2024-09-21T02:20:34.527052+0000","last_fullsized":"2024-09-21T02:20:34.527052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:33.463036+0000","last_clean_scrub_stamp":"2024-09-21T02:20:33.463036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:25.761361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"129'3","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.552526+0000","last_change":"2024-09-21T02:20:33.518340+0000","last_active":"2024-09-21T02:20:34.552526+0000","last_peered":"2024-09-21T02:20:34.552526+0000","last_clean":"2024-09-21T02:20:34.552526+0000","last_became_active":"2024-09-21T02:20:33.517987+0000","last_became_peered":"2024-09-21T02:20:33.517987+0000","last_unstale":"2024-09-21T02:20:34.552526+0000","last_undegraded":"2024-09-21T02:20:34.552526+0000","last_fullsized":"2024-09-21T02:20:34.552526+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:20:32.459122+0000","last_clean_scrub_stamp":"2024-09-21T02:20:32.459122+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:10:50.644522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":331,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T02:20:34.485229+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:20:34.485229+0000","last_peered":"2024-09-21T02:20:34.485229+0000","last_clean":"2024-09-21T02:20:34.485229+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:20:34.485229+0000","last_undegraded":"2024-09-21T02:20:34.485229+0000","last_fullsized":"2024-09-21T02:20:34.485229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1609728,"data_stored":708396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2191360,"data_stored":1291961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509532,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1613824,"data_stored":708418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41652,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330188,"statfs":{"total":96636764160,"available":96594112512,"internally_reserved":0,"allocated":2191360,"data_stored":1291961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1597440,"data_stored":701593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:20:39.094 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:20:39.094 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d746d640 1 -- 172.21.15.170:0/3737580058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54bc004e70 msgr2=0x7f54bc0052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d746d640 1 --2- 172.21.15.170:0/3737580058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54bc004e70 0x7f54bc0052c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f54d00b5e70 tx=0x7f54c801d280 comp rx=0 tx=0).stop 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d5a0a640 1 -- 172.21.15.170:0/3737580058 reap_dead start 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d746d640 1 -- 172.21.15.170:0/3737580058 shutdown_connections 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d746d640 1 -- 172.21.15.170:0/3737580058 >> 172.21.15.170:0/3737580058 conn(0x7f54d00d0170 msgr2=0x7f54d00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d746d640 1 -- 172.21.15.170:0/3737580058 shutdown_connections 2024-09-21T02:20:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.281+0000 7f54d746d640 1 -- 172.21.15.170:0/3737580058 wait complete. 2024-09-21T02:20:39.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 Processor -- start 2024-09-21T02:20:39.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 -- start start 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54d00b0c00 0x7f54d01f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 0x7f54d0147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54d01571e0 0x7f54d01575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f54d00b3550 con 0x7f54d014a910 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f54d00b33d0 con 0x7f54d01571e0 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f54d00ce540 con 0x7f54d00b0c00 2024-09-21T02:20:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 0x7f54d0147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 0x7f54d0147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50688/0 (socket says 172.21.15.170:50688) 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 -- 172.21.15.170:0/1429330163 learned_addr learned my addr 172.21.15.170:0/1429330163 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d5209640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54d00b0c00 0x7f54d01f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d5a0a640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54d01571e0 0x7f54d01575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 -- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54d00b0c00 msgr2=0x7f54d01f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54d00b0c00 0x7f54d01f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 -- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54d01571e0 msgr2=0x7f54d01575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54d01571e0 0x7f54d01575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 -- 172.21.15.170:0/1429330163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d01f4830 con 0x7f54d014a910 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d5a0a640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54d01571e0 0x7f54d01575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d4a08640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 0x7f54d0147e70 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f54c0004da0 tx=0x7f54c0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0001d50 con 0x7f54d014a910 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c0016070 con 0x7f54d014a910 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d014aeb0 con 0x7f54d014a910 2024-09-21T02:20:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0010b80 con 0x7f54d014a910 2024-09-21T02:20:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.285+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d014b470 con 0x7f54d014a910 2024-09-21T02:20:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.289+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d00cee90 con 0x7f54d014a910 2024-09-21T02:20:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.289+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f54c001b460 con 0x7f54d014a910 2024-09-21T02:20:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.289+0000 7f54c5ffb640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f54b4073860 0x7f54b4075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.289+0000 7f54d5209640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f54b4073860 0x7f54b4075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.289+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f54c0095650 con 0x7f54d014a910 2024-09-21T02:20:39.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.289+0000 7f54d5209640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f54b4073860 0x7f54b4075d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54d009dc50 tx=0x7f54cc00f530 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:39.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.293+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f54d01f4830 con 0x7f54d014a910 2024-09-21T02:20:39.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.513+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f54d015d070 con 0x7f54d014a910 2024-09-21T02:20:39.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.517+0000 7f54c5ffb640 1 -- 172.21.15.170:0/1429330163 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7f54c005fca0 con 0x7f54d014a910 2024-09-21T02:20:39.518 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:20:39.518 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:20:39.518 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:20:39.518 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:20:39.518 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:20:39.518 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:20:39.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f54b4073860 msgr2=0x7f54b4075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f54b4073860 0x7f54b4075d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54d009dc50 tx=0x7f54cc00f530 comp rx=0 tx=0).stop 2024-09-21T02:20:39.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 msgr2=0x7f54d0147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 0x7f54d0147e70 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f54c0004da0 tx=0x7f54c0001640 comp rx=0 tx=0).stop 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 shutdown_connections 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f54b4073860 0x7f54b4075d70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54d01571e0 0x7f54d01575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d014a910 0x7f54d0147e70 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 --2- 172.21.15.170:0/1429330163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f54d00b0c00 0x7f54d01f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 >> 172.21.15.170:0/1429330163 conn(0x7f54d00d0170 msgr2=0x7f54d015b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 shutdown_connections 2024-09-21T02:20:39.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.521+0000 7f54d746d640 1 -- 172.21.15.170:0/1429330163 wait complete. 2024-09-21T02:20:39.541 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 -- 172.21.15.170:0/2008796197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64940b25f0 msgr2=0x7f64940cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 --2- 172.21.15.170:0/2008796197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64940b25f0 0x7f64940cf950 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f64940d1240 tx=0x7f648c01cce0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 -- 172.21.15.170:0/2008796197 shutdown_connections 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 --2- 172.21.15.170:0/2008796197 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6494160440 0x7f64940cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 --2- 172.21.15.170:0/2008796197 >> v2:172.21.15.191:3300/0 conn(0x7f649415faf0 0x7f649415ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 --2- 172.21.15.170:0/2008796197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64940b25f0 0x7f64940cf950 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 -- 172.21.15.170:0/2008796197 >> 172.21.15.170:0/2008796197 conn(0x7f649409a9c0 msgr2=0x7f64940d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 -- 172.21.15.170:0/2008796197 shutdown_connections 2024-09-21T02:20:39.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.689+0000 7f649956c640 1 -- 172.21.15.170:0/2008796197 wait complete. 2024-09-21T02:20:39.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 Processor -- start 2024-09-21T02:20:39.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 -- start start 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64940b25f0 0x7f64941581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 0x7f64941476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6494160440 0x7f6494159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64940b4bc0 con 0x7f649415faf0 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64940b4a40 con 0x7f64940b25f0 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f64940b4d40 con 0x7f6494160440 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 0x7f64941476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 0x7f64941476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50712/0 (socket says 172.21.15.170:50712) 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 -- 172.21.15.170:0/2988280355 learned_addr learned my addr 172.21.15.170:0/2988280355 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f6492ffd640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64940b25f0 0x7f64941581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64937fe640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6494160440 0x7f6494159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 -- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6494160440 msgr2=0x7f6494159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6494160440 0x7f6494159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 -- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64940b25f0 msgr2=0x7f64941581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64940b25f0 0x7f64941581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 -- 172.21.15.170:0/2988280355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6494158700 con 0x7f649415faf0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f64927fc640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 0x7f64941476d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c7b0 tx=0x7f647c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c007bb0 con 0x7f649415faf0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f647c007e40 con 0x7f649415faf0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6494158b80 con 0x7f649415faf0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c00f6b0 con 0x7f649415faf0 2024-09-21T02:20:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.693+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6494159140 con 0x7f649415faf0 2024-09-21T02:20:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.697+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64941588d0 con 0x7f649415faf0 2024-09-21T02:20:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.697+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f647c00f850 con 0x7f649415faf0 2024-09-21T02:20:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.697+0000 7f648b7fe640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6474073830 0x7f6474075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.697+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f647c0955a0 con 0x7f649415faf0 2024-09-21T02:20:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.697+0000 7f6492ffd640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6474073830 0x7f6474075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.697+0000 7f6492ffd640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6474073830 0x7f6474075d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f648c0af000 tx=0x7f648c01b520 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.701+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f647c014030 con 0x7f649415faf0 2024-09-21T02:20:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.905+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64940ce9d0 con 0x7f649415faf0 2024-09-21T02:20:39.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.905+0000 7f648b7fe640 1 -- 172.21.15.170:0/2988280355 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5236 (secure 0 0 0) 0x7f647c05fbe0 con 0x7f649415faf0 2024-09-21T02:20:39.906 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:39.906 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":90,"btime":"2024-09-21T02:20:37:507464+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:20:33.897736+0000","modified":"2024-09-21T02:20:36.929189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/169289834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":169289834},{"type":"v1","addr":"172.21.15.191:6835","nonce":169289834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8061,"qdb_cluster":[8061]},"id":10}]} 2024-09-21T02:20:39.907 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 90 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6474073830 msgr2=0x7f6474075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6474073830 0x7f6474075d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f648c0af000 tx=0x7f648c01b520 comp rx=0 tx=0).stop 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 msgr2=0x7f64941476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 0x7f64941476d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c7b0 tx=0x7f647c00cc70 comp rx=0 tx=0).stop 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 shutdown_connections 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6474073830 0x7f6474075d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6494160440 0x7f6494159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f649415faf0 0x7f64941476d0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 --2- 172.21.15.170:0/2988280355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64940b25f0 0x7f64941581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 >> 172.21.15.170:0/2988280355 conn(0x7f649409a9c0 msgr2=0x7f6494155190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 shutdown_connections 2024-09-21T02:20:39.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:39.909+0000 7f649956c640 1 -- 172.21.15.170:0/2988280355 wait complete. 2024-09-21T02:20:40.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:20:33.897736+0000', 'modified': '2024-09-21T02:20:36.929189+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8061': {'gid': 8061, 'name': 'd', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6835/169289834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 169289834}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 169289834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8061, 'qdb_cluster': [8061]} 2024-09-21T02:20:40.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:20:40.331 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T02:20:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd25ced4640 1 -- 172.21.15.170:0/2753728850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244003a40 msgr2=0x7fd244003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd25ced4640 1 --2- 172.21.15.170:0/2753728850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd244003a40 0x7fd244003eb0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd248009b40 tx=0x7fd24801d270 comp rx=0 tx=0).stop 2024-09-21T02:20:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd256d9d640 1 -- 172.21.15.170:0/2753728850 reap_dead start 2024-09-21T02:20:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd25ced4640 1 -- 172.21.15.170:0/2753728850 shutdown_connections 2024-09-21T02:20:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd25ced4640 1 -- 172.21.15.170:0/2753728850 >> 172.21.15.170:0/2753728850 conn(0x7fd2580b1c40 msgr2=0x7fd2580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd25ced4640 1 -- 172.21.15.170:0/2753728850 shutdown_connections 2024-09-21T02:20:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.497+0000 7fd25ced4640 1 -- 172.21.15.170:0/2753728850 wait complete. 2024-09-21T02:20:40.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 Processor -- start 2024-09-21T02:20:40.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 -- start start 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd244003a40 0x7fd2580c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd25815fd60 0x7fd2580b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 0x7fd25815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581690e0 con 0x7fd2580b4520 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd258168f60 con 0x7fd25815fd60 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd258169260 con 0x7fd244003a40 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 0x7fd25815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 0x7fd25815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50736/0 (socket says 172.21.15.170:50736) 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 -- 172.21.15.170:0/3072614622 learned_addr learned my addr 172.21.15.170:0/3072614622 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25659c640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd244003a40 0x7fd2580c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd255d9b640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd25815fd60 0x7fd2580b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 -- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd244003a40 msgr2=0x7fd2580c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd244003a40 0x7fd2580c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 -- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd25815fd60 msgr2=0x7fd2580b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd25815fd60 0x7fd2580b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 -- 172.21.15.170:0/3072614622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2580b4930 con 0x7fd2580b4520 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd256d9d640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 0x7fd25815e680 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd258070c40 tx=0x7fd25000dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd24effd640 1 -- 172.21.15.170:0/3072614622 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250003d90 con 0x7fd2580b4520 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd24effd640 1 -- 172.21.15.170:0/3072614622 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd25000b910 con 0x7fd2580b4520 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd258144210 con 0x7fd2580b4520 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd24effd640 1 -- 172.21.15.170:0/3072614622 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25000bd60 con 0x7fd2580b4520 2024-09-21T02:20:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.501+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2580c69f0 con 0x7fd2580b4520 2024-09-21T02:20:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd24effd640 1 -- 172.21.15.170:0/3072614622 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fd250006070 con 0x7fd2580b4520 2024-09-21T02:20:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd24effd640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd238073830 0x7fd238075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd25659c640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd238073830 0x7fd238075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd24effd640 1 -- 172.21.15.170:0/3072614622 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fd2500953d0 con 0x7fd2580b4520 2024-09-21T02:20:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd25659c640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd238073830 0x7fd238075d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2480ad000 tx=0x7fd24801a010 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd26419f000 1 Processor -- start 2024-09-21T02:20:40.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.505+0000 7fd26419f000 1 -- start start 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9784b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf9785040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf9785580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cf9655410 con 0x562cf923f460 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562cf9592b40 con 0x562cf9618ca0 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x562cf9581f70 con 0x562cf962f950 2024-09-21T02:20:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9784b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9784b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50750/0 (socket says 172.21.15.170:50750) 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 -- 172.21.15.170:0/3607176311 learned_addr learned my addr 172.21.15.170:0/3607176311 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd255d9b640 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf9785040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd256d9d640 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf9785580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 -- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 msgr2=0x562cf9785040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf9785040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 -- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 msgr2=0x562cf9785580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf9785580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 -- 172.21.15.170:0/3607176311 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf95495b0 con 0x562cf923f460 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd256d9d640 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf9785580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd25659c640 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9784b00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd248009b40 tx=0x7fd2480b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c61add4d037170 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd237fff640 1 -- 172.21.15.170:0/3607176311 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480c9070 con 0x562cf923f460 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd237fff640 1 -- 172.21.15.170:0/3607176311 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd248004b40 con 0x562cf923f460 2024-09-21T02:20:40.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd237fff640 1 -- 172.21.15.170:0/3607176311 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480c5450 con 0x562cf923f460 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- 172.21.15.170:0/3607176311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 msgr2=0x562cf9784b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9784b00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd248009b40 tx=0x7fd2480b9000 comp rx=0 tx=0).stop 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- 172.21.15.170:0/3607176311 shutdown_connections 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf9785040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf9785580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 --2- 172.21.15.170:0/3607176311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9784b00 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- 172.21.15.170:0/3607176311 >> 172.21.15.170:0/3607176311 conn(0x562cf9705d90 msgr2=0x562cf9735690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- 172.21.15.170:0/3607176311 shutdown_connections 2024-09-21T02:20:40.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- 172.21.15.170:0/3607176311 wait complete. 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 -- start start 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.509+0000 7fd26419f000 1 Processor -- start 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf923f460 0x562cf9773120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf9618ca0 0x562cf9773660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 0x562cf9773ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cf951c5f0 con 0x562cf962f950 2024-09-21T02:20:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562cf9585d80 con 0x562cf923f460 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x562cf9528520 con 0x562cf9618ca0 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 10 client.0 _ll_register_callbacks cb 0x562cf952c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 0x562cf9773ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 0x562cf9773ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50756/0 (socket says 172.21.15.170:50756) 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 -- 172.21.15.170:0/2906332529 learned_addr learned my addr 172.21.15.170:0/2906332529 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd255d9b640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf9618ca0 0x562cf9773660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd25659c640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf923f460 0x562cf9773120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 -- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf9618ca0 msgr2=0x562cf9773660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf9618ca0 0x562cf9773660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 -- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf923f460 msgr2=0x562cf9773120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf923f460 0x562cf9773120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 -- 172.21.15.170:0/2906332529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf9520190 con 0x562cf962f950 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd25659c640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf923f460 0x562cf9773120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd256d9d640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 0x562cf9773ba0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd2500015d0 tx=0x7fd25005e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513ec5439b17e9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 1 -- 172.21.15.170:0/2906332529 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250068a90 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 1 -- 172.21.15.170:0/2906332529 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fd25005e210 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cf951a110 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 1 -- 172.21.15.170:0/2906332529 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd25005ee40 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 10 client.8193 fetch_fsmap learned FSMap version 90 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x562cf9606f10 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 1 -- 172.21.15.170:0/2906332529 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562cf951a110 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 1 -- 172.21.15.170:0/2906332529 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7fd250064ef0 con 0x562cf962f950 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 10 client.8193 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 10 client.8193 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 20 client.8193 populate_metadata read hostname 'smithi170' 2024-09-21T02:20:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf92247d0 0x562cf97745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 4 client.8193 mds_command: new command op to 8061 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:20:40.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 --> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf9594a90 con 0x562cf92247d0 2024-09-21T02:20:40.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd25659c640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf92247d0 0x562cf97745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd2357fa640 10 client.8193 ms_handle_connect on v2:172.21.15.191:6834/169289834 2024-09-21T02:20:40.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.513+0000 7fd25659c640 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf92247d0 0x562cf97745b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6db72e2c73c7970 server_cookie=56284c8486cf4ac6 in_seq=0 out_seq=0 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd2357fa640 1 -- 172.21.15.170:0/2906332529 <== mds.0 v2:172.21.15.191:6834/169289834 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562cf9594a90 con 0x562cf92247d0 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd2357fa640 10 client.8193 handle_command_reply: tid=0 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 client.8193 shutdown 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 msgr2=0x562cf9773ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 0x562cf9773ba0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd2500015d0 tx=0x7fd25005e960 comp rx=0 tx=0).stop 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 shutdown_connections 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf92247d0 0x562cf97745b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf9618ca0 0x562cf9773660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf962f950 0x562cf9773ba0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd2357fa640 0 client.8193 ms_handle_reset on v2:172.21.15.191:6834/169289834 2024-09-21T02:20:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 --2- 172.21.15.170:0/2906332529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf923f460 0x562cf9773120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 >> 172.21.15.170:0/2906332529 conn(0x562cf9705d90 msgr2=0x562cf973d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 shutdown_connections 2024-09-21T02:20:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 1 -- 172.21.15.170:0/2906332529 wait complete. 2024-09-21T02:20:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.517+0000 7fd26419f000 20 client.8193 trim_cache size 0 max 16384 2024-09-21T02:20:40.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 Processor -- start 2024-09-21T02:20:40.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- start start 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9742190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf979b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd258201450 con 0x562cf923f460 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580746b0 con 0x562cf9618ca0 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd2581fa700 con 0x562cf962f950 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9742190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9742190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50764/0 (socket says 172.21.15.170:50764) 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 -- 172.21.15.170:0/3681076425 learned_addr learned my addr 172.21.15.170:0/3681076425 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd256d9d640 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf979b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd255d9b640 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 -- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 msgr2=0x562cf979b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf979b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 -- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 msgr2=0x562cf97426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 -- 172.21.15.170:0/3681076425 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf95dddb0 con 0x562cf923f460 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd255d9b640 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd25659c640 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9742190 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd24801cce0 tx=0x7fd2480091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a428f2bc68153b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd2377fe640 1 -- 172.21.15.170:0/3681076425 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480bc040 con 0x562cf923f460 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd2377fe640 1 -- 172.21.15.170:0/3681076425 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd248004e30 con 0x562cf923f460 2024-09-21T02:20:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd2377fe640 1 -- 172.21.15.170:0/3681076425 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480c9040 con 0x562cf923f460 2024-09-21T02:20:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- 172.21.15.170:0/3681076425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 msgr2=0x562cf9742190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9742190 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd24801cce0 tx=0x7fd2480091e0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- 172.21.15.170:0/3681076425 shutdown_connections 2024-09-21T02:20:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf979b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 --2- 172.21.15.170:0/3681076425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf9742190 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- 172.21.15.170:0/3681076425 >> 172.21.15.170:0/3681076425 conn(0x562cf9705d90 msgr2=0x562cf9741a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- 172.21.15.170:0/3681076425 shutdown_connections 2024-09-21T02:20:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.537+0000 7fd26419f000 1 -- 172.21.15.170:0/3681076425 wait complete. 2024-09-21T02:20:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- start start 2024-09-21T02:20:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 Processor -- start 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf97fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97fac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf97fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cf951c5f0 con 0x562cf923f460 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562cf9773f60 con 0x562cf9618ca0 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x562cf9585090 con 0x562cf962f950 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 10 client.0 _ll_register_callbacks cb 0x7fd250001520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf97fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf97fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50776/0 (socket says 172.21.15.170:50776) 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 -- 172.21.15.170:0/197189438 learned_addr learned my addr 172.21.15.170:0/197189438 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd256d9d640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf97fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd255d9b640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97fac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 -- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 msgr2=0x562cf97fb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf97fb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 -- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 msgr2=0x562cf97fac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97fac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 -- 172.21.15.170:0/197189438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf9548ab0 con 0x562cf923f460 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd256d9d640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf97fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd255d9b640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd25659c640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf97fa6d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd24801cce0 tx=0x7fd248009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b437926a87ff9855 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd2357fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd2357fa640 1 -- 172.21.15.170:0/197189438 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480bc070 con 0x562cf923f460 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd2357fa640 1 -- 172.21.15.170:0/197189438 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fd2480d9660 con 0x562cf923f460 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248004e30 con 0x562cf923f460 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd2357fa640 1 -- 172.21.15.170:0/197189438 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd2480ef050 con 0x562cf923f460 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 10 client.8199 fetch_fsmap learned FSMap version 90 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x562cf9527070 con 0x562cf923f460 2024-09-21T02:20:40.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd2357fa640 1 -- 172.21.15.170:0/197189438 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd248004e30 con 0x562cf923f460 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd2357fa640 1 -- 172.21.15.170:0/197189438 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7fd2480f2020 con 0x562cf923f460 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 10 client.8199 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 10 client.8199 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 20 client.8199 populate_metadata read hostname 'smithi170' 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf9212830 0x562cf9797b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 4 client.8199 mds_command: new command op to 8061 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 --> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] -- command(tid 0: {"prefix": "status"}) -- 0x562cf9590540 con 0x562cf9212830 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.541+0000 7fd255d9b640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf9212830 0x562cf9797b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd2357fa640 10 client.8199 ms_handle_connect on v2:172.21.15.191:6834/169289834 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd255d9b640 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf9212830 0x562cf9797b00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89756cba5296f44e server_cookie=c0d2975700549a6d in_seq=0 out_seq=0 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd2357fa640 1 -- 172.21.15.170:0/197189438 <== mds.0 v2:172.21.15.191:6834/169289834 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562cf9590540 con 0x562cf9212830 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd2357fa640 10 client.8199 handle_command_reply: tid=0 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 client.8199 shutdown 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 msgr2=0x562cf97fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf97fa6d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd24801cce0 tx=0x7fd248009710 comp rx=0 tx=0).stop 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 shutdown_connections 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:6834/169289834,v1:172.21.15.191:6835/169289834] conn(0x562cf9212830 0x562cf9797b00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x562cf962f950 0x562cf97fb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562cf9618ca0 0x562cf97fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd2357fa640 0 client.8199 ms_handle_reset on v2:172.21.15.191:6834/169289834 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 --2- 172.21.15.170:0/197189438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cf923f460 0x562cf97fa6d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 >> 172.21.15.170:0/197189438 conn(0x562cf9705d90 msgr2=0x562cf9742440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 shutdown_connections 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 1 -- 172.21.15.170:0/197189438 wait complete. 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd26419f000 20 client.8199 trim_cache size 0 max 16384 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:20:40.548 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8061, 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 6.0071038300000001, 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 90, 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 130, 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 130, 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.541838943 2024-09-21T02:20:40.549 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd238073830 msgr2=0x7fd238075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd238073830 0x7fd238075d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2480ad000 tx=0x7fd24801a010 comp rx=0 tx=0).stop 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 msgr2=0x7fd25815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 0x7fd25815e680 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd258070c40 tx=0x7fd25000dd80 comp rx=0 tx=0).stop 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 shutdown_connections 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd238073830 0x7fd238075d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2580b4520 0x7fd25815e680 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd25815fd60 0x7fd2580b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 --2- 172.21.15.170:0/3072614622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd244003a40 0x7fd2580c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 >> 172.21.15.170:0/3072614622 conn(0x7fd2580b1c40 msgr2=0x7fd2580ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 shutdown_connections 2024-09-21T02:20:40.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.545+0000 7fd25ced4640 1 -- 172.21.15.170:0/3072614622 wait complete. 2024-09-21T02:20:40.564 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:20:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 -- 172.21.15.170:0/4034380984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300cd210 msgr2=0x7f52300b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 --2- 172.21.15.170:0/4034380984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300cd210 0x7f52300b3350 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5228009bc0 tx=0x7f522801cba0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 -- 172.21.15.170:0/4034380984 shutdown_connections 2024-09-21T02:20:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 --2- 172.21.15.170:0/4034380984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300cd210 0x7f52300b3350 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 -- 172.21.15.170:0/4034380984 >> 172.21.15.170:0/4034380984 conn(0x7f52300b1c40 msgr2=0x7f52300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 -- 172.21.15.170:0/4034380984 shutdown_connections 2024-09-21T02:20:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 -- 172.21.15.170:0/4034380984 wait complete. 2024-09-21T02:20:40.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 Processor -- start 2024-09-21T02:20:40.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 -- start start 2024-09-21T02:20:40.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.701+0000 7f523486b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 0x7f52300c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52300b5410 0x7f52300b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f523009ba10 0x7f52300c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f523016b070 con 0x7f52300b2f40 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52300cf9a0 con 0x7f523009ba10 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f523016b1f0 con 0x7f52300b5410 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 0x7f52300c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 0x7f52300c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50784/0 (socket says 172.21.15.170:50784) 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 -- 172.21.15.170:0/2015961795 learned_addr learned my addr 172.21.15.170:0/2015961795 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522ed9d640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f523009ba10 0x7f52300c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522dd9b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52300b5410 0x7f52300b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 -- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52300b5410 msgr2=0x7f52300b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52300b5410 0x7f52300b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 -- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f523009ba10 msgr2=0x7f52300c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f523009ba10 0x7f52300c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 -- 172.21.15.170:0/2015961795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5230155be0 con 0x7f52300b2f40 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522ed9d640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f523009ba10 0x7f52300c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522e59c640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 0x7f52300c6410 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f522400c400 tx=0x7f522400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224011790 con 0x7f52300b2f40 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5224017070 con 0x7f52300b2f40 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301f68b0 con 0x7f52300b2f40 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224011cc0 con 0x7f52300b2f40 2024-09-21T02:20:40.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523015c3f0 con 0x7f52300b2f40 2024-09-21T02:20:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52300b3350 con 0x7f52300b2f40 2024-09-21T02:20:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f522401c460 con 0x7f52300b2f40 2024-09-21T02:20:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f521effd640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5210073830 0x7f5210075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f5224095640 con 0x7f52300b2f40 2024-09-21T02:20:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.705+0000 7f522dd9b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5210073830 0x7f5210075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.709+0000 7f522dd9b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5210073830 0x7f5210075d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f52300c6950 tx=0x7f5220008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.709+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f522405fd80 con 0x7f52300b2f40 2024-09-21T02:20:40.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.905+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f523009df80 con 0x7f52300b2f40 2024-09-21T02:20:40.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.905+0000 7f521effd640 1 -- 172.21.15.170:0/2015961795 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) ==== 53+0+73 (secure 0 0 0) 0x7f522405f680 con 0x7f52300b2f40 2024-09-21T02:20:40.907 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:20:40.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5210073830 msgr2=0x7f5210075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5210073830 0x7f5210075d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f52300c6950 tx=0x7f5220008040 comp rx=0 tx=0).stop 2024-09-21T02:20:40.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 msgr2=0x7f52300c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:40.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 0x7f52300c6410 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f522400c400 tx=0x7f522400c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 shutdown_connections 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5210073830 0x7f5210075d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52300b5410 0x7f52300b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f523009ba10 0x7f52300c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 --2- 172.21.15.170:0/2015961795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52300b2f40 0x7f52300c6410 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 >> 172.21.15.170:0/2015961795 conn(0x7f52300b1c40 msgr2=0x7f52300cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 shutdown_connections 2024-09-21T02:20:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:40.909+0000 7f523486b640 1 -- 172.21.15.170:0/2015961795 wait complete. 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:20:40.928 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:20:40.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:40.929 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:20:40.934 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:20:40.934 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:20:40.934 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:20:40.934 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:20:40.935 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42074sec preferred_lft 42074sec 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:40.936 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:40.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:40.937 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:20:40.986 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:20:40.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:40.988 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:20:41.040 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:41.041 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:20:41.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:20:41.042 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:20:41.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:20:41.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:20:41.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:20:41.042 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:20:41.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:20:41.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:41.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:41.162 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:20:41.208 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:20:41.208 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:20:41.209 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:20:41.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:41.209 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:20:41.265 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:20:41.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:41.265 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:41.313 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:20:41.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:41.313 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:41.360 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:20:41.418 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:41.465 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:20:41.465 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:20:41.465 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:20:41.465 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:20:41.465 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:20:41.465 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:20:41.466 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:41.467 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:41.468 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:41.468 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:41.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:41.469 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:20:41.520 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:20:41.520 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:20:41.565 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:20:41.581 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:41.632 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:20:41.632 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:20:41.633 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:20:41.634 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:41.635 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:41.635 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:41.635 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:41.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:41.636 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:20:41.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:20:41.677+0000 7f9866235580 -1 init, newargv = 0x7f98540ad020 newargc=15 2024-09-21T02:20:41.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[48841]: starting ceph client 2024-09-21T02:20:41.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[48841]: starting fuse 2024-09-21T02:20:42.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:20:42.801 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:42.850 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:42.851 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:20:42.852 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:20:42.853 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:42.853 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:20:42.853 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:42.853 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:20:42.853 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:42.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:42.854 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:20:42.900 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:20:42.900 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:20:42.901 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:20:42.902 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:20:43.025 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.48841.asok 2024-09-21T02:20:43.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:43.037 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48841.asok 2024-09-21T02:20:43.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48841.asok status 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:20:43.220 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "48841", 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8217, 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 8217 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:43.221 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3724081653 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3724081653 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.8217 192.168.0.1:0/3724081653", 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/3724081653", 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 90, 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 130, 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:20:43.222 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:20:43.223 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:20:43.223 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:20:43.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.237 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:20:43.237 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:43.243 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:20:43.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:43.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.243 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:43.301 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 -- 172.21.15.170:0/1172822694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6788005310 msgr2=0x7f678800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/1172822694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6788005310 0x7f678800b9d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6798009e30 tx=0x7f679801d1b0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 -- 172.21.15.170:0/1172822694 shutdown_connections 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/1172822694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6788005310 0x7f678800b9d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 -- 172.21.15.170:0/1172822694 >> 172.21.15.170:0/1172822694 conn(0x7f67a009a9c0 msgr2=0x7f67a00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 -- 172.21.15.170:0/1172822694 shutdown_connections 2024-09-21T02:20:43.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.501+0000 7f67a6cd6640 1 -- 172.21.15.170:0/1172822694 wait complete. 2024-09-21T02:20:43.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 Processor -- start 2024-09-21T02:20:43.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 -- start start 2024-09-21T02:20:43.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6788005310 0x7f67a0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:43.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a01f6700 0x7f67a01f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:43.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 0x7f67a01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:43.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f67a00cfce0 con 0x7f67a01500c0 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f67a00cfb60 con 0x7f6788005310 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a6cd6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f67a00cfe60 con 0x7f67a01f6700 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a5273640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 0x7f67a01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a5273640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 0x7f67a01504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50812/0 (socket says 172.21.15.170:50812) 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a5273640 1 -- 172.21.15.170:0/425065548 learned_addr learned my addr 172.21.15.170:0/425065548 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f679ffff640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a01f6700 0x7f67a01f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.505+0000 7f67a4a72640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6788005310 0x7f67a0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a5273640 1 -- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a01f6700 msgr2=0x7f67a01f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a5273640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a01f6700 0x7f67a01f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a5273640 1 -- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6788005310 msgr2=0x7f67a0153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:43.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a5273640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6788005310 0x7f67a0153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a5273640 1 -- 172.21.15.170:0/425065548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a01f36b0 con 0x7f67a01500c0 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a5273640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 0x7f67a01504d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f679801cc20 tx=0x7f67980045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67980bd070 con 0x7f67a01500c0 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6798003980 con 0x7f67a01500c0 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a0150a10 con 0x7f67a01500c0 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798004a50 con 0x7f67a01500c0 2024-09-21T02:20:43.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a0152300 con 0x7f67a01500c0 2024-09-21T02:20:43.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a01547d0 con 0x7f67a01500c0 2024-09-21T02:20:43.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f67980c0070 con 0x7f67a01500c0 2024-09-21T02:20:43.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f679d7fa640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f678c073830 0x7f678c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:43.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f67a4a72640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f678c073830 0x7f678c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:43.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.509+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f679812f830 con 0x7f67a01500c0 2024-09-21T02:20:43.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.513+0000 7f67a4a72640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f678c073830 0x7f678c075d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6790001ec0 tx=0x7f6790001070 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:43.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.513+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f67a0150a10 con 0x7f67a01500c0 2024-09-21T02:20:43.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.725+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67a00cffe0 con 0x7f67a01500c0 2024-09-21T02:20:43.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.725+0000 7f679d7fa640 1 -- 172.21.15.170:0/425065548 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) ==== 53+0+73 (secure 0 0 0) 0x7f6798016ec0 con 0x7f67a01500c0 2024-09-21T02:20:43.726 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:20:43.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f678c073830 msgr2=0x7f678c075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f678c073830 0x7f678c075d40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6790001ec0 tx=0x7f6790001070 comp rx=0 tx=0).stop 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 msgr2=0x7f67a01504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 0x7f67a01504d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f679801cc20 tx=0x7f67980045e0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 shutdown_connections 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f678c073830 0x7f678c075d40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a01f6700 0x7f67a01f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a01500c0 0x7f67a01504d0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 --2- 172.21.15.170:0/425065548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6788005310 0x7f67a0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 >> 172.21.15.170:0/425065548 conn(0x7f67a009a9c0 msgr2=0x7f67a015b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 shutdown_connections 2024-09-21T02:20:43.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:43.729+0000 7f67a6cd6640 1 -- 172.21.15.170:0/425065548 wait complete. 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:20:43.749 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:20:43.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:20:43.750 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:20:43.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.750 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:43.757 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42065sec preferred_lft 42065sec 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:20:43.758 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:20:43.759 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:43.760 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:20:43.760 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:20:43.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.760 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:20:43.810 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:20:43.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.811 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:20:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:20:43.864 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:20:43.864 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:20:43.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:20:43.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:20:43.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:20:43.864 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:20:43.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:20:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:20:43.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:20:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:43.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.987 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:20:43.993 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:20:43.993 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:20:43.993 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:20:43.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:43.994 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:20:44.049 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:20:44.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:44.049 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:44.094 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:20:44.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:20:44.094 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:44.142 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:20:44.200 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:44.247 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:20:44.247 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:20:44.247 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:20:44.247 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:20:44.247 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:20:44.247 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:20:44.248 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:44.249 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:44.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:44.250 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:20:44.298 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:20:44.298 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:20:44.343 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:20:44.359 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:44.406 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:20:44.406 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:20:44.407 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:44.408 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:44.409 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:44.409 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:44.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:44.410 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:20:44.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:20:44.452+0000 7fda05b04580 -1 init, newargv = 0x561429256fd0 newargc=15 2024-09-21T02:20:44.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35541]: starting ceph client 2024-09-21T02:20:44.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35541]: starting fuse 2024-09-21T02:20:45.558 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:20:45.575 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:20:45.629 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:20:45.629 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:20:45.629 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:20:45.629 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:20:45.629 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:20:45.629 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:20:45.630 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:20:45.631 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:45.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:45.632 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:20:45.679 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:20:45.679 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:20:45.679 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:20:45.680 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:20:45.681 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:20:45.681 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:20:45.681 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:20:45.681 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:20:45.808 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.35541.asok 2024-09-21T02:20:45.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:20:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:20:45.820 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35541.asok 2024-09-21T02:20:45.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:45.820 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35541.asok status 2024-09-21T02:20:46.006 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "35541", 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:20:46.007 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6167, 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6167 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3020924614 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3020924614 2024-09-21T02:20:46.008 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6167 192.168.0.1:0/3020924614", 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/3020924614", 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 90, 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 130, 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:20:46.009 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:20:46.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:46.023 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:20:46.023 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:46.029 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:20:46.029 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:46.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:46.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:46.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1707278791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca84003a10 msgr2=0x7fca84003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1707278791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca84003a10 0x7fca84003e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009b50 tx=0x7fca8c01d290 comp rx=0 tx=0).stop 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa1102640 1 -- 172.21.15.170:0/1707278791 reap_dead start 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1707278791 shutdown_connections 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1707278791 >> 172.21.15.170:0/1707278791 conn(0x7fca9c0b1c40 msgr2=0x7fca9c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1707278791 shutdown_connections 2024-09-21T02:20:46.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1707278791 wait complete. 2024-09-21T02:20:46.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 Processor -- start 2024-09-21T02:20:46.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.261+0000 7fcaa2b65640 1 -- start start 2024-09-21T02:20:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84003a10 0x7fca9c163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca9c09d7c0 0x7fca9c09dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 0x7fca9c15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c166570 con 0x7fca9c15dec0 2024-09-21T02:20:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c1663f0 con 0x7fca84003a10 2024-09-21T02:20:46.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fca9c1666f0 con 0x7fca9c09d7c0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 0x7fca9c15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 0x7fca9c15e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41586/0 (socket says 172.21.15.170:41586) 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 -- 172.21.15.170:0/1419625298 learned_addr learned my addr 172.21.15.170:0/1419625298 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa0901640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84003a10 0x7fca9c163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca9bfff640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca9c09d7c0 0x7fca9c09dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 -- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca9c09d7c0 msgr2=0x7fca9c09dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca9c09d7c0 0x7fca9c09dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 -- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84003a10 msgr2=0x7fca9c163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84003a10 0x7fca9c163b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 -- 172.21.15.170:0/1419625298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c1640c0 con 0x7fca9c15dec0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa1102640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 0x7fca9c15e2d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0ce3a0 tx=0x7fca9400fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9400dca0 con 0x7fca9c15dec0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca9401a070 con 0x7fca9c15dec0 2024-09-21T02:20:46.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c15e810 con 0x7fca9c15dec0 2024-09-21T02:20:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940116a0 con 0x7fca9c15dec0 2024-09-21T02:20:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c0c76b0 con 0x7fca9c15dec0 2024-09-21T02:20:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c0cabc0 con 0x7fca9c15dec0 2024-09-21T02:20:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fca9400c070 con 0x7fca9c15dec0 2024-09-21T02:20:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca997fa640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca88073860 0x7fca88075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:46.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fca940958f0 con 0x7fca9c15dec0 2024-09-21T02:20:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.265+0000 7fcaa0901640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca88073860 0x7fca88075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:46.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.269+0000 7fcaa0901640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca88073860 0x7fca88075d70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ad000 tx=0x7fca8c0af040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:46.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.269+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fca9c15e810 con 0x7fca9c15dec0 2024-09-21T02:20:46.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.485+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fca9c0b2fe0 con 0x7fca9c15dec0 2024-09-21T02:20:46.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.977+0000 7fca997fa640 1 -- 172.21.15.170:0/1419625298 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v91) ==== 136+0+0 (secure 0 0 0) 0x7fca94009e20 con 0x7fca9c15dec0 2024-09-21T02:20:46.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca88073860 msgr2=0x7fca88075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:46.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca88073860 0x7fca88075d70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ad000 tx=0x7fca8c0af040 comp rx=0 tx=0).stop 2024-09-21T02:20:46.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 msgr2=0x7fca9c15e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:46.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 0x7fca9c15e2d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0ce3a0 tx=0x7fca9400fd90 comp rx=0 tx=0).stop 2024-09-21T02:20:46.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 shutdown_connections 2024-09-21T02:20:46.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fca88073860 0x7fca88075d70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:46.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca9c09d7c0 0x7fca9c09dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:46.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca9c15dec0 0x7fca9c15e2d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:46.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 --2- 172.21.15.170:0/1419625298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84003a10 0x7fca9c163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:46.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 >> 172.21.15.170:0/1419625298 conn(0x7fca9c0b1c40 msgr2=0x7fca9c15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:46.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 shutdown_connections 2024-09-21T02:20:46.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:46.981+0000 7fcaa2b65640 1 -- 172.21.15.170:0/1419625298 wait complete. 2024-09-21T02:20:47.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:20:47.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 -- 172.21.15.170:0/547522455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc0b4540 msgr2=0x7f14fc15c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 --2- 172.21.15.170:0/547522455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc0b4540 0x7f14fc15c850 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f14fc075080 tx=0x7f14e401cb80 comp rx=0 tx=0).stop 2024-09-21T02:20:47.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 -- 172.21.15.170:0/547522455 shutdown_connections 2024-09-21T02:20:47.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 --2- 172.21.15.170:0/547522455 >> v2:172.21.15.191:3300/0 conn(0x7f14fc0d0aa0 0x7f14fc0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 --2- 172.21.15.170:0/547522455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc0b4540 0x7f14fc15c850 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 -- 172.21.15.170:0/547522455 >> 172.21.15.170:0/547522455 conn(0x7f14fc0cee30 msgr2=0x7f14fc0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:47.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 -- 172.21.15.170:0/547522455 shutdown_connections 2024-09-21T02:20:47.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.165+0000 7f1502487640 1 -- 172.21.15.170:0/547522455 wait complete. 2024-09-21T02:20:47.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 Processor -- start 2024-09-21T02:20:47.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 -- start start 2024-09-21T02:20:47.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14fc0b7ab0 0x7f14fc153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14fc0d0aa0 0x7f14fc151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 0x7f14fc1fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc16a470 con 0x7f14fc153640 2024-09-21T02:20:47.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc16a2f0 con 0x7f14fc0d0aa0 2024-09-21T02:20:47.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1502487640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc16a5f0 con 0x7f14fc0b7ab0 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1500a24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 0x7f14fc1fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1500a24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 0x7f14fc1fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41612/0 (socket says 172.21.15.170:41612) 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f1500a24640 1 -- 172.21.15.170:0/2713743440 learned_addr learned my addr 172.21.15.170:0/2713743440 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f14fb7fe640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14fc0d0aa0 0x7f14fc151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.169+0000 7f14fbfff640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14fc0b7ab0 0x7f14fc153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1500a24640 1 -- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14fc0b7ab0 msgr2=0x7f14fc153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1500a24640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14fc0b7ab0 0x7f14fc153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1500a24640 1 -- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14fc0d0aa0 msgr2=0x7f14fc151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1500a24640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14fc0d0aa0 0x7f14fc151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1500a24640 1 -- 172.21.15.170:0/2713743440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc1fb0d0 con 0x7f14fc153640 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1500a24640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 0x7f14fc1fa190 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f14f400d660 tx=0x7f14f400db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4017070 con 0x7f14fc153640 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f4005310 con 0x7f14fc153640 2024-09-21T02:20:47.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4009820 con 0x7f14fc153640 2024-09-21T02:20:47.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc0b6040 con 0x7f14fc153640 2024-09-21T02:20:47.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1522e0 con 0x7f14fc153640 2024-09-21T02:20:47.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc15c850 con 0x7f14fc153640 2024-09-21T02:20:47.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f14f40099c0 con 0x7f14fc153640 2024-09-21T02:20:47.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14f8ff9640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14dc073830 0x7f14dc075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f14f4094db0 con 0x7f14fc153640 2024-09-21T02:20:47.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14fbfff640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14dc073830 0x7f14dc075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.173+0000 7f14fbfff640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14dc073830 0x7f14dc075d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0d0480 tx=0x7f14ec0013a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.177+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f14f405ee30 con 0x7f14fc153640 2024-09-21T02:20:47.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.369+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f14fc0c8d50 con 0x7f14dc073830 2024-09-21T02:20:47.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f14f8ff9640 1 -- 172.21.15.170:0/2713743440 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f14fc0c8d50 con 0x7f14dc073830 2024-09-21T02:20:47.373 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:20:47.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:20:47.373 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:20:47.373 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:20:47.373 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:20:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14dc073830 msgr2=0x7f14dc075d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14dc073830 0x7f14dc075d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0d0480 tx=0x7f14ec0013a0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 msgr2=0x7f14fc1fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 0x7f14fc1fa190 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f14f400d660 tx=0x7f14f400db20 comp rx=0 tx=0).stop 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 shutdown_connections 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14dc073830 0x7f14dc075d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14fc153640 0x7f14fc1fa190 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14fc0d0aa0 0x7f14fc151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 --2- 172.21.15.170:0/2713743440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14fc0b7ab0 0x7f14fc153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.373+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 >> 172.21.15.170:0/2713743440 conn(0x7f14fc0cee30 msgr2=0x7f14fc0b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.377+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 shutdown_connections 2024-09-21T02:20:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.377+0000 7f1502487640 1 -- 172.21.15.170:0/2713743440 wait complete. 2024-09-21T02:20:47.393 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:20:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 -- 172.21.15.170:0/3576832152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0700b0 msgr2=0x7f266c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 --2- 172.21.15.170:0/3576832152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0700b0 0x7f266c0ce5b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2664009bc0 tx=0x7f266401cb10 comp rx=0 tx=0).stop 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 -- 172.21.15.170:0/3576832152 shutdown_connections 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 --2- 172.21.15.170:0/3576832152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0700b0 0x7f266c0ce5b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 --2- 172.21.15.170:0/3576832152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 0x7f266c0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 -- 172.21.15.170:0/3576832152 >> 172.21.15.170:0/3576832152 conn(0x7f266c0d0170 msgr2=0x7f266c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 -- 172.21.15.170:0/3576832152 shutdown_connections 2024-09-21T02:20:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.537+0000 7f26744ac640 1 -- 172.21.15.170:0/3576832152 wait complete. 2024-09-21T02:20:47.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 Processor -- start 2024-09-21T02:20:47.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 -- start start 2024-09-21T02:20:47.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c0700b0 0x7f266c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 0x7f266c09d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 0x7f266c1fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f266c0746a0 con 0x7f266c0cb790 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f266c074520 con 0x7f266c0700b0 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f26744ac640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f266c1684d0 con 0x7f266c0cb380 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 0x7f266c1fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 0x7f266c1fa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41634/0 (socket says 172.21.15.170:41634) 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 -- 172.21.15.170:0/1346671478 learned_addr learned my addr 172.21.15.170:0/1346671478 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2671a47640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 0x7f266c09d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672248640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c0700b0 0x7f266c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 -- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 msgr2=0x7f266c09d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 0x7f266c09d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 -- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c0700b0 msgr2=0x7f266c15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c0700b0 0x7f266c15b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 -- 172.21.15.170:0/1346671478 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c15e000 con 0x7f266c0cb790 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2671a47640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 0x7f266c09d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2672a49640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 0x7f266c1fa7e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2664009b10 tx=0x7f2664003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.541+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640bb070 con 0x7f266c0cb790 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2664004e00 con 0x7f266c0cb790 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640b2040 con 0x7f266c0cb790 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c15e2e0 con 0x7f266c0cb790 2024-09-21T02:20:47.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c15da90 con 0x7f266c0cb790 2024-09-21T02:20:47.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c15e800 con 0x7f266c0cb790 2024-09-21T02:20:47.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f26640be070 con 0x7f266c0cb790 2024-09-21T02:20:47.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2662ffd640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2650073860 0x7f2650075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2672248640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2650073860 0x7f2650075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f266412ed10 con 0x7f266c0cb790 2024-09-21T02:20:47.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.545+0000 7f2672248640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2650073860 0x7f2650075d70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26680058d0 tx=0x7f2668005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:47.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.549+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f266c15e800 con 0x7f266c0cb790 2024-09-21T02:20:47.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.721+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f266c06f9a0 con 0x7f266c0cb790 2024-09-21T02:20:47.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.721+0000 7f2662ffd640 1 -- 172.21.15.170:0/1346671478 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f2664016790 con 0x7f266c0cb790 2024-09-21T02:20:47.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2650073860 msgr2=0x7f2650075d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2650073860 0x7f2650075d70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26680058d0 tx=0x7f2668005860 comp rx=0 tx=0).stop 2024-09-21T02:20:47.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 msgr2=0x7f266c1fa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 0x7f266c1fa7e0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2664009b10 tx=0x7f2664003f60 comp rx=0 tx=0).stop 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 shutdown_connections 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2650073860 0x7f2650075d70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c0cb380 0x7f266c09d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f266c0cb790 0x7f266c1fa7e0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 --2- 172.21.15.170:0/1346671478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c0700b0 0x7f266c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 >> 172.21.15.170:0/1346671478 conn(0x7f266c0d0170 msgr2=0x7f266c165530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 shutdown_connections 2024-09-21T02:20:47.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.725+0000 7f26744ac640 1 -- 172.21.15.170:0/1346671478 wait complete. 2024-09-21T02:20:47.745 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 -- 172.21.15.170:0/1159579665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d4c20 msgr2=0x7f29741694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 --2- 172.21.15.170:0/1159579665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d4c20 0x7f29741694f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f296c009bc0 tx=0x7f296c01cba0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 -- 172.21.15.170:0/1159579665 shutdown_connections 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 --2- 172.21.15.170:0/1159579665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d4c20 0x7f29741694f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 -- 172.21.15.170:0/1159579665 >> 172.21.15.170:0/1159579665 conn(0x7f297408bf50 msgr2=0x7f297409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 -- 172.21.15.170:0/1159579665 shutdown_connections 2024-09-21T02:20:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 -- 172.21.15.170:0/1159579665 wait complete. 2024-09-21T02:20:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 Processor -- start 2024-09-21T02:20:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.889+0000 7f297b728640 1 -- start start 2024-09-21T02:20:47.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29740d17c0 0x7f29740cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d1bd0 0x7f29740c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2974158b20 0x7f2974158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f297416d3c0 con 0x7f29740d1bd0 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f297416d240 con 0x7f29740d17c0 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f297416d540 con 0x7f2974158b20 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d1bd0 0x7f29740c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d1bd0 0x7f29740c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41660/0 (socket says 172.21.15.170:41660) 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 -- 172.21.15.170:0/1299457576 learned_addr learned my addr 172.21.15.170:0/1299457576 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2979cc5640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2974158b20 0x7f2974158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f29794c4640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29740d17c0 0x7f29740cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 -- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2974158b20 msgr2=0x7f2974158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2974158b20 0x7f2974158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 -- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29740d17c0 msgr2=0x7f29740cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29740d17c0 0x7f29740cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 -- 172.21.15.170:0/1299457576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2974157da0 con 0x7f29740d1bd0 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f29794c4640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29740d17c0 0x7f29740cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2978cc3640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29740d1bd0 0x7f29740c9fd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f297000c3f0 tx=0x7f297000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970017070 con 0x7f29740d1bd0 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29700053c0 con 0x7f29740d1bd0 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970003860 con 0x7f29740d1bd0 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 -- 172.21.15.170:0/1299457576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2974158040 con 0x7f29740d1bd0 2024-09-21T02:20:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 -- 172.21.15.170:0/1299457576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2974158600 con 0x7f29740d1bd0 2024-09-21T02:20:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f297b728640 1 -- 172.21.15.170:0/1299457576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29740ca8e0 con 0x7f29740d1bd0 2024-09-21T02:20:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f297001a070 con 0x7f29740d1bd0 2024-09-21T02:20:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.893+0000 7f2969ffb640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2958073830 0x7f2958075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.897+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f2970094240 con 0x7f29740d1bd0 2024-09-21T02:20:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.897+0000 7f29794c4640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2958073830 0x7f2958075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.897+0000 7f29794c4640 1 --2- 172.21.15.170:0/1299457576 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2958073830 0x7f2958075d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f29640046c0 tx=0x7f2964009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:47.901+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f297005e7f0 con 0x7f29740d1bd0 2024-09-21T02:20:48.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.101+0000 7f297b728640 1 -- 172.21.15.170:0/1299457576 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f29740b3a10 con 0x7f2958073830 2024-09-21T02:20:48.104 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:48.105+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:48.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.105+0000 7f2969ffb640 1 -- 172.21.15.170:0/1299457576 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f29740b3a10 con 0x7f2958073830 2024-09-21T02:20:48.105 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:48.114 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:48.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 -- 172.21.15.170:0/536361261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c0cb790 msgr2=0x7fcd9c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 --2- 172.21.15.170:0/536361261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c0cb790 0x7fcd9c0ce5b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0ca750 tx=0x7fcd9001d920 comp rx=0 tx=0).stop 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 -- 172.21.15.170:0/536361261 shutdown_connections 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 --2- 172.21.15.170:0/536361261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c0cb790 0x7fcd9c0ce5b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 --2- 172.21.15.170:0/536361261 >> v2:172.21.15.191:3300/0 conn(0x7fcd9c0cb380 0x7fcd9c0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 -- 172.21.15.170:0/536361261 >> 172.21.15.170:0/536361261 conn(0x7fcd9c0d0170 msgr2=0x7fcd9c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 -- 172.21.15.170:0/536361261 shutdown_connections 2024-09-21T02:20:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 -- 172.21.15.170:0/536361261 wait complete. 2024-09-21T02:20:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 Processor -- start 2024-09-21T02:20:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.277+0000 7fcda209e640 1 -- start start 2024-09-21T02:20:48.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c06f9a0 0x7fcd9c1f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd9c0cb380 0x7fcd9c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd9c0cb790 0x7fcd9c0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c0746f0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c074570 con 0x7fcd9c0cb790 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c1684d0 con 0x7fcd9c0cb380 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c06f9a0 0x7fcd9c1f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c06f9a0 0x7fcd9c1f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41664/0 (socket says 172.21.15.170:41664) 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 -- 172.21.15.170:0/3597537595 learned_addr learned my addr 172.21.15.170:0/3597537595 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 -- 172.21.15.170:0/3597537595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd9c0cb380 msgr2=0x7fcd9c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 --2- 172.21.15.170:0/3597537595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd9c0cb380 0x7fcd9c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 -- 172.21.15.170:0/3597537595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd9c0cb790 msgr2=0x7fcd9c0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 --2- 172.21.15.170:0/3597537595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd9c0cb790 0x7fcd9c0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 -- 172.21.15.170:0/3597537595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c15fb10 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd9b7fe640 1 --2- 172.21.15.170:0/3597537595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c06f9a0 0x7fcd9c1f8af0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00b440 tx=0x7fcd8c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c015020 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd8c0119a0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c011df0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 -- 172.21.15.170:0/3597537595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c15fdf0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 -- 172.21.15.170:0/3597537595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c1603b0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.281+0000 7fcda209e640 1 -- 172.21.15.170:0/3597537595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c06fdb0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.285+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fcd8c018020 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.285+0000 7fcd83fff640 1 --2- 172.21.15.170:0/3597537595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd88073830 0x7fcd88075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.285+0000 7fcd9affd640 1 --2- 172.21.15.170:0/3597537595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd88073830 0x7fcd88075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:48.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.285+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fcd8c094cf0 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.285+0000 7fcd9affd640 1 --2- 172.21.15.170:0/3597537595 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcd88073830 0x7fcd88075d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd900b0000 tx=0x7fcd9001a6c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:48.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.289+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcd8c00d040 con 0x7fcd9c06f9a0 2024-09-21T02:20:48.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.473+0000 7fcda209e640 1 -- 172.21.15.170:0/3597537595 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fcd9c151bc0 con 0x7fcd88073830 2024-09-21T02:20:48.477 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:48.477+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.477+0000 7fcd83fff640 1 -- 172.21.15.170:0/3597537595 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fcd90004020 con 0x7fcd88073830 2024-09-21T02:20:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:48.485 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:48.486 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-21T02:20:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 -- 172.21.15.170:0/938980371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 msgr2=0x7fd9980d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 --2- 172.21.15.170:0/938980371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 0x7fd9980d0a40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd990009c80 tx=0x7fd99001cce0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 -- 172.21.15.170:0/938980371 shutdown_connections 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 --2- 172.21.15.170:0/938980371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 0x7fd9980d0a40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 --2- 172.21.15.170:0/938980371 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9980b51d0 0x7fd9980b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 -- 172.21.15.170:0/938980371 >> 172.21.15.170:0/938980371 conn(0x7fd9980cee00 msgr2=0x7fd9980cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 -- 172.21.15.170:0/938980371 shutdown_connections 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.637+0000 7fd99d1a5640 1 -- 172.21.15.170:0/938980371 wait complete. 2024-09-21T02:20:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 Processor -- start 2024-09-21T02:20:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- start start 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd99806e330 0x7fd9980b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9980b51d0 0x7fd998157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 0x7fd99815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9980cf920 con 0x7fd9980d0630 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd9980b81b0 con 0x7fd99806e330 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd9980cfaa0 con 0x7fd9980b51d0 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 0x7fd99815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 0x7fd99815c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41680/0 (socket says 172.21.15.170:41680) 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 -- 172.21.15.170:0/543434032 learned_addr learned my addr 172.21.15.170:0/543434032 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 -- 172.21.15.170:0/543434032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9980b51d0 msgr2=0x7fd998157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9980b51d0 0x7fd998157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd996d9d640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd99806e330 0x7fd9980b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 -- 172.21.15.170:0/543434032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd99806e330 msgr2=0x7fd9980b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd99806e330 0x7fd9980b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 -- 172.21.15.170:0/543434032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9980cdb90 con 0x7fd9980d0630 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99759e640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9980d0630 0x7fd99815c230 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd990009bd0 tx=0x7fd9900093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900bc070 con 0x7fd9980d0630 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9900042f0 con 0x7fd9980d0630 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- 172.21.15.170:0/543434032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9980cde70 con 0x7fd9980d0630 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900b2040 con 0x7fd9980d0630 2024-09-21T02:20:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- 172.21.15.170:0/543434032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9980ce430 con 0x7fd9980d0630 2024-09-21T02:20:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.641+0000 7fd99d1a5640 1 -- 172.21.15.170:0/543434032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd998159e30 con 0x7fd9980d0630 2024-09-21T02:20:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.645+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fd9900bf070 con 0x7fd9980d0630 2024-09-21T02:20:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.645+0000 7fd97f7fe640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073830 0x7fd984075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.645+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fd99012edb0 con 0x7fd9980d0630 2024-09-21T02:20:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.645+0000 7fd996d9d640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073830 0x7fd984075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:48.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.645+0000 7fd996d9d640 1 --2- 172.21.15.170:0/543434032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073830 0x7fd984075d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd98800a750 tx=0x7fd988008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.649+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd990016990 con 0x7fd9980d0630 2024-09-21T02:20:48.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.833+0000 7fd99d1a5640 1 -- 172.21.15.170:0/543434032 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fd9980b55e0 con 0x7fd984073830 2024-09-21T02:20:48.835 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:48.833+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:48.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:48.833+0000 7fd97f7fe640 1 -- 172.21.15.170:0/543434032 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd9980b55e0 con 0x7fd984073830 2024-09-21T02:20:48.836 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:48.845 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:48.846 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 -- 172.21.15.170:0/4227109762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 msgr2=0x7f76300ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 --2- 172.21.15.170:0/4227109762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 0x7f76300ce070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f76300ca750 tx=0x7f762801caf0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 -- 172.21.15.170:0/4227109762 shutdown_connections 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 --2- 172.21.15.170:0/4227109762 >> v2:172.21.15.191:3300/0 conn(0x7f76300cb790 0x7f76300ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 --2- 172.21.15.170:0/4227109762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 0x7f76300ce070 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 -- 172.21.15.170:0/4227109762 >> 172.21.15.170:0/4227109762 conn(0x7f76300d0170 msgr2=0x7f76300d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 -- 172.21.15.170:0/4227109762 shutdown_connections 2024-09-21T02:20:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 -- 172.21.15.170:0/4227109762 wait complete. 2024-09-21T02:20:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 Processor -- start 2024-09-21T02:20:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.001+0000 7f7637cb7640 1 -- start start 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f763006f9a0 0x7f763015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 0x7f763009c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76300cb790 0x7f763009ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76300746f0 con 0x7f76300cb380 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7630074570 con 0x7f76300cb790 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f76301695e0 con 0x7f763006f9a0 2024-09-21T02:20:49.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 0x7f763009c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 0x7f763009c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41688/0 (socket says 172.21.15.170:41688) 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 -- 172.21.15.170:0/2916544097 learned_addr learned my addr 172.21.15.170:0/2916544097 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7636254640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76300cb790 0x7f763009ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635a53640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f763006f9a0 0x7f763015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 -- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f763006f9a0 msgr2=0x7f763015c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f763006f9a0 0x7f763015c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 -- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76300cb790 msgr2=0x7f763009ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76300cb790 0x7f763009ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 -- 172.21.15.170:0/2916544097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76300c4c80 con 0x7f76300cb380 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635a53640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f763006f9a0 0x7f763015c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7635252640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76300cb380 0x7f763009c4c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f762400d720 tx=0x7f762400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624017070 con 0x7f76300cb380 2024-09-21T02:20:49.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7624009ab0 con 0x7f76300cb380 2024-09-21T02:20:49.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 -- 172.21.15.170:0/2916544097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76301635b0 con 0x7f76300cb380 2024-09-21T02:20:49.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624010460 con 0x7f76300cb380 2024-09-21T02:20:49.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 -- 172.21.15.170:0/2916544097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7630163b70 con 0x7f76300cb380 2024-09-21T02:20:49.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f7637cb7640 1 -- 172.21.15.170:0/2916544097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763006fdb0 con 0x7f76300cb380 2024-09-21T02:20:49.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f762401a070 con 0x7f76300cb380 2024-09-21T02:20:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.005+0000 7f762e7fc640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f761c073860 0x7f761c075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.009+0000 7f7635a53640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f761c073860 0x7f761c075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.009+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f7624094fd0 con 0x7f76300cb380 2024-09-21T02:20:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.009+0000 7f7635a53640 1 --2- 172.21.15.170:0/2916544097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f761c073860 0x7f761c075d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f76280af000 tx=0x7f762801b330 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:49.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.013+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f762405f580 con 0x7f76300cb380 2024-09-21T02:20:49.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.205+0000 7f7637cb7640 1 -- 172.21.15.170:0/2916544097 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f763015c940 con 0x7f761c073860 2024-09-21T02:20:49.208 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:49.209+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.209+0000 7f762e7fc640 1 -- 172.21.15.170:0/2916544097 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f763015c940 con 0x7f761c073860 2024-09-21T02:20:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:49.218 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:49.219 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b6331d640 1 -- 172.21.15.170:0/550579700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b48004d50 msgr2=0x7f1b48005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b6331d640 1 --2- 172.21.15.170:0/550579700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b48004d50 0x7f1b48005160 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c09afc0 tx=0x7f1b5401d280 comp rx=0 tx=0).stop 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b618ba640 1 -- 172.21.15.170:0/550579700 reap_dead start 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b6331d640 1 -- 172.21.15.170:0/550579700 shutdown_connections 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b6331d640 1 -- 172.21.15.170:0/550579700 >> 172.21.15.170:0/550579700 conn(0x7f1b5c09a9c0 msgr2=0x7f1b5c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b6331d640 1 -- 172.21.15.170:0/550579700 shutdown_connections 2024-09-21T02:20:49.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.381+0000 7f1b6331d640 1 -- 172.21.15.170:0/550579700 wait complete. 2024-09-21T02:20:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 Processor -- start 2024-09-21T02:20:49.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 -- start start 2024-09-21T02:20:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b48004d50 0x7f1b5c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b5c1f7790 0x7f1b5c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b5c159450 0x7f1b5c1f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b6810 con 0x7f1b48004d50 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0cea80 con 0x7f1b5c1f7790 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b6990 con 0x7f1b5c159450 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b48004d50 0x7f1b5c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b48004d50 0x7f1b5c14bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41710/0 (socket says 172.21.15.170:41710) 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 -- 172.21.15.170:0/169463967 learned_addr learned my addr 172.21.15.170:0/169463967 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b608b8640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b5c1f7790 0x7f1b5c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b618ba640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b5c159450 0x7f1b5c1f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 -- 172.21.15.170:0/169463967 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b5c159450 msgr2=0x7f1b5c1f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b5c159450 0x7f1b5c1f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 -- 172.21.15.170:0/169463967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b5c1f7790 msgr2=0x7f1b5c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b5c1f7790 0x7f1b5c158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 -- 172.21.15.170:0/169463967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c159860 con 0x7f1b48004d50 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b610b9640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b48004d50 0x7f1b5c14bd60 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c010a40 tx=0x7f1b4c010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00ed40 con 0x7f1b48004d50 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b4c005020 con 0x7f1b48004d50 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c005450 con 0x7f1b48004d50 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 -- 172.21.15.170:0/169463967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c09b420 con 0x7f1b48004d50 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.385+0000 7f1b6331d640 1 -- 172.21.15.170:0/169463967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c09b9e0 con 0x7f1b48004d50 2024-09-21T02:20:49.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.389+0000 7f1b6331d640 1 -- 172.21.15.170:0/169463967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c15b520 con 0x7f1b48004d50 2024-09-21T02:20:49.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.389+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f1b4c0047b0 con 0x7f1b48004d50 2024-09-21T02:20:49.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.389+0000 7f1b51ffb640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b40073860 0x7f1b40075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.389+0000 7f1b608b8640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b40073860 0x7f1b40075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.389+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f1b4c095b40 con 0x7f1b48004d50 2024-09-21T02:20:49.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.389+0000 7f1b608b8640 1 --2- 172.21.15.170:0/169463967 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b40073860 0x7f1b40075d70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009470 tx=0x7f1b580093a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:49.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.393+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1b5c159860 con 0x7f1b48004d50 2024-09-21T02:20:49.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.585+0000 7f1b6331d640 1 -- 172.21.15.170:0/169463967 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f1b5c0b63a0 con 0x7f1b40073860 2024-09-21T02:20:49.590 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:49.589+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:49.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.589+0000 7f1b51ffb640 1 -- 172.21.15.170:0/169463967 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1b5c0b63a0 con 0x7f1b40073860 2024-09-21T02:20:49.590 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:49.598 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:49.599 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 -- 172.21.15.170:0/2788505635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768073640 msgr2=0x7f8768073a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 --2- 172.21.15.170:0/2788505635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768073640 0x7f8768073a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8760009c80 tx=0x7f876001cda0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 -- 172.21.15.170:0/2788505635 shutdown_connections 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 --2- 172.21.15.170:0/2788505635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768073640 0x7f8768073a50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 --2- 172.21.15.170:0/2788505635 >> v2:172.21.15.191:3300/0 conn(0x7f87680706d0 0x7f87680cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 -- 172.21.15.170:0/2788505635 >> 172.21.15.170:0/2788505635 conn(0x7f87680b1c40 msgr2=0x7f87680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:49.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 -- 172.21.15.170:0/2788505635 shutdown_connections 2024-09-21T02:20:49.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.761+0000 7f876ccae640 1 -- 172.21.15.170:0/2788505635 wait complete. 2024-09-21T02:20:49.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 Processor -- start 2024-09-21T02:20:49.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 -- start start 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768073640 0x7f87680ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87680ca7b0 0x7f87681f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f876814c860 0x7f87681f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8768168f30 con 0x7f8768073640 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8768168db0 con 0x7f87680ca7b0 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876ccae640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f87681690b0 con 0x7f876814c860 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768073640 0x7f87680ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768073640 0x7f87680ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41716/0 (socket says 172.21.15.170:41716) 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 -- 172.21.15.170:0/1523804703 learned_addr learned my addr 172.21.15.170:0/1523804703 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f8766d9d640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f876814c860 0x7f87681f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f8765d9b640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87680ca7b0 0x7f87681f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 -- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f876814c860 msgr2=0x7f87681f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f876814c860 0x7f87681f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 -- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87680ca7b0 msgr2=0x7f87681f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87680ca7b0 0x7f87681f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f876659c640 1 -- 172.21.15.170:0/1523804703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876814ecb0 con 0x7f8768073640 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.765+0000 7f8765d9b640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87680ca7b0 0x7f87681f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f876659c640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768073640 0x7f87680ae9d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f875800e710 tx=0x7f875800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580052d0 con 0x7f8768073640 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f875800ee20 con 0x7f8768073640 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f876ccae640 1 -- 172.21.15.170:0/1523804703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876814bdb0 con 0x7f8768073640 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758001b10 con 0x7f8768073640 2024-09-21T02:20:49.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f876ccae640 1 -- 172.21.15.170:0/1523804703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876814c020 con 0x7f8768073640 2024-09-21T02:20:49.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f876ccae640 1 -- 172.21.15.170:0/1523804703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87680cabc0 con 0x7f8768073640 2024-09-21T02:20:49.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f8758003650 con 0x7f8768073640 2024-09-21T02:20:49.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8756ffd640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8748073830 0x7f8748075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:49.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f8758097660 con 0x7f8768073640 2024-09-21T02:20:49.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8765d9b640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8748073830 0x7f8748075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:49.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.769+0000 7f8765d9b640 1 --2- 172.21.15.170:0/1523804703 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8748073830 0x7f8748075d40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f876809dd20 tx=0x7f875c0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:49.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.773+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8758016020 con 0x7f8768073640 2024-09-21T02:20:49.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.981+0000 7f876ccae640 1 -- 172.21.15.170:0/1523804703 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f876809b1b0 con 0x7f8748073830 2024-09-21T02:20:49.982 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:49.981+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:49.981+0000 7f8756ffd640 1 -- 172.21.15.170:0/1523804703 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f876809b1b0 con 0x7f8748073830 2024-09-21T02:20:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:49.994 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:49.994 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-21T02:20:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 -- 172.21.15.170:0/3020224209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 msgr2=0x7f1a40003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 --2- 172.21.15.170:0/3020224209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 0x7f1a40003eb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009e00 tx=0x7f1a5001d130 comp rx=0 tx=0).stop 2024-09-21T02:20:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 -- 172.21.15.170:0/3020224209 shutdown_connections 2024-09-21T02:20:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 --2- 172.21.15.170:0/3020224209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 0x7f1a40003eb0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 -- 172.21.15.170:0/3020224209 >> 172.21.15.170:0/3020224209 conn(0x7f1a540d2a30 msgr2=0x7f1a540cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 -- 172.21.15.170:0/3020224209 shutdown_connections 2024-09-21T02:20:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 -- 172.21.15.170:0/3020224209 wait complete. 2024-09-21T02:20:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 Processor -- start 2024-09-21T02:20:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.141+0000 7f1a5c31e640 1 -- start start 2024-09-21T02:20:50.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 0x7f1a541543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a54158560 0x7f1a54158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a540b1e30 0x7f1a540b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54071890 con 0x7f1a40003a60 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1a54071710 con 0x7f1a54158560 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1a54071a10 con 0x7f1a540b1e30 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 0x7f1a541543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 0x7f1a541543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41732/0 (socket says 172.21.15.170:41732) 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 -- 172.21.15.170:0/19439967 learned_addr learned my addr 172.21.15.170:0/19439967 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a598b9640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a54158560 0x7f1a54158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 -- 172.21.15.170:0/19439967 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a540b1e30 msgr2=0x7f1a540b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a540b1e30 0x7f1a540b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 -- 172.21.15.170:0/19439967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a54158560 msgr2=0x7f1a54158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a54158560 0x7f1a54158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 -- 172.21.15.170:0/19439967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a540b2780 con 0x7f1a40003a60 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a598b9640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a54158560 0x7f1a54158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5a0ba640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a40003a60 0x7f1a541543a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f1a500af040 tx=0x7f1a500ace30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500b1040 con 0x7f1a40003a60 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a500be070 con 0x7f1a40003a60 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500b76d0 con 0x7f1a40003a60 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 -- 172.21.15.170:0/19439967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a54159250 con 0x7f1a40003a60 2024-09-21T02:20:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 -- 172.21.15.170:0/19439967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a54159810 con 0x7f1a40003a60 2024-09-21T02:20:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a5c31e640 1 -- 172.21.15.170:0/19439967 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a540b4180 con 0x7f1a40003a60 2024-09-21T02:20:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f1a500b7870 con 0x7f1a40003a60 2024-09-21T02:20:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.145+0000 7f1a4affd640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1a38073830 0x7f1a38075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.149+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f1a50131380 con 0x7f1a40003a60 2024-09-21T02:20:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.149+0000 7f1a598b9640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1a38073830 0x7f1a38075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.149+0000 7f1a598b9640 1 --2- 172.21.15.170:0/19439967 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1a38073830 0x7f1a38075d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a44001890 tx=0x7f1a44009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:50.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.149+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1a540b2780 con 0x7f1a40003a60 2024-09-21T02:20:50.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.333+0000 7f1a5c31e640 1 -- 172.21.15.170:0/19439967 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f1a54070120 con 0x7f1a38073830 2024-09-21T02:20:50.335 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:50.333+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:50.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.337+0000 7f1a4affd640 1 -- 172.21.15.170:0/19439967 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1a54070120 con 0x7f1a38073830 2024-09-21T02:20:50.336 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:50.343 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:50.344 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 -- 172.21.15.170:0/1668689410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00cf8f0 msgr2=0x7ff9c00d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 --2- 172.21.15.170:0/1668689410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00cf8f0 0x7ff9c00d1f20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00d13d0 tx=0x7ff9b801cc80 comp rx=0 tx=0).stop 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 -- 172.21.15.170:0/1668689410 shutdown_connections 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 --2- 172.21.15.170:0/1668689410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9c00ae0d0 0x7ff9c00ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 --2- 172.21.15.170:0/1668689410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00cf8f0 0x7ff9c00d1f20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 -- 172.21.15.170:0/1668689410 >> 172.21.15.170:0/1668689410 conn(0x7ff9c0074320 msgr2=0x7ff9c00d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 -- 172.21.15.170:0/1668689410 shutdown_connections 2024-09-21T02:20:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.501+0000 7ff9c6827640 1 -- 172.21.15.170:0/1668689410 wait complete. 2024-09-21T02:20:50.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 Processor -- start 2024-09-21T02:20:50.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- start start 2024-09-21T02:20:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00ae0d0 0x7ff9c014fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9c00d1b10 0x7ff9c00cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9c0151990 0x7ff9c00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0168cf0 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0168b70 con 0x7ff9c00d1b10 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c0168e70 con 0x7ff9c0151990 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00ae0d0 0x7ff9c014fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00ae0d0 0x7ff9c014fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41736/0 (socket says 172.21.15.170:41736) 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 -- 172.21.15.170:0/2045285056 learned_addr learned my addr 172.21.15.170:0/2045285056 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 -- 172.21.15.170:0/2045285056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9c0151990 msgr2=0x7ff9c00ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 --2- 172.21.15.170:0/2045285056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9c0151990 0x7ff9c00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 -- 172.21.15.170:0/2045285056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9c00d1b10 msgr2=0x7ff9c00cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 --2- 172.21.15.170:0/2045285056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9c00d1b10 0x7ff9c00cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 -- 172.21.15.170:0/2045285056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c0157480 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bffff640 1 --2- 172.21.15.170:0/2045285056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9c00ae0d0 0x7ff9c014fda0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000b3e0 tx=0x7ff9b000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0015020 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- 172.21.15.170:0/2045285056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c01528d0 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b0011940 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0011d90 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- 172.21.15.170:0/2045285056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c0152e90 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.505+0000 7ff9c6827640 1 -- 172.21.15.170:0/2045285056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c00ae4e0 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.509+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7ff9b0018020 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.509+0000 7ff9bcff9640 1 --2- 172.21.15.170:0/2045285056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9ac073830 0x7ff9ac075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.509+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7ff9b0094e10 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.509+0000 7ff9bf7fe640 1 --2- 172.21.15.170:0/2045285056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9ac073830 0x7ff9ac075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.509+0000 7ff9bf7fe640 1 --2- 172.21.15.170:0/2045285056 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff9ac073830 0x7ff9ac075d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80af000 tx=0x7ff9b8019a80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:50.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.513+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff9b001f020 con 0x7ff9c00ae0d0 2024-09-21T02:20:50.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.709+0000 7ff9c6827640 1 -- 172.21.15.170:0/2045285056 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7ff9c00cebf0 con 0x7ff9ac073830 2024-09-21T02:20:50.712 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:50.713+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:50.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.713+0000 7ff9bcff9640 1 -- 172.21.15.170:0/2045285056 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff9c00cebf0 con 0x7ff9ac073830 2024-09-21T02:20:50.713 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:50.722 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:50.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T02:20:50.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 -- 172.21.15.170:0/434663049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b4d80 msgr2=0x7f00e00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 --2- 172.21.15.170:0/434663049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b4d80 0x7f00e00ccae0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f00e0072900 tx=0x7f00d001c570 comp rx=0 tx=0).stop 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 -- 172.21.15.170:0/434663049 shutdown_connections 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 --2- 172.21.15.170:0/434663049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00e00cd020 0x7f00e00cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 --2- 172.21.15.170:0/434663049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b4d80 0x7f00e00ccae0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 --2- 172.21.15.170:0/434663049 >> v2:172.21.15.191:3300/0 conn(0x7f00e00b52d0 0x7f00e00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 -- 172.21.15.170:0/434663049 >> 172.21.15.170:0/434663049 conn(0x7f00e00b1c40 msgr2=0x7f00e00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 -- 172.21.15.170:0/434663049 shutdown_connections 2024-09-21T02:20:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 -- 172.21.15.170:0/434663049 wait complete. 2024-09-21T02:20:50.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 Processor -- start 2024-09-21T02:20:50.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.885+0000 7f00e7a36640 1 -- start start 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00e00b4d80 0x7f00e00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b52d0 0x7f00e0164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00e00cd020 0x7f00e009ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00e006f630 con 0x7f00e00b52d0 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00e006f4b0 con 0x7f00e00cd020 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f00e006f7b0 con 0x7f00e00b4d80 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b52d0 0x7f00e0164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b52d0 0x7f00e0164470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41740/0 (socket says 172.21.15.170:41740) 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 -- 172.21.15.170:0/2975279796 learned_addr learned my addr 172.21.15.170:0/2975279796 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:50.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e57d2640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00e00b4d80 0x7f00e00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 -- 172.21.15.170:0/2975279796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00e00b4d80 msgr2=0x7f00e00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00e00b4d80 0x7f00e00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 -- 172.21.15.170:0/2975279796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00e00cd020 msgr2=0x7f00e009ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00e00cd020 0x7f00e009ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 -- 172.21.15.170:0/2975279796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00c9ef0 con 0x7f00e00b52d0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e4fd1640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e00b52d0 0x7f00e0164470 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f00d001c3a0 tx=0x7f00d00a9520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00ad070 con 0x7f00e00b52d0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00d00a9cc0 con 0x7f00e00b52d0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 -- 172.21.15.170:0/2975279796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01f6e00 con 0x7f00e00b52d0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0003860 con 0x7f00e00b52d0 2024-09-21T02:20:50.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 -- 172.21.15.170:0/2975279796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01f73c0 con 0x7f00e00b52d0 2024-09-21T02:20:50.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e7a36640 1 -- 172.21.15.170:0/2975279796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e006fd10 con 0x7f00e00b52d0 2024-09-21T02:20:50.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f00d00b0070 con 0x7f00e00b52d0 2024-09-21T02:20:50.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00d5ffb640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f00c4073830 0x7f00c4075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:50.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e57d2640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f00c4073830 0x7f00c4075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:50.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f00d012f810 con 0x7f00e00b52d0 2024-09-21T02:20:50.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.889+0000 7f00e57d2640 1 --2- 172.21.15.170:0/2975279796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f00c4073830 0x7f00c4075d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00dc005870 tx=0x7f00dc00c440 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:50.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:50.893+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f00d00164b0 con 0x7f00e00b52d0 2024-09-21T02:20:51.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.105+0000 7f00e7a36640 1 -- 172.21.15.170:0/2975279796 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f00e00b41d0 con 0x7f00c4073830 2024-09-21T02:20:51.107 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:51.105+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:51.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.105+0000 7f00d5ffb640 1 -- 172.21.15.170:0/2975279796 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f00e00b41d0 con 0x7f00c4073830 2024-09-21T02:20:51.107 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:51.119 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:51.120 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T02:20:51.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 -- 172.21.15.170:0/1905291720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 msgr2=0x7f8cd40b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 --2- 172.21.15.170:0/1905291720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 0x7f8cd40b0a90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009c80 tx=0x7f8ccc01cda0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 -- 172.21.15.170:0/1905291720 shutdown_connections 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 --2- 172.21.15.170:0/1905291720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 0x7f8cd40b0a90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 --2- 172.21.15.170:0/1905291720 >> v2:172.21.15.191:3300/0 conn(0x7f8cd40b6480 0x7f8cd415eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 -- 172.21.15.170:0/1905291720 >> 172.21.15.170:0/1905291720 conn(0x7f8cd409a9c0 msgr2=0x7f8cd40d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 -- 172.21.15.170:0/1905291720 shutdown_connections 2024-09-21T02:20:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.265+0000 7f8cd9273640 1 -- 172.21.15.170:0/1905291720 wait complete. 2024-09-21T02:20:51.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 Processor -- start 2024-09-21T02:20:51.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 -- start start 2024-09-21T02:20:51.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd406ce00 0x7f8cd4070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 0x7f8cd4070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cd40b6480 0x7f8cd4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd416c190 con 0x7f8cd406ce00 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd416c010 con 0x7f8cd40b6480 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd416c310 con 0x7f8cd40b0680 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd406ce00 0x7f8cd4070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd406ce00 0x7f8cd4070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41744/0 (socket says 172.21.15.170:41744) 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 -- 172.21.15.170:0/1405464678 learned_addr learned my addr 172.21.15.170:0/1405464678 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd37fe640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cd40b6480 0x7f8cd4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd27fc640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 0x7f8cd4070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 -- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 msgr2=0x7f8cd4070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 0x7f8cd4070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 -- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cd40b6480 msgr2=0x7f8cd4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cd40b6480 0x7f8cd4159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 -- 172.21.15.170:0/1405464678 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd415b660 con 0x7f8cd406ce00 2024-09-21T02:20:51.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd27fc640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cd40b0680 0x7f8cd4070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd2ffd640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd406ce00 0x7f8cd4070540 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400ea20 tx=0x7f8cc400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cd80 con 0x7f8cd406ce00 2024-09-21T02:20:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cc4010430 con 0x7f8cd406ce00 2024-09-21T02:20:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 -- 172.21.15.170:0/1405464678 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd415b940 con 0x7f8cd406ce00 2024-09-21T02:20:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4010880 con 0x7f8cd406ce00 2024-09-21T02:20:51.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.269+0000 7f8cd9273640 1 -- 172.21.15.170:0/1405464678 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd409c910 con 0x7f8cd406ce00 2024-09-21T02:20:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.273+0000 7f8cd9273640 1 -- 172.21.15.170:0/1405464678 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd40b0a90 con 0x7f8cd406ce00 2024-09-21T02:20:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.273+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f8cc4010a20 con 0x7f8cd406ce00 2024-09-21T02:20:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.273+0000 7f8cbb7fe640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8cc0073830 0x7f8cc0075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.273+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f8cc4095a30 con 0x7f8cd406ce00 2024-09-21T02:20:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.273+0000 7f8cd27fc640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8cc0073830 0x7f8cc0075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.273+0000 7f8cd27fc640 1 --2- 172.21.15.170:0/1405464678 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8cc0073830 0x7f8cc0075d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8cd409bbf0 tx=0x7f8cc8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:51.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.277+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8cc405ffe0 con 0x7f8cd406ce00 2024-09-21T02:20:51.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.473+0000 7f8cd9273640 1 -- 172.21.15.170:0/1405464678 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f8cd41552a0 con 0x7f8cc0073830 2024-09-21T02:20:51.475 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:51.473+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:51.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.473+0000 7f8cbb7fe640 1 -- 172.21.15.170:0/1405464678 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8cd41552a0 con 0x7f8cc0073830 2024-09-21T02:20:51.475 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:51.488 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:51.488 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.641+0000 7f790b19a640 1 -- 172.21.15.170:0/3689489262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790406f2d0 msgr2=0x7f790406f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.641+0000 7f790b19a640 1 --2- 172.21.15.170:0/3689489262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790406f2d0 0x7f790406f6e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00acd0 tx=0x7f78fc01def0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.641+0000 7f790b19a640 1 -- 172.21.15.170:0/3689489262 shutdown_connections 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.641+0000 7f790b19a640 1 --2- 172.21.15.170:0/3689489262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790406f2d0 0x7f790406f6e0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.641+0000 7f790b19a640 1 --2- 172.21.15.170:0/3689489262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79040ca720 0x7f79040cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.641+0000 7f790b19a640 1 -- 172.21.15.170:0/3689489262 >> 172.21.15.170:0/3689489262 conn(0x7f79040b1c40 msgr2=0x7f79040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 -- 172.21.15.170:0/3689489262 shutdown_connections 2024-09-21T02:20:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 -- 172.21.15.170:0/3689489262 wait complete. 2024-09-21T02:20:51.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 Processor -- start 2024-09-21T02:20:51.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 -- start start 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f790406f2d0 0x7f790414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790409dc00 0x7f790409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79040ca720 0x7f790409af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7904167d70 con 0x7f790409dc00 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f79040705b0 con 0x7f790406f2d0 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f790b19a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7904167ef0 con 0x7f79040ca720 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790409dc00 0x7f790409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790409dc00 0x7f790409a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41760/0 (socket says 172.21.15.170:41760) 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 -- 172.21.15.170:0/2083383094 learned_addr learned my addr 172.21.15.170:0/2083383094 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7909737640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79040ca720 0x7f790409af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 -- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79040ca720 msgr2=0x7f790409af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79040ca720 0x7f790409af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7908f36640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f790406f2d0 0x7f790414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 -- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f790406f2d0 msgr2=0x7f790414a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f790406f2d0 0x7f790414a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 -- 172.21.15.170:0/2083383094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790414abd0 con 0x7f790409dc00 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7909737640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79040ca720 0x7f790409af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f7903fff640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790409dc00 0x7f790409a9c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f78f80058d0 tx=0x7f78f800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.645+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8017070 con 0x7f790409dc00 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f790b19a640 1 -- 172.21.15.170:0/2083383094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790409d080 con 0x7f790409dc00 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78f8009d20 con 0x7f790409dc00 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80106c0 con 0x7f790409dc00 2024-09-21T02:20:51.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f790b19a640 1 -- 172.21.15.170:0/2083383094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790415b900 con 0x7f790409dc00 2024-09-21T02:20:51.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f790b19a640 1 -- 172.21.15.170:0/2083383094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790406f6e0 con 0x7f790409dc00 2024-09-21T02:20:51.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f78f801a070 con 0x7f790409dc00 2024-09-21T02:20:51.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f79017fa640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78f0073860 0x7f78f0075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:51.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f78f8094f60 con 0x7f790409dc00 2024-09-21T02:20:51.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f7908f36640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78f0073860 0x7f78f0075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:51.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.649+0000 7f7908f36640 1 --2- 172.21.15.170:0/2083383094 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78f0073860 0x7f78f0075d70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f78f4001560 tx=0x7f78f4009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:51.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.653+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f78f805f5b0 con 0x7f790409dc00 2024-09-21T02:20:51.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.845+0000 7f790b19a640 1 -- 172.21.15.170:0/2083383094 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f790409d360 con 0x7f78f0073860 2024-09-21T02:20:51.847 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:51.845+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:51.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:51.845+0000 7f79017fa640 1 -- 172.21.15.170:0/2083383094 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f790409d360 con 0x7f78f0073860 2024-09-21T02:20:51.847 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:51.861 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:51.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T02:20:52.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4707efa640 1 -- 172.21.15.170:0/572768125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46ec003a40 msgr2=0x7f46ec003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4707efa640 1 --2- 172.21.15.170:0/572768125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46ec003a40 0x7f46ec003eb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009b50 tx=0x7f46fc01d290 comp rx=0 tx=0).stop 2024-09-21T02:20:52.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4706497640 1 -- 172.21.15.170:0/572768125 reap_dead start 2024-09-21T02:20:52.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4707efa640 1 -- 172.21.15.170:0/572768125 shutdown_connections 2024-09-21T02:20:52.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4707efa640 1 -- 172.21.15.170:0/572768125 >> 172.21.15.170:0/572768125 conn(0x7f470009d800 msgr2=0x7f47000d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:52.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4707efa640 1 -- 172.21.15.170:0/572768125 shutdown_connections 2024-09-21T02:20:52.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.017+0000 7f4707efa640 1 -- 172.21.15.170:0/572768125 wait complete. 2024-09-21T02:20:52.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 Processor -- start 2024-09-21T02:20:52.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 -- start start 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46ec003a40 0x7f4700160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47000a1ac0 0x7f47000c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47000c9950 0x7f47001fc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47000d4f90 con 0x7f47000c9950 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47000d4e10 con 0x7f47000a1ac0 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f470016af20 con 0x7f46ec003a40 2024-09-21T02:20:52.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47000c9950 0x7f47001fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47000c9950 0x7f47001fc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41780/0 (socket says 172.21.15.170:41780) 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 -- 172.21.15.170:0/3368249823 learned_addr learned my addr 172.21.15.170:0/3368249823 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4705c96640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46ec003a40 0x7f4700160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4705495640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47000a1ac0 0x7f47000c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 -- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46ec003a40 msgr2=0x7f4700160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46ec003a40 0x7f4700160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 -- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47000a1ac0 msgr2=0x7f47000c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47000a1ac0 0x7f47000c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 -- 172.21.15.170:0/3368249823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47000c8370 con 0x7f47000c9950 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4705c96640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46ec003a40 0x7f4700160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4706497640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47000c9950 0x7f47001fc450 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4700162570 tx=0x7f46f800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f800f040 con 0x7f47000c9950 2024-09-21T02:20:52.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46f8018070 con 0x7f47000c9950 2024-09-21T02:20:52.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 -- 172.21.15.170:0/3368249823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47000c8650 con 0x7f47000c9950 2024-09-21T02:20:52.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f800c900 con 0x7f47000c9950 2024-09-21T02:20:52.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.021+0000 7f4707efa640 1 -- 172.21.15.170:0/3368249823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47000c8c10 con 0x7f47000c9950 2024-09-21T02:20:52.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.025+0000 7f4707efa640 1 -- 172.21.15.170:0/3368249823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47000d17b0 con 0x7f47000c9950 2024-09-21T02:20:52.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.025+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f46f800caa0 con 0x7f47000c9950 2024-09-21T02:20:52.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.025+0000 7f46f67fc640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f46e4073830 0x7f46e4075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.025+0000 7f4705c96640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f46e4073830 0x7f46e4075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.025+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f46f8094990 con 0x7f47000c9950 2024-09-21T02:20:52.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.025+0000 7f4705c96640 1 --2- 172.21.15.170:0/3368249823 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f46e4073830 0x7f46e4075d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0ad000 tx=0x7f46fc0af040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:52.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.029+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f46f805f120 con 0x7f47000c9950 2024-09-21T02:20:52.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.225+0000 7f4707efa640 1 -- 172.21.15.170:0/3368249823 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f47000d14d0 con 0x7f46e4073830 2024-09-21T02:20:52.229 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.229+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:52.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.229+0000 7f46f67fc640 1 -- 172.21.15.170:0/3368249823 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f47000d14d0 con 0x7f46e4073830 2024-09-21T02:20:52.229 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:52.239 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:52.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 -- 172.21.15.170:0/3142691484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 msgr2=0x7f77440af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 --2- 172.21.15.170:0/3142691484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 0x7f77440af440 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f7740005fd0 tx=0x7f774001c790 comp rx=0 tx=0).stop 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 -- 172.21.15.170:0/3142691484 shutdown_connections 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 --2- 172.21.15.170:0/3142691484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440cd1f0 0x7f774406ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 --2- 172.21.15.170:0/3142691484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 0x7f77440af440 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 -- 172.21.15.170:0/3142691484 >> 172.21.15.170:0/3142691484 conn(0x7f77440b1c40 msgr2=0x7f77440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 -- 172.21.15.170:0/3142691484 shutdown_connections 2024-09-21T02:20:52.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.385+0000 7f774bf80640 1 -- 172.21.15.170:0/3142691484 wait complete. 2024-09-21T02:20:52.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 Processor -- start 2024-09-21T02:20:52.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- start start 2024-09-21T02:20:52.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440b4d80 0x7f77441530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 0x7f77441f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441536c0 0x7f77441fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7744169f70 con 0x7f77440b52d0 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7744169df0 con 0x7f77441536c0 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f774416a0f0 con 0x7f77440b4d80 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 0x7f77441f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 0x7f77441f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41790/0 (socket says 172.21.15.170:41790) 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 -- 172.21.15.170:0/2657987669 learned_addr learned my addr 172.21.15.170:0/2657987669 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 -- 172.21.15.170:0/2657987669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440b4d80 msgr2=0x7f77441530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 --2- 172.21.15.170:0/2657987669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440b4d80 0x7f77441530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 -- 172.21.15.170:0/2657987669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441536c0 msgr2=0x7f77441fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 --2- 172.21.15.170:0/2657987669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441536c0 0x7f77441fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 -- 172.21.15.170:0/2657987669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77440c80f0 con 0x7f77440b52d0 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774951b640 1 --2- 172.21.15.170:0/2657987669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77440b52d0 0x7f77441f8a10 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f773400c3e0 tx=0x7f773400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734011790 con 0x7f77440b52d0 2024-09-21T02:20:52.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7734017070 con 0x7f77440b52d0 2024-09-21T02:20:52.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- 172.21.15.170:0/2657987669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77440c83d0 con 0x7f77440b52d0 2024-09-21T02:20:52.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734011ce0 con 0x7f77440b52d0 2024-09-21T02:20:52.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- 172.21.15.170:0/2657987669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441fc380 con 0x7f77440b52d0 2024-09-21T02:20:52.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.389+0000 7f774bf80640 1 -- 172.21.15.170:0/2657987669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77440af440 con 0x7f77440b52d0 2024-09-21T02:20:52.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.393+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f773401c460 con 0x7f77440b52d0 2024-09-21T02:20:52.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.393+0000 7f773a7fc640 1 --2- 172.21.15.170:0/2657987669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7728073860 0x7f7728075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.393+0000 7f7749d1c640 1 --2- 172.21.15.170:0/2657987669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7728073860 0x7f7728075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.393+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f7734094c10 con 0x7f77440b52d0 2024-09-21T02:20:52.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.393+0000 7f7749d1c640 1 --2- 172.21.15.170:0/2657987669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7728073860 0x7f7728075d70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f77400ae000 tx=0x7f7740019530 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:52.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.397+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f773405f210 con 0x7f77440b52d0 2024-09-21T02:20:52.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.581+0000 7f774bf80640 1 -- 172.21.15.170:0/2657987669 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f77440c7e80 con 0x7f7728073860 2024-09-21T02:20:52.585 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.585+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:52.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.585+0000 7f773a7fc640 1 -- 172.21.15.170:0/2657987669 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f77440c7e80 con 0x7f7728073860 2024-09-21T02:20:52.586 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:52.598 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:52.599 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-21T02:20:52.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 -- 172.21.15.170:0/1993802671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00740a0 msgr2=0x7f99b00744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 --2- 172.21.15.170:0/1993802671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00740a0 0x7f99b00744b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009840 tx=0x7f99a401c7c0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 -- 172.21.15.170:0/1993802671 shutdown_connections 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 --2- 172.21.15.170:0/1993802671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00740a0 0x7f99b00744b0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 --2- 172.21.15.170:0/1993802671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00cf7d0 0x7f99b0073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 -- 172.21.15.170:0/1993802671 >> 172.21.15.170:0/1993802671 conn(0x7f99b00d0170 msgr2=0x7f99b00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 -- 172.21.15.170:0/1993802671 shutdown_connections 2024-09-21T02:20:52.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 -- 172.21.15.170:0/1993802671 wait complete. 2024-09-21T02:20:52.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 Processor -- start 2024-09-21T02:20:52.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.765+0000 7f99b65e8640 1 -- start start 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00740a0 0x7f99b015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00ce070 0x7f99b00c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00cf7d0 0x7f99b00c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0169040 con 0x7f99b00cf7d0 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0168ec0 con 0x7f99b00ce070 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f99b01691c0 con 0x7f99b00740a0 2024-09-21T02:20:52.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00cf7d0 0x7f99b00c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00cf7d0 0x7f99b00c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41800/0 (socket says 172.21.15.170:41800) 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 -- 172.21.15.170:0/221324367 learned_addr learned my addr 172.21.15.170:0/221324367 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99affff640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00740a0 0x7f99b015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99af7fe640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00ce070 0x7f99b00c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 -- 172.21.15.170:0/221324367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00740a0 msgr2=0x7f99b015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00740a0 0x7f99b015a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 -- 172.21.15.170:0/221324367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00ce070 msgr2=0x7f99b00c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00ce070 0x7f99b00c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 -- 172.21.15.170:0/221324367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b00afad0 con 0x7f99b00cf7d0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b4b85640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b00cf7d0 0x7f99b00c9930 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f99a800ca20 tx=0x7f99a800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8016020 con 0x7f99b00cf7d0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99a800a950 con 0x7f99b00cf7d0 2024-09-21T02:20:52.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 -- 172.21.15.170:0/221324367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b01f98a0 con 0x7f99b00cf7d0 2024-09-21T02:20:52.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a800ada0 con 0x7f99b00cf7d0 2024-09-21T02:20:52.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 -- 172.21.15.170:0/221324367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b015ca20 con 0x7f99b00cf7d0 2024-09-21T02:20:52.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.769+0000 7f99b65e8640 1 -- 172.21.15.170:0/221324367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b00744b0 con 0x7f99b00cf7d0 2024-09-21T02:20:52.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.773+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f99a8019020 con 0x7f99b00cf7d0 2024-09-21T02:20:52.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.773+0000 7f99acff9640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f999c073830 0x7f999c075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:52.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.773+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f99a80953c0 con 0x7f99b00cf7d0 2024-09-21T02:20:52.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.773+0000 7f99affff640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f999c073830 0x7f999c075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:52.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.773+0000 7f99affff640 1 --2- 172.21.15.170:0/221324367 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f999c073830 0x7f999c075d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b9a0 tx=0x7f99a0005c30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:52.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.777+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f99a8020020 con 0x7f99b00cf7d0 2024-09-21T02:20:52.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:52.985+0000 7f99b65e8640 1 -- 172.21.15.170:0/221324367 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f99b006dec0 con 0x7f999c073830 2024-09-21T02:20:52.992 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.993+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:52.992 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.993+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:52.992 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.993+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:52.992 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.993+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:52.992 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:52.993+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:20:53.002 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:53.001+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:53.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.001+0000 7f99acff9640 1 -- 172.21.15.170:0/221324367 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f99b006dec0 con 0x7f999c073830 2024-09-21T02:20:53.003 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:53.013 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:53.013 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 -- 172.21.15.170:0/2722089725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10004e70 msgr2=0x7fcb100052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 --2- 172.21.15.170:0/2722089725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10004e70 0x7fcb100052e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009e00 tx=0x7fcb1c01d150 comp rx=0 tx=0).stop 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2a52e640 1 -- 172.21.15.170:0/2722089725 reap_dead start 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 -- 172.21.15.170:0/2722089725 shutdown_connections 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 -- 172.21.15.170:0/2722089725 >> 172.21.15.170:0/2722089725 conn(0x7fcb240d0170 msgr2=0x7fcb240d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 -- 172.21.15.170:0/2722089725 shutdown_connections 2024-09-21T02:20:53.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 -- 172.21.15.170:0/2722089725 wait complete. 2024-09-21T02:20:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 Processor -- start 2024-09-21T02:20:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.157+0000 7fcb2bf91640 1 -- start start 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb2414cc10 0x7fcb24137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2409a9c0 0x7fcb241380e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb24138620 0x7fcb2413b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb240b5bb0 con 0x7fcb2414cc10 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb240ce780 con 0x7fcb2409a9c0 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcb240ce900 con 0x7fcb24138620 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb2414cc10 0x7fcb24137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb2414cc10 0x7fcb24137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41830/0 (socket says 172.21.15.170:41830) 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 -- 172.21.15.170:0/382251479 learned_addr learned my addr 172.21.15.170:0/382251479 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2a52e640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb24138620 0x7fcb2413b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2952c640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2409a9c0 0x7fcb241380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 -- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb24138620 msgr2=0x7fcb2413b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb24138620 0x7fcb2413b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 -- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2409a9c0 msgr2=0x7fcb241380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2409a9c0 0x7fcb241380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 -- 172.21.15.170:0/382251479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb241509e0 con 0x7fcb2414cc10 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2952c640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2409a9c0 0x7fcb241380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb29d2d640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb2414cc10 0x7fcb24137ba0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000da50 tx=0x7fcb2000df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20008d20 con 0x7fcb2414cc10 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb20004020 con 0x7fcb2414cc10 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20004450 con 0x7fcb2414cc10 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 -- 172.21.15.170:0/382251479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2409af80 con 0x7fcb2414cc10 2024-09-21T02:20:53.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 -- 172.21.15.170:0/382251479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2409d3d0 con 0x7fcb2414cc10 2024-09-21T02:20:53.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb2bf91640 1 -- 172.21.15.170:0/382251479 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb241573f0 con 0x7fcb2414cc10 2024-09-21T02:20:53.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.161+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fcb2000c040 con 0x7fcb2414cc10 2024-09-21T02:20:53.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.165+0000 7fcb1a7fc640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb08073830 0x7fcb08075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.165+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fcb200954a0 con 0x7fcb2414cc10 2024-09-21T02:20:53.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.165+0000 7fcb2952c640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb08073830 0x7fcb08075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.165+0000 7fcb2952c640 1 --2- 172.21.15.170:0/382251479 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcb08073830 0x7fcb08075d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcb24072ff0 tx=0x7fcb14001580 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:53.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.169+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcb241509e0 con 0x7fcb2414cc10 2024-09-21T02:20:53.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.361+0000 7fcb2bf91640 1 -- 172.21.15.170:0/382251479 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fcb24073b20 con 0x7fcb08073830 2024-09-21T02:20:53.363 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:53.361+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:53.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.361+0000 7fcb1a7fc640 1 -- 172.21.15.170:0/382251479 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fcb24073b20 con 0x7fcb08073830 2024-09-21T02:20:53.363 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:53.373 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:53.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 -- 172.21.15.170:0/1691202195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c0cb380 msgr2=0x7fbc7c0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 --2- 172.21.15.170:0/1691202195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c0cb380 0x7fbc7c0ce070 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0c71d0 tx=0x7fbc7001c830 comp rx=0 tx=0).stop 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 -- 172.21.15.170:0/1691202195 shutdown_connections 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 --2- 172.21.15.170:0/1691202195 >> v2:172.21.15.191:3300/0 conn(0x7fbc7c06f9a0 0x7fbc7c06fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 --2- 172.21.15.170:0/1691202195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc7c0cb790 0x7fbc7c0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 --2- 172.21.15.170:0/1691202195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c0cb380 0x7fbc7c0ce070 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 -- 172.21.15.170:0/1691202195 >> 172.21.15.170:0/1691202195 conn(0x7fbc7c0d0170 msgr2=0x7fbc7c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:53.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 -- 172.21.15.170:0/1691202195 shutdown_connections 2024-09-21T02:20:53.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.533+0000 7fbc84673640 1 -- 172.21.15.170:0/1691202195 wait complete. 2024-09-21T02:20:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 Processor -- start 2024-09-21T02:20:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 -- start start 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c06f9a0 0x7fbc7c149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc7c0cb380 0x7fbc7c14c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc7c0cb790 0x7fbc7c14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c073710 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c073590 con 0x7fbc7c0cb790 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc84673640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c073890 con 0x7fbc7c0cb380 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c06f9a0 0x7fbc7c149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c06f9a0 0x7fbc7c149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41846/0 (socket says 172.21.15.170:41846) 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 -- 172.21.15.170:0/1871420127 learned_addr learned my addr 172.21.15.170:0/1871420127 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc82c10640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc7c0cb790 0x7fbc7c14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc81c0e640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc7c0cb380 0x7fbc7c14c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 -- 172.21.15.170:0/1871420127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc7c0cb380 msgr2=0x7fbc7c14c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc7c0cb380 0x7fbc7c14c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 -- 172.21.15.170:0/1871420127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc7c0cb790 msgr2=0x7fbc7c14cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc7c0cb790 0x7fbc7c14cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 -- 172.21.15.170:0/1871420127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c0b1440 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.537+0000 7fbc8240f640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c06f9a0 0x7fbc7c149e90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbc700b0040 tx=0x7fbc700a6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70004340 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc700bc070 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc84673640 1 -- 172.21.15.170:0/1871420127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c14d3b0 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc700052b0 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc84673640 1 -- 172.21.15.170:0/1871420127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c0b19e0 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc84673640 1 -- 172.21.15.170:0/1871420127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7c06fdb0 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fbc70005450 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc6effd640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc60073830 0x7fbc60075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fbc7012fa90 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc81c0e640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc60073830 0x7fbc60075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.541+0000 7fbc81c0e640 1 --2- 172.21.15.170:0/1871420127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc60073830 0x7fbc60075d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc780058d0 tx=0x7fbc78005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.545+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbc700164b0 con 0x7fbc7c06f9a0 2024-09-21T02:20:53.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.729+0000 7fbc84673640 1 -- 172.21.15.170:0/1871420127 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fbc7c0b0a30 con 0x7fbc60073830 2024-09-21T02:20:53.730 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:53.729+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.729+0000 7fbc6effd640 1 -- 172.21.15.170:0/1871420127 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbc7c0b0a30 con 0x7fbc60073830 2024-09-21T02:20:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:53.745 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:53.745 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-21T02:20:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 -- 172.21.15.170:0/2551603084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 msgr2=0x7f4644003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 --2- 172.21.15.170:0/2551603084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 0x7f4644003e60 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f465400c310 tx=0x7f465401cf20 comp rx=0 tx=0).stop 2024-09-21T02:20:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 -- 172.21.15.170:0/2551603084 shutdown_connections 2024-09-21T02:20:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 --2- 172.21.15.170:0/2551603084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 0x7f4644003e60 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 -- 172.21.15.170:0/2551603084 >> 172.21.15.170:0/2551603084 conn(0x7f46580d0170 msgr2=0x7f46580d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 -- 172.21.15.170:0/2551603084 shutdown_connections 2024-09-21T02:20:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.897+0000 7f4660105640 1 -- 172.21.15.170:0/2551603084 wait complete. 2024-09-21T02:20:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 Processor -- start 2024-09-21T02:20:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- start start 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 0x7f4658155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46580b1e60 0x7f46580adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46580ae090 0x7f465815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f465806f750 con 0x7f4644003a10 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f46580b35e0 con 0x7f46580b1e60 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f465806f8d0 con 0x7f46580ae090 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 0x7f4658155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 0x7f4658155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41864/0 (socket says 172.21.15.170:41864) 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 -- 172.21.15.170:0/1907931393 learned_addr learned my addr 172.21.15.170:0/1907931393 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465d6a0640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46580b1e60 0x7f46580adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465e6a2640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46580ae090 0x7f465815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 -- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46580ae090 msgr2=0x7f465815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46580ae090 0x7f465815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 -- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46580b1e60 msgr2=0x7f46580adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46580b1e60 0x7f46580adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 -- 172.21.15.170:0/1907931393 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46580ae4a0 con 0x7f4644003a10 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465d6a0640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46580b1e60 0x7f46580adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f465dea1640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644003a10 0x7f4658155290 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4654098860 tx=0x7f4654000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654001500 con 0x7f4644003a10 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46540016a0 con 0x7f4644003a10 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540a5da0 con 0x7f4644003a10 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- 172.21.15.170:0/1907931393 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46581fa290 con 0x7f4644003a10 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- 172.21.15.170:0/1907931393 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581fa850 con 0x7f4644003a10 2024-09-21T02:20:53.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.901+0000 7f4660105640 1 -- 172.21.15.170:0/1907931393 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465806fdd0 con 0x7f4644003a10 2024-09-21T02:20:53.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.905+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7f46540b6030 con 0x7f4644003a10 2024-09-21T02:20:53.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.905+0000 7f464e7fc640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f463c073890 0x7f463c075da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:53.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.905+0000 7f465d6a0640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f463c073890 0x7f463c075da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:53.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.905+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f465412e960 con 0x7f4644003a10 2024-09-21T02:20:53.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.905+0000 7f465d6a0640 1 --2- 172.21.15.170:0/1907931393 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f463c073890 0x7f463c075da0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f465815acf0 tx=0x7f4648006b00 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:53.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:53.909+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f46580ae4a0 con 0x7f4644003a10 2024-09-21T02:20:54.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.101+0000 7f4660105640 1 -- 172.21.15.170:0/1907931393 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f465815b120 con 0x7f463c073890 2024-09-21T02:20:54.104 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:54.105+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:54.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.105+0000 7f464e7fc640 1 -- 172.21.15.170:0/1907931393 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f465815b120 con 0x7f463c073890 2024-09-21T02:20:54.105 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:54.119 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:54.119 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-21T02:20:54.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6f5535640 1 -- 172.21.15.170:0/3890196430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6d8004090 msgr2=0x7fc6d8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:54.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6f5535640 1 --2- 172.21.15.170:0/3890196430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6d8004090 0x7fc6d8004810 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00d10f0 tx=0x7fc6e401f6e0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6ef7fe640 1 -- 172.21.15.170:0/3890196430 reap_dead start 2024-09-21T02:20:54.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6f5535640 1 -- 172.21.15.170:0/3890196430 shutdown_connections 2024-09-21T02:20:54.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6f5535640 1 -- 172.21.15.170:0/3890196430 >> 172.21.15.170:0/3890196430 conn(0x7fc6f00d2a30 msgr2=0x7fc6f00cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:54.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6f5535640 1 -- 172.21.15.170:0/3890196430 shutdown_connections 2024-09-21T02:20:54.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.269+0000 7fc6f5535640 1 -- 172.21.15.170:0/3890196430 wait complete. 2024-09-21T02:20:54.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 Processor -- start 2024-09-21T02:20:54.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 -- start start 2024-09-21T02:20:54.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d8004090 0x7fc6f00c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6f014ac00 0x7fc6f014a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6f00c7c10 0x7fc6f00c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0160af0 con 0x7fc6f014ac00 2024-09-21T02:20:54.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0160c70 con 0x7fc6d8004090 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f0160df0 con 0x7fc6f00c7c10 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6f014ac00 0x7fc6f014a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6f014ac00 0x7fc6f014a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41870/0 (socket says 172.21.15.170:41870) 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 -- 172.21.15.170:0/1306907636 learned_addr learned my addr 172.21.15.170:0/1306907636 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ef7fe640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6f00c7c10 0x7fc6f00c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6eeffd640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d8004090 0x7fc6f00c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 -- 172.21.15.170:0/1306907636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6f00c7c10 msgr2=0x7fc6f00c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6f00c7c10 0x7fc6f00c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 -- 172.21.15.170:0/1306907636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d8004090 msgr2=0x7fc6f00c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6d8004090 0x7fc6f00c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 -- 172.21.15.170:0/1306907636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f00c8560 con 0x7fc6f014ac00 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6ee7fc640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6f014ac00 0x7fc6f014a400 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c10 tx=0x7fc6e4001db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:54.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40bf070 con 0x7fc6f014ac00 2024-09-21T02:20:54.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6e4006090 con 0x7fc6f014ac00 2024-09-21T02:20:54.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40b3070 con 0x7fc6f014ac00 2024-09-21T02:20:54.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 -- 172.21.15.170:0/1306907636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f00c9410 con 0x7fc6f014ac00 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.273+0000 7fc6f5535640 1 -- 172.21.15.170:0/1306907636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f014a940 con 0x7fc6f014ac00 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.277+0000 7fc6f5535640 1 -- 172.21.15.170:0/1306907636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f0071880 con 0x7fc6f014ac00 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.277+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fc6e4003860 con 0x7fc6f014ac00 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.277+0000 7fc6d77fe640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc6dc073830 0x7fc6dc075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.277+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fc6e4132630 con 0x7fc6f014ac00 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.277+0000 7fc6eeffd640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc6dc073830 0x7fc6dc075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.277+0000 7fc6eeffd640 1 --2- 172.21.15.170:0/1306907636 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc6dc073830 0x7fc6dc075d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0161030 tx=0x7fc6e0005f30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:54.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.281+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc6f00c8560 con 0x7fc6f014ac00 2024-09-21T02:20:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.469+0000 7fc6f5535640 1 -- 172.21.15.170:0/1306907636 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fc6f00c6ef0 con 0x7fc6dc073830 2024-09-21T02:20:54.474 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:54.473+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:54.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.473+0000 7fc6d77fe640 1 -- 172.21.15.170:0/1306907636 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc6f00c6ef0 con 0x7fc6dc073830 2024-09-21T02:20:54.474 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:54.482 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:54.482 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 -- 172.21.15.170:0/2725749785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ccae0 msgr2=0x7fddd80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 --2- 172.21.15.170:0/2725749785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ccae0 0x7fddd80ccef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fddd000b6c0 tx=0x7fddd001d410 comp rx=0 tx=0).stop 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 -- 172.21.15.170:0/2725749785 shutdown_connections 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 --2- 172.21.15.170:0/2725749785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ccae0 0x7fddd80ccef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 --2- 172.21.15.170:0/2725749785 >> v2:172.21.15.191:3300/0 conn(0x7fddd80ca720 0x7fddd80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 -- 172.21.15.170:0/2725749785 >> 172.21.15.170:0/2725749785 conn(0x7fddd80b1c40 msgr2=0x7fddd80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 -- 172.21.15.170:0/2725749785 shutdown_connections 2024-09-21T02:20:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.629+0000 7fdde0735640 1 -- 172.21.15.170:0/2725749785 wait complete. 2024-09-21T02:20:54.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 Processor -- start 2024-09-21T02:20:54.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 -- start start 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ca720 0x7fddd809cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddd80ccae0 0x7fddd809d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd80cdfd0 0x7fddd80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80ce560 con 0x7fddd80cdfd0 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80ce3e0 con 0x7fddd80ccae0 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fddd815bbd0 con 0x7fddd80ca720 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd80cdfd0 0x7fddd80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd80cdfd0 0x7fddd80ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41882/0 (socket says 172.21.15.170:41882) 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 -- 172.21.15.170:0/14510343 learned_addr learned my addr 172.21.15.170:0/14510343 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fddde4d1640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ca720 0x7fddd809cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fddddcd0640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddd80ccae0 0x7fddd809d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 -- 172.21.15.170:0/14510343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ca720 msgr2=0x7fddd809cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd80ca720 0x7fddd809cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 -- 172.21.15.170:0/14510343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddd80ccae0 msgr2=0x7fddd809d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddd80ccae0 0x7fddd809d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 -- 172.21.15.170:0/14510343 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd8149f90 con 0x7fddd80cdfd0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdddecd2640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd80cdfd0 0x7fddd80ce6e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fddd001bac0 tx=0x7fddd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00bb070 con 0x7fddd80cdfd0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddd00b0040 con 0x7fddd80cdfd0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 -- 172.21.15.170:0/14510343 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd815d2c0 con 0x7fddd80cdfd0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00af070 con 0x7fddd80cdfd0 2024-09-21T02:20:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.633+0000 7fdde0735640 1 -- 172.21.15.170:0/14510343 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd815d880 con 0x7fddd80cdfd0 2024-09-21T02:20:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.637+0000 7fdde0735640 1 -- 172.21.15.170:0/14510343 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd80cab30 con 0x7fddd80cdfd0 2024-09-21T02:20:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.637+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 22) ==== 97795+0+0 (secure 0 0 0) 0x7fddd00be070 con 0x7fddd80cdfd0 2024-09-21T02:20:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.637+0000 7fddceffd640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fddbc073830 0x7fddbc075d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:54.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.637+0000 7fddde4d1640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fddbc073830 0x7fddbc075d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:54.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.637+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fddd012ef80 con 0x7fddd80cdfd0 2024-09-21T02:20:54.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.637+0000 7fddde4d1640 1 --2- 172.21.15.170:0/14510343 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fddbc073830 0x7fddbc075d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fddc8007fd0 tx=0x7fddc8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:54.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.641+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fddd0017030 con 0x7fddd80cdfd0 2024-09-21T02:20:54.803 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:20:54.806 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:20:54.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.825+0000 7fdde0735640 1 -- 172.21.15.170:0/14510343 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7fddd8154870 con 0x7fddbc073830 2024-09-21T02:20:54.828 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:54.829+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:54.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:54.829+0000 7fddceffd640 1 -- 172.21.15.170:0/14510343 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fddd8154870 con 0x7fddbc073830 2024-09-21T02:20:54.828 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:54.837 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:54.837 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.033+0000 7f182aa1c640 1 -- 172.21.15.170:0/2778026698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 msgr2=0x7f18240b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.033+0000 7f182aa1c640 1 --2- 172.21.15.170:0/2778026698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 0x7f18240b2940 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f18240ccd30 tx=0x7f181c01ccf0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- 172.21.15.170:0/2778026698 shutdown_connections 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 --2- 172.21.15.170:0/2778026698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 0x7f18240b2940 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 --2- 172.21.15.170:0/2778026698 >> v2:172.21.15.191:3300/0 conn(0x7f18240cafd0 0x7f18240b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- 172.21.15.170:0/2778026698 >> 172.21.15.170:0/2778026698 conn(0x7f18240cd5d0 msgr2=0x7f18240cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- 172.21.15.170:0/2778026698 shutdown_connections 2024-09-21T02:20:55.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- 172.21.15.170:0/2778026698 wait complete. 2024-09-21T02:20:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 Processor -- start 2024-09-21T02:20:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- start start 2024-09-21T02:20:55.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18240cafd0 0x7f18240b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 0x7f1824151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18240ce550 0x7f18241523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f182416c080 con 0x7f18240cb3e0 2024-09-21T02:20:55.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f182416bf00 con 0x7f18240ce550 2024-09-21T02:20:55.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f182aa1c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f182416c200 con 0x7f18240cafd0 2024-09-21T02:20:55.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f18237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 0x7f1824151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f18237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 0x7f1824151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41894/0 (socket says 172.21.15.170:41894) 2024-09-21T02:20:55.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f18237fe640 1 -- 172.21.15.170:0/3763269279 learned_addr learned my addr 172.21.15.170:0/3763269279 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:55.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f1828fb9640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18240ce550 0x7f18241523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.037+0000 7f1823fff640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18240cafd0 0x7f18240b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f18237fe640 1 -- 172.21.15.170:0/3763269279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18240cafd0 msgr2=0x7f18240b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f18237fe640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18240cafd0 0x7f18240b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f18237fe640 1 -- 172.21.15.170:0/3763269279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18240ce550 msgr2=0x7f18241523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f18237fe640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18240ce550 0x7f18241523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f18237fe640 1 -- 172.21.15.170:0/3763269279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18240b1160 con 0x7f18240cb3e0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f18237fe640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18240cb3e0 0x7f1824151e70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f181800b3f0 tx=0x7f181800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818015020 con 0x7f18240cb3e0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1818011920 con 0x7f18240cb3e0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818011d70 con 0x7f18240cb3e0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f182aa1c640 1 -- 172.21.15.170:0/3763269279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18240b1440 con 0x7f18240cb3e0 2024-09-21T02:20:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f182aa1c640 1 -- 172.21.15.170:0/3763269279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182415d030 con 0x7f18240cb3e0 2024-09-21T02:20:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f182aa1c640 1 -- 172.21.15.170:0/3763269279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182415c310 con 0x7f18240cb3e0 2024-09-21T02:20:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f1818018020 con 0x7f18240cb3e0 2024-09-21T02:20:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1820ff9640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f18100738c0 0x7f1810075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f1818095290 con 0x7f18240cb3e0 2024-09-21T02:20:55.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.041+0000 7f1823fff640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f18100738c0 0x7f1810075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.045+0000 7f1823fff640 1 --2- 172.21.15.170:0/3763269279 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f18100738c0 0x7f1810075dd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f18140098e0 tx=0x7f1814009060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:55.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.045+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f181801f020 con 0x7f18240cb3e0 2024-09-21T02:20:55.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.229+0000 7f182aa1c640 1 -- 172.21.15.170:0/3763269279 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f18240b6900 con 0x7f18100738c0 2024-09-21T02:20:55.232 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:55.233+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:55.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.233+0000 7f1820ff9640 1 -- 172.21.15.170:0/3763269279 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f18240b6900 con 0x7f18100738c0 2024-09-21T02:20:55.232 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:55.242 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:55.242 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-21T02:20:55.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9423873640 1 -- 172.21.15.170:0/2545267680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9408003a10 msgr2=0x7f9408003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9423873640 1 --2- 172.21.15.170:0/2545267680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9408003a10 0x7f9408003e80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f941c09dbf0 tx=0x7f941801cf80 comp rx=0 tx=0).stop 2024-09-21T02:20:55.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9421e10640 1 -- 172.21.15.170:0/2545267680 reap_dead start 2024-09-21T02:20:55.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9423873640 1 -- 172.21.15.170:0/2545267680 shutdown_connections 2024-09-21T02:20:55.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9423873640 1 -- 172.21.15.170:0/2545267680 >> 172.21.15.170:0/2545267680 conn(0x7f941c0b1c40 msgr2=0x7f941c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:55.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9423873640 1 -- 172.21.15.170:0/2545267680 shutdown_connections 2024-09-21T02:20:55.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.405+0000 7f9423873640 1 -- 172.21.15.170:0/2545267680 wait complete. 2024-09-21T02:20:55.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 Processor -- start 2024-09-21T02:20:55.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- start start 2024-09-21T02:20:55.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9408003a10 0x7f941c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c154910 0x7f941c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f941c09a9c0 0x7f941c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f941c167790 con 0x7f941c154910 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f941c167610 con 0x7f941c09a9c0 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f941c167910 con 0x7f9408003a10 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c154910 0x7f941c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c154910 0x7f941c15a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53730/0 (socket says 172.21.15.170:53730) 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 -- 172.21.15.170:0/1853174257 learned_addr learned my addr 172.21.15.170:0/1853174257 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f942160f640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9408003a10 0x7f941c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9421e10640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f941c09a9c0 0x7f941c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 -- 172.21.15.170:0/1853174257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9408003a10 msgr2=0x7f941c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9408003a10 0x7f941c0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 -- 172.21.15.170:0/1853174257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f941c09a9c0 msgr2=0x7f941c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f941c09a9c0 0x7f941c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 -- 172.21.15.170:0/1853174257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c15c130 con 0x7f941c154910 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9420e0e640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c154910 0x7f941c15a450 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9a0 tx=0x7f940c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00ccc0 con 0x7f941c154910 2024-09-21T02:20:55.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f940c00ce60 con 0x7f941c154910 2024-09-21T02:20:55.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- 172.21.15.170:0/1853174257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c159690 con 0x7f941c154910 2024-09-21T02:20:55.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c005280 con 0x7f941c154910 2024-09-21T02:20:55.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- 172.21.15.170:0/1853174257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c159c50 con 0x7f941c154910 2024-09-21T02:20:55.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.409+0000 7f9423873640 1 -- 172.21.15.170:0/1853174257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941c0af440 con 0x7f941c154910 2024-09-21T02:20:55.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.413+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f940c005420 con 0x7f941c154910 2024-09-21T02:20:55.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.413+0000 7f9411ffb640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f94000738c0 0x7f9400075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.413+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f940c095b50 con 0x7f941c154910 2024-09-21T02:20:55.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.413+0000 7f942160f640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f94000738c0 0x7f9400075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.413+0000 7f942160f640 1 --2- 172.21.15.170:0/1853174257 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f94000738c0 0x7f9400075dd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94180ac000 tx=0x7f941c06ff50 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:55.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.417+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f940c060340 con 0x7f941c154910 2024-09-21T02:20:55.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.597+0000 7f9423873640 1 -- 172.21.15.170:0/1853174257 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f941c0b2330 con 0x7f94000738c0 2024-09-21T02:20:55.601 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:55.601+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:55.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.601+0000 7f9411ffb640 1 -- 172.21.15.170:0/1853174257 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f941c0b2330 con 0x7f94000738c0 2024-09-21T02:20:55.602 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:55.611 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:55.611 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-21T02:20:55.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 -- 172.21.15.170:0/1105821150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc0b4290 msgr2=0x7f9dbc0b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 --2- 172.21.15.170:0/1105821150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc0b4290 0x7f9dbc0b5120 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009c80 tx=0x7f9db401cb80 comp rx=0 tx=0).stop 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 -- 172.21.15.170:0/1105821150 shutdown_connections 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 --2- 172.21.15.170:0/1105821150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc0b4290 0x7f9dbc0b5120 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 --2- 172.21.15.170:0/1105821150 >> v2:172.21.15.191:3300/0 conn(0x7f9dbc15c770 0x7f9dbc0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 -- 172.21.15.170:0/1105821150 >> 172.21.15.170:0/1105821150 conn(0x7f9dbc0ced50 msgr2=0x7f9dbc0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 -- 172.21.15.170:0/1105821150 shutdown_connections 2024-09-21T02:20:55.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.769+0000 7f9dc1718640 1 -- 172.21.15.170:0/1105821150 wait complete. 2024-09-21T02:20:55.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 Processor -- start 2024-09-21T02:20:55.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- start start 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9dbc0b4290 0x7f9dbc14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dbc0b7f20 0x7f9dbc0b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc15c770 0x7f9dbc0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc169eb0 con 0x7f9dbc15c770 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc169d30 con 0x7f9dbc0b7f20 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc16a030 con 0x7f9dbc0b4290 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc15c770 0x7f9dbc0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc15c770 0x7f9dbc0b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53748/0 (socket says 172.21.15.170:53748) 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 -- 172.21.15.170:0/3763111 learned_addr learned my addr 172.21.15.170:0/3763111 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dba7fc640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dbc0b7f20 0x7f9dbc0b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbaffd640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9dbc0b4290 0x7f9dbc14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 -- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9dbc0b4290 msgr2=0x7f9dbc14c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9dbc0b4290 0x7f9dbc14c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 -- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dbc0b7f20 msgr2=0x7f9dbc0b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dbc0b7f20 0x7f9dbc0b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 -- 172.21.15.170:0/3763111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc152590 con 0x7f9dbc15c770 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dba7fc640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dbc0b7f20 0x7f9dbc0b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbaffd640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9dbc0b4290 0x7f9dbc14c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dbb7fe640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9dbc15c770 0x7f9dbc0b6680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009bd0 tx=0x7f9db40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40bb070 con 0x7f9dbc15c770 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9db4004e00 con 0x7f9dbc15c770 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- 172.21.15.170:0/3763111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc14dcc0 con 0x7f9dbc15c770 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40b2040 con 0x7f9dbc15c770 2024-09-21T02:20:55.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- 172.21.15.170:0/3763111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc14d9e0 con 0x7f9dbc15c770 2024-09-21T02:20:55.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.773+0000 7f9dc1718640 1 -- 172.21.15.170:0/3763111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc152760 con 0x7f9dbc15c770 2024-09-21T02:20:55.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.777+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f9db40be070 con 0x7f9dbc15c770 2024-09-21T02:20:55.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.777+0000 7f9da37fe640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9da80738c0 0x7f9da8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:55.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.777+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f9db412eb40 con 0x7f9dbc15c770 2024-09-21T02:20:55.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.777+0000 7f9dbaffd640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9da80738c0 0x7f9da8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:55.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.777+0000 7f9dbaffd640 1 --2- 172.21.15.170:0/3763111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9da80738c0 0x7f9da8075dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00a830 tx=0x7f9dac008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:55.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.781+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9dbc0b46a0 con 0x7f9dbc15c770 2024-09-21T02:20:55.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.977+0000 7f9dc1718640 1 -- 172.21.15.170:0/3763111 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f9dbc14d770 con 0x7f9da80738c0 2024-09-21T02:20:55.978 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:55.977+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:55.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:55.977+0000 7f9da37fe640 1 -- 172.21.15.170:0/3763111 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9dbc14d770 con 0x7f9da80738c0 2024-09-21T02:20:55.979 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:55.992 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:55.993 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc85b8eb640 1 -- 172.21.15.170:0/2502725720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc840003a10 msgr2=0x7fc840003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc85b8eb640 1 --2- 172.21.15.170:0/2502725720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc840003a10 0x7fc840003e80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009e80 tx=0x7fc84c01d140 comp rx=0 tx=0).stop 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc859e88640 1 -- 172.21.15.170:0/2502725720 reap_dead start 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc85b8eb640 1 -- 172.21.15.170:0/2502725720 shutdown_connections 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc85b8eb640 1 -- 172.21.15.170:0/2502725720 >> 172.21.15.170:0/2502725720 conn(0x7fc8540b1c40 msgr2=0x7fc8540b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc85b8eb640 1 -- 172.21.15.170:0/2502725720 shutdown_connections 2024-09-21T02:20:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.141+0000 7fc85b8eb640 1 -- 172.21.15.170:0/2502725720 wait complete. 2024-09-21T02:20:56.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 Processor -- start 2024-09-21T02:20:56.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 -- start start 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc840003a10 0x7fc85415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc854151090 0x7fc85415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc85415dba0 0x7fc8540b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc854166570 con 0x7fc85415dba0 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541663f0 con 0x7fc854151090 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc8541666f0 con 0x7fc840003a10 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc85415dba0 0x7fc8540b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc85415dba0 0x7fc8540b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53774/0 (socket says 172.21.15.170:53774) 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 -- 172.21.15.170:0/1010054537 learned_addr learned my addr 172.21.15.170:0/1010054537 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859687640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc840003a10 0x7fc85415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 -- 172.21.15.170:0/1010054537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc840003a10 msgr2=0x7fc85415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc858e86640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc854151090 0x7fc85415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc840003a10 0x7fc85415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 -- 172.21.15.170:0/1010054537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc854151090 msgr2=0x7fc85415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc854151090 0x7fc85415d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 -- 172.21.15.170:0/1010054537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8541f5660 con 0x7fc85415dba0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc859e88640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc85415dba0 0x7fc8540b3f90 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fc8540ce3a0 tx=0x7fc85000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850016030 con 0x7fc85415dba0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc850005110 con 0x7fc85415dba0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850005520 con 0x7fc85415dba0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 -- 172.21.15.170:0/1010054537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8541f5940 con 0x7fc85415dba0 2024-09-21T02:20:56.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.145+0000 7fc85b8eb640 1 -- 172.21.15.170:0/1010054537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541f5f00 con 0x7fc85415dba0 2024-09-21T02:20:56.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.149+0000 7fc85b8eb640 1 -- 172.21.15.170:0/1010054537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85415d240 con 0x7fc85415dba0 2024-09-21T02:20:56.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.149+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fc850019070 con 0x7fc85415dba0 2024-09-21T02:20:56.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.149+0000 7fc849ffb640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8380738f0 0x7fc838075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.149+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fc850094b60 con 0x7fc85415dba0 2024-09-21T02:20:56.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.149+0000 7fc859687640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8380738f0 0x7fc838075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.149+0000 7fc859687640 1 --2- 172.21.15.170:0/1010054537 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc8380738f0 0x7fc838075e00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0b0000 tx=0x7fc84c019ee0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:56.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.153+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc85005f2e0 con 0x7fc85415dba0 2024-09-21T02:20:56.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.353+0000 7fc85b8eb640 1 -- 172.21.15.170:0/1010054537 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fc85415c3f0 con 0x7fc8380738f0 2024-09-21T02:20:56.355 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:20:56.353+0000 7efb9f2c4640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T02:20:56.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.353+0000 7fc849ffb640 1 -- 172.21.15.170:0/1010054537 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc85415c3f0 con 0x7fc8380738f0 2024-09-21T02:20:56.355 INFO:teuthology.orchestra.run.smithi170.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T02:20:56.368 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T02:20:56.368 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:20:56.369 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:20:56.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:20:56.519 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:20:56.519 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:56.519 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:56.519 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:56.525 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:56.526 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:56.572 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 -- 172.21.15.170:0/431200224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 msgr2=0x7f1f8806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/431200224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8806fa30 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f1f880c7150 tx=0x7f1f8401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 -- 172.21.15.170:0/431200224 shutdown_connections 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/431200224 >> v2:172.21.15.191:3300/0 conn(0x7f1f880cd210 0x7f1f880b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/431200224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8806fa30 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 -- 172.21.15.170:0/431200224 >> 172.21.15.170:0/431200224 conn(0x7f1f880b1c40 msgr2=0x7f1f880b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:56.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 -- 172.21.15.170:0/431200224 shutdown_connections 2024-09-21T02:20:56.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.781+0000 7f1f8fe18640 1 -- 172.21.15.170:0/431200224 wait complete. 2024-09-21T02:20:56.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 Processor -- start 2024-09-21T02:20:56.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 -- start start 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f880b2f40 0x7f1f880c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f8809af00 0x7f1f881f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1f88169fd0 con 0x7f1f880b5410 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f88169e50 con 0x7f1f880b2f40 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8fe18640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8816a150 con 0x7f1f8809af00 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8809a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53794/0 (socket says 172.21.15.170:53794) 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 -- 172.21.15.170:0/1449293462 learned_addr learned my addr 172.21.15.170:0/1449293462 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:56.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8dbb4640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f880b2f40 0x7f1f880c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8e3b5640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f8809af00 0x7f1f881f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 -- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f8809af00 msgr2=0x7f1f881f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f8809af00 0x7f1f881f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 -- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f880b2f40 msgr2=0x7f1f880c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f880b2f40 0x7f1f880c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.785+0000 7f1f8d3b3640 1 -- 172.21.15.170:0/1449293462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f88155290 con 0x7f1f880b5410 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8dbb4640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f880b2f40 0x7f1f880c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8d3b3640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8809a9c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800b3e0 tx=0x7f1f7800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78015020 con 0x7f1f880b5410 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f78011940 con 0x7f1f880b5410 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f88155570 con 0x7f1f880b5410 2024-09-21T02:20:56.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78011d90 con 0x7f1f880b5410 2024-09-21T02:20:56.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8809c7f0 con 0x7f1f880b5410 2024-09-21T02:20:56.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8806fa30 con 0x7f1f880b5410 2024-09-21T02:20:56.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f1f78018020 con 0x7f1f880b5410 2024-09-21T02:20:56.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f7e7fc640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f6c0738c0 0x7f1f6c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:56.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f1f78095350 con 0x7f1f880b5410 2024-09-21T02:20:56.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8dbb4640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f6c0738c0 0x7f1f6c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:56.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.789+0000 7f1f8dbb4640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f6c0738c0 0x7f1f6c075dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f840af000 tx=0x7f1f8401b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:56.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.793+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1f7801f020 con 0x7f1f880b5410 2024-09-21T02:20:56.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.977+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f881564d0 con 0x7f1f880b5410 2024-09-21T02:20:57.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.977+0000 7f1f7e7fc640 1 -- 172.21.15.170:0/1449293462 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7f1f7805f9b0 con 0x7f1f880b5410 2024-09-21T02:20:57.038 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:20:57.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f6c0738c0 msgr2=0x7f1f6c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f6c0738c0 0x7f1f6c075dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f840af000 tx=0x7f1f8401b100 comp rx=0 tx=0).stop 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 msgr2=0x7f1f8809a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8809a9c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800b3e0 tx=0x7f1f7800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 shutdown_connections 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1f6c0738c0 0x7f1f6c075dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f8809af00 0x7f1f881f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f880b5410 0x7f1f8809a9c0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 --2- 172.21.15.170:0/1449293462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f880b2f40 0x7f1f880c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 >> 172.21.15.170:0/1449293462 conn(0x7f1f880b1c40 msgr2=0x7f1f880c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 shutdown_connections 2024-09-21T02:20:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:56.981+0000 7f1f8fe18640 1 -- 172.21.15.170:0/1449293462 wait complete. 2024-09-21T02:20:57.040 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:20:57.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:57.041 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:20:57.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[48841]: fuse finished with error 0 and tester_r 0 2024-09-21T02:20:57.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:20:57.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:57.055 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:20:57.217 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:57.217 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:57.217 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:57.224 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:57.224 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:57.271 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:20:57.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f33385af640 1 -- 172.21.15.170:0/3905458693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f331c005190 msgr2=0x7f331c005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f33385af640 1 --2- 172.21.15.170:0/3905458693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f331c005190 0x7f331c005920 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3320009b30 tx=0x7f332001d250 comp rx=0 tx=0).stop 2024-09-21T02:20:57.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f3336b4c640 1 -- 172.21.15.170:0/3905458693 reap_dead start 2024-09-21T02:20:57.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f33385af640 1 -- 172.21.15.170:0/3905458693 shutdown_connections 2024-09-21T02:20:57.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f33385af640 1 -- 172.21.15.170:0/3905458693 >> 172.21.15.170:0/3905458693 conn(0x7f33300b1c40 msgr2=0x7f33300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:57.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f33385af640 1 -- 172.21.15.170:0/3905458693 shutdown_connections 2024-09-21T02:20:57.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.429+0000 7f33385af640 1 -- 172.21.15.170:0/3905458693 wait complete. 2024-09-21T02:20:57.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 Processor -- start 2024-09-21T02:20:57.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- start start 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f331c005190 0x7f33300c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 0x7f333014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33300c9a30 0x7f333014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33300ce8d0 con 0x7f33300c5320 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3330074f00 con 0x7f331c005190 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f33300cea50 con 0x7f33300c9a30 2024-09-21T02:20:57.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 0x7f333014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 0x7f333014d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53822/0 (socket says 172.21.15.170:53822) 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 -- 172.21.15.170:0/4196125517 learned_addr learned my addr 172.21.15.170:0/4196125517 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 -- 172.21.15.170:0/4196125517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33300c9a30 msgr2=0x7f333014d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33300c9a30 0x7f333014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 -- 172.21.15.170:0/4196125517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f331c005190 msgr2=0x7f33300c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f331c005190 0x7f33300c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 -- 172.21.15.170:0/4196125517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300b2540 con 0x7f33300c5320 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3335b4a640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 0x7f333014d410 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3320009a80 tx=0x7f3320005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200bb020 con 0x7f33300c5320 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3320004020 con 0x7f33300c5320 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33300c6980 con 0x7f33300c5320 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200b1040 con 0x7f33300c5320 2024-09-21T02:20:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33300c4b20 con 0x7f33300c5320 2024-09-21T02:20:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.433+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33300c66f0 con 0x7f33300c5320 2024-09-21T02:20:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.437+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f33200be070 con 0x7f33300c5320 2024-09-21T02:20:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.437+0000 7f3326ffd640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33140738c0 0x7f3314075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.437+0000 7f333634b640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33140738c0 0x7f3314075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.437+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f332012f630 con 0x7f33300c5320 2024-09-21T02:20:57.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.437+0000 7f333634b640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33140738c0 0x7f3314075dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f332c001ec0 tx=0x7f332c001500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:57.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.441+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3320016f60 con 0x7f33300c5320 2024-09-21T02:20:57.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3330070030 con 0x7f33300c5320 2024-09-21T02:20:57.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f3326ffd640 1 -- 172.21.15.170:0/4196125517 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7f3320004470 con 0x7f33300c5320 2024-09-21T02:20:57.621 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:20:57.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33140738c0 msgr2=0x7f3314075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33140738c0 0x7f3314075dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f332c001ec0 tx=0x7f332c001500 comp rx=0 tx=0).stop 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 msgr2=0x7f333014d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 0x7f333014d410 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3320009a80 tx=0x7f3320005500 comp rx=0 tx=0).stop 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 shutdown_connections 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f33140738c0 0x7f3314075dd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33300c9a30 0x7f333014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33300c5320 0x7f333014d410 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 --2- 172.21.15.170:0/4196125517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f331c005190 0x7f33300c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.621+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 >> 172.21.15.170:0/4196125517 conn(0x7f33300b1c40 msgr2=0x7f33300d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.625+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 shutdown_connections 2024-09-21T02:20:57.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:57.625+0000 7f33385af640 1 -- 172.21.15.170:0/4196125517 wait complete. 2024-09-21T02:20:57.642 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:20:57.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:57.643 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:20:57.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35541]: fuse finished with error 0 and tester_r 0 2024-09-21T02:20:57.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:20:57.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:20:57.658 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:20:57.826 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:57.826 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:20:57.826 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:57.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:20:57.832 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:20:57.832 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:20:57.832 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:20:57.879 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:20:57.879 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:58.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 -- 172.21.15.170:0/2246317533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f877415f990 msgr2=0x7f8774070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 --2- 172.21.15.170:0/2246317533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f877415f990 0x7f8774070030 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f87740cdc00 tx=0x7f877001c7d0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 -- 172.21.15.170:0/2246317533 shutdown_connections 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 --2- 172.21.15.170:0/2246317533 >> v2:172.21.15.191:3300/0 conn(0x7f87740cadf0 0x7f87740cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 --2- 172.21.15.170:0/2246317533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f877415f990 0x7f8774070030 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 -- 172.21.15.170:0/2246317533 >> 172.21.15.170:0/2246317533 conn(0x7f87740b1c40 msgr2=0x7f87740b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 -- 172.21.15.170:0/2246317533 shutdown_connections 2024-09-21T02:20:58.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.041+0000 7f877bdb8640 1 -- 172.21.15.170:0/2246317533 wait complete. 2024-09-21T02:20:58.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 Processor -- start 2024-09-21T02:20:58.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 -- start start 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8774070570 0x7f87740b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 0x7f8774156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877415f990 0x7f87741572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8774165fe0 con 0x7f87740cadf0 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87740cede0 con 0x7f877415f990 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877bdb8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8774166160 con 0x7f8774070570 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 0x7f8774156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 0x7f8774156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53842/0 (socket says 172.21.15.170:53842) 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 -- 172.21.15.170:0/1543533867 learned_addr learned my addr 172.21.15.170:0/1543533867 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779b54640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8774070570 0x7f87740b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f877a355640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877415f990 0x7f87741572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 -- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8774070570 msgr2=0x7f87740b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8774070570 0x7f87740b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 -- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877415f990 msgr2=0x7f87741572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877415f990 0x7f87741572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 -- 172.21.15.170:0/1543533867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877415f370 con 0x7f87740cadf0 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779b54640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8774070570 0x7f87740b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f8779353640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 0x7f8774156d80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f876400c430 tx=0x7f876400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764011790 con 0x7f87740cadf0 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.045+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8764017070 con 0x7f87740cadf0 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8774165bd0 con 0x7f87740cadf0 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764011cc0 con 0x7f87740cadf0 2024-09-21T02:20:58.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877415eb70 con 0x7f87740cadf0 2024-09-21T02:20:58.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87740cb200 con 0x7f87740cadf0 2024-09-21T02:20:58.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f876401c460 con 0x7f87740cadf0 2024-09-21T02:20:58.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f876a7fc640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f87580738f0 0x7f8758075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f8764094c10 con 0x7f87740cadf0 2024-09-21T02:20:58.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f8779b54640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f87580738f0 0x7f8758075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.049+0000 7f8779b54640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f87580738f0 0x7f8758075e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87700af000 tx=0x7f877001b010 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:58.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.053+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f87740cb740 con 0x7f87740cadf0 2024-09-21T02:20:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.245+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87740ce2a0 con 0x7f87740cadf0 2024-09-21T02:20:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f876a7fc640 1 -- 172.21.15.170:0/1543533867 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5236 (secure 0 0 0) 0x7f876405f270 con 0x7f87740cadf0 2024-09-21T02:20:58.248 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:58.248 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":91,"btime":"2024-09-21T02:20:46:972016+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:20:33.897736+0000","modified":"2024-09-21T02:20:36.929189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/169289834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":169289834},{"type":"v1","addr":"172.21.15.191:6835","nonce":169289834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8061,"qdb_cluster":[8061]},"id":10}]} 2024-09-21T02:20:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 91 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f87580738f0 msgr2=0x7f8758075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f87580738f0 0x7f8758075e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87700af000 tx=0x7f877001b010 comp rx=0 tx=0).stop 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 msgr2=0x7f8774156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 0x7f8774156d80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f876400c430 tx=0x7f876400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 shutdown_connections 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f87580738f0 0x7f8758075e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877415f990 0x7f87741572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87740cadf0 0x7f8774156d80 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 --2- 172.21.15.170:0/1543533867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8774070570 0x7f87740b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 >> 172.21.15.170:0/1543533867 conn(0x7f87740b1c40 msgr2=0x7f87740cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:58.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 shutdown_connections 2024-09-21T02:20:58.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.249+0000 7f877bdb8640 1 -- 172.21.15.170:0/1543533867 wait complete. 2024-09-21T02:20:58.271 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 -- 172.21.15.170:0/879987230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cf7e0 msgr2=0x7fcc34070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/879987230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cf7e0 0x7fcc34070340 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcc34070cc0 tx=0x7fcc2401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 -- 172.21.15.170:0/879987230 shutdown_connections 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/879987230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc340cc8c0 0x7fcc340cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/879987230 >> v2:172.21.15.191:3300/0 conn(0x7fcc340cb460 0x7fcc340cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/879987230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cf7e0 0x7fcc34070340 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 -- 172.21.15.170:0/879987230 >> 172.21.15.170:0/879987230 conn(0x7fcc340b1c40 msgr2=0x7fcc340b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 -- 172.21.15.170:0/879987230 shutdown_connections 2024-09-21T02:20:58.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.417+0000 7fcc39dd2640 1 -- 172.21.15.170:0/879987230 wait complete. 2024-09-21T02:20:58.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 Processor -- start 2024-09-21T02:20:58.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- start start 2024-09-21T02:20:58.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 0x7fcc3409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc340cc8c0 0x7fcc34148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc340cf7e0 0x7fcc34156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3406fa50 con 0x7fcc340cb460 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3406f8d0 con 0x7fcc340cc8c0 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3406fbd0 con 0x7fcc340cf7e0 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 0x7fcc3409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 0x7fcc3409a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53868/0 (socket says 172.21.15.170:53868) 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 -- 172.21.15.170:0/2019609947 learned_addr learned my addr 172.21.15.170:0/2019609947 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 -- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc340cf7e0 msgr2=0x7fcc34156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc32ffd640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc340cc8c0 0x7fcc34148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc340cf7e0 0x7fcc34156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 -- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc340cc8c0 msgr2=0x7fcc34148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc340cc8c0 0x7fcc34148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 -- 172.21.15.170:0/2019609947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc341565a0 con 0x7fcc340cb460 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc337fe640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 0x7fcc3409a9c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcc240b0040 tx=0x7fcc240a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc240b2040 con 0x7fcc340cb460 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc240044b0 con 0x7fcc340cb460 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3409b5e0 con 0x7fcc340cb460 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24003860 con 0x7fcc340cb460 2024-09-21T02:20:58.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc340c8c90 con 0x7fcc340cb460 2024-09-21T02:20:58.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.421+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3409bb00 con 0x7fcc340cb460 2024-09-21T02:20:58.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.425+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fcc24003a00 con 0x7fcc340cb460 2024-09-21T02:20:58.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.425+0000 7fcc1bfff640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcc200738f0 0x7fcc20075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.425+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fcc2412e860 con 0x7fcc340cb460 2024-09-21T02:20:58.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.425+0000 7fcc32ffd640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcc200738f0 0x7fcc20075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.425+0000 7fcc32ffd640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcc200738f0 0x7fcc20075e00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc340b4990 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:58.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.429+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcc240bd060 con 0x7fcc340cb460 2024-09-21T02:20:58.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc340cccd0 con 0x7fcc340cb460 2024-09-21T02:20:58.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc1bfff640 1 -- 172.21.15.170:0/2019609947 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5236 (secure 0 0 0) 0x7fcc240164b0 con 0x7fcc340cb460 2024-09-21T02:20:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:58.629 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":91,"btime":"2024-09-21T02:20:46:972016+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:20:33.897736+0000","modified":"2024-09-21T02:20:36.929189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/169289834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":169289834},{"type":"v1","addr":"172.21.15.191:6835","nonce":169289834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8061,"qdb_cluster":[8061]},"id":10}]} 2024-09-21T02:20:58.629 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 91 2024-09-21T02:20:58.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcc200738f0 msgr2=0x7fcc20075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcc200738f0 0x7fcc20075e00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc340b4990 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2024-09-21T02:20:58.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 msgr2=0x7fcc3409a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 0x7fcc3409a9c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcc240b0040 tx=0x7fcc240a57a0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 shutdown_connections 2024-09-21T02:20:58.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcc200738f0 0x7fcc20075e00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc340cf7e0 0x7fcc34156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc340cc8c0 0x7fcc34148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 --2- 172.21.15.170:0/2019609947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc340cb460 0x7fcc3409a9c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 >> 172.21.15.170:0/2019609947 conn(0x7fcc340b1c40 msgr2=0x7fcc340d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:58.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 shutdown_connections 2024-09-21T02:20:58.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.629+0000 7fcc39dd2640 1 -- 172.21.15.170:0/2019609947 wait complete. 2024-09-21T02:20:58.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:58.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 -- 172.21.15.170:0/2171660189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160b50 msgr2=0x7f48300cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 --2- 172.21.15.170:0/2171660189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160b50 0x7f48300cca40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4828009840 tx=0x7f482801c820 comp rx=0 tx=0).stop 2024-09-21T02:20:58.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 -- 172.21.15.170:0/2171660189 shutdown_connections 2024-09-21T02:20:58.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 --2- 172.21.15.170:0/2171660189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160b50 0x7f48300cca40 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 -- 172.21.15.170:0/2171660189 >> 172.21.15.170:0/2171660189 conn(0x7f48300711c0 msgr2=0x7f48300715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:58.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 -- 172.21.15.170:0/2171660189 shutdown_connections 2024-09-21T02:20:58.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 -- 172.21.15.170:0/2171660189 wait complete. 2024-09-21T02:20:58.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 Processor -- start 2024-09-21T02:20:58.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.805+0000 7f48365a7640 1 -- start start 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48300ccf80 0x7f48301f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4830160b50 0x7f483015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 0x7f4830154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f483016af00 con 0x7f4830160f60 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f483016ad80 con 0x7f4830160b50 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f483016b080 con 0x7f48300ccf80 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 0x7f4830154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 0x7f4830154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53888/0 (socket says 172.21.15.170:53888) 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 -- 172.21.15.170:0/3212234413 learned_addr learned my addr 172.21.15.170:0/3212234413 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482ffff640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48300ccf80 0x7f48301f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482f7fe640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4830160b50 0x7f483015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 -- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48300ccf80 msgr2=0x7f48301f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48300ccf80 0x7f48301f0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 -- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4830160b50 msgr2=0x7f483015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4830160b50 0x7f483015bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 -- 172.21.15.170:0/3212234413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48301f0250 con 0x7f4830160f60 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482ffff640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48300ccf80 0x7f48301f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f4834b44640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 0x7f4830154620 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f482000c3c0 tx=0x7f482000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:58.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820017070 con 0x7f4830160f60 2024-09-21T02:20:58.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4820003600 con 0x7f4830160f60 2024-09-21T02:20:58.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483015a300 con 0x7f4830160f60 2024-09-21T02:20:58.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820003a50 con 0x7f4830160f60 2024-09-21T02:20:58.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48301f04f0 con 0x7f4830160f60 2024-09-21T02:20:58.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483015a070 con 0x7f4830160f60 2024-09-21T02:20:58.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.809+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f482001a070 con 0x7f4830160f60 2024-09-21T02:20:58.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.813+0000 7f482cff9640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f481c0738f0 0x7f481c075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:58.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.813+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f4820094170 con 0x7f4830160f60 2024-09-21T02:20:58.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.813+0000 7f482ffff640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f481c0738f0 0x7f481c075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:58.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.813+0000 7f482ffff640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f481c0738f0 0x7f481c075e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f48280af000 tx=0x7f482801b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:58.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.817+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f482005e6d0 con 0x7f4830160f60 2024-09-21T02:20:58.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.981+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48300cca40 con 0x7f4830160f60 2024-09-21T02:20:58.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.981+0000 7f482cff9640 1 -- 172.21.15.170:0/3212234413 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16831 (secure 0 0 0) 0x7f482005dfd0 con 0x7f4830160f60 2024-09-21T02:20:58.983 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:58.984 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":130,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:20:35.480836+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:20:31.581917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T02:20:32.865501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f481c0738f0 msgr2=0x7f481c075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f481c0738f0 0x7f481c075e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f48280af000 tx=0x7f482801b060 comp rx=0 tx=0).stop 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 msgr2=0x7f4830154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 0x7f4830154620 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f482000c3c0 tx=0x7f482000c7c0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 shutdown_connections 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f481c0738f0 0x7f481c075e00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4830160f60 0x7f4830154620 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4830160b50 0x7f483015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 --2- 172.21.15.170:0/3212234413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48300ccf80 0x7f48301f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:58.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 >> 172.21.15.170:0/3212234413 conn(0x7f48300711c0 msgr2=0x7f4830075f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:58.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 shutdown_connections 2024-09-21T02:20:58.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:58.985+0000 7f48365a7640 1 -- 172.21.15.170:0/3212234413 wait complete. 2024-09-21T02:20:59.006 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:20:59.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 -- 172.21.15.170:0/1696318306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 msgr2=0x7f6e0c15c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1696318306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15c850 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c075080 tx=0x7f6e0001c800 comp rx=0 tx=0).stop 2024-09-21T02:20:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 -- 172.21.15.170:0/1696318306 shutdown_connections 2024-09-21T02:20:59.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1696318306 >> v2:172.21.15.191:3300/0 conn(0x7f6e0c0d0aa0 0x7f6e0c0d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1696318306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15c850 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 -- 172.21.15.170:0/1696318306 >> 172.21.15.170:0/1696318306 conn(0x7f6e0c0cee30 msgr2=0x7f6e0c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 -- 172.21.15.170:0/1696318306 shutdown_connections 2024-09-21T02:20:59.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 -- 172.21.15.170:0/1696318306 wait complete. 2024-09-21T02:20:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.161+0000 7f6e1354f640 1 Processor -- start 2024-09-21T02:20:59.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- start start 2024-09-21T02:20:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e0c0b3e30 0x7f6e0c155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e0c0d0aa0 0x7f6e0c15be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c16a470 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c16a2f0 con 0x7f6e0c0d0aa0 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c16a5f0 con 0x7f6e0c0b3e30 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53900/0 (socket says 172.21.15.170:53900) 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 -- 172.21.15.170:0/1753169441 learned_addr learned my addr 172.21.15.170:0/1753169441 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e112eb640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e0c0b3e30 0x7f6e0c155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e11aec640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e0c0d0aa0 0x7f6e0c15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 -- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e0c0b3e30 msgr2=0x7f6e0c155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e0c0b3e30 0x7f6e0c155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 -- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e0c0d0aa0 msgr2=0x7f6e0c15be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e0c0d0aa0 0x7f6e0c15be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 -- 172.21.15.170:0/1753169441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c0b6ca0 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e10aea640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15b8f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c3e0 tx=0x7f6e0800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08011790 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e08017070 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c0b6f80 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08011ce0 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c06f910 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.165+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c06fbf0 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.169+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f6e0801c460 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.169+0000 7f6dfdffb640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6df00738c0 0x7f6df0075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.169+0000 7f6e112eb640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6df00738c0 0x7f6df0075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.169+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f6e08094ca0 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.169+0000 7f6e112eb640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6df00738c0 0x7f6df0075dd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6e000af000 tx=0x7f6e0001b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:59.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.173+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6e0805f2c0 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.365+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e0c15c850 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.365+0000 7f6dfdffb640 1 -- 172.21.15.170:0/1753169441 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5236 (secure 0 0 0) 0x7f6e0805ebc0 con 0x7f6e0c0b7ab0 2024-09-21T02:20:59.367 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:59.367 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":91,"btime":"2024-09-21T02:20:46:972016+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:20:33.897736+0000","modified":"2024-09-21T02:20:36.929189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/169289834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":169289834},{"type":"v1","addr":"172.21.15.191:6835","nonce":169289834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8061,"qdb_cluster":[8061]},"id":10}]} 2024-09-21T02:20:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 91 2024-09-21T02:20:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6df00738c0 msgr2=0x7f6df0075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6df00738c0 0x7f6df0075dd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6e000af000 tx=0x7f6e0001b040 comp rx=0 tx=0).stop 2024-09-21T02:20:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 msgr2=0x7f6e0c15b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15b8f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c3e0 tx=0x7f6e0800c8a0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 shutdown_connections 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6df00738c0 0x7f6df0075dd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e0c0d0aa0 0x7f6e0c15be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e0c0b7ab0 0x7f6e0c15b8f0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 --2- 172.21.15.170:0/1753169441 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e0c0b3e30 0x7f6e0c155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 >> 172.21.15.170:0/1753169441 conn(0x7f6e0c0cee30 msgr2=0x7f6e0c15fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 shutdown_connections 2024-09-21T02:20:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.369+0000 7f6e1354f640 1 -- 172.21.15.170:0/1753169441 wait complete. 2024-09-21T02:20:59.389 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:20:59.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 -- 172.21.15.170:0/1181341437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100ce4c0 msgr2=0x7f7e100ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 --2- 172.21.15.170:0/1181341437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100ce4c0 0x7f7e100ce8d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009900 tx=0x7f7e0001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 -- 172.21.15.170:0/1181341437 shutdown_connections 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 --2- 172.21.15.170:0/1181341437 >> v2:172.21.15.191:3300/0 conn(0x7f7e100cadf0 0x7f7e100cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 --2- 172.21.15.170:0/1181341437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100ce4c0 0x7f7e100ce8d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 -- 172.21.15.170:0/1181341437 >> 172.21.15.170:0/1181341437 conn(0x7f7e100b1c40 msgr2=0x7f7e100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 -- 172.21.15.170:0/1181341437 shutdown_connections 2024-09-21T02:20:59.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.529+0000 7f7e163b9640 1 -- 172.21.15.170:0/1181341437 wait complete. 2024-09-21T02:20:59.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 Processor -- start 2024-09-21T02:20:59.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 -- start start 2024-09-21T02:20:59.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100cadf0 0x7f7e10151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 0x7f7e10152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e1015f990 0x7f7e10157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10166160 con 0x7f7e100ce4c0 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10165fe0 con 0x7f7e1015f990 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e163b9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7e101662e0 con 0x7f7e100cadf0 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.533+0000 7f7e0f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 0x7f7e10152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 0x7f7e10152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53910/0 (socket says 172.21.15.170:53910) 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 -- 172.21.15.170:0/652533009 learned_addr learned my addr 172.21.15.170:0/652533009 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0ffff640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100cadf0 0x7f7e10151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e14956640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e1015f990 0x7f7e10157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 -- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100cadf0 msgr2=0x7f7e10151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100cadf0 0x7f7e10151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 -- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e1015f990 msgr2=0x7f7e10157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e1015f990 0x7f7e10157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 -- 172.21.15.170:0/652533009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e10151d00 con 0x7f7e100ce4c0 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0ffff640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100cadf0 0x7f7e10151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0f7fe640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 0x7f7e10152490 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009850 tx=0x7f7e00005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e000b2070 con 0x7f7e100ce4c0 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e000b3040 con 0x7f7e100ce4c0 2024-09-21T02:20:59.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e10155190 con 0x7f7e100ce4c0 2024-09-21T02:20:59.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00005280 con 0x7f7e100ce4c0 2024-09-21T02:20:59.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e100d0590 con 0x7f7e100ce4c0 2024-09-21T02:20:59.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e100cb200 con 0x7f7e100ce4c0 2024-09-21T02:20:59.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f7e00005420 con 0x7f7e100ce4c0 2024-09-21T02:20:59.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0cff9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dfc0738c0 0x7f7dfc075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7f7e00130680 con 0x7f7e100ce4c0 2024-09-21T02:20:59.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0ffff640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dfc0738c0 0x7f7dfc075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.537+0000 7f7e0ffff640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dfc0738c0 0x7f7dfc075dd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e040016b0 tx=0x7f7e04009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:59.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.541+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7e000164d0 con 0x7f7e100ce4c0 2024-09-21T02:20:59.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.737+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e100af850 con 0x7f7e100ce4c0 2024-09-21T02:20:59.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.737+0000 7f7e0cff9640 1 -- 172.21.15.170:0/652533009 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5236 (secure 0 0 0) 0x7f7e000fade0 con 0x7f7e100ce4c0 2024-09-21T02:20:59.738 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:20:59.738 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":91,"btime":"2024-09-21T02:20:46:972016+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:20:33.897736+0000","modified":"2024-09-21T02:20:36.929189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/169289834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":169289834},{"type":"v1","addr":"172.21.15.191:6835","nonce":169289834}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8061,"qdb_cluster":[8061]},"id":10}]} 2024-09-21T02:20:59.739 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 91 2024-09-21T02:20:59.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dfc0738c0 msgr2=0x7f7dfc075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dfc0738c0 0x7f7dfc075dd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e040016b0 tx=0x7f7e04009040 comp rx=0 tx=0).stop 2024-09-21T02:20:59.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 msgr2=0x7f7e10152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 0x7f7e10152490 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009850 tx=0x7f7e00005de0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 shutdown_connections 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7dfc0738c0 0x7f7dfc075dd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e1015f990 0x7f7e10157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e100ce4c0 0x7f7e10152490 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 --2- 172.21.15.170:0/652533009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e100cadf0 0x7f7e10151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 >> 172.21.15.170:0/652533009 conn(0x7f7e100b1c40 msgr2=0x7f7e10075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 shutdown_connections 2024-09-21T02:20:59.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.741+0000 7f7e163b9640 1 -- 172.21.15.170:0/652533009 wait complete. 2024-09-21T02:20:59.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:20:59.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 -- 172.21.15.170:0/1902124907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc0cadf0 msgr2=0x7fc0bc0cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 --2- 172.21.15.170:0/1902124907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc0cadf0 0x7fc0bc0cb200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400ad90 tx=0x7fc0b401dcb0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 -- 172.21.15.170:0/1902124907 shutdown_connections 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 --2- 172.21.15.170:0/1902124907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc0cadf0 0x7fc0bc0cb200 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 --2- 172.21.15.170:0/1902124907 >> v2:172.21.15.191:3300/0 conn(0x7fc0bc0ce4c0 0x7fc0bc0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 -- 172.21.15.170:0/1902124907 >> 172.21.15.170:0/1902124907 conn(0x7fc0bc0b1c40 msgr2=0x7fc0bc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 -- 172.21.15.170:0/1902124907 shutdown_connections 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.917+0000 7fc0c1985640 1 -- 172.21.15.170:0/1902124907 wait complete. 2024-09-21T02:20:59.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 Processor -- start 2024-09-21T02:20:59.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- start start 2024-09-21T02:20:59.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0bc0cadf0 0x7fc0bc1641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 0x7fc0bc156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc15f990 0x7fc0bc1472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc168170 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc167ff0 con 0x7fc0bc0cadf0 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc0bc1682f0 con 0x7fc0bc15f990 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 0x7fc0bc156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 0x7fc0bc156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53918/0 (socket says 172.21.15.170:53918) 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 -- 172.21.15.170:0/3549531470 learned_addr learned my addr 172.21.15.170:0/3549531470 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0bb7fe640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc15f990 0x7fc0bc1472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0baffd640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0bc0cadf0 0x7fc0bc1641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 -- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc15f990 msgr2=0x7fc0bc1472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc15f990 0x7fc0bc1472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 -- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0bc0cadf0 msgr2=0x7fc0bc1641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0bc0cadf0 0x7fc0bc1641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 -- 172.21.15.170:0/3549531470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc1f9140 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0baffd640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0bc0cadf0 0x7fc0bc1641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:20:59.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0ba7fc640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 0x7fc0bc156160 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00e970 tx=0x7fc0ac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00cd80 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0ac004020 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc148170 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac004470 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc158920 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.921+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc147c00 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.925+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fc0ac003600 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.925+0000 7fc0a3fff640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0a80738c0 0x7fc0a8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:20:59.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.925+0000 7fc0baffd640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0a80738c0 0x7fc0a8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:20:59.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.925+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5737+0+0 (secure 0 0 0) 0x7fc0ac0958e0 con 0x7fc0bc0ce4c0 2024-09-21T02:20:59.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.925+0000 7fc0baffd640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0a80738c0 0x7fc0a8075dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0000f70 tx=0x7fc0b0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:20:59.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:20:59.929+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc0bc1f9140 con 0x7fc0bc0ce4c0 2024-09-21T02:21:00.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.109+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0bc0cb200 con 0x7fc0bc0ce4c0 2024-09-21T02:21:00.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.113+0000 7fc0a3fff640 1 -- 172.21.15.170:0/3549531470 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16869 (secure 0 0 0) 0x7fc0ac0600d0 con 0x7fc0bc0ce4c0 2024-09-21T02:21:00.113 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:00.114 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":131,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:00.050240+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:20:31.581917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T02:20:32.865501+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1b","mappings":[{"from":2,"to":5}]},{"pgid":"21.34","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0a80738c0 msgr2=0x7fc0a8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0a80738c0 0x7fc0a8075dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0000f70 tx=0x7fc0b0009040 comp rx=0 tx=0).stop 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 msgr2=0x7fc0bc156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 0x7fc0bc156160 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00e970 tx=0x7fc0ac00ee30 comp rx=0 tx=0).stop 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 shutdown_connections 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0a80738c0 0x7fc0a8075dd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0bc15f990 0x7fc0bc1472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0bc0ce4c0 0x7fc0bc156160 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 --2- 172.21.15.170:0/3549531470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0bc0cadf0 0x7fc0bc1641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 >> 172.21.15.170:0/3549531470 conn(0x7fc0bc0b1c40 msgr2=0x7fc0bc0aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:00.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 shutdown_connections 2024-09-21T02:21:00.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.117+0000 7fc0c1985640 1 -- 172.21.15.170:0/3549531470 wait complete. 2024-09-21T02:21:00.137 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 -- 172.21.15.170:0/1570386962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f666006d510 msgr2=0x7f6660160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 --2- 172.21.15.170:0/1570386962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f666006d510 0x7f6660160390 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f6650009900 tx=0x7f665001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 -- 172.21.15.170:0/1570386962 shutdown_connections 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 --2- 172.21.15.170:0/1570386962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f666006d510 0x7f6660160390 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 --2- 172.21.15.170:0/1570386962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66600b22b0 0x7f66600b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 -- 172.21.15.170:0/1570386962 >> 172.21.15.170:0/1570386962 conn(0x7f666009a9c0 msgr2=0x7f66600d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:00.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 -- 172.21.15.170:0/1570386962 shutdown_connections 2024-09-21T02:21:00.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.289+0000 7f6664d21640 1 -- 172.21.15.170:0/1570386962 wait complete. 2024-09-21T02:21:00.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 Processor -- start 2024-09-21T02:21:00.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 -- start start 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66600b22b0 0x7f666014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66600b6930 0x7f66600af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 0x7f6660157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f666015c670 con 0x7f6660156c20 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f666015c4f0 con 0x7f66600b6930 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f666015c7f0 con 0x7f66600b22b0 2024-09-21T02:21:00.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 0x7f6660157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 0x7f6660157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53942/0 (socket says 172.21.15.170:53942) 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 -- 172.21.15.170:0/174441951 learned_addr learned my addr 172.21.15.170:0/174441951 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665dd9b640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66600b6930 0x7f66600af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665e59c640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66600b22b0 0x7f666014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 -- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66600b22b0 msgr2=0x7f666014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66600b22b0 0x7f666014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 -- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66600b6930 msgr2=0x7f66600af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66600b6930 0x7f66600af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 -- 172.21.15.170:0/174441951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6660157570 con 0x7f6660156c20 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f665ed9d640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 0x7f6660157030 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f66500b0040 tx=0x7f6650005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500b2040 con 0x7f6660156c20 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6650004330 con 0x7f6660156c20 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6660152bf0 con 0x7f6660156c20 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650003860 con 0x7f6660156c20 2024-09-21T02:21:00.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.293+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601fa930 con 0x7f6660156c20 2024-09-21T02:21:00.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.297+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660160390 con 0x7f6660156c20 2024-09-21T02:21:00.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.297+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f66500bf060 con 0x7f6660156c20 2024-09-21T02:21:00.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.297+0000 7f664effd640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f66400738c0 0x7f6640075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.297+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5795+0+0 (secure 0 0 0) 0x7f665012ec70 con 0x7f6660156c20 2024-09-21T02:21:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.297+0000 7f665e59c640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f66400738c0 0x7f6640075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.297+0000 7f665e59c640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f66400738c0 0x7f6640075dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6658009e40 tx=0x7f66580094a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.301+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6660157570 con 0x7f6660156c20 2024-09-21T02:21:00.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.509+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f66600b4dc0 con 0x7f6660156c20 2024-09-21T02:21:00.512 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:21:00.513+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:21:00.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.513+0000 7f664effd640 1 -- 172.21.15.170:0/174441951 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) ==== 161+0+0 (secure 0 0 0) 0x7f66500164d0 con 0x7f6660156c20 2024-09-21T02:21:00.515 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:21:00.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f66400738c0 msgr2=0x7f6640075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f66400738c0 0x7f6640075dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6658009e40 tx=0x7f66580094a0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 msgr2=0x7f6660157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 0x7f6660157030 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f66500b0040 tx=0x7f6650005e90 comp rx=0 tx=0).stop 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 shutdown_connections 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f66400738c0 0x7f6640075dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6660156c20 0x7f6660157030 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66600b6930 0x7f66600af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 --2- 172.21.15.170:0/174441951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66600b22b0 0x7f666014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 >> 172.21.15.170:0/174441951 conn(0x7f666009a9c0 msgr2=0x7f6660070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.517+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 shutdown_connections 2024-09-21T02:21:00.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.521+0000 7f6664d21640 1 -- 172.21.15.170:0/174441951 wait complete. 2024-09-21T02:21:00.530 INFO:tasks.ceph.mds.d.smithi191.stderr: -1867> 2024-09-21T02:20:36.452+0000 7f66d3c75640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:21:00.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:21:00.606 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:21:00.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 -- 172.21.15.170:0/3503286417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa378004010 msgr2=0x7fa378004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 --2- 172.21.15.170:0/3503286417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa378004010 0x7fa378004810 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa384001690 tx=0x7fa38401c7a0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa38f7fe640 1 -- 172.21.15.170:0/3503286417 reap_dead start 2024-09-21T02:21:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 -- 172.21.15.170:0/3503286417 shutdown_connections 2024-09-21T02:21:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 -- 172.21.15.170:0/3503286417 >> 172.21.15.170:0/3503286417 conn(0x7fa3900d0170 msgr2=0x7fa3900d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 -- 172.21.15.170:0/3503286417 shutdown_connections 2024-09-21T02:21:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 -- 172.21.15.170:0/3503286417 wait complete. 2024-09-21T02:21:00.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 Processor -- start 2024-09-21T02:21:00.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.701+0000 7fa3957d5640 1 -- start start 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa378004010 0x7fa3900b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3900c6700 0x7fa390159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 0x7fa3900b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900b5bb0 con 0x7fa39009d5e0 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3900caad0 con 0x7fa378004010 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa390073b60 con 0x7fa3900c6700 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 0x7fa3900b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 0x7fa3900b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53954/0 (socket says 172.21.15.170:53954) 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 -- 172.21.15.170:0/1223059104 learned_addr learned my addr 172.21.15.170:0/1223059104 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38e7fc640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3900c6700 0x7fa390159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38effd640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa378004010 0x7fa3900b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 -- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3900c6700 msgr2=0x7fa390159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3900c6700 0x7fa390159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 -- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa378004010 msgr2=0x7fa3900b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa378004010 0x7fa3900b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 -- 172.21.15.170:0/1223059104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3900b25d0 con 0x7fa39009d5e0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38e7fc640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3900c6700 0x7fa390159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38effd640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa378004010 0x7fa3900b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa38f7fe640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 0x7fa3900b2fc0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa3900d0050 tx=0x7fa388010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38800ed00 con 0x7fa39009d5e0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa388013480 con 0x7fa39009d5e0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3900b28b0 con 0x7fa39009d5e0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3880138d0 con 0x7fa39009d5e0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39009d0c0 con 0x7fa39009d5e0 2024-09-21T02:21:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.705+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39009cbd0 con 0x7fa39009d5e0 2024-09-21T02:21:00.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.709+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fa388013a70 con 0x7fa39009d5e0 2024-09-21T02:21:00.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.709+0000 7fa3777fe640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa37c0738c0 0x7fa37c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:00.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.709+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5881+0+0 (secure 0 0 0) 0x7fa388099850 con 0x7fa39009d5e0 2024-09-21T02:21:00.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.709+0000 7fa38effd640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa37c0738c0 0x7fa37c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:00.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.709+0000 7fa38effd640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa37c0738c0 0x7fa37c075dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa380009a80 tx=0x7fa3800091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:00.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.713+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa388063fb0 con 0x7fa39009d5e0 2024-09-21T02:21:00.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:00.921+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa3900b2a80 con 0x7fa39009d5e0 2024-09-21T02:21:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3777fe640 1 -- 172.21.15.170:0/1223059104 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) ==== 104+0+0 (secure 0 0 0) 0x7fa3880638b0 con 0x7fa39009d5e0 2024-09-21T02:21:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa37c0738c0 msgr2=0x7fa37c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa37c0738c0 0x7fa37c075dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa380009a80 tx=0x7fa3800091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 msgr2=0x7fa3900b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 0x7fa3900b2fc0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa3900d0050 tx=0x7fa388010e80 comp rx=0 tx=0).stop 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 shutdown_connections 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa37c0738c0 0x7fa37c075dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3900c6700 0x7fa390159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39009d5e0 0x7fa3900b2fc0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 --2- 172.21.15.170:0/1223059104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa378004010 0x7fa3900b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 >> 172.21.15.170:0/1223059104 conn(0x7fa3900d0170 msgr2=0x7fa3900b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 shutdown_connections 2024-09-21T02:21:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.529+0000 7fa3957d5640 1 -- 172.21.15.170:0/1223059104 wait complete. 2024-09-21T02:21:01.558 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:21:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 -- 172.21.15.170:0/2737418610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 msgr2=0x7f3d8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 --2- 172.21.15.170:0/2737418610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d8800b9d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009eb0 tx=0x7f3d9401d5f0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 -- 172.21.15.170:0/2737418610 shutdown_connections 2024-09-21T02:21:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 --2- 172.21.15.170:0/2737418610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d8800b9d0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 -- 172.21.15.170:0/2737418610 >> 172.21.15.170:0/2737418610 conn(0x7f3d9c0b1c40 msgr2=0x7f3d9c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 -- 172.21.15.170:0/2737418610 shutdown_connections 2024-09-21T02:21:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.721+0000 7f3da118a640 1 -- 172.21.15.170:0/2737418610 wait complete. 2024-09-21T02:21:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 Processor -- start 2024-09-21T02:21:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- start start 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d9c15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d9c09b590 0x7f3d9c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d9c151320 0x7f3d9c09c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0af7d0 con 0x7f3d88005360 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0af650 con 0x7f3d9c151320 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9c0ccae0 con 0x7f3d9c09b590 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d9c15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d9c15edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53962/0 (socket says 172.21.15.170:53962) 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 -- 172.21.15.170:0/982796271 learned_addr learned my addr 172.21.15.170:0/982796271 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9b59e640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d9c151320 0x7f3d9c09c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9a59c640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d9c09b590 0x7f3d9c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 -- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d9c09b590 msgr2=0x7f3d9c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d9c09b590 0x7f3d9c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 -- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d9c151320 msgr2=0x7f3d9c09c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d9c151320 0x7f3d9c09c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 -- 172.21.15.170:0/982796271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c15f640 con 0x7f3d88005360 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9b59e640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d9c151320 0x7f3d9c09c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9a59c640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d9c09b590 0x7f3d9c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d9ad9d640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d9c15edf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c004e80 tx=0x7f3d8c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c015020 con 0x7f3d88005360 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c15f920 con 0x7f3d88005360 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d8c011960 con 0x7f3d88005360 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c011db0 con 0x7f3d88005360 2024-09-21T02:21:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c09bf70 con 0x7f3d88005360 2024-09-21T02:21:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.725+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c0cd040 con 0x7f3d88005360 2024-09-21T02:21:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.729+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f3d8c018020 con 0x7f3d88005360 2024-09-21T02:21:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.729+0000 7f3d937fe640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d7c0738c0 0x7f3d7c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.729+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5881+0+0 (secure 0 0 0) 0x7f3d8c095ab0 con 0x7f3d88005360 2024-09-21T02:21:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.729+0000 7f3d9a59c640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d7c0738c0 0x7f3d7c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.729+0000 7f3d9a59c640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d7c0738c0 0x7f3d7c075dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001020 tx=0x7f3d84001160 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:01.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.733+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d8c01f020 con 0x7f3d88005360 2024-09-21T02:21:01.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:01.905+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3d9c0b4030 con 0x7f3d88005360 2024-09-21T02:21:02.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.529+0000 7f3d937fe640 1 -- 172.21.15.170:0/982796271 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7f3d8c05ff90 con 0x7f3d88005360 2024-09-21T02:21:02.530 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:21:02.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.533+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d7c0738c0 msgr2=0x7f3d7c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:02.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.533+0000 7f3da118a640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d7c0738c0 0x7f3d7c075dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001020 tx=0x7f3d84001160 comp rx=0 tx=0).stop 2024-09-21T02:21:02.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.533+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 msgr2=0x7f3d9c15edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:02.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.533+0000 7f3da118a640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d9c15edf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c004e80 tx=0x7f3d8c001630 comp rx=0 tx=0).stop 2024-09-21T02:21:02.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 shutdown_connections 2024-09-21T02:21:02.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d7c0738c0 0x7f3d7c075dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d9c09b590 0x7f3d9c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d9c151320 0x7f3d9c09c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 --2- 172.21.15.170:0/982796271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d88005360 0x7f3d9c15edf0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 >> 172.21.15.170:0/982796271 conn(0x7f3d9c0b1c40 msgr2=0x7f3d9c0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:02.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 shutdown_connections 2024-09-21T02:21:02.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.537+0000 7f3da118a640 1 -- 172.21.15.170:0/982796271 wait complete. 2024-09-21T02:21:02.563 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 -- 172.21.15.170:0/1353503407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb44c003a60 msgr2=0x7fb44c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 --2- 172.21.15.170:0/1353503407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb44c003a60 0x7fb44c003ed0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb454001720 tx=0x7fb45401c730 comp rx=0 tx=0).stop 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 -- 172.21.15.170:0/1353503407 shutdown_connections 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 --2- 172.21.15.170:0/1353503407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb44c003a60 0x7fb44c003ed0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 -- 172.21.15.170:0/1353503407 >> 172.21.15.170:0/1353503407 conn(0x7fb4640b1c40 msgr2=0x7fb4640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 -- 172.21.15.170:0/1353503407 shutdown_connections 2024-09-21T02:21:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.725+0000 7fb469529640 1 -- 172.21.15.170:0/1353503407 wait complete. 2024-09-21T02:21:02.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 Processor -- start 2024-09-21T02:21:02.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 -- start start 2024-09-21T02:21:02.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb44c003a60 0x7fb46414c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:02.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb464154770 0x7fb4641f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:02.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 0x7fb46415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb46415eec0 con 0x7fb4641f8af0 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb46415ed40 con 0x7fb464154770 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb46415f040 con 0x7fb44c003a60 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 0x7fb46415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 0x7fb46415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53980/0 (socket says 172.21.15.170:53980) 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 -- 172.21.15.170:0/392355728 learned_addr learned my addr 172.21.15.170:0/392355728 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb462ffd640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb44c003a60 0x7fb46414c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4627fc640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb464154770 0x7fb4641f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 -- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb44c003a60 msgr2=0x7fb46414c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb44c003a60 0x7fb46414c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 -- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb464154770 msgr2=0x7fb4641f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb464154770 0x7fb4641f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:02.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 -- 172.21.15.170:0/392355728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46415d750 con 0x7fb4641f8af0 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4627fc640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb464154770 0x7fb4641f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb4637fe640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 0x7fb46415b4f0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb4640ce1a0 tx=0x7fb45c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003980 con 0x7fb4641f8af0 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb45c003b40 con 0x7fb4641f8af0 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641f9280 con 0x7fb4641f8af0 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004030 con 0x7fb4641f8af0 2024-09-21T02:21:02.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.729+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46409bbf0 con 0x7fb4641f8af0 2024-09-21T02:21:02.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.733+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46415f5a0 con 0x7fb4641f8af0 2024-09-21T02:21:02.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.733+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fb45c0041d0 con 0x7fb4641f8af0 2024-09-21T02:21:02.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.733+0000 7fb44b7fe640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb4500738c0 0x7fb450075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:02.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.733+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5443+0+0 (secure 0 0 0) 0x7fb45c096320 con 0x7fb4641f8af0 2024-09-21T02:21:02.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.733+0000 7fb462ffd640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb4500738c0 0x7fb450075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:02.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.733+0000 7fb462ffd640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb4500738c0 0x7fb450075dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4540ab000 tx=0x7fb4540aa040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:02.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.737+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb45c01e060 con 0x7fb4641f8af0 2024-09-21T02:21:02.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:02.913+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb4640cb3c0 con 0x7fb4641f8af0 2024-09-21T02:21:03.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.541+0000 7fb44b7fe640 1 -- 172.21.15.170:0/392355728 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) ==== 169+0+0 (secure 0 0 0) 0x7fb45c060b30 con 0x7fb4641f8af0 2024-09-21T02:21:03.543 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:21:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb4500738c0 msgr2=0x7fb450075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb4500738c0 0x7fb450075dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4540ab000 tx=0x7fb4540aa040 comp rx=0 tx=0).stop 2024-09-21T02:21:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 msgr2=0x7fb46415b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 0x7fb46415b4f0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fb4640ce1a0 tx=0x7fb45c00c7a0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 shutdown_connections 2024-09-21T02:21:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb4500738c0 0x7fb450075dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4641f8af0 0x7fb46415b4f0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb464154770 0x7fb4641f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 --2- 172.21.15.170:0/392355728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb44c003a60 0x7fb46414c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.545+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 >> 172.21.15.170:0/392355728 conn(0x7fb4640b1c40 msgr2=0x7fb464073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.549+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 shutdown_connections 2024-09-21T02:21:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.553+0000 7fb469529640 1 -- 172.21.15.170:0/392355728 wait complete. 2024-09-21T02:21:03.582 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.737+0000 7f65ce42e640 1 -- 172.21.15.170:0/2659780198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 msgr2=0x7f65b4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.737+0000 7f65ce42e640 1 --2- 172.21.15.170:0/2659780198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65b4004c20 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f65b000c260 tx=0x7f65b001f6a0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65cc9cb640 1 -- 172.21.15.170:0/2659780198 reap_dead start 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- 172.21.15.170:0/2659780198 shutdown_connections 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- 172.21.15.170:0/2659780198 >> 172.21.15.170:0/2659780198 conn(0x7f65c809a9c0 msgr2=0x7f65c80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- 172.21.15.170:0/2659780198 shutdown_connections 2024-09-21T02:21:03.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- 172.21.15.170:0/2659780198 wait complete. 2024-09-21T02:21:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 Processor -- start 2024-09-21T02:21:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- start start 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65c80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c81f3d00 0x7f65c81f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c80b5730 0x7f65c80b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80b1ac0 con 0x7f65b4004810 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80b1940 con 0x7f65c81f3d00 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65ce42e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f65c8160aa0 con 0x7f65c80b5730 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65c80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65c80c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53996/0 (socket says 172.21.15.170:53996) 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 -- 172.21.15.170:0/1979348683 learned_addr learned my addr 172.21.15.170:0/1979348683 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65bf7fe640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c81f3d00 0x7f65c81f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:03.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65cc9cb640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c80b5730 0x7f65c80b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 -- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c80b5730 msgr2=0x7f65c80b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c80b5730 0x7f65c80b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 -- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c81f3d00 msgr2=0x7f65c81f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c81f3d00 0x7f65c81f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65c7fff640 1 -- 172.21.15.170:0/1979348683 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c80b6080 con 0x7f65b4004810 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.741+0000 7f65bf7fe640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c81f3d00 0x7f65c81f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c7fff640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65c80c9e70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f65b8011ab0 tx=0x7f65b8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b801f070 con 0x7f65b4004810 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65b8012450 con 0x7f65b4004810 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b80128a0 con 0x7f65b4004810 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c80ca910 con 0x7f65b4004810 2024-09-21T02:21:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c80c86f0 con 0x7f65b4004810 2024-09-21T02:21:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c8161000 con 0x7f65b4004810 2024-09-21T02:21:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f65b8022070 con 0x7f65b4004810 2024-09-21T02:21:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c57fa640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65a80738c0 0x7f65a8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65bf7fe640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65a80738c0 0x7f65a8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7f65b8095670 con 0x7f65b4004810 2024-09-21T02:21:03.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.745+0000 7f65bf7fe640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65a80738c0 0x7f65a8075dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f65b00b0000 tx=0x7f65b001c4e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:03.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.749+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f65c80b6080 con 0x7f65b4004810 2024-09-21T02:21:03.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.925+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f65c8071ae0 con 0x7f65b4004810 2024-09-21T02:21:03.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.929+0000 7f65c57fa640 1 -- 172.21.15.170:0/1979348683 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f65b805fa10 con 0x7f65b4004810 2024-09-21T02:21:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65a80738c0 msgr2=0x7f65a8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65a80738c0 0x7f65a8075dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f65b00b0000 tx=0x7f65b001c4e0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 msgr2=0x7f65c80c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65c80c9e70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f65b8011ab0 tx=0x7f65b8011f70 comp rx=0 tx=0).stop 2024-09-21T02:21:03.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 shutdown_connections 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65a80738c0 0x7f65a8075dd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c80b5730 0x7f65c80b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c81f3d00 0x7f65c81f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 --2- 172.21.15.170:0/1979348683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b4004810 0x7f65c80c9e70 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 >> 172.21.15.170:0/1979348683 conn(0x7f65c809a9c0 msgr2=0x7f65c80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 shutdown_connections 2024-09-21T02:21:03.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:03.933+0000 7f65ce42e640 1 -- 172.21.15.170:0/1979348683 wait complete. 2024-09-21T02:21:03.954 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-21T02:21:04.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3846891908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac070470 msgr2=0x7fd7ac0cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3846891908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac070470 0x7fd7ac0cd0f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac072c20 tx=0x7fd7a801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3846891908 shutdown_connections 2024-09-21T02:21:04.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3846891908 >> v2:172.21.15.191:3300/0 conn(0x7fd7ac15e6e0 0x7fd7ac15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3846891908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac070470 0x7fd7ac0cd0f0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3846891908 >> 172.21.15.170:0/3846891908 conn(0x7fd7ac0b1c40 msgr2=0x7fd7ac0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:04.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3846891908 shutdown_connections 2024-09-21T02:21:04.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3846891908 wait complete. 2024-09-21T02:21:04.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 Processor -- start 2024-09-21T02:21:04.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.117+0000 7fd7b40f8640 1 -- start start 2024-09-21T02:21:04.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7ac070470 0x7fd7ac1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 0x7fd7ac158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7ac15e6e0 0x7fd7ac168bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac16a7b0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac16a630 con 0x7fd7ac070470 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ac16a930 con 0x7fd7ac15e6e0 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 0x7fd7ac158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 0x7fd7ac158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54022/0 (socket says 172.21.15.170:54022) 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 -- 172.21.15.170:0/3128626150 learned_addr learned my addr 172.21.15.170:0/3128626150 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1e94640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7ac070470 0x7fd7ac1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 -- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7ac15e6e0 msgr2=0x7fd7ac168bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b2695640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7ac15e6e0 0x7fd7ac168bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7ac15e6e0 0x7fd7ac168bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 -- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7ac070470 msgr2=0x7fd7ac1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7ac070470 0x7fd7ac1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 -- 172.21.15.170:0/3128626150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac0b55c0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1e94640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7ac070470 0x7fd7ac1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b1693640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 0x7fd7ac158880 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c3f0 tx=0x7fd79c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c017070 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd79c0053c0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac169c90 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c003860 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac1683b0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac0cd0f0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fd79c01a070 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.121+0000 7fd7a27fc640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7900738f0 0x7fd790075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.125+0000 7fd7b1e94640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7900738f0 0x7fd790075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.125+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7fd79c093f50 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.125+0000 7fd7b1e94640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7900738f0 0x7fd790075e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80af000 tx=0x7fd7a801b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:04.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.129+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd79c05e7b0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.369+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fd7ac152d00 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.545+0000 7fd7a27fc640 1 -- 172.21.15.170:0/3128626150 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v406) ==== 151+0+0 (secure 0 0 0) 0x7fd79c05e0b0 con 0x7fd7ac15c5e0 2024-09-21T02:21:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7900738f0 msgr2=0x7fd790075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7900738f0 0x7fd790075e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80af000 tx=0x7fd7a801b100 comp rx=0 tx=0).stop 2024-09-21T02:21:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 msgr2=0x7fd7ac158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 0x7fd7ac158880 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00c3f0 tx=0x7fd79c00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 shutdown_connections 2024-09-21T02:21:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd7900738f0 0x7fd790075e00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd7ac15e6e0 0x7fd7ac168bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7ac15c5e0 0x7fd7ac158880 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 --2- 172.21.15.170:0/3128626150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7ac070470 0x7fd7ac1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 >> 172.21.15.170:0/3128626150 conn(0x7fd7ac0b1c40 msgr2=0x7fd7ac0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:04.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 shutdown_connections 2024-09-21T02:21:04.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.549+0000 7fd7b40f8640 1 -- 172.21.15.170:0/3128626150 wait complete. 2024-09-21T02:21:04.572 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-21T02:21:04.572 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-21T02:21:04.572 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:21:04.573 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:04.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 -- 172.21.15.170:0/685858783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840b2f40 msgr2=0x7fc7840b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 --2- 172.21.15.170:0/685858783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840b2f40 0x7fc7840b3350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7840b3a10 tx=0x7fc78001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 -- 172.21.15.170:0/685858783 shutdown_connections 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 --2- 172.21.15.170:0/685858783 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840b2f40 0x7fc7840b3350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 --2- 172.21.15.170:0/685858783 >> v2:172.21.15.191:3300/0 conn(0x7fc7840b5410 0x7fc78406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 -- 172.21.15.170:0/685858783 >> 172.21.15.170:0/685858783 conn(0x7fc7840b1c40 msgr2=0x7fc7840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 -- 172.21.15.170:0/685858783 shutdown_connections 2024-09-21T02:21:04.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.729+0000 7fc78be9d640 1 -- 172.21.15.170:0/685858783 wait complete. 2024-09-21T02:21:04.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 Processor -- start 2024-09-21T02:21:04.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 -- start start 2024-09-21T02:21:04.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7840b2f40 0x7fc784156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 0x7fc784154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840ccb00 0x7fc784155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc784169fd0 con 0x7fc7840b5410 2024-09-21T02:21:04.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc784169e50 con 0x7fc7840b2f40 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc78416a150 con 0x7fc7840ccb00 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 0x7fc784154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 0x7fc784154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54026/0 (socket says 172.21.15.170:54026) 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 -- 172.21.15.170:0/3168570770 learned_addr learned my addr 172.21.15.170:0/3168570770 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789c39640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7840b2f40 0x7fc784156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78a43a640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840ccb00 0x7fc784155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 -- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840ccb00 msgr2=0x7fc784155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840ccb00 0x7fc784155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 -- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7840b2f40 msgr2=0x7fc784156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7840b2f40 0x7fc784156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 -- 172.21.15.170:0/3168570770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc784157140 con 0x7fc7840b5410 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc789438640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 0x7fc784154e90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc7800b0040 tx=0x7fc780005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800b2040 con 0x7fc7840b5410 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc780004330 con 0x7fc7840b5410 2024-09-21T02:21:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780005290 con 0x7fc7840b5410 2024-09-21T02:21:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7841553d0 con 0x7fc7840b5410 2024-09-21T02:21:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.733+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc784154690 con 0x7fc7840b5410 2024-09-21T02:21:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.737+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7841544a0 con 0x7fc7840b5410 2024-09-21T02:21:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.737+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fc780005430 con 0x7fc7840b5410 2024-09-21T02:21:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.737+0000 7fc77a7fc640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc7680738f0 0x7fc768075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:04.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.737+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7fc78012ed50 con 0x7fc7840b5410 2024-09-21T02:21:04.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.737+0000 7fc789c39640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc7680738f0 0x7fc768075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:04.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.737+0000 7fc789c39640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc7680738f0 0x7fc768075e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ad0 tx=0x7fc7740091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.741+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc7800be060 con 0x7fc7840b5410 2024-09-21T02:21:04.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.913+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7840b3350 con 0x7fc7840b5410 2024-09-21T02:21:04.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc77a7fc640 1 -- 172.21.15.170:0/3168570770 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+13134 (secure 0 0 0) 0x7fc7800164d0 con 0x7fc7840b5410 2024-09-21T02:21:04.917 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:04.917 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":134,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:03.533717+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6835/169289834":"2024-09-22T02:21:00.515250+0000","172.21.15.191:6834/169289834":"2024-09-22T02:21:00.515250+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:04.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc7680738f0 msgr2=0x7fc768075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc7680738f0 0x7fc768075e00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ad0 tx=0x7fc7740091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 msgr2=0x7fc784154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:04.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 0x7fc784154e90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc7800b0040 tx=0x7fc780005d60 comp rx=0 tx=0).stop 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 shutdown_connections 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc7680738f0 0x7fc768075e00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7840ccb00 0x7fc784155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840b5410 0x7fc784154e90 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 --2- 172.21.15.170:0/3168570770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7840b2f40 0x7fc784156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 >> 172.21.15.170:0/3168570770 conn(0x7fc7840b1c40 msgr2=0x7fc7840b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 shutdown_connections 2024-09-21T02:21:04.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:04.917+0000 7fc78be9d640 1 -- 172.21.15.170:0/3168570770 wait complete. 2024-09-21T02:21:04.938 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:21:04.938 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:21:05.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc456e8640 1 -- 172.21.15.170:0/3006051575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc28004e70 msgr2=0x7fbc280052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3006051575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc28004e70 0x7fbc280052e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fbc400b60f0 tx=0x7fbc3801d260 comp rx=0 tx=0).stop 2024-09-21T02:21:05.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc3f7fe640 1 -- 172.21.15.170:0/3006051575 reap_dead start 2024-09-21T02:21:05.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc456e8640 1 -- 172.21.15.170:0/3006051575 shutdown_connections 2024-09-21T02:21:05.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc456e8640 1 -- 172.21.15.170:0/3006051575 >> 172.21.15.170:0/3006051575 conn(0x7fbc400d0170 msgr2=0x7fbc400d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:05.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc456e8640 1 -- 172.21.15.170:0/3006051575 shutdown_connections 2024-09-21T02:21:05.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.097+0000 7fbc456e8640 1 -- 172.21.15.170:0/3006051575 wait complete. 2024-09-21T02:21:05.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 Processor -- start 2024-09-21T02:21:05.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- start start 2024-09-21T02:21:05.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40145750 0x7fbc4009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc400b1d20 0x7fbc400b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 0x7fbc401f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc400b3550 con 0x7fbc401f6f90 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc400b33d0 con 0x7fbc40145750 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4006ff50 con 0x7fbc400b1d20 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 0x7fbc401f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 0x7fbc401f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54054/0 (socket says 172.21.15.170:54054) 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 -- 172.21.15.170:0/3282820151 learned_addr learned my addr 172.21.15.170:0/3282820151 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3e7fc640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc400b1d20 0x7fbc400b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3effd640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40145750 0x7fbc4009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 -- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc400b1d20 msgr2=0x7fbc400b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc400b1d20 0x7fbc400b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 -- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40145750 msgr2=0x7fbc4009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40145750 0x7fbc4009a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 -- 172.21.15.170:0/3282820151 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc400b2670 con 0x7fbc401f6f90 2024-09-21T02:21:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc3f7fe640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 0x7fbc401f73a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801b960 tx=0x7fbc380014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38001210 con 0x7fbc401f6f90 2024-09-21T02:21:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc38006070 con 0x7fbc401f6f90 2024-09-21T02:21:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4009b940 con 0x7fbc401f6f90 2024-09-21T02:21:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38006440 con 0x7fbc401f6f90 2024-09-21T02:21:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4009b660 con 0x7fbc401f6f90 2024-09-21T02:21:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.101+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc400704e0 con 0x7fbc401f6f90 2024-09-21T02:21:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.105+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fbc380065e0 con 0x7fbc401f6f90 2024-09-21T02:21:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.105+0000 7fbc277fe640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc2c0738c0 0x7fbc2c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.105+0000 7fbc3effd640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc2c0738c0 0x7fbc2c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.105+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7fbc3812f8a0 con 0x7fbc401f6f90 2024-09-21T02:21:05.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.105+0000 7fbc3effd640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc2c0738c0 0x7fbc2c075dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbc30000c10 tx=0x7fbc30006b50 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:05.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.109+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbc400b2670 con 0x7fbc401f6f90 2024-09-21T02:21:05.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.273+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc400b3c00 con 0x7fbc401f6f90 2024-09-21T02:21:05.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc277fe640 1 -- 172.21.15.170:0/3282820151 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7fbc38017010 con 0x7fbc401f6f90 2024-09-21T02:21:05.275 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:21:05.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc2c0738c0 msgr2=0x7fbc2c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc2c0738c0 0x7fbc2c075dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbc30000c10 tx=0x7fbc30006b50 comp rx=0 tx=0).stop 2024-09-21T02:21:05.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 msgr2=0x7fbc401f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 0x7fbc401f73a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801b960 tx=0x7fbc380014a0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 shutdown_connections 2024-09-21T02:21:05.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbc2c0738c0 0x7fbc2c075dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc400b1d20 0x7fbc400b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc401f6f90 0x7fbc401f73a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 --2- 172.21.15.170:0/3282820151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40145750 0x7fbc4009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 >> 172.21.15.170:0/3282820151 conn(0x7fbc400d0170 msgr2=0x7fbc4015f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:05.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 shutdown_connections 2024-09-21T02:21:05.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.277+0000 7fbc456e8640 1 -- 172.21.15.170:0/3282820151 wait complete. 2024-09-21T02:21:05.297 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-21T02:21:05.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b96cf640 1 -- 172.21.15.170:0/2666826882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a0004d50 msgr2=0x7f05a0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b96cf640 1 --2- 172.21.15.170:0/2666826882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a0004d50 0x7f05a0005160 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f05a400afc0 tx=0x7f05a401e430 comp rx=0 tx=0).stop 2024-09-21T02:21:05.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b37fe640 1 -- 172.21.15.170:0/2666826882 reap_dead start 2024-09-21T02:21:05.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b96cf640 1 -- 172.21.15.170:0/2666826882 shutdown_connections 2024-09-21T02:21:05.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b96cf640 1 -- 172.21.15.170:0/2666826882 >> 172.21.15.170:0/2666826882 conn(0x7f05b40b1c40 msgr2=0x7f05b40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:05.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b96cf640 1 -- 172.21.15.170:0/2666826882 shutdown_connections 2024-09-21T02:21:05.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.449+0000 7f05b96cf640 1 -- 172.21.15.170:0/2666826882 wait complete. 2024-09-21T02:21:05.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 Processor -- start 2024-09-21T02:21:05.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- start start 2024-09-21T02:21:05.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 0x7f05b41f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41f96a0 0x7f05b415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415e0e0 0x7f05b41f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4074e70 con 0x7f05b415bed0 2024-09-21T02:21:05.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f05b4074cf0 con 0x7f05b415e0e0 2024-09-21T02:21:05.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f05b40ce8d0 con 0x7f05b41f96a0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 0x7f05b41f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 0x7f05b41f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52702/0 (socket says 172.21.15.170:52702) 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 -- 172.21.15.170:0/1838101339 learned_addr learned my addr 172.21.15.170:0/1838101339 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b37fe640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415e0e0 0x7f05b41f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b27fc640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41f96a0 0x7f05b415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 -- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41f96a0 msgr2=0x7f05b415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41f96a0 0x7f05b415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 -- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415e0e0 msgr2=0x7f05b41f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415e0e0 0x7f05b41f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 -- 172.21.15.170:0/1838101339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b41f9ab0 con 0x7f05b415bed0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b37fe640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415e0e0 0x7f05b41f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b2ffd640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 0x7f05b41f9160 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00ca70 tx=0x7f05ac00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac017070 con 0x7f05b415bed0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05ac0036e0 con 0x7f05b415bed0 2024-09-21T02:21:05.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b415e4f0 con 0x7f05b415bed0 2024-09-21T02:21:05.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac003b10 con 0x7f05b415bed0 2024-09-21T02:21:05.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b415ea80 con 0x7f05b415bed0 2024-09-21T02:21:05.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.453+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b409ba10 con 0x7f05b415bed0 2024-09-21T02:21:05.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.457+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f05ac01a070 con 0x7f05b415bed0 2024-09-21T02:21:05.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.457+0000 7f05ab7fe640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05940738c0 0x7f0594075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:05.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.457+0000 7f05b27fc640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05940738c0 0x7f0594075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:05.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.457+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7f05ac094320 con 0x7f05b415bed0 2024-09-21T02:21:05.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.457+0000 7f05b27fc640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05940738c0 0x7f0594075dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f05b40af760 tx=0x7f059c0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:05.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.461+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f05b41f9ab0 con 0x7f05b415bed0 2024-09-21T02:21:05.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:05.693+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f05b40cb200 con 0x7f05b415bed0 2024-09-21T02:21:06.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.549+0000 7f05ab7fe640 1 -- 172.21.15.170:0/1838101339 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v408) ==== 152+0+0 (secure 0 0 0) 0x7f05b40cb740 con 0x7f05b415bed0 2024-09-21T02:21:06.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05940738c0 msgr2=0x7f0594075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05940738c0 0x7f0594075dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f05b40af760 tx=0x7f059c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 msgr2=0x7f05b41f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 0x7f05b41f9160 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00ca70 tx=0x7f05ac00cf30 comp rx=0 tx=0).stop 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 shutdown_connections 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f05940738c0 0x7f0594075dd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41f96a0 0x7f05b415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415e0e0 0x7f05b41f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 --2- 172.21.15.170:0/1838101339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b415bed0 0x7f05b41f9160 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 >> 172.21.15.170:0/1838101339 conn(0x7f05b40b1c40 msgr2=0x7f05b40aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 shutdown_connections 2024-09-21T02:21:06.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.553+0000 7f05b96cf640 1 -- 172.21.15.170:0/1838101339 wait complete. 2024-09-21T02:21:06.579 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 -- 172.21.15.170:0/3865175394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a40b5410 msgr2=0x7fb3a406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/3865175394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a40b5410 0x7fb3a406fa30 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fb394009900 tx=0x7fb39401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 -- 172.21.15.170:0/3865175394 shutdown_connections 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/3865175394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40cd210 0x7fb3a40b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/3865175394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a40b5410 0x7fb3a406fa30 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 -- 172.21.15.170:0/3865175394 >> 172.21.15.170:0/3865175394 conn(0x7fb3a40b1c40 msgr2=0x7fb3a40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 -- 172.21.15.170:0/3865175394 shutdown_connections 2024-09-21T02:21:06.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 -- 172.21.15.170:0/3865175394 wait complete. 2024-09-21T02:21:06.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 Processor -- start 2024-09-21T02:21:06.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.741+0000 7fb3a9b97640 1 -- start start 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40b2f40 0x7fb3a4153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 0x7fb3a40adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a409c7d0 0x7fb3a40ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a416a060 con 0x7fb3a41f66d0 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40701b0 con 0x7fb3a409c7d0 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a416a1e0 con 0x7fb3a40b2f40 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 0x7fb3a40adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 0x7fb3a40adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52720/0 (socket says 172.21.15.170:52720) 2024-09-21T02:21:06.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 -- 172.21.15.170:0/1935448011 learned_addr learned my addr 172.21.15.170:0/1935448011 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a37fe640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40b2f40 0x7fb3a4153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a3fff640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a409c7d0 0x7fb3a40ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 -- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40b2f40 msgr2=0x7fb3a4153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40b2f40 0x7fb3a4153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 -- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a409c7d0 msgr2=0x7fb3a40ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a409c7d0 0x7fb3a40ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 -- 172.21.15.170:0/1935448011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a4153b60 con 0x7fb3a41f66d0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a37fe640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40b2f40 0x7fb3a4153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a3fff640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a409c7d0 0x7fb3a40ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a2ffd640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 0x7fb3a40adcf0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb39800c420 tx=0x7fb39800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980052d0 con 0x7fb3a41f66d0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb398007020 con 0x7fb3a41f66d0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a409d6b0 con 0x7fb3a41f66d0 2024-09-21T02:21:06.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398007470 con 0x7fb3a41f66d0 2024-09-21T02:21:06.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a409add0 con 0x7fb3a41f66d0 2024-09-21T02:21:06.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a409d960 con 0x7fb3a41f66d0 2024-09-21T02:21:06.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fb3980067e0 con 0x7fb3a41f66d0 2024-09-21T02:21:06.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.745+0000 7fb38bfff640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3900738f0 0x7fb390075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:06.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.749+0000 7fb3a37fe640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3900738f0 0x7fb390075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:06.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.749+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7fb398094100 con 0x7fb3a41f66d0 2024-09-21T02:21:06.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.749+0000 7fb3a37fe640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3900738f0 0x7fb390075e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb3940af000 tx=0x7fb39401b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:06.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.753+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb39805e960 con 0x7fb3a41f66d0 2024-09-21T02:21:06.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.925+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb3a406fa30 con 0x7fb3a41f66d0 2024-09-21T02:21:06.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.925+0000 7fb38bfff640 1 -- 172.21.15.170:0/1935448011 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+2294 (secure 0 0 0) 0x7fb398001090 con 0x7fb3a41f66d0 2024-09-21T02:21:06.927 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:21:06.927 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:21:06.927 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:21:06.927 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:21:06.928 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:21:06.929 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:21:06.930 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3900738f0 msgr2=0x7fb390075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3900738f0 0x7fb390075e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb3940af000 tx=0x7fb39401b120 comp rx=0 tx=0).stop 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 msgr2=0x7fb3a40adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 0x7fb3a40adcf0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb39800c420 tx=0x7fb39800c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 shutdown_connections 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3900738f0 0x7fb390075e00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a409c7d0 0x7fb3a40ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a41f66d0 0x7fb3a40adcf0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 --2- 172.21.15.170:0/1935448011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a40b2f40 0x7fb3a4153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 >> 172.21.15.170:0/1935448011 conn(0x7fb3a40b1c40 msgr2=0x7fb3a40c73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 shutdown_connections 2024-09-21T02:21:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:06.929+0000 7fb3a9b97640 1 -- 172.21.15.170:0/1935448011 wait complete. 2024-09-21T02:21:06.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:21:07.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 -- 172.21.15.170:0/2248481603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0144004810 msgr2=0x7f0144004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 --2- 172.21.15.170:0/2248481603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0144004810 0x7f0144004c20 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f01480016c0 tx=0x7f014801c790 comp rx=0 tx=0).stop 2024-09-21T02:21:07.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015e265640 1 -- 172.21.15.170:0/2248481603 reap_dead start 2024-09-21T02:21:07.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 -- 172.21.15.170:0/2248481603 shutdown_connections 2024-09-21T02:21:07.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 -- 172.21.15.170:0/2248481603 >> 172.21.15.170:0/2248481603 conn(0x7f01580d2a30 msgr2=0x7f01580cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:07.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 -- 172.21.15.170:0/2248481603 shutdown_connections 2024-09-21T02:21:07.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 -- 172.21.15.170:0/2248481603 wait complete. 2024-09-21T02:21:07.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 Processor -- start 2024-09-21T02:21:07.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.109+0000 7f015fcc8640 1 -- start start 2024-09-21T02:21:07.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 0x7f015806f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01580cb6f0 0x7f01580ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01580cbb00 0x7f01580ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0158071820 con 0x7f01580b0c90 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f01580719a0 con 0x7f01580cb6f0 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0158160af0 con 0x7f01580cbb00 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 0x7f015806f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 0x7f015806f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52744/0 (socket says 172.21.15.170:52744) 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 -- 172.21.15.170:0/1230957268 learned_addr learned my addr 172.21.15.170:0/1230957268 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015d263640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01580cb6f0 0x7f01580ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015e265640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01580cbb00 0x7f01580ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 -- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01580cbb00 msgr2=0x7f01580ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01580cbb00 0x7f01580ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 -- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01580cb6f0 msgr2=0x7f01580ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01580cb6f0 0x7f01580ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 -- 172.21.15.170:0/1230957268 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01580b8370 con 0x7f01580b0c90 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015e265640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01580cbb00 0x7f01580ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:07.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015da64640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 0x7f015806f340 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f015400e940 tx=0x7f015400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015400ccc0 con 0x7f01580b0c90 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0154005020 con 0x7f01580b0c90 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581f5ad0 con 0x7f01580b0c90 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154005470 con 0x7f01580b0c90 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01580b8090 con 0x7f01580b0c90 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.113+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015815d3e0 con 0x7f01580b0c90 2024-09-21T02:21:07.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.117+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f01540047b0 con 0x7f01580b0c90 2024-09-21T02:21:07.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.117+0000 7f014e7fc640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f013c0738c0 0x7f013c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.117+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7f01540958c0 con 0x7f01580b0c90 2024-09-21T02:21:07.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.117+0000 7f015d263640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f013c0738c0 0x7f013c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.117+0000 7f015d263640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f013c0738c0 0x7f013c075dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01580cf6c0 tx=0x7f01480a7040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:07.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.121+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f01540603b0 con 0x7f01580b0c90 2024-09-21T02:21:07.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.289+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0158154490 con 0x7f01580b0c90 2024-09-21T02:21:07.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.293+0000 7f014e7fc640 1 -- 172.21.15.170:0/1230957268 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f015405fcb0 con 0x7f01580b0c90 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f013c0738c0 msgr2=0x7f013c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f013c0738c0 0x7f013c075dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01580cf6c0 tx=0x7f01480a7040 comp rx=0 tx=0).stop 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 msgr2=0x7f015806f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 0x7f015806f340 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f015400e940 tx=0x7f015400ee00 comp rx=0 tx=0).stop 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 shutdown_connections 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f013c0738c0 0x7f013c075dd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01580cbb00 0x7f01580ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01580cb6f0 0x7f01580ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 --2- 172.21.15.170:0/1230957268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01580b0c90 0x7f015806f340 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 >> 172.21.15.170:0/1230957268 conn(0x7f01580d2a30 msgr2=0x7f0158076120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 shutdown_connections 2024-09-21T02:21:07.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.297+0000 7f015fcc8640 1 -- 172.21.15.170:0/1230957268 wait complete. 2024-09-21T02:21:07.318 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:21:07.318 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:21:07.417 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:21:07.495 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:21:07.495 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:07.495 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:07.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:21:07.501 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:07.502 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:07.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:21:07.508 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:07.508 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:07.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:21:07.549 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:07.549 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:07.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:21:07.554 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:21:07.555 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 -- 172.21.15.170:0/2112627741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff750004810 msgr2=0x7ff750004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 --2- 172.21.15.170:0/2112627741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff750004810 0x7ff750004c20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009b60 tx=0x7ff75c01d270 comp rx=0 tx=0).stop 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 -- 172.21.15.170:0/2112627741 shutdown_connections 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 --2- 172.21.15.170:0/2112627741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff750004810 0x7ff750004c20 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 -- 172.21.15.170:0/2112627741 >> 172.21.15.170:0/2112627741 conn(0x7ff7680b1c40 msgr2=0x7ff7680b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 -- 172.21.15.170:0/2112627741 shutdown_connections 2024-09-21T02:21:07.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.756+0000 7ff76d86c640 1 -- 172.21.15.170:0/2112627741 wait complete. 2024-09-21T02:21:07.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 Processor -- start 2024-09-21T02:21:07.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- start start 2024-09-21T02:21:07.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff750004810 0x7ff7681540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 0x7ff7681f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7680c9080 0x7ff7680c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680cb9f0 con 0x7ff7681f6930 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680cb870 con 0x7ff7680c9080 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff7680cbb70 con 0x7ff750004810 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 0x7ff7681f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 0x7ff7681f9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52768/0 (socket says 172.21.15.170:52768) 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 -- 172.21.15.170:0/870288287 learned_addr learned my addr 172.21.15.170:0/870288287 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7677fe640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7680c9080 0x7ff7680c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff766ffd640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff750004810 0x7ff7681540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 -- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff750004810 msgr2=0x7ff7681540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff750004810 0x7ff7681540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 -- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7680c9080 msgr2=0x7ff7680c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7680c9080 0x7ff7680c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 -- 172.21.15.170:0/870288287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7680c99d0 con 0x7ff7681f6930 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7677fe640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7680c9080 0x7ff7680c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:07.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff7667fc640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 0x7ff7681f9d40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff75c01d0a0 tx=0x7ff75c0056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:07.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0b1040 con 0x7ff7681f6930 2024-09-21T02:21:07.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff75c0ac7b0 con 0x7ff7681f6930 2024-09-21T02:21:07.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76809b5d0 con 0x7ff7681f6930 2024-09-21T02:21:07.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0acbc0 con 0x7ff7681f6930 2024-09-21T02:21:07.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681f6130 con 0x7ff7681f6930 2024-09-21T02:21:07.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.760+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76809b340 con 0x7ff7681f6930 2024-09-21T02:21:07.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.764+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7ff75c0be020 con 0x7ff7681f6930 2024-09-21T02:21:07.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.764+0000 7ff74ffff640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff7540738c0 0x7ff754075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:07.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.764+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7ff75c12ff40 con 0x7ff7681f6930 2024-09-21T02:21:07.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.764+0000 7ff766ffd640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff7540738c0 0x7ff754075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:07.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.764+0000 7ff766ffd640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff7540738c0 0x7ff754075dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff75800e970 tx=0x7ff758001520 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:07.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.768+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff75c0c5020 con 0x7ff7681f6930 2024-09-21T02:21:07.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.968+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7680cbd50 con 0x7ff7681f6930 2024-09-21T02:21:07.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff74ffff640 1 -- 172.21.15.170:0/870288287 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+3923 (secure 0 0 0) 0x7ff7680c99d0 con 0x7ff7681f6930 2024-09-21T02:21:07.972 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:07.972 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":93,"btime":"2024-09-21T02:21:01:518784+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1927593685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1927593685},{"type":"v1","addr":"172.21.15.170:6835","nonce":1927593685}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1117687547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1117687547},{"type":"v1","addr":"172.21.15.191:6801","nonce":1117687547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3635501462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3635501462},{"type":"v1","addr":"172.21.15.191:6835","nonce":3635501462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2853899592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2853899592},{"type":"v1","addr":"172.21.15.170:6833","nonce":2853899592}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[]} 2024-09-21T02:21:07.972 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 93 2024-09-21T02:21:07.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff7540738c0 msgr2=0x7ff754075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff7540738c0 0x7ff754075dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff75800e970 tx=0x7ff758001520 comp rx=0 tx=0).stop 2024-09-21T02:21:07.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 msgr2=0x7ff7681f9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:07.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 0x7ff7681f9d40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff75c01d0a0 tx=0x7ff75c0056c0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 shutdown_connections 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff7540738c0 0x7ff754075dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7680c9080 0x7ff7680c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7681f6930 0x7ff7681f9d40 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 --2- 172.21.15.170:0/870288287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff750004810 0x7ff7681540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 >> 172.21.15.170:0/870288287 conn(0x7ff7680b1c40 msgr2=0x7ff7680adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 shutdown_connections 2024-09-21T02:21:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:07.972+0000 7ff76d86c640 1 -- 172.21.15.170:0/870288287 wait complete. 2024-09-21T02:21:07.993 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:21:07.993 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:21:07.993 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:21:07.993 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:07.993 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:21:07.993 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:21:07.993 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:21:07.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:07.994 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:21:07.994 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:21:07.994 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:21:07.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:07.994 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:21:07.994 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:21:07.995 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:21:07.995 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:08.007 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:21:08.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:21:08.009 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:21:08.009 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:21:08.010 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:21:08.010 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:21:08.015 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:21:08.015 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:21:08.052 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:21:08.052 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:08.053 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:21:08.053 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:08.058 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:21:08.059 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:08.060 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:21:08.060 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:08.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:21:08.141 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:21:08.144 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:21:08.158 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:21:08.159 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 -- 172.21.15.170:0/1745294681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cb460 msgr2=0x7fb3140cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 --2- 172.21.15.170:0/1745294681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cb460 0x7fb3140cb870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb308009840 tx=0x7fb30801c740 comp rx=0 tx=0).stop 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 -- 172.21.15.170:0/1745294681 shutdown_connections 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 --2- 172.21.15.170:0/1745294681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cb460 0x7fb3140cb870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 --2- 172.21.15.170:0/1745294681 >> v2:172.21.15.191:3300/0 conn(0x7fb3140cf7e0 0x7fb314070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 -- 172.21.15.170:0/1745294681 >> 172.21.15.170:0/1745294681 conn(0x7fb3140b1c40 msgr2=0x7fb3140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 -- 172.21.15.170:0/1745294681 shutdown_connections 2024-09-21T02:21:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 -- 172.21.15.170:0/1745294681 wait complete. 2024-09-21T02:21:08.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 Processor -- start 2024-09-21T02:21:08.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.208+0000 7fb31967b640 1 -- start start 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3140cb460 0x7fb314146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 0x7fb3141544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cf7e0 0x7fb3141581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb31406f240 con 0x7fb3140cc8c0 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb31406f0c0 con 0x7fb3140cb460 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb31406f3c0 con 0x7fb3140cf7e0 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 0x7fb3141544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 0x7fb3141544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52808/0 (socket says 172.21.15.170:52808) 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 -- 172.21.15.170:0/3211490448 learned_addr learned my addr 172.21.15.170:0/3211490448 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3137fe640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cf7e0 0x7fb3141581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb312ffd640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3140cb460 0x7fb314146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 -- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cf7e0 msgr2=0x7fb3141581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cf7e0 0x7fb3141581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 -- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3140cb460 msgr2=0x7fb314146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3140cb460 0x7fb314146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 -- 172.21.15.170:0/3211490448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb314155250 con 0x7fb3140cc8c0 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb312ffd640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3140cb460 0x7fb314146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb3127fc640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 0x7fb3141544e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb308009790 tx=0x7fb308005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3080ac030 con 0x7fb3140cc8c0 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb308004ac0 con 0x7fb3140cc8c0 2024-09-21T02:21:08.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb314155530 con 0x7fb3140cc8c0 2024-09-21T02:21:08.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3080a6700 con 0x7fb3140cc8c0 2024-09-21T02:21:08.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31415c440 con 0x7fb3140cc8c0 2024-09-21T02:21:08.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb314154fc0 con 0x7fb3140cc8c0 2024-09-21T02:21:08.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fb3080af070 con 0x7fb3140cc8c0 2024-09-21T02:21:08.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb2fb7fe640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3000738c0 0x7fb300075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4971+0+0 (secure 0 0 0) 0x7fb30812e7d0 con 0x7fb3140cc8c0 2024-09-21T02:21:08.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.212+0000 7fb312ffd640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3000738c0 0x7fb300075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.216+0000 7fb312ffd640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3000738c0 0x7fb300075dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb304009b90 tx=0x7fb3040091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:08.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.220+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb3080bf070 con 0x7fb3140cc8c0 2024-09-21T02:21:08.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.388+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb314155700 con 0x7fb3140cc8c0 2024-09-21T02:21:08.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb2fb7fe640 1 -- 172.21.15.170:0/3211490448 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) ==== 97+0+0 (secure 0 0 0) 0x7fb308016330 con 0x7fb3140cc8c0 2024-09-21T02:21:08.584 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:21:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3000738c0 msgr2=0x7fb300075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3000738c0 0x7fb300075dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb304009b90 tx=0x7fb3040091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 msgr2=0x7fb3141544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 0x7fb3141544e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb308009790 tx=0x7fb308005750 comp rx=0 tx=0).stop 2024-09-21T02:21:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 shutdown_connections 2024-09-21T02:21:08.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb3000738c0 0x7fb300075dd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3140cf7e0 0x7fb3141581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3140cc8c0 0x7fb3141544e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 --2- 172.21.15.170:0/3211490448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3140cb460 0x7fb314146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 >> 172.21.15.170:0/3211490448 conn(0x7fb3140b1c40 msgr2=0x7fb3140ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:08.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 shutdown_connections 2024-09-21T02:21:08.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.584+0000 7fb31967b640 1 -- 172.21.15.170:0/3211490448 wait complete. 2024-09-21T02:21:08.611 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:21:08.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 -- 172.21.15.170:0/1880920734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b2f40 msgr2=0x7f312c0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1880920734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b2f40 0x7f312c0b3350 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f312c071d80 tx=0x7f311c01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 -- 172.21.15.170:0/1880920734 shutdown_connections 2024-09-21T02:21:08.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1880920734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b2f40 0x7f312c0b3350 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1880920734 >> v2:172.21.15.191:3300/0 conn(0x7f312c0b5410 0x7f312c06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 -- 172.21.15.170:0/1880920734 >> 172.21.15.170:0/1880920734 conn(0x7f312c0b1c40 msgr2=0x7f312c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:08.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 -- 172.21.15.170:0/1880920734 shutdown_connections 2024-09-21T02:21:08.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 -- 172.21.15.170:0/1880920734 wait complete. 2024-09-21T02:21:08.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 Processor -- start 2024-09-21T02:21:08.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.760+0000 7f3130fc4640 1 -- start start 2024-09-21T02:21:08.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f312c0b2f40 0x7f312c1f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 0x7f312c0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f312c0ccb00 0x7f312c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f312c169fd0 con 0x7f312c0b5410 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f312c169e50 con 0x7f312c0b2f40 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f312c16a150 con 0x7f312c0ccb00 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 0x7f312c0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 0x7f312c0c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52824/0 (socket says 172.21.15.170:52824) 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 -- 172.21.15.170:0/1415006792 learned_addr learned my addr 172.21.15.170:0/1415006792 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 -- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f312c0ccb00 msgr2=0x7f312c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f312c0ccb00 0x7f312c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 -- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f312c0b2f40 msgr2=0x7f312c1f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f312a59c640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f312c0b2f40 0x7f312c1f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f312c0b2f40 0x7f312c1f9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:08.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 -- 172.21.15.170:0/1415006792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c09db40 con 0x7f312c0b5410 2024-09-21T02:21:08.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3129d9b640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 0x7f312c0c4f10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f311c0b0040 tx=0x7f311c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:08.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c0b2040 con 0x7f312c0b5410 2024-09-21T02:21:08.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f311c0a5c30 con 0x7f312c0b5410 2024-09-21T02:21:08.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1f9b60 con 0x7f312c0b5410 2024-09-21T02:21:08.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c003860 con 0x7f312c0b5410 2024-09-21T02:21:08.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c09ddb0 con 0x7f312c0b5410 2024-09-21T02:21:08.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c0b3350 con 0x7f312c0b5410 2024-09-21T02:21:08.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.764+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f311c003a00 con 0x7f312c0b5410 2024-09-21T02:21:08.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.768+0000 7f3122ffd640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f310c0738c0 0x7f310c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:08.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.768+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4885+0+0 (secure 0 0 0) 0x7f311c12e2f0 con 0x7f312c0b5410 2024-09-21T02:21:08.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.768+0000 7f312a59c640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f310c0738c0 0x7f310c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:08.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.768+0000 7f312a59c640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f310c0738c0 0x7f310c075dd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f31140058d0 tx=0x7f3114005bd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:08.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:08.772+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f312c1f9b60 con 0x7f312c0b5410 2024-09-21T02:21:09.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.004+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f312c0b4670 con 0x7f312c0b5410 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3122ffd640 1 -- 172.21.15.170:0/1415006792 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) ==== 80+0+5304 (secure 0 0 0) 0x7f311c0bc060 con 0x7f312c0b5410 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:21:09.008 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:09.009 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.011 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.012 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:21:09.013 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.017 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.019 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:21:09.021 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f310c0738c0 msgr2=0x7f310c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f310c0738c0 0x7f310c075dd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f31140058d0 tx=0x7f3114005bd0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 msgr2=0x7f312c0c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 0x7f312c0c4f10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f311c0b0040 tx=0x7f311c005790 comp rx=0 tx=0).stop 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 shutdown_connections 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f310c0738c0 0x7f310c075dd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f312c0ccb00 0x7f312c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312c0b5410 0x7f312c0c4f10 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 --2- 172.21.15.170:0/1415006792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f312c0b2f40 0x7f312c1f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.008+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 >> 172.21.15.170:0/1415006792 conn(0x7f312c0b1c40 msgr2=0x7f312c0c8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:09.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.012+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 shutdown_connections 2024-09-21T02:21:09.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.012+0000 7f3130fc4640 1 -- 172.21.15.170:0/1415006792 wait complete. 2024-09-21T02:21:09.032 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:21:09.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b30ca640 1 -- 172.21.15.170:0/544900551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff998005160 msgr2=0x7ff99800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/544900551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff998005160 0x7ff99800b9d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c01f6a0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b1667640 1 -- 172.21.15.170:0/544900551 reap_dead start 2024-09-21T02:21:09.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b30ca640 1 -- 172.21.15.170:0/544900551 shutdown_connections 2024-09-21T02:21:09.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b30ca640 1 -- 172.21.15.170:0/544900551 >> 172.21.15.170:0/544900551 conn(0x7ff9ac09a9c0 msgr2=0x7ff9ac0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:09.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b30ca640 1 -- 172.21.15.170:0/544900551 shutdown_connections 2024-09-21T02:21:09.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.184+0000 7ff9b30ca640 1 -- 172.21.15.170:0/544900551 wait complete. 2024-09-21T02:21:09.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 Processor -- start 2024-09-21T02:21:09.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 -- start start 2024-09-21T02:21:09.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac15d100 0x7ff9ac1f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0b57c0 0x7ff9ac0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 0x7ff9ac15b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac15eda0 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac15ec20 con 0x7ff9ac0b57c0 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ac06d510 con 0x7ff9ac15d100 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 0x7ff9ac15b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 0x7ff9ac15b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52848/0 (socket says 172.21.15.170:52848) 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 -- 172.21.15.170:0/31330354 learned_addr learned my addr 172.21.15.170:0/31330354 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b0e66640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac15d100 0x7ff9ac1f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 -- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac15d100 msgr2=0x7ff9ac1f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9abfff640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0b57c0 0x7ff9ac0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac15d100 0x7ff9ac1f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 -- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0b57c0 msgr2=0x7ff9ac0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0b57c0 0x7ff9ac0b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 -- 172.21.15.170:0/31330354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac0b6110 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b1667640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 0x7ff9ac15b9c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff99c01dda0 tx=0x7ff99c01f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0be070 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff99c006510 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac15d5e0 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b3040 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.188+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac15dba0 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.192+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac0ae910 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.192+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7ff99c0c1070 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.192+0000 7ff9a97fa640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff98c0738c0 0x7ff98c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.192+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4885+0+0 (secure 0 0 0) 0x7ff99c131870 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.192+0000 7ff9b0e66640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff98c0738c0 0x7ff98c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.192+0000 7ff9b0e66640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff98c0738c0 0x7ff98c075dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4000f90 tx=0x7ff9a4006e20 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:09.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.196+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff9ac0b6110 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.372+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff9ac0cee40 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.372+0000 7ff9a97fa640 1 -- 172.21.15.170:0/31330354 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v138) ==== 84+0+65 (secure 0 0 0) 0x7ff99c019450 con 0x7ff9ac15b5b0 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:09.374 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:21:09.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff98c0738c0 msgr2=0x7ff98c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff98c0738c0 0x7ff98c075dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4000f90 tx=0x7ff9a4006e20 comp rx=0 tx=0).stop 2024-09-21T02:21:09.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 msgr2=0x7ff9ac15b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 0x7ff9ac15b9c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff99c01dda0 tx=0x7ff99c01f8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 shutdown_connections 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff98c0738c0 0x7ff98c075dd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9ac15b5b0 0x7ff9ac15b9c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac0b57c0 0x7ff9ac0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 --2- 172.21.15.170:0/31330354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9ac15d100 0x7ff9ac1f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 >> 172.21.15.170:0/31330354 conn(0x7ff9ac09a9c0 msgr2=0x7ff9ac0af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 shutdown_connections 2024-09-21T02:21:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.376+0000 7ff9b30ca640 1 -- 172.21.15.170:0/31330354 wait complete. 2024-09-21T02:21:09.396 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:21:09.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 -- 172.21.15.170:0/2865609500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b52d0 msgr2=0x7f02ec0af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 --2- 172.21.15.170:0/2865609500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b52d0 0x7f02ec0af440 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f02ec072900 tx=0x7f02e801c810 comp rx=0 tx=0).stop 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 -- 172.21.15.170:0/2865609500 shutdown_connections 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 --2- 172.21.15.170:0/2865609500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec0cd1f0 0x7f02ec06ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 --2- 172.21.15.170:0/2865609500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b52d0 0x7f02ec0af440 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 -- 172.21.15.170:0/2865609500 >> 172.21.15.170:0/2865609500 conn(0x7f02ec0b1c40 msgr2=0x7f02ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:09.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 -- 172.21.15.170:0/2865609500 shutdown_connections 2024-09-21T02:21:09.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.556+0000 7f02f4784640 1 -- 172.21.15.170:0/2865609500 wait complete. 2024-09-21T02:21:09.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 Processor -- start 2024-09-21T02:21:09.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 -- start start 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 0x7f02ec155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec0b52d0 0x7f02ec0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec0cb890 0x7f02ec153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec16a000 con 0x7f02ec0b4d80 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec169e80 con 0x7f02ec0cb890 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec16a180 con 0x7f02ec0b52d0 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 0x7f02ec155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 0x7f02ec155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52874/0 (socket says 172.21.15.170:52874) 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 -- 172.21.15.170:0/2049072115 learned_addr learned my addr 172.21.15.170:0/2049072115 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2d21640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec0cb890 0x7f02ec153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f1d1f640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec0b52d0 0x7f02ec0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 -- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec0b52d0 msgr2=0x7f02ec0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec0b52d0 0x7f02ec0cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 -- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec0cb890 msgr2=0x7f02ec153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec0cb890 0x7f02ec153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 -- 172.21.15.170:0/2049072115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec15c570 con 0x7f02ec0b4d80 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f2520640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 0x7f02ec155280 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f02e801d0b0 tx=0x7f02e8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80b2040 con 0x7f02ec0b4d80 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02e8004330 con 0x7f02ec0b4d80 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec15c850 con 0x7f02ec0b4d80 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8005280 con 0x7f02ec0b4d80 2024-09-21T02:21:09.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.560+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec0c8010 con 0x7f02ec0b4d80 2024-09-21T02:21:09.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.564+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec15ca90 con 0x7f02ec0b4d80 2024-09-21T02:21:09.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.564+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f02e80be060 con 0x7f02ec0b4d80 2024-09-21T02:21:09.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.564+0000 7f02e2ffd640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f02d00738c0 0x7f02d0075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:09.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.564+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4885+0+0 (secure 0 0 0) 0x7f02e812e480 con 0x7f02ec0b4d80 2024-09-21T02:21:09.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.564+0000 7f02f1d1f640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f02d00738c0 0x7f02d0075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:09.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.564+0000 7f02f1d1f640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f02d00738c0 0x7f02d0075dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cc0c0 tx=0x7f02dc008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:09.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.568+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f02e80f3290 con 0x7f02ec0b4d80 2024-09-21T02:21:09.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:09.736+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f02ec0af440 con 0x7f02ec0b4d80 2024-09-21T02:21:10.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.608+0000 7f02e2ffd640 1 -- 172.21.15.170:0/2049072115 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v139) ==== 138+0+0 (secure 0 0 0) 0x7f02ec15ca90 con 0x7f02ec0b4d80 2024-09-21T02:21:10.608 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:21:10.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f02d00738c0 msgr2=0x7f02d0075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:10.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f02d00738c0 0x7f02d0075dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f02ec0cc0c0 tx=0x7f02dc008040 comp rx=0 tx=0).stop 2024-09-21T02:21:10.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 msgr2=0x7f02ec155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:10.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 0x7f02ec155280 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f02e801d0b0 tx=0x7f02e8005e10 comp rx=0 tx=0).stop 2024-09-21T02:21:10.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 shutdown_connections 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f02d00738c0 0x7f02d0075dd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec0b52d0 0x7f02ec0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec0cb890 0x7f02ec153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 --2- 172.21.15.170:0/2049072115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0b4d80 0x7f02ec155280 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 >> 172.21.15.170:0/2049072115 conn(0x7f02ec0b1c40 msgr2=0x7f02ec152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 shutdown_connections 2024-09-21T02:21:10.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.612+0000 7f02f4784640 1 -- 172.21.15.170:0/2049072115 wait complete. 2024-09-21T02:21:10.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 -- 172.21.15.170:0/3881347781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb790 msgr2=0x7f52200ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 --2- 172.21.15.170:0/3881347781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb790 0x7f52200ce5b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5208009900 tx=0x7f520801c850 comp rx=0 tx=0).stop 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 -- 172.21.15.170:0/3881347781 shutdown_connections 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 --2- 172.21.15.170:0/3881347781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb790 0x7f52200ce5b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 --2- 172.21.15.170:0/3881347781 >> v2:172.21.15.191:3300/0 conn(0x7f52200cb380 0x7f52200ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 -- 172.21.15.170:0/3881347781 >> 172.21.15.170:0/3881347781 conn(0x7f52200d0170 msgr2=0x7f52200d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 -- 172.21.15.170:0/3881347781 shutdown_connections 2024-09-21T02:21:10.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 -- 172.21.15.170:0/3881347781 wait complete. 2024-09-21T02:21:10.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 Processor -- start 2024-09-21T02:21:10.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.808+0000 7f52259fc640 1 -- start start 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f522006f9a0 0x7f5220164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb380 0x7f52201629f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 0x7f5220162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5220074640 con 0x7f52200cb790 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52200744c0 con 0x7f522006f9a0 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f522015d920 con 0x7f52200cb380 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 0x7f5220162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 0x7f5220162f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52890/0 (socket says 172.21.15.170:52890) 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 -- 172.21.15.170:0/99692854 learned_addr learned my addr 172.21.15.170:0/99692854 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521effd640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f522006f9a0 0x7f5220164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f5217fff640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb380 0x7f52201629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 -- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb380 msgr2=0x7f52201629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:10.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb380 0x7f52201629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 -- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f522006f9a0 msgr2=0x7f5220164ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f522006f9a0 0x7f5220164ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 -- 172.21.15.170:0/99692854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52200b46c0 con 0x7f52200cb790 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f5217fff640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb380 0x7f52201629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521f7fe640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 0x7f5220162f30 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f521800c730 tx=0x7f521800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218016020 con 0x7f52200cb790 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52180039a0 con 0x7f52200cb790 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52200ae1d0 con 0x7f52200cb790 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218003df0 con 0x7f52200cb790 2024-09-21T02:21:10.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52200ae790 con 0x7f52200cb790 2024-09-21T02:21:10.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.812+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52200b4930 con 0x7f52200cb790 2024-09-21T02:21:10.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.816+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f52180052e0 con 0x7f52200cb790 2024-09-21T02:21:10.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.816+0000 7f521cff9640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52000738c0 0x7f5200075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:10.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.816+0000 7f521effd640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52000738c0 0x7f5200075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:10.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.816+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5255+0+0 (secure 0 0 0) 0x7f5218095470 con 0x7f52200cb790 2024-09-21T02:21:10.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.816+0000 7f521effd640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52000738c0 0x7f5200075dd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5210009b40 tx=0x7f52100091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.820+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5218020020 con 0x7f52200cb790 2024-09-21T02:21:10.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:10.980+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f522006fdb0 con 0x7f52200cb790 2024-09-21T02:21:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.620+0000 7f521cff9640 1 -- 172.21.15.170:0/99692854 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v140) ==== 170+0+0 (secure 0 0 0) 0x7f521805fd40 con 0x7f52200cb790 2024-09-21T02:21:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:21:11.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52000738c0 msgr2=0x7f5200075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:11.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52000738c0 0x7f5200075dd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5210009b40 tx=0x7f52100091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 msgr2=0x7f5220162f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:11.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 0x7f5220162f30 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f521800c730 tx=0x7f521800cbf0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 shutdown_connections 2024-09-21T02:21:11.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f52000738c0 0x7f5200075dd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52200cb380 0x7f52201629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52200cb790 0x7f5220162f30 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 --2- 172.21.15.170:0/99692854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f522006f9a0 0x7f5220164ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 >> 172.21.15.170:0/99692854 conn(0x7f52200d0170 msgr2=0x7f52200b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 shutdown_connections 2024-09-21T02:21:11.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.624+0000 7f52259fc640 1 -- 172.21.15.170:0/99692854 wait complete. 2024-09-21T02:21:11.670 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:21:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 -- 172.21.15.170:0/3550007030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a300b2fc0 msgr2=0x7f0a300ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 --2- 172.21.15.170:0/3550007030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a300b2fc0 0x7f0a300ce540 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009870 tx=0x7f0a2c01c820 comp rx=0 tx=0).stop 2024-09-21T02:21:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 -- 172.21.15.170:0/3550007030 shutdown_connections 2024-09-21T02:21:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 --2- 172.21.15.170:0/3550007030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a300cea80 0x7f0a300cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 --2- 172.21.15.170:0/3550007030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a300b2fc0 0x7f0a300ce540 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 -- 172.21.15.170:0/3550007030 >> 172.21.15.170:0/3550007030 conn(0x7f0a300d0170 msgr2=0x7f0a300d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 -- 172.21.15.170:0/3550007030 shutdown_connections 2024-09-21T02:21:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.828+0000 7f0a3806c640 1 -- 172.21.15.170:0/3550007030 wait complete. 2024-09-21T02:21:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 Processor -- start 2024-09-21T02:21:11.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 -- start start 2024-09-21T02:21:11.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a3006ef40 0x7f0a300b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a300b2fc0 0x7f0a30153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 0x7f0a30153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30169fe0 con 0x7f0a300cea80 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300b35f0 con 0x7f0a3006ef40 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3016a160 con 0x7f0a300b2fc0 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 0x7f0a30153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 0x7f0a30153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52892/0 (socket says 172.21.15.170:52892) 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 -- 172.21.15.170:0/2976679590 learned_addr learned my addr 172.21.15.170:0/2976679590 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a35e08640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a3006ef40 0x7f0a300b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a35607640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a300b2fc0 0x7f0a30153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 -- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a300b2fc0 msgr2=0x7f0a30153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a300b2fc0 0x7f0a30153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 -- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a3006ef40 msgr2=0x7f0a300b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a3006ef40 0x7f0a300b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 -- 172.21.15.170:0/2976679590 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a300cc130 con 0x7f0a300cea80 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a36609640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 0x7f0a30153780 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800c760 tx=0x7f0a2800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28003b80 con 0x7f0a300cea80 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a28003d20 con 0x7f0a300cea80 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a301fa1a0 con 0x7f0a300cea80 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800aab0 con 0x7f0a300cea80 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.832+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a301f9ec0 con 0x7f0a300cea80 2024-09-21T02:21:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.836+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a300b0830 con 0x7f0a300cea80 2024-09-21T02:21:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.836+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f0a2800ac50 con 0x7f0a300cea80 2024-09-21T02:21:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.836+0000 7f0a267fc640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a140738c0 0x7f0a14075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.836+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5637+0+0 (secure 0 0 0) 0x7f0a28095470 con 0x7f0a300cea80 2024-09-21T02:21:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.836+0000 7f0a35e08640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a140738c0 0x7f0a14075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.836+0000 7f0a35e08640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a140738c0 0x7f0a14075dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0af000 tx=0x7f0a2c01b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:11.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:11.840+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0a2805fcc0 con 0x7f0a300cea80 2024-09-21T02:21:12.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.052+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0a300adfd0 con 0x7f0a300cea80 2024-09-21T02:21:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.624+0000 7f0a267fc640 1 -- 172.21.15.170:0/2976679590 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v96) ==== 174+0+0 (secure 0 0 0) 0x7f0a2805f5c0 con 0x7f0a300cea80 2024-09-21T02:21:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 22 and data pool 23 2024-09-21T02:21:12.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a140738c0 msgr2=0x7f0a14075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:12.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a140738c0 0x7f0a14075dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0af000 tx=0x7f0a2c01b060 comp rx=0 tx=0).stop 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 msgr2=0x7f0a30153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 0x7f0a30153780 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800c760 tx=0x7f0a2800cc20 comp rx=0 tx=0).stop 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 shutdown_connections 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0a140738c0 0x7f0a14075dd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a300b2fc0 0x7f0a30153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a300cea80 0x7f0a30153780 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 --2- 172.21.15.170:0/2976679590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a3006ef40 0x7f0a300b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 >> 172.21.15.170:0/2976679590 conn(0x7f0a300d0170 msgr2=0x7f0a30159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 shutdown_connections 2024-09-21T02:21:12.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.628+0000 7f0a3806c640 1 -- 172.21.15.170:0/2976679590 wait complete. 2024-09-21T02:21:12.677 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.824+0000 7f2ec88db640 1 -- 172.21.15.170:0/2264343234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb0005360 msgr2=0x7f2eb000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.824+0000 7f2ec88db640 1 --2- 172.21.15.170:0/2264343234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb0005360 0x7f2eb000b9d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009b60 tx=0x7f2ebc01d2b0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.824+0000 7f2ec88db640 1 -- 172.21.15.170:0/2264343234 shutdown_connections 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.824+0000 7f2ec88db640 1 --2- 172.21.15.170:0/2264343234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb0005360 0x7f2eb000b9d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.824+0000 7f2ec88db640 1 -- 172.21.15.170:0/2264343234 >> 172.21.15.170:0/2264343234 conn(0x7f2ec409a9c0 msgr2=0x7f2ec40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 -- 172.21.15.170:0/2264343234 shutdown_connections 2024-09-21T02:21:12.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 -- 172.21.15.170:0/2264343234 wait complete. 2024-09-21T02:21:12.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 Processor -- start 2024-09-21T02:21:12.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 -- start start 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eb0005360 0x7f2ec415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41f5bd0 0x7f2ec41619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 0x7f2ec415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec40d0fe0 con 0x7f2ec415d160 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec40d0e60 con 0x7f2ec41f5bd0 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec88db640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec4163290 con 0x7f2eb0005360 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 0x7f2ec415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 0x7f2ec415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52904/0 (socket says 172.21.15.170:52904) 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 -- 172.21.15.170:0/1573708951 learned_addr learned my addr 172.21.15.170:0/1573708951 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec1d9b640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41f5bd0 0x7f2ec41619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec259c640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eb0005360 0x7f2ec415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 -- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eb0005360 msgr2=0x7f2ec415b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:12.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eb0005360 0x7f2ec415b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 -- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41f5bd0 msgr2=0x7f2ec41619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41f5bd0 0x7f2ec41619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 -- 172.21.15.170:0/1573708951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec415dab0 con 0x7f2ec415d160 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.828+0000 7f2ec2d9d640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 0x7f2ec415d570 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc01db50 tx=0x7f2ebc003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0bd070 con 0x7f2ec415d160 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ebc004020 con 0x7f2ec415d160 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc003d50 con 0x7f2ec415d160 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec415ba60 con 0x7f2ec415d160 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec415c020 con 0x7f2ec415d160 2024-09-21T02:21:12.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec415b7d0 con 0x7f2ec415d160 2024-09-21T02:21:12.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f2ebc0b5040 con 0x7f2ec415d160 2024-09-21T02:21:12.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ebaffd640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ea40738f0 0x7f2ea4075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:12.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5737+0+0 (secure 0 0 0) 0x7f2ebc12f8b0 con 0x7f2ec415d160 2024-09-21T02:21:12.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ec259c640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ea40738f0 0x7f2ea4075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:12.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.832+0000 7f2ec259c640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ea40738f0 0x7f2ea4075e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000c10 tx=0x7f2eb4006a80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:12.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:12.836+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2ebc0c6080 con 0x7f2ec415d160 2024-09-21T02:21:13.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.012+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ec40d0560 con 0x7f2ec415d160 2024-09-21T02:21:13.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.012+0000 7f2ebaffd640 1 -- 172.21.15.170:0/1573708951 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16844 (secure 0 0 0) 0x7f2ec415dab0 con 0x7f2ec415d160 2024-09-21T02:21:13.015 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:13.015 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":141,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:12.056709+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:09.751144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:10.994981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ea40738f0 msgr2=0x7f2ea4075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ea40738f0 0x7f2ea4075e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000c10 tx=0x7f2eb4006a80 comp rx=0 tx=0).stop 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 msgr2=0x7f2ec415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 0x7f2ec415d570 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc01db50 tx=0x7f2ebc003be0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 shutdown_connections 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ea40738f0 0x7f2ea4075e00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec415d160 0x7f2ec415d570 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41f5bd0 0x7f2ec41619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 --2- 172.21.15.170:0/1573708951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2eb0005360 0x7f2ec415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 >> 172.21.15.170:0/1573708951 conn(0x7f2ec409a9c0 msgr2=0x7f2ec40b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:13.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 shutdown_connections 2024-09-21T02:21:13.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.016+0000 7f2ec88db640 1 -- 172.21.15.170:0/1573708951 wait complete. 2024-09-21T02:21:13.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f2914008640 1 -- 172.21.15.170:0/321028303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4004d50 msgr2=0x7f28f4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f2914008640 1 --2- 172.21.15.170:0/321028303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4004d50 0x7f28f4005160 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00c2e0 tx=0x7f28fc01cf80 comp rx=0 tx=0).stop 2024-09-21T02:21:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f29125a5640 1 -- 172.21.15.170:0/321028303 reap_dead start 2024-09-21T02:21:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f2914008640 1 -- 172.21.15.170:0/321028303 shutdown_connections 2024-09-21T02:21:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f2914008640 1 -- 172.21.15.170:0/321028303 >> 172.21.15.170:0/321028303 conn(0x7f290c0b1c40 msgr2=0x7f290c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f2914008640 1 -- 172.21.15.170:0/321028303 shutdown_connections 2024-09-21T02:21:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.196+0000 7f2914008640 1 -- 172.21.15.170:0/321028303 wait complete. 2024-09-21T02:21:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 Processor -- start 2024-09-21T02:21:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- start start 2024-09-21T02:21:13.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28f4004d50 0x7f290c0ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 0x7f290c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c0afff0 0x7f290c09cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f290c0cb500 con 0x7f290c15c9b0 2024-09-21T02:21:13.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f290c074f00 con 0x7f28f4004d50 2024-09-21T02:21:13.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f290c0cb680 con 0x7f290c0afff0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 0x7f290c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 0x7f290c0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52926/0 (socket says 172.21.15.170:52926) 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 -- 172.21.15.170:0/417584029 learned_addr learned my addr 172.21.15.170:0/417584029 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29125a5640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c0afff0 0x7f290c09cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2911da4640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28f4004d50 0x7f290c0ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 -- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c0afff0 msgr2=0x7f290c09cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c0afff0 0x7f290c09cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 -- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28f4004d50 msgr2=0x7f290c0ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28f4004d50 0x7f290c0ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 -- 172.21.15.170:0/417584029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c0ae0d0 con 0x7f290c15c9b0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29125a5640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c0afff0 0x7f290c09cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f29115a3640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 0x7f290c0afab0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f2900011850 tx=0x7f2900011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290001f070 con 0x7f290c15c9b0 2024-09-21T02:21:13.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f290000fe40 con 0x7f290c15c9b0 2024-09-21T02:21:13.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290c0ae3b0 con 0x7f290c15c9b0 2024-09-21T02:21:13.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900012690 con 0x7f290c15c9b0 2024-09-21T02:21:13.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290c15c1b0 con 0x7f290c15c9b0 2024-09-21T02:21:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.200+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290c0ade20 con 0x7f290c15c9b0 2024-09-21T02:21:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.204+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f2900022070 con 0x7f290c15c9b0 2024-09-21T02:21:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.204+0000 7f290a7fc640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f28f80738c0 0x7f28f8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.204+0000 7f2911da4640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f28f80738c0 0x7f28f8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.204+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5737+0+0 (secure 0 0 0) 0x7f29000957e0 con 0x7f290c15c9b0 2024-09-21T02:21:13.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.204+0000 7f2911da4640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f28f80738c0 0x7f28f8075dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f290400ae20 tx=0x7f290400a610 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:13.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.208+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f290005f880 con 0x7f290c15c9b0 2024-09-21T02:21:13.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.380+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f290c0701f0 con 0x7f290c15c9b0 2024-09-21T02:21:13.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f290a7fc640 1 -- 172.21.15.170:0/417584029 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16844 (secure 0 0 0) 0x7f29000642b0 con 0x7f290c15c9b0 2024-09-21T02:21:13.384 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:13.384 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":141,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:12.056709+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:09.751144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:10.994981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:13.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f28f80738c0 msgr2=0x7f28f8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f28f80738c0 0x7f28f8075dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f290400ae20 tx=0x7f290400a610 comp rx=0 tx=0).stop 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 msgr2=0x7f290c0afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 0x7f290c0afab0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f2900011850 tx=0x7f2900011d10 comp rx=0 tx=0).stop 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 shutdown_connections 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f28f80738c0 0x7f28f8075dd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c0afff0 0x7f290c09cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c15c9b0 0x7f290c0afab0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 --2- 172.21.15.170:0/417584029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28f4004d50 0x7f290c0ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 >> 172.21.15.170:0/417584029 conn(0x7f290c0b1c40 msgr2=0x7f290c06f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 shutdown_connections 2024-09-21T02:21:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.384+0000 7f2914008640 1 -- 172.21.15.170:0/417584029 wait complete. 2024-09-21T02:21:13.407 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:21:13.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- 172.21.15.170:0/1497606852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 msgr2=0x7ff25c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/1497606852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff25c005220 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff2740b60f0 tx=0x7ff26c01cad0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff279661640 1 -- 172.21.15.170:0/1497606852 reap_dead start 2024-09-21T02:21:13.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- 172.21.15.170:0/1497606852 shutdown_connections 2024-09-21T02:21:13.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- 172.21.15.170:0/1497606852 >> 172.21.15.170:0/1497606852 conn(0x7ff2740d0170 msgr2=0x7ff2740d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:13.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- 172.21.15.170:0/1497606852 shutdown_connections 2024-09-21T02:21:13.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- 172.21.15.170:0/1497606852 wait complete. 2024-09-21T02:21:13.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 Processor -- start 2024-09-21T02:21:13.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- start start 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff27414a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27415e960 0x7ff27415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff27409cfa0 0x7ff27409d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740b3550 con 0x7ff25c004d50 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740b33d0 con 0x7ff27409cfa0 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff27b0c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff27406ff50 con 0x7ff27415e960 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff278e60640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff27414a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff278e60640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff27414a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52932/0 (socket says 172.21.15.170:52932) 2024-09-21T02:21:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.564+0000 7ff278e60640 1 -- 172.21.15.170:0/2370994300 learned_addr learned my addr 172.21.15.170:0/2370994300 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff279661640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff27409cfa0 0x7ff27409d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff273fff640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27415e960 0x7ff27415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff278e60640 1 -- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27415e960 msgr2=0x7ff27415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff278e60640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27415e960 0x7ff27415ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff278e60640 1 -- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff27409cfa0 msgr2=0x7ff27409d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff278e60640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff27409cfa0 0x7ff27409d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff278e60640 1 -- 172.21.15.170:0/2370994300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27415f2b0 con 0x7ff25c004d50 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff279661640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff27409cfa0 0x7ff27409d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff278e60640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff27414a640 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7ff26400e590 tx=0x7ff26400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264015070 con 0x7ff25c004d50 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff264010980 con 0x7ff25c004d50 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2741f7330 con 0x7ff25c004d50 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264010db0 con 0x7ff25c004d50 2024-09-21T02:21:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2741f78f0 con 0x7ff25c004d50 2024-09-21T02:21:13.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27414abd0 con 0x7ff25c004d50 2024-09-21T02:21:13.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7ff264018070 con 0x7ff25c004d50 2024-09-21T02:21:13.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff2717fa640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2600738f0 0x7ff260075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:13.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5737+0+0 (secure 0 0 0) 0x7ff2640953c0 con 0x7ff25c004d50 2024-09-21T02:21:13.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff273fff640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2600738f0 0x7ff260075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:13.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.568+0000 7ff273fff640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2600738f0 0x7ff260075e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2740c9d50 tx=0x7ff2680091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:13.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.572+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff27415f2b0 con 0x7ff25c004d50 2024-09-21T02:21:13.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:13.780+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff2740704e0 con 0x7ff25c004d50 2024-09-21T02:21:14.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.668+0000 7ff2717fa640 1 -- 172.21.15.170:0/2370994300 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7ff26405f930 con 0x7ff25c004d50 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2600738f0 msgr2=0x7ff260075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2600738f0 0x7ff260075e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff2740c9d50 tx=0x7ff2680091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 msgr2=0x7ff27414a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff27414a640 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7ff26400e590 tx=0x7ff26400ea50 comp rx=0 tx=0).stop 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 shutdown_connections 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff2600738f0 0x7ff260075e00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff27415e960 0x7ff27415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff27409cfa0 0x7ff27409d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 --2- 172.21.15.170:0/2370994300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff25c004d50 0x7ff27414a640 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 >> 172.21.15.170:0/2370994300 conn(0x7ff2740d0170 msgr2=0x7ff27415f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 shutdown_connections 2024-09-21T02:21:14.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.672+0000 7ff27b0c4640 1 -- 172.21.15.170:0/2370994300 wait complete. 2024-09-21T02:21:14.698 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:21:14.698 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:21:14.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 -- 172.21.15.170:0/605345953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0740a0 msgr2=0x7f5ffc0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 --2- 172.21.15.170:0/605345953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0740a0 0x7f5ffc0744b0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0726b0 tx=0x7f5ff001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 -- 172.21.15.170:0/605345953 shutdown_connections 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 --2- 172.21.15.170:0/605345953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ffc0ce070 0x7f5ffc0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 --2- 172.21.15.170:0/605345953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0740a0 0x7f5ffc0744b0 secure :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0726b0 tx=0x7f5ff001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 --2- 172.21.15.170:0/605345953 >> v2:172.21.15.191:3300/0 conn(0x7f5ffc0cf7d0 0x7f5ffc073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 -- 172.21.15.170:0/605345953 >> 172.21.15.170:0/605345953 conn(0x7f5ffc0d0170 msgr2=0x7f5ffc0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 -- 172.21.15.170:0/605345953 shutdown_connections 2024-09-21T02:21:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.860+0000 7f60021c5640 1 -- 172.21.15.170:0/605345953 wait complete. 2024-09-21T02:21:14.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 Processor -- start 2024-09-21T02:21:14.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 -- start start 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ffc0ce070 0x7f5ffc0c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 0x7f5ffc145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ffc163880 0x7f5ffc163c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cbe60 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cbce0 con 0x7f5ffc163880 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cbfe0 con 0x7f5ffc0ce070 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 0x7f5ffc145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 0x7f5ffc145fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52944/0 (socket says 172.21.15.170:52944) 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 -- 172.21.15.170:0/2743517597 learned_addr learned my addr 172.21.15.170:0/2743517597 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffb7fe640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ffc0ce070 0x7f5ffc0c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffbfff640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ffc163880 0x7f5ffc163c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 -- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ffc0ce070 msgr2=0x7f5ffc0c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ffc0ce070 0x7f5ffc0c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 -- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ffc163880 msgr2=0x7f5ffc163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ffc163880 0x7f5ffc163c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 -- 172.21.15.170:0/2743517597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc0c54c0 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffbfff640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ffc163880 0x7f5ffc163c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5ffaffd640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 0x7f5ffc145fb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f5ff001a4f0 tx=0x7f5ff00a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00bb070 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ff0005320 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc164640 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00b2040 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.864+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc164e90 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.868+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc0ce480 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.868+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f5ff00be070 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.868+0000 7f5fe3fff640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fe80738c0 0x7f5fe8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:14.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.868+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f5ff012ec10 con 0x7f5ffc0cf7d0 2024-09-21T02:21:14.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.868+0000 7f5ffb7fe640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fe80738c0 0x7f5fe8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:14.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.868+0000 7f5ffb7fe640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fe80738c0 0x7f5fe8075dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fec005c50 tx=0x7f5fec00a390 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:14.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:14.872+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5ff00164b0 con 0x7f5ffc0cf7d0 2024-09-21T02:21:15.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.080+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5ffc1643d0 con 0x7f5ffc0cf7d0 2024-09-21T02:21:15.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.680+0000 7f5fe3fff640 1 -- 172.21.15.170:0/2743517597 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7f5ff00f9490 con 0x7f5ffc0cf7d0 2024-09-21T02:21:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fe80738c0 msgr2=0x7f5fe8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fe80738c0 0x7f5fe8075dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5fec005c50 tx=0x7f5fec00a390 comp rx=0 tx=0).stop 2024-09-21T02:21:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 msgr2=0x7f5ffc145fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 0x7f5ffc145fb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f5ff001a4f0 tx=0x7f5ff00a6ac0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 shutdown_connections 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fe80738c0 0x7f5fe8075dd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ffc163880 0x7f5ffc163c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0cf7d0 0x7f5ffc145fb0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 --2- 172.21.15.170:0/2743517597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ffc0ce070 0x7f5ffc0c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 >> 172.21.15.170:0/2743517597 conn(0x7f5ffc0d0170 msgr2=0x7f5ffc15ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 shutdown_connections 2024-09-21T02:21:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.684+0000 7f60021c5640 1 -- 172.21.15.170:0/2743517597 wait complete. 2024-09-21T02:21:15.714 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:21:15.714 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:15.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- 172.21.15.170:0/95207778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f340c004d50 msgr2=0x7f340c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:15.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 --2- 172.21.15.170:0/95207778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f340c004d50 0x7f340c005200 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f342009b010 tx=0x7f341801cac0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3426d12640 1 -- 172.21.15.170:0/95207778 reap_dead start 2024-09-21T02:21:15.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- 172.21.15.170:0/95207778 shutdown_connections 2024-09-21T02:21:15.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- 172.21.15.170:0/95207778 >> 172.21.15.170:0/95207778 conn(0x7f342009a9c0 msgr2=0x7f34200d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:15.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- 172.21.15.170:0/95207778 shutdown_connections 2024-09-21T02:21:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- 172.21.15.170:0/95207778 wait complete. 2024-09-21T02:21:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 Processor -- start 2024-09-21T02:21:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- start start 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420159780 0x7f3420149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342014c6e0 0x7f342014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 0x7f34200c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f34200b1ac0 con 0x7f342014d030 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34200b1940 con 0x7f3420159780 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3428775640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3420160aa0 con 0x7f342014c6e0 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3426d12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 0x7f34200c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3426d12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 0x7f34200c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43098/0 (socket says 172.21.15.170:43098) 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3426d12640 1 -- 172.21.15.170:0/2132744587 learned_addr learned my addr 172.21.15.170:0/2132744587 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:15.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3426511640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420159780 0x7f3420149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.876+0000 7f3425d10640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342014c6e0 0x7f342014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426d12640 1 -- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342014c6e0 msgr2=0x7f342014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426d12640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342014c6e0 0x7f342014caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426d12640 1 -- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420159780 msgr2=0x7f3420149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426d12640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420159780 0x7f3420149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426d12640 1 -- 172.21.15.170:0/2132744587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34200c60e0 con 0x7f342014d030 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426d12640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 0x7f34200c4df0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3418000f90 tx=0x7f3418003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34180013f0 con 0x7f342014d030 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34180a5df0 con 0x7f342014d030 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418006cb0 con 0x7f342014d030 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34200c63c0 con 0x7f342014d030 2024-09-21T02:21:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342014a650 con 0x7f342014d030 2024-09-21T02:21:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3420161000 con 0x7f342014d030 2024-09-21T02:21:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f3418006330 con 0x7f342014d030 2024-09-21T02:21:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3416ffd640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34040738f0 0x7f3404075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f341812e6c0 con 0x7f342014d030 2024-09-21T02:21:15.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426511640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34040738f0 0x7f3404075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:15.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.880+0000 7f3426511640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34040738f0 0x7f3404075e00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3420070ce0 tx=0x7f341c005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:15.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:15.884+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f34180be070 con 0x7f342014d030 2024-09-21T02:21:16.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.088+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34200ce7b0 con 0x7f342014d030 2024-09-21T02:21:16.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.088+0000 7f3416ffd640 1 -- 172.21.15.170:0/2132744587 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5238 (secure 0 0 0) 0x7f34180167d0 con 0x7f342014d030 2024-09-21T02:21:16.091 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:16.091 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":100,"btime":"2024-09-21T02:21:15:682112+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:12.056653+0000","modified":"2024-09-21T02:21:15.085591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2032631050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2032631050},{"type":"v1","addr":"172.21.15.170:6835","nonce":2032631050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":11}]} 2024-09-21T02:21:16.091 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 100 2024-09-21T02:21:16.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34040738f0 msgr2=0x7f3404075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34040738f0 0x7f3404075e00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3420070ce0 tx=0x7f341c005f40 comp rx=0 tx=0).stop 2024-09-21T02:21:16.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 msgr2=0x7f34200c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 0x7f34200c4df0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3418000f90 tx=0x7f3418003f10 comp rx=0 tx=0).stop 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 shutdown_connections 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f34040738f0 0x7f3404075e00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342014c6e0 0x7f342014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342014d030 0x7f34200c4df0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 --2- 172.21.15.170:0/2132744587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420159780 0x7f3420149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 >> 172.21.15.170:0/2132744587 conn(0x7f342009a9c0 msgr2=0x7f34200b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 shutdown_connections 2024-09-21T02:21:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.092+0000 7f3428775640 1 -- 172.21.15.170:0/2132744587 wait complete. 2024-09-21T02:21:16.112 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 -- 172.21.15.170:0/779346787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 msgr2=0x7f5e1806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/779346787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e1806fa30 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009840 tx=0x7f5e0c01c820 comp rx=0 tx=0).stop 2024-09-21T02:21:16.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 -- 172.21.15.170:0/779346787 shutdown_connections 2024-09-21T02:21:16.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/779346787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e1806fa30 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 -- 172.21.15.170:0/779346787 >> 172.21.15.170:0/779346787 conn(0x7f5e180b1c40 msgr2=0x7f5e180b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:16.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 -- 172.21.15.170:0/779346787 shutdown_connections 2024-09-21T02:21:16.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 -- 172.21.15.170:0/779346787 wait complete. 2024-09-21T02:21:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 Processor -- start 2024-09-21T02:21:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.252+0000 7f5e1cef8640 1 -- start start 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e180b2f40 0x7f5e1809cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e18158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e180b3a40 0x7f5e180b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1816b570 con 0x7f5e180b5410 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18070180 con 0x7f5e180b3a40 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1816b6f0 con 0x7f5e180b2f40 2024-09-21T02:21:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e18158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e18158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43108/0 (socket says 172.21.15.170:43108) 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 -- 172.21.15.170:0/675968420 learned_addr learned my addr 172.21.15.170:0/675968420 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 -- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e180b2f40 msgr2=0x7f5e1809cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e180b2f40 0x7f5e1809cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 -- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e180b3a40 msgr2=0x7f5e180b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e16d9d640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e180b3a40 0x7f5e180b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e180b3a40 0x7f5e180b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 -- 172.21.15.170:0/675968420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e180b4390 con 0x7f5e180b5410 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e15d9b640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e18158bc0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c430 tx=0x7f5e0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08011790 con 0x7f5e180b5410 2024-09-21T02:21:16.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e08017070 con 0x7f5e180b5410 2024-09-21T02:21:16.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e18159100 con 0x7f5e180b5410 2024-09-21T02:21:16.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08011ce0 con 0x7f5e180b5410 2024-09-21T02:21:16.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e180acfc0 con 0x7f5e180b5410 2024-09-21T02:21:16.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1806fa30 con 0x7f5e180b5410 2024-09-21T02:21:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.256+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f5e0801c460 con 0x7f5e180b5410 2024-09-21T02:21:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.260+0000 7f5e06ffd640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5df80738c0 0x7f5df8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.260+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f5e08095670 con 0x7f5e180b5410 2024-09-21T02:21:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.260+0000 7f5e1659c640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5df80738c0 0x7f5df8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.260+0000 7f5e1659c640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5df80738c0 0x7f5df8075dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0af000 tx=0x7f5e0c0195c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:16.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.264+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5e180b4390 con 0x7f5e180b5410 2024-09-21T02:21:16.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.424+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e1809d300 con 0x7f5e180b5410 2024-09-21T02:21:16.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e06ffd640 1 -- 172.21.15.170:0/675968420 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16831 (secure 0 0 0) 0x7f5e0805fe90 con 0x7f5e180b5410 2024-09-21T02:21:16.428 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:16.428 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:13.627328+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:09.751144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:10.994981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5df80738c0 msgr2=0x7f5df8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5df80738c0 0x7f5df8075dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0af000 tx=0x7f5e0c0195c0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 msgr2=0x7f5e18158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e18158bc0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c430 tx=0x7f5e0800c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 shutdown_connections 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5df80738c0 0x7f5df8075dd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e180b3a40 0x7f5e180b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e180b5410 0x7f5e18158bc0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 --2- 172.21.15.170:0/675968420 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e180b2f40 0x7f5e1809cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 >> 172.21.15.170:0/675968420 conn(0x7f5e180b1c40 msgr2=0x7f5e180cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 shutdown_connections 2024-09-21T02:21:16.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.428+0000 7f5e1cef8640 1 -- 172.21.15.170:0/675968420 wait complete. 2024-09-21T02:21:16.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:21:16.449 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:21:16.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.588+0000 7fa226912640 1 -- 172.21.15.170:0/674838045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa208003a60 msgr2=0x7fa208003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.588+0000 7fa226912640 1 --2- 172.21.15.170:0/674838045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa208003a60 0x7fa208003ed0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa210009b60 tx=0x7fa21001d270 comp rx=0 tx=0).stop 2024-09-21T02:21:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.588+0000 7fa226912640 1 -- 172.21.15.170:0/674838045 shutdown_connections 2024-09-21T02:21:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.588+0000 7fa226912640 1 --2- 172.21.15.170:0/674838045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa208003a60 0x7fa208003ed0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.588+0000 7fa226912640 1 -- 172.21.15.170:0/674838045 >> 172.21.15.170:0/674838045 conn(0x7fa2200b1c40 msgr2=0x7fa2200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.588+0000 7fa226912640 1 -- 172.21.15.170:0/674838045 shutdown_connections 2024-09-21T02:21:16.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- 172.21.15.170:0/674838045 wait complete. 2024-09-21T02:21:16.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 Processor -- start 2024-09-21T02:21:16.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- start start 2024-09-21T02:21:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa220151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 0x7fa2200c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2200c55c0 0x7fa22015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa220072e50 con 0x7fa2200c4c70 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa220074f00 con 0x7fa2200c55c0 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa220072fd0 con 0x7fa208003a60 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 0x7fa2200c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 0x7fa2200c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43124/0 (socket says 172.21.15.170:43124) 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 -- 172.21.15.170:0/1304825588 learned_addr learned my addr 172.21.15.170:0/1304825588 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa224eaf640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2200c55c0 0x7fa22015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21ffff640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa220151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 -- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 msgr2=0x7fa220151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa220151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 -- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2200c55c0 msgr2=0x7fa22015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2200c55c0 0x7fa22015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 -- 172.21.15.170:0/1304825588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa220151ad0 con 0x7fa2200c4c70 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa224eaf640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2200c55c0 0x7fa22015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21f7fe640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 0x7fa2200c5080 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c430 tx=0x7fa21400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140052d0 con 0x7fa2200c4c70 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa214007020 con 0x7fa2200c4c70 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22015d090 con 0x7fa2200c4c70 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214007470 con 0x7fa2200c4c70 2024-09-21T02:21:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.592+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22015cdb0 con 0x7fa2200c4c70 2024-09-21T02:21:16.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.596+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2200c74b0 con 0x7fa2200c4c70 2024-09-21T02:21:16.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.596+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fa2140067e0 con 0x7fa2200c4c70 2024-09-21T02:21:16.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.596+0000 7fa21cff9640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa20c0738c0 0x7fa20c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.596+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fa2140948e0 con 0x7fa2200c4c70 2024-09-21T02:21:16.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.596+0000 7fa21ffff640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa20c0738c0 0x7fa20c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.596+0000 7fa21ffff640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa20c0738c0 0x7fa20c075dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ae000 tx=0x7fa21001a010 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:16.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.600+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa21405ef40 con 0x7fa2200c4c70 2024-09-21T02:21:16.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.756+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa220073530 con 0x7fa20c0738c0 2024-09-21T02:21:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.760+0000 7fa21cff9640 1 -- 172.21.15.170:0/1304825588 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7fa220073530 con 0x7fa20c0738c0 2024-09-21T02:21:16.765 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:16.766 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:21:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa20c0738c0 msgr2=0x7fa20c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa20c0738c0 0x7fa20c075dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ae000 tx=0x7fa21001a010 comp rx=0 tx=0).stop 2024-09-21T02:21:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 msgr2=0x7fa2200c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 0x7fa2200c5080 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c430 tx=0x7fa21400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 shutdown_connections 2024-09-21T02:21:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa20c0738c0 0x7fa20c075dd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2200c55c0 0x7fa22015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2200c4c70 0x7fa2200c5080 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 --2- 172.21.15.170:0/1304825588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa220151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 >> 172.21.15.170:0/1304825588 conn(0x7fa2200b1c40 msgr2=0x7fa22009c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:16.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 shutdown_connections 2024-09-21T02:21:16.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.764+0000 7fa226912640 1 -- 172.21.15.170:0/1304825588 wait complete. 2024-09-21T02:21:16.785 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-21T02:21:14.920961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289560,"kb_used_data":15076,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754685160,"statfs":{"total":773094113280,"available":772797603840,"internally_reserved":0,"allocated":15437824,"data_stored":7442180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":280874545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.308759"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640934+0000","last_change":"2024-09-21T02:21:12.657140+0000","last_active":"2024-09-21T02:21:14.640934+0000","last_peered":"2024-09-21T02:21:14.640934+0000","last_clean":"2024-09-21T02:21:14.640934+0000","last_became_active":"2024-09-21T02:21:12.656783+0000","last_became_peered":"2024-09-21T02:21:12.656783+0000","last_unstale":"2024-09-21T02:21:14.640934+0000","last_undegraded":"2024-09-21T02:21:14.640934+0000","last_fullsized":"2024-09-21T02:21:14.640934+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:47.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640725+0000","last_change":"2024-09-21T02:21:12.646566+0000","last_active":"2024-09-21T02:21:14.640725+0000","last_peered":"2024-09-21T02:21:14.640725+0000","last_clean":"2024-09-21T02:21:14.640725+0000","last_became_active":"2024-09-21T02:21:12.646230+0000","last_became_peered":"2024-09-21T02:21:12.646230+0000","last_unstale":"2024-09-21T02:21:14.640725+0000","last_undegraded":"2024-09-21T02:21:14.640725+0000","last_fullsized":"2024-09-21T02:21:14.640725+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:22.082763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640246+0000","last_change":"2024-09-21T02:21:12.650516+0000","last_active":"2024-09-21T02:21:14.640246+0000","last_peered":"2024-09-21T02:21:14.640246+0000","last_clean":"2024-09-21T02:21:14.640246+0000","last_became_active":"2024-09-21T02:21:12.650136+0000","last_became_peered":"2024-09-21T02:21:12.650136+0000","last_unstale":"2024-09-21T02:21:14.640246+0000","last_undegraded":"2024-09-21T02:21:14.640246+0000","last_fullsized":"2024-09-21T02:21:14.640246+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:01.115540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862522+0000","last_change":"2024-09-21T02:21:12.686783+0000","last_active":"2024-09-21T02:21:14.862522+0000","last_peered":"2024-09-21T02:21:14.862522+0000","last_clean":"2024-09-21T02:21:14.862522+0000","last_became_active":"2024-09-21T02:21:12.686227+0000","last_became_peered":"2024-09-21T02:21:12.686227+0000","last_unstale":"2024-09-21T02:21:14.862522+0000","last_undegraded":"2024-09-21T02:21:14.862522+0000","last_fullsized":"2024-09-21T02:21:14.862522+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:15.635175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861701+0000","last_change":"2024-09-21T02:21:12.648176+0000","last_active":"2024-09-21T02:21:14.861701+0000","last_peered":"2024-09-21T02:21:14.861701+0000","last_clean":"2024-09-21T02:21:14.861701+0000","last_became_active":"2024-09-21T02:21:12.647830+0000","last_became_peered":"2024-09-21T02:21:12.647830+0000","last_unstale":"2024-09-21T02:21:14.861701+0000","last_undegraded":"2024-09-21T02:21:14.861701+0000","last_fullsized":"2024-09-21T02:21:14.861701+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:48.046120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635893+0000","last_change":"2024-09-21T02:21:12.691351+0000","last_active":"2024-09-21T02:21:13.635893+0000","last_peered":"2024-09-21T02:21:13.635893+0000","last_clean":"2024-09-21T02:21:13.635893+0000","last_became_active":"2024-09-21T02:21:12.689586+0000","last_became_peered":"2024-09-21T02:21:12.689586+0000","last_unstale":"2024-09-21T02:21:13.635893+0000","last_undegraded":"2024-09-21T02:21:13.635893+0000","last_fullsized":"2024-09-21T02:21:13.635893+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:59.902935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864538+0000","last_change":"2024-09-21T02:21:12.661496+0000","last_active":"2024-09-21T02:21:14.864538+0000","last_peered":"2024-09-21T02:21:14.864538+0000","last_clean":"2024-09-21T02:21:14.864538+0000","last_became_active":"2024-09-21T02:21:12.659318+0000","last_became_peered":"2024-09-21T02:21:12.659318+0000","last_unstale":"2024-09-21T02:21:14.864538+0000","last_undegraded":"2024-09-21T02:21:14.864538+0000","last_fullsized":"2024-09-21T02:21:14.864538+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:52.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517564+0000","last_change":"2024-09-21T02:21:12.649101+0000","last_active":"2024-09-21T02:21:14.517564+0000","last_peered":"2024-09-21T02:21:14.517564+0000","last_clean":"2024-09-21T02:21:14.517564+0000","last_became_active":"2024-09-21T02:21:12.648733+0000","last_became_peered":"2024-09-21T02:21:12.648733+0000","last_unstale":"2024-09-21T02:21:14.517564+0000","last_undegraded":"2024-09-21T02:21:14.517564+0000","last_fullsized":"2024-09-21T02:21:14.517564+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:47.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518030+0000","last_change":"2024-09-21T02:21:12.693775+0000","last_active":"2024-09-21T02:21:14.518030+0000","last_peered":"2024-09-21T02:21:14.518030+0000","last_clean":"2024-09-21T02:21:14.518030+0000","last_became_active":"2024-09-21T02:21:12.693465+0000","last_became_peered":"2024-09-21T02:21:12.693465+0000","last_unstale":"2024-09-21T02:21:14.518030+0000","last_undegraded":"2024-09-21T02:21:14.518030+0000","last_fullsized":"2024-09-21T02:21:14.518030+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:26.309641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518033+0000","last_change":"2024-09-21T02:21:12.687333+0000","last_active":"2024-09-21T02:21:14.518033+0000","last_peered":"2024-09-21T02:21:14.518033+0000","last_clean":"2024-09-21T02:21:14.518033+0000","last_became_active":"2024-09-21T02:21:12.687057+0000","last_became_peered":"2024-09-21T02:21:12.687057+0000","last_unstale":"2024-09-21T02:21:14.518033+0000","last_undegraded":"2024-09-21T02:21:14.518033+0000","last_fullsized":"2024-09-21T02:21:14.518033+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:04.829698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861994+0000","last_change":"2024-09-21T02:21:12.684093+0000","last_active":"2024-09-21T02:21:14.861994+0000","last_peered":"2024-09-21T02:21:14.861994+0000","last_clean":"2024-09-21T02:21:14.861994+0000","last_became_active":"2024-09-21T02:21:12.683615+0000","last_became_peered":"2024-09-21T02:21:12.683615+0000","last_unstale":"2024-09-21T02:21:14.861994+0000","last_undegraded":"2024-09-21T02:21:14.861994+0000","last_fullsized":"2024-09-21T02:21:14.861994+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:27.588267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633971+0000","last_change":"2024-09-21T02:21:12.660970+0000","last_active":"2024-09-21T02:21:13.633971+0000","last_peered":"2024-09-21T02:21:13.633971+0000","last_clean":"2024-09-21T02:21:13.633971+0000","last_became_active":"2024-09-21T02:21:12.660621+0000","last_became_peered":"2024-09-21T02:21:12.660621+0000","last_unstale":"2024-09-21T02:21:13.633971+0000","last_undegraded":"2024-09-21T02:21:13.633971+0000","last_fullsized":"2024-09-21T02:21:13.633971+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:33.945102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633523+0000","last_change":"2024-09-21T02:21:12.680944+0000","last_active":"2024-09-21T02:21:13.633523+0000","last_peered":"2024-09-21T02:21:13.633523+0000","last_clean":"2024-09-21T02:21:13.633523+0000","last_became_active":"2024-09-21T02:21:12.680642+0000","last_became_peered":"2024-09-21T02:21:12.680642+0000","last_unstale":"2024-09-21T02:21:13.633523+0000","last_undegraded":"2024-09-21T02:21:13.633523+0000","last_fullsized":"2024-09-21T02:21:13.633523+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:01.368078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518012+0000","last_change":"2024-09-21T02:21:12.645681+0000","last_active":"2024-09-21T02:21:14.518012+0000","last_peered":"2024-09-21T02:21:14.518012+0000","last_clean":"2024-09-21T02:21:14.518012+0000","last_became_active":"2024-09-21T02:21:12.645150+0000","last_became_peered":"2024-09-21T02:21:12.645150+0000","last_unstale":"2024-09-21T02:21:14.518012+0000","last_undegraded":"2024-09-21T02:21:14.518012+0000","last_fullsized":"2024-09-21T02:21:14.518012+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:02.832696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517815+0000","last_change":"2024-09-21T02:21:12.690790+0000","last_active":"2024-09-21T02:21:14.517815+0000","last_peered":"2024-09-21T02:21:14.517815+0000","last_clean":"2024-09-21T02:21:14.517815+0000","last_became_active":"2024-09-21T02:21:12.690538+0000","last_became_peered":"2024-09-21T02:21:12.690538+0000","last_unstale":"2024-09-21T02:21:14.517815+0000","last_undegraded":"2024-09-21T02:21:14.517815+0000","last_fullsized":"2024-09-21T02:21:14.517815+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:48.769696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862691+0000","last_change":"2024-09-21T02:21:12.658615+0000","last_active":"2024-09-21T02:21:14.862691+0000","last_peered":"2024-09-21T02:21:14.862691+0000","last_clean":"2024-09-21T02:21:14.862691+0000","last_became_active":"2024-09-21T02:21:12.657389+0000","last_became_peered":"2024-09-21T02:21:12.657389+0000","last_unstale":"2024-09-21T02:21:14.862691+0000","last_undegraded":"2024-09-21T02:21:14.862691+0000","last_fullsized":"2024-09-21T02:21:14.862691+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.353809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.637362+0000","last_change":"2024-09-21T02:21:12.687582+0000","last_active":"2024-09-21T02:21:13.637362+0000","last_peered":"2024-09-21T02:21:13.637362+0000","last_clean":"2024-09-21T02:21:13.637362+0000","last_became_active":"2024-09-21T02:21:12.686744+0000","last_became_peered":"2024-09-21T02:21:12.686744+0000","last_unstale":"2024-09-21T02:21:13.637362+0000","last_undegraded":"2024-09-21T02:21:13.637362+0000","last_fullsized":"2024-09-21T02:21:13.637362+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:19.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862761+0000","last_change":"2024-09-21T02:21:12.678221+0000","last_active":"2024-09-21T02:21:14.862761+0000","last_peered":"2024-09-21T02:21:14.862761+0000","last_clean":"2024-09-21T02:21:14.862761+0000","last_became_active":"2024-09-21T02:21:12.677445+0000","last_became_peered":"2024-09-21T02:21:12.677445+0000","last_unstale":"2024-09-21T02:21:14.862761+0000","last_undegraded":"2024-09-21T02:21:14.862761+0000","last_fullsized":"2024-09-21T02:21:14.862761+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:17.092471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865545+0000","last_change":"2024-09-21T02:21:12.657681+0000","last_active":"2024-09-21T02:21:14.865545+0000","last_peered":"2024-09-21T02:21:14.865545+0000","last_clean":"2024-09-21T02:21:14.865545+0000","last_became_active":"2024-09-21T02:21:12.656726+0000","last_became_peered":"2024-09-21T02:21:12.656726+0000","last_unstale":"2024-09-21T02:21:14.865545+0000","last_undegraded":"2024-09-21T02:21:14.865545+0000","last_fullsized":"2024-09-21T02:21:14.865545+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:04.828657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.634209+0000","last_change":"2024-09-21T02:21:12.654055+0000","last_active":"2024-09-21T02:21:13.634209+0000","last_peered":"2024-09-21T02:21:13.634209+0000","last_clean":"2024-09-21T02:21:13.634209+0000","last_became_active":"2024-09-21T02:21:12.653691+0000","last_became_peered":"2024-09-21T02:21:12.653691+0000","last_unstale":"2024-09-21T02:21:13.634209+0000","last_undegraded":"2024-09-21T02:21:13.634209+0000","last_fullsized":"2024-09-21T02:21:13.634209+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:15.586055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636157+0000","last_change":"2024-09-21T02:21:12.687239+0000","last_active":"2024-09-21T02:21:13.636157+0000","last_peered":"2024-09-21T02:21:13.636157+0000","last_clean":"2024-09-21T02:21:13.636157+0000","last_became_active":"2024-09-21T02:21:12.686136+0000","last_became_peered":"2024-09-21T02:21:12.686136+0000","last_unstale":"2024-09-21T02:21:13.636157+0000","last_undegraded":"2024-09-21T02:21:13.636157+0000","last_fullsized":"2024-09-21T02:21:13.636157+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:39.643004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861651+0000","last_change":"2024-09-21T02:21:12.683850+0000","last_active":"2024-09-21T02:21:14.861651+0000","last_peered":"2024-09-21T02:21:14.861651+0000","last_clean":"2024-09-21T02:21:14.861651+0000","last_became_active":"2024-09-21T02:21:12.683521+0000","last_became_peered":"2024-09-21T02:21:12.683521+0000","last_unstale":"2024-09-21T02:21:14.861651+0000","last_undegraded":"2024-09-21T02:21:14.861651+0000","last_fullsized":"2024-09-21T02:21:14.861651+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:26.955870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638365+0000","last_change":"2024-09-21T02:21:12.644867+0000","last_active":"2024-09-21T02:21:13.638365+0000","last_peered":"2024-09-21T02:21:13.638365+0000","last_clean":"2024-09-21T02:21:13.638365+0000","last_became_active":"2024-09-21T02:21:12.643984+0000","last_became_peered":"2024-09-21T02:21:12.643984+0000","last_unstale":"2024-09-21T02:21:13.638365+0000","last_undegraded":"2024-09-21T02:21:13.638365+0000","last_fullsized":"2024-09-21T02:21:13.638365+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:25.151852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861509+0000","last_change":"2024-09-21T02:21:12.648414+0000","last_active":"2024-09-21T02:21:14.861509+0000","last_peered":"2024-09-21T02:21:14.861509+0000","last_clean":"2024-09-21T02:21:14.861509+0000","last_became_active":"2024-09-21T02:21:12.648066+0000","last_became_peered":"2024-09-21T02:21:12.648066+0000","last_unstale":"2024-09-21T02:21:14.861509+0000","last_undegraded":"2024-09-21T02:21:14.861509+0000","last_fullsized":"2024-09-21T02:21:14.861509+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:15.965614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862207+0000","last_change":"2024-09-21T02:21:12.692205+0000","last_active":"2024-09-21T02:21:14.862207+0000","last_peered":"2024-09-21T02:21:14.862207+0000","last_clean":"2024-09-21T02:21:14.862207+0000","last_became_active":"2024-09-21T02:21:12.691837+0000","last_became_peered":"2024-09-21T02:21:12.691837+0000","last_unstale":"2024-09-21T02:21:14.862207+0000","last_undegraded":"2024-09-21T02:21:14.862207+0000","last_fullsized":"2024-09-21T02:21:14.862207+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:30.686170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638655+0000","last_change":"2024-09-21T02:21:12.678295+0000","last_active":"2024-09-21T02:21:13.638655+0000","last_peered":"2024-09-21T02:21:13.638655+0000","last_clean":"2024-09-21T02:21:13.638655+0000","last_became_active":"2024-09-21T02:21:12.677499+0000","last_became_peered":"2024-09-21T02:21:12.677499+0000","last_unstale":"2024-09-21T02:21:13.638655+0000","last_undegraded":"2024-09-21T02:21:13.638655+0000","last_fullsized":"2024-09-21T02:21:13.638655+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:22:33.803430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862073+0000","last_change":"2024-09-21T02:21:12.657619+0000","last_active":"2024-09-21T02:21:14.862073+0000","last_peered":"2024-09-21T02:21:14.862073+0000","last_clean":"2024-09-21T02:21:14.862073+0000","last_became_active":"2024-09-21T02:21:12.657298+0000","last_became_peered":"2024-09-21T02:21:12.657298+0000","last_unstale":"2024-09-21T02:21:14.862073+0000","last_undegraded":"2024-09-21T02:21:14.862073+0000","last_fullsized":"2024-09-21T02:21:14.862073+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:14.425805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865346+0000","last_change":"2024-09-21T02:21:12.657779+0000","last_active":"2024-09-21T02:21:14.865346+0000","last_peered":"2024-09-21T02:21:14.865346+0000","last_clean":"2024-09-21T02:21:14.865346+0000","last_became_active":"2024-09-21T02:21:12.656796+0000","last_became_peered":"2024-09-21T02:21:12.656796+0000","last_unstale":"2024-09-21T02:21:14.865346+0000","last_undegraded":"2024-09-21T02:21:14.865346+0000","last_fullsized":"2024-09-21T02:21:14.865346+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:20.656513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639985+0000","last_change":"2024-09-21T02:21:12.655801+0000","last_active":"2024-09-21T02:21:14.639985+0000","last_peered":"2024-09-21T02:21:14.639985+0000","last_clean":"2024-09-21T02:21:14.639985+0000","last_became_active":"2024-09-21T02:21:12.655459+0000","last_became_peered":"2024-09-21T02:21:12.655459+0000","last_unstale":"2024-09-21T02:21:14.639985+0000","last_undegraded":"2024-09-21T02:21:14.639985+0000","last_fullsized":"2024-09-21T02:21:14.639985+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:21.852819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517556+0000","last_change":"2024-09-21T02:21:12.645458+0000","last_active":"2024-09-21T02:21:14.517556+0000","last_peered":"2024-09-21T02:21:14.517556+0000","last_clean":"2024-09-21T02:21:14.517556+0000","last_became_active":"2024-09-21T02:21:12.644827+0000","last_became_peered":"2024-09-21T02:21:12.644827+0000","last_unstale":"2024-09-21T02:21:14.517556+0000","last_undegraded":"2024-09-21T02:21:14.517556+0000","last_fullsized":"2024-09-21T02:21:14.517556+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:10.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639685+0000","last_change":"2024-09-21T02:21:12.680088+0000","last_active":"2024-09-21T02:21:14.639685+0000","last_peered":"2024-09-21T02:21:14.639685+0000","last_clean":"2024-09-21T02:21:14.639685+0000","last_became_active":"2024-09-21T02:21:12.679587+0000","last_became_peered":"2024-09-21T02:21:12.679587+0000","last_unstale":"2024-09-21T02:21:14.639685+0000","last_undegraded":"2024-09-21T02:21:14.639685+0000","last_fullsized":"2024-09-21T02:21:14.639685+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:19.183751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635763+0000","last_change":"2024-09-21T02:21:12.688250+0000","last_active":"2024-09-21T02:21:13.635763+0000","last_peered":"2024-09-21T02:21:13.635763+0000","last_clean":"2024-09-21T02:21:13.635763+0000","last_became_active":"2024-09-21T02:21:12.687644+0000","last_became_peered":"2024-09-21T02:21:12.687644+0000","last_unstale":"2024-09-21T02:21:13.635763+0000","last_undegraded":"2024-09-21T02:21:13.635763+0000","last_fullsized":"2024-09-21T02:21:13.635763+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:01.983454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862992+0000","last_change":"2024-09-21T02:21:12.690510+0000","last_active":"2024-09-21T02:21:14.862992+0000","last_peered":"2024-09-21T02:21:14.862992+0000","last_clean":"2024-09-21T02:21:14.862992+0000","last_became_active":"2024-09-21T02:21:12.686384+0000","last_became_peered":"2024-09-21T02:21:12.686384+0000","last_unstale":"2024-09-21T02:21:14.862992+0000","last_undegraded":"2024-09-21T02:21:14.862992+0000","last_fullsized":"2024-09-21T02:21:14.862992+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:01.174469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"141'4","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.641134+0000","last_change":"2024-09-21T02:21:11.672513+0000","last_active":"2024-09-21T02:21:14.641134+0000","last_peered":"2024-09-21T02:21:14.641134+0000","last_clean":"2024-09-21T02:21:14.641134+0000","last_became_active":"2024-09-21T02:21:11.672016+0000","last_became_peered":"2024-09-21T02:21:11.672016+0000","last_unstale":"2024-09-21T02:21:14.641134+0000","last_undegraded":"2024-09-21T02:21:14.641134+0000","last_fullsized":"2024-09-21T02:21:14.641134+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:20.387646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.641001+0000","last_change":"2024-09-21T02:21:12.657757+0000","last_active":"2024-09-21T02:21:14.641001+0000","last_peered":"2024-09-21T02:21:14.641001+0000","last_clean":"2024-09-21T02:21:14.641001+0000","last_became_active":"2024-09-21T02:21:12.657198+0000","last_became_peered":"2024-09-21T02:21:12.657198+0000","last_unstale":"2024-09-21T02:21:14.641001+0000","last_undegraded":"2024-09-21T02:21:14.641001+0000","last_fullsized":"2024-09-21T02:21:14.641001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:32.705267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518034+0000","last_change":"2024-09-21T02:21:11.649938+0000","last_active":"2024-09-21T02:21:14.518034+0000","last_peered":"2024-09-21T02:21:14.518034+0000","last_clean":"2024-09-21T02:21:14.518034+0000","last_became_active":"2024-09-21T02:21:11.649055+0000","last_became_peered":"2024-09-21T02:21:11.649055+0000","last_unstale":"2024-09-21T02:21:14.518034+0000","last_undegraded":"2024-09-21T02:21:14.518034+0000","last_fullsized":"2024-09-21T02:21:14.518034+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:13.560595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636605+0000","last_change":"2024-09-21T02:21:12.683714+0000","last_active":"2024-09-21T02:21:13.636605+0000","last_peered":"2024-09-21T02:21:13.636605+0000","last_clean":"2024-09-21T02:21:13.636605+0000","last_became_active":"2024-09-21T02:21:12.680809+0000","last_became_peered":"2024-09-21T02:21:12.680809+0000","last_unstale":"2024-09-21T02:21:13.636605+0000","last_undegraded":"2024-09-21T02:21:13.636605+0000","last_fullsized":"2024-09-21T02:21:13.636605+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:02.163749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635997+0000","last_change":"2024-09-21T02:21:11.666571+0000","last_active":"2024-09-21T02:21:13.635997+0000","last_peered":"2024-09-21T02:21:13.635997+0000","last_clean":"2024-09-21T02:21:13.635997+0000","last_became_active":"2024-09-21T02:21:11.665745+0000","last_became_peered":"2024-09-21T02:21:11.665745+0000","last_unstale":"2024-09-21T02:21:13.635997+0000","last_undegraded":"2024-09-21T02:21:13.635997+0000","last_fullsized":"2024-09-21T02:21:13.635997+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:05.063185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518214+0000","last_change":"2024-09-21T02:21:12.683442+0000","last_active":"2024-09-21T02:21:14.518214+0000","last_peered":"2024-09-21T02:21:14.518214+0000","last_clean":"2024-09-21T02:21:14.518214+0000","last_became_active":"2024-09-21T02:21:12.683037+0000","last_became_peered":"2024-09-21T02:21:12.683037+0000","last_unstale":"2024-09-21T02:21:14.518214+0000","last_undegraded":"2024-09-21T02:21:14.518214+0000","last_fullsized":"2024-09-21T02:21:14.518214+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:25.151852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638027+0000","last_change":"2024-09-21T02:21:11.652063+0000","last_active":"2024-09-21T02:21:13.638027+0000","last_peered":"2024-09-21T02:21:13.638027+0000","last_clean":"2024-09-21T02:21:13.638027+0000","last_became_active":"2024-09-21T02:21:11.651183+0000","last_became_peered":"2024-09-21T02:21:11.651183+0000","last_unstale":"2024-09-21T02:21:13.638027+0000","last_undegraded":"2024-09-21T02:21:13.638027+0000","last_fullsized":"2024-09-21T02:21:13.638027+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.390907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863232+0000","last_change":"2024-09-21T02:21:12.691108+0000","last_active":"2024-09-21T02:21:14.863232+0000","last_peered":"2024-09-21T02:21:14.863232+0000","last_clean":"2024-09-21T02:21:14.863232+0000","last_became_active":"2024-09-21T02:21:12.690566+0000","last_became_peered":"2024-09-21T02:21:12.690566+0000","last_unstale":"2024-09-21T02:21:14.863232+0000","last_undegraded":"2024-09-21T02:21:14.863232+0000","last_fullsized":"2024-09-21T02:21:14.863232+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:30.178084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640183+0000","last_change":"2024-09-21T02:21:11.663613+0000","last_active":"2024-09-21T02:21:14.640183+0000","last_peered":"2024-09-21T02:21:14.640183+0000","last_clean":"2024-09-21T02:21:14.640183+0000","last_became_active":"2024-09-21T02:21:11.663194+0000","last_became_peered":"2024-09-21T02:21:11.663194+0000","last_unstale":"2024-09-21T02:21:14.640183+0000","last_undegraded":"2024-09-21T02:21:14.640183+0000","last_fullsized":"2024-09-21T02:21:14.640183+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.094488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636803+0000","last_change":"2024-09-21T02:21:12.694301+0000","last_active":"2024-09-21T02:21:13.636803+0000","last_peered":"2024-09-21T02:21:13.636803+0000","last_clean":"2024-09-21T02:21:13.636803+0000","last_became_active":"2024-09-21T02:21:12.693692+0000","last_became_peered":"2024-09-21T02:21:12.693692+0000","last_unstale":"2024-09-21T02:21:13.636803+0000","last_undegraded":"2024-09-21T02:21:13.636803+0000","last_fullsized":"2024-09-21T02:21:13.636803+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:39.725490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862772+0000","last_change":"2024-09-21T02:21:11.655128+0000","last_active":"2024-09-21T02:21:14.862772+0000","last_peered":"2024-09-21T02:21:14.862772+0000","last_clean":"2024-09-21T02:21:14.862772+0000","last_became_active":"2024-09-21T02:21:11.654374+0000","last_became_peered":"2024-09-21T02:21:11.654374+0000","last_unstale":"2024-09-21T02:21:14.862772+0000","last_undegraded":"2024-09-21T02:21:14.862772+0000","last_fullsized":"2024-09-21T02:21:14.862772+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:59.628633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636552+0000","last_change":"2024-09-21T02:21:12.653304+0000","last_active":"2024-09-21T02:21:13.636552+0000","last_peered":"2024-09-21T02:21:13.636552+0000","last_clean":"2024-09-21T02:21:13.636552+0000","last_became_active":"2024-09-21T02:21:12.649190+0000","last_became_peered":"2024-09-21T02:21:12.649190+0000","last_unstale":"2024-09-21T02:21:13.636552+0000","last_undegraded":"2024-09-21T02:21:13.636552+0000","last_fullsized":"2024-09-21T02:21:13.636552+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:04.829698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639713+0000","last_change":"2024-09-21T02:21:11.673015+0000","last_active":"2024-09-21T02:21:14.639713+0000","last_peered":"2024-09-21T02:21:14.639713+0000","last_clean":"2024-09-21T02:21:14.639713+0000","last_became_active":"2024-09-21T02:21:11.671967+0000","last_became_peered":"2024-09-21T02:21:11.671967+0000","last_unstale":"2024-09-21T02:21:14.639713+0000","last_undegraded":"2024-09-21T02:21:14.639713+0000","last_fullsized":"2024-09-21T02:21:14.639713+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:23.690995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862420+0000","last_change":"2024-09-21T02:21:12.692476+0000","last_active":"2024-09-21T02:21:14.862420+0000","last_peered":"2024-09-21T02:21:14.862420+0000","last_clean":"2024-09-21T02:21:14.862420+0000","last_became_active":"2024-09-21T02:21:12.691881+0000","last_became_peered":"2024-09-21T02:21:12.691881+0000","last_unstale":"2024-09-21T02:21:14.862420+0000","last_undegraded":"2024-09-21T02:21:14.862420+0000","last_fullsized":"2024-09-21T02:21:14.862420+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:36.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636753+0000","last_change":"2024-09-21T02:21:11.668220+0000","last_active":"2024-09-21T02:21:13.636753+0000","last_peered":"2024-09-21T02:21:13.636753+0000","last_clean":"2024-09-21T02:21:13.636753+0000","last_became_active":"2024-09-21T02:21:11.664451+0000","last_became_peered":"2024-09-21T02:21:11.664451+0000","last_unstale":"2024-09-21T02:21:13.636753+0000","last_undegraded":"2024-09-21T02:21:13.636753+0000","last_fullsized":"2024-09-21T02:21:13.636753+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:30.242334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865538+0000","last_change":"2024-09-21T02:21:12.676467+0000","last_active":"2024-09-21T02:21:14.865538+0000","last_peered":"2024-09-21T02:21:14.865538+0000","last_clean":"2024-09-21T02:21:14.865538+0000","last_became_active":"2024-09-21T02:21:12.674933+0000","last_became_peered":"2024-09-21T02:21:12.674933+0000","last_unstale":"2024-09-21T02:21:14.865538+0000","last_undegraded":"2024-09-21T02:21:14.865538+0000","last_fullsized":"2024-09-21T02:21:14.865538+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:10.947619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.637030+0000","last_change":"2024-09-21T02:21:11.673836+0000","last_active":"2024-09-21T02:21:13.637030+0000","last_peered":"2024-09-21T02:21:13.637030+0000","last_clean":"2024-09-21T02:21:13.637030+0000","last_became_active":"2024-09-21T02:21:11.673065+0000","last_became_peered":"2024-09-21T02:21:11.673065+0000","last_unstale":"2024-09-21T02:21:13.637030+0000","last_undegraded":"2024-09-21T02:21:13.637030+0000","last_fullsized":"2024-09-21T02:21:13.637030+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:08.601736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865713+0000","last_change":"2024-09-21T02:21:12.675693+0000","last_active":"2024-09-21T02:21:14.865713+0000","last_peered":"2024-09-21T02:21:14.865713+0000","last_clean":"2024-09-21T02:21:14.865713+0000","last_became_active":"2024-09-21T02:21:12.674822+0000","last_became_peered":"2024-09-21T02:21:12.674822+0000","last_unstale":"2024-09-21T02:21:14.865713+0000","last_undegraded":"2024-09-21T02:21:14.865713+0000","last_fullsized":"2024-09-21T02:21:14.865713+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:08.443398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"141'3","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638286+0000","last_change":"2024-09-21T02:21:11.671036+0000","last_active":"2024-09-21T02:21:13.638286+0000","last_peered":"2024-09-21T02:21:13.638286+0000","last_clean":"2024-09-21T02:21:13.638286+0000","last_became_active":"2024-09-21T02:21:11.670106+0000","last_became_peered":"2024-09-21T02:21:11.670106+0000","last_unstale":"2024-09-21T02:21:13.638286+0000","last_undegraded":"2024-09-21T02:21:13.638286+0000","last_fullsized":"2024-09-21T02:21:13.638286+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:27.959717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640842+0000","last_change":"2024-09-21T02:21:12.652675+0000","last_active":"2024-09-21T02:21:14.640842+0000","last_peered":"2024-09-21T02:21:14.640842+0000","last_clean":"2024-09-21T02:21:14.640842+0000","last_became_active":"2024-09-21T02:21:12.652287+0000","last_became_peered":"2024-09-21T02:21:12.652287+0000","last_unstale":"2024-09-21T02:21:14.640842+0000","last_undegraded":"2024-09-21T02:21:14.640842+0000","last_fullsized":"2024-09-21T02:21:14.640842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:19.836373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"141'6","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861863+0000","last_change":"2024-09-21T02:21:11.668119+0000","last_active":"2024-09-21T02:21:14.861863+0000","last_peered":"2024-09-21T02:21:14.861863+0000","last_clean":"2024-09-21T02:21:14.861863+0000","last_became_active":"2024-09-21T02:21:11.665334+0000","last_became_peered":"2024-09-21T02:21:11.665334+0000","last_unstale":"2024-09-21T02:21:14.861863+0000","last_undegraded":"2024-09-21T02:21:14.861863+0000","last_fullsized":"2024-09-21T02:21:14.861863+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:00.112531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864412+0000","last_change":"2024-09-21T02:21:12.657519+0000","last_active":"2024-09-21T02:21:14.864412+0000","last_peered":"2024-09-21T02:21:14.864412+0000","last_clean":"2024-09-21T02:21:14.864412+0000","last_became_active":"2024-09-21T02:21:12.656650+0000","last_became_peered":"2024-09-21T02:21:12.656650+0000","last_unstale":"2024-09-21T02:21:14.864412+0000","last_undegraded":"2024-09-21T02:21:14.864412+0000","last_fullsized":"2024-09-21T02:21:14.864412+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:23.619967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862470+0000","last_change":"2024-09-21T02:21:11.664189+0000","last_active":"2024-09-21T02:21:14.862470+0000","last_peered":"2024-09-21T02:21:14.862470+0000","last_clean":"2024-09-21T02:21:14.862470+0000","last_became_active":"2024-09-21T02:21:11.663121+0000","last_became_peered":"2024-09-21T02:21:11.663121+0000","last_unstale":"2024-09-21T02:21:14.862470+0000","last_undegraded":"2024-09-21T02:21:14.862470+0000","last_fullsized":"2024-09-21T02:21:14.862470+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:46.443462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638939+0000","last_change":"2024-09-21T02:21:12.661821+0000","last_active":"2024-09-21T02:21:13.638939+0000","last_peered":"2024-09-21T02:21:13.638939+0000","last_clean":"2024-09-21T02:21:13.638939+0000","last_became_active":"2024-09-21T02:21:12.654243+0000","last_became_peered":"2024-09-21T02:21:12.654243+0000","last_unstale":"2024-09-21T02:21:13.638939+0000","last_undegraded":"2024-09-21T02:21:13.638939+0000","last_fullsized":"2024-09-21T02:21:13.638939+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:31.011889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636465+0000","last_change":"2024-09-21T02:21:11.668855+0000","last_active":"2024-09-21T02:21:13.636465+0000","last_peered":"2024-09-21T02:21:13.636465+0000","last_clean":"2024-09-21T02:21:13.636465+0000","last_became_active":"2024-09-21T02:21:11.668145+0000","last_became_peered":"2024-09-21T02:21:11.668145+0000","last_unstale":"2024-09-21T02:21:13.636465+0000","last_undegraded":"2024-09-21T02:21:13.636465+0000","last_fullsized":"2024-09-21T02:21:13.636465+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.680181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635850+0000","last_change":"2024-09-21T02:21:12.681547+0000","last_active":"2024-09-21T02:21:13.635850+0000","last_peered":"2024-09-21T02:21:13.635850+0000","last_clean":"2024-09-21T02:21:13.635850+0000","last_became_active":"2024-09-21T02:21:12.680756+0000","last_became_peered":"2024-09-21T02:21:12.680756+0000","last_unstale":"2024-09-21T02:21:13.635850+0000","last_undegraded":"2024-09-21T02:21:13.635850+0000","last_fullsized":"2024-09-21T02:21:13.635850+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:35.062212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865126+0000","last_change":"2024-09-21T02:21:11.670891+0000","last_active":"2024-09-21T02:21:14.865126+0000","last_peered":"2024-09-21T02:21:14.865126+0000","last_clean":"2024-09-21T02:21:14.865126+0000","last_became_active":"2024-09-21T02:21:11.670034+0000","last_became_peered":"2024-09-21T02:21:11.670034+0000","last_unstale":"2024-09-21T02:21:14.865126+0000","last_undegraded":"2024-09-21T02:21:14.865126+0000","last_fullsized":"2024-09-21T02:21:14.865126+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:16.771406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517663+0000","last_change":"2024-09-21T02:21:12.667739+0000","last_active":"2024-09-21T02:21:14.517663+0000","last_peered":"2024-09-21T02:21:14.517663+0000","last_clean":"2024-09-21T02:21:14.517663+0000","last_became_active":"2024-09-21T02:21:12.667455+0000","last_became_peered":"2024-09-21T02:21:12.667455+0000","last_unstale":"2024-09-21T02:21:14.517663+0000","last_undegraded":"2024-09-21T02:21:14.517663+0000","last_fullsized":"2024-09-21T02:21:14.517663+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:38.161318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"141'8","reported_seq":23,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861492+0000","last_change":"2024-09-21T02:21:11.669023+0000","last_active":"2024-09-21T02:21:14.861492+0000","last_peered":"2024-09-21T02:21:14.861492+0000","last_clean":"2024-09-21T02:21:14.861492+0000","last_became_active":"2024-09-21T02:21:11.665723+0000","last_became_peered":"2024-09-21T02:21:11.665723+0000","last_unstale":"2024-09-21T02:21:14.861492+0000","last_undegraded":"2024-09-21T02:21:14.861492+0000","last_fullsized":"2024-09-21T02:21:14.861492+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:48.330139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865414+0000","last_change":"2024-09-21T02:21:12.684723+0000","last_active":"2024-09-21T02:21:14.865414+0000","last_peered":"2024-09-21T02:21:14.865414+0000","last_clean":"2024-09-21T02:21:14.865414+0000","last_became_active":"2024-09-21T02:21:12.684059+0000","last_became_peered":"2024-09-21T02:21:12.684059+0000","last_unstale":"2024-09-21T02:21:14.865414+0000","last_undegraded":"2024-09-21T02:21:14.865414+0000","last_fullsized":"2024-09-21T02:21:14.865414+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:07.221606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862355+0000","last_change":"2024-09-21T02:21:11.668215+0000","last_active":"2024-09-21T02:21:14.862355+0000","last_peered":"2024-09-21T02:21:14.862355+0000","last_clean":"2024-09-21T02:21:14.862355+0000","last_became_active":"2024-09-21T02:21:11.667087+0000","last_became_peered":"2024-09-21T02:21:11.667087+0000","last_unstale":"2024-09-21T02:21:14.862355+0000","last_undegraded":"2024-09-21T02:21:14.862355+0000","last_fullsized":"2024-09-21T02:21:14.862355+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:21.695038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864027+0000","last_change":"2024-09-21T02:21:12.685002+0000","last_active":"2024-09-21T02:21:14.864027+0000","last_peered":"2024-09-21T02:21:14.864027+0000","last_clean":"2024-09-21T02:21:14.864027+0000","last_became_active":"2024-09-21T02:21:12.678126+0000","last_became_peered":"2024-09-21T02:21:12.678126+0000","last_unstale":"2024-09-21T02:21:14.864027+0000","last_undegraded":"2024-09-21T02:21:14.864027+0000","last_fullsized":"2024-09-21T02:21:14.864027+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:28.280903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518260+0000","last_change":"2024-09-21T02:21:11.669801+0000","last_active":"2024-09-21T02:21:14.518260+0000","last_peered":"2024-09-21T02:21:14.518260+0000","last_clean":"2024-09-21T02:21:14.518260+0000","last_became_active":"2024-09-21T02:21:11.669043+0000","last_became_peered":"2024-09-21T02:21:11.669043+0000","last_unstale":"2024-09-21T02:21:14.518260+0000","last_undegraded":"2024-09-21T02:21:14.518260+0000","last_fullsized":"2024-09-21T02:21:14.518260+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:49.124823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863520+0000","last_change":"2024-09-21T02:21:12.691615+0000","last_active":"2024-09-21T02:21:14.863520+0000","last_peered":"2024-09-21T02:21:14.863520+0000","last_clean":"2024-09-21T02:21:14.863520+0000","last_became_active":"2024-09-21T02:21:12.690630+0000","last_became_peered":"2024-09-21T02:21:12.690630+0000","last_unstale":"2024-09-21T02:21:14.863520+0000","last_undegraded":"2024-09-21T02:21:14.863520+0000","last_fullsized":"2024-09-21T02:21:14.863520+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:55.636074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862070+0000","last_change":"2024-09-21T02:21:11.666644+0000","last_active":"2024-09-21T02:21:14.862070+0000","last_peered":"2024-09-21T02:21:14.862070+0000","last_clean":"2024-09-21T02:21:14.862070+0000","last_became_active":"2024-09-21T02:21:11.665755+0000","last_became_peered":"2024-09-21T02:21:11.665755+0000","last_unstale":"2024-09-21T02:21:14.862070+0000","last_undegraded":"2024-09-21T02:21:14.862070+0000","last_fullsized":"2024-09-21T02:21:14.862070+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:30.663362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638797+0000","last_change":"2024-09-21T02:21:12.669916+0000","last_active":"2024-09-21T02:21:13.638797+0000","last_peered":"2024-09-21T02:21:13.638797+0000","last_clean":"2024-09-21T02:21:13.638797+0000","last_became_active":"2024-09-21T02:21:12.668825+0000","last_became_peered":"2024-09-21T02:21:12.668825+0000","last_unstale":"2024-09-21T02:21:13.638797+0000","last_undegraded":"2024-09-21T02:21:13.638797+0000","last_fullsized":"2024-09-21T02:21:13.638797+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:16.073590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.634281+0000","last_change":"2024-09-21T02:21:11.648087+0000","last_active":"2024-09-21T02:21:13.634281+0000","last_peered":"2024-09-21T02:21:13.634281+0000","last_clean":"2024-09-21T02:21:13.634281+0000","last_became_active":"2024-09-21T02:21:11.647677+0000","last_became_peered":"2024-09-21T02:21:11.647677+0000","last_unstale":"2024-09-21T02:21:13.634281+0000","last_undegraded":"2024-09-21T02:21:13.634281+0000","last_fullsized":"2024-09-21T02:21:13.634281+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:18.412204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639759+0000","last_change":"2024-09-21T02:21:12.694126+0000","last_active":"2024-09-21T02:21:14.639759+0000","last_peered":"2024-09-21T02:21:14.639759+0000","last_clean":"2024-09-21T02:21:14.639759+0000","last_became_active":"2024-09-21T02:21:12.693728+0000","last_became_peered":"2024-09-21T02:21:12.693728+0000","last_unstale":"2024-09-21T02:21:14.639759+0000","last_undegraded":"2024-09-21T02:21:14.639759+0000","last_fullsized":"2024-09-21T02:21:14.639759+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:05.491955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.639300+0000","last_change":"2024-09-21T02:21:11.658742+0000","last_active":"2024-09-21T02:21:13.639300+0000","last_peered":"2024-09-21T02:21:13.639300+0000","last_clean":"2024-09-21T02:21:13.639300+0000","last_became_active":"2024-09-21T02:21:11.657887+0000","last_became_peered":"2024-09-21T02:21:11.657887+0000","last_unstale":"2024-09-21T02:21:13.639300+0000","last_undegraded":"2024-09-21T02:21:13.639300+0000","last_fullsized":"2024-09-21T02:21:13.639300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:06.656862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638986+0000","last_change":"2024-09-21T02:21:12.667823+0000","last_active":"2024-09-21T02:21:13.638986+0000","last_peered":"2024-09-21T02:21:13.638986+0000","last_clean":"2024-09-21T02:21:13.638986+0000","last_became_active":"2024-09-21T02:21:12.666931+0000","last_became_peered":"2024-09-21T02:21:12.666931+0000","last_unstale":"2024-09-21T02:21:13.638986+0000","last_undegraded":"2024-09-21T02:21:13.638986+0000","last_fullsized":"2024-09-21T02:21:13.638986+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.441996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633574+0000","last_change":"2024-09-21T02:21:11.660664+0000","last_active":"2024-09-21T02:21:13.633574+0000","last_peered":"2024-09-21T02:21:13.633574+0000","last_clean":"2024-09-21T02:21:13.633574+0000","last_became_active":"2024-09-21T02:21:11.659805+0000","last_became_peered":"2024-09-21T02:21:11.659805+0000","last_unstale":"2024-09-21T02:21:13.633574+0000","last_undegraded":"2024-09-21T02:21:13.633574+0000","last_fullsized":"2024-09-21T02:21:13.633574+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:20:52.021191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639426+0000","last_change":"2024-09-21T02:21:12.650755+0000","last_active":"2024-09-21T02:21:14.639426+0000","last_peered":"2024-09-21T02:21:14.639426+0000","last_clean":"2024-09-21T02:21:14.639426+0000","last_became_active":"2024-09-21T02:21:12.650491+0000","last_became_peered":"2024-09-21T02:21:12.650491+0000","last_unstale":"2024-09-21T02:21:14.639426+0000","last_undegraded":"2024-09-21T02:21:14.639426+0000","last_fullsized":"2024-09-21T02:21:14.639426+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:42.573239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862778+0000","last_change":"2024-09-21T02:21:11.674291+0000","last_active":"2024-09-21T02:21:14.862778+0000","last_peered":"2024-09-21T02:21:14.862778+0000","last_clean":"2024-09-21T02:21:14.862778+0000","last_became_active":"2024-09-21T02:21:11.673583+0000","last_became_peered":"2024-09-21T02:21:11.673583+0000","last_unstale":"2024-09-21T02:21:14.862778+0000","last_undegraded":"2024-09-21T02:21:14.862778+0000","last_fullsized":"2024-09-21T02:21:14.862778+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.119498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864588+0000","last_change":"2024-09-21T02:21:12.663749+0000","last_active":"2024-09-21T02:21:14.864588+0000","last_peered":"2024-09-21T02:21:14.864588+0000","last_clean":"2024-09-21T02:21:14.864588+0000","last_became_active":"2024-09-21T02:21:12.662912+0000","last_became_peered":"2024-09-21T02:21:12.662912+0000","last_unstale":"2024-09-21T02:21:14.864588+0000","last_undegraded":"2024-09-21T02:21:14.864588+0000","last_fullsized":"2024-09-21T02:21:14.864588+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:28.936634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633657+0000","last_change":"2024-09-21T02:21:11.647147+0000","last_active":"2024-09-21T02:21:13.633657+0000","last_peered":"2024-09-21T02:21:13.633657+0000","last_clean":"2024-09-21T02:21:13.633657+0000","last_became_active":"2024-09-21T02:21:11.646267+0000","last_became_peered":"2024-09-21T02:21:11.646267+0000","last_unstale":"2024-09-21T02:21:13.633657+0000","last_undegraded":"2024-09-21T02:21:13.633657+0000","last_fullsized":"2024-09-21T02:21:13.633657+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:35.455966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638834+0000","last_change":"2024-09-21T02:21:12.680202+0000","last_active":"2024-09-21T02:21:13.638834+0000","last_peered":"2024-09-21T02:21:13.638834+0000","last_clean":"2024-09-21T02:21:13.638834+0000","last_became_active":"2024-09-21T02:21:12.679372+0000","last_became_peered":"2024-09-21T02:21:12.679372+0000","last_unstale":"2024-09-21T02:21:13.638834+0000","last_undegraded":"2024-09-21T02:21:13.638834+0000","last_fullsized":"2024-09-21T02:21:13.638834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:28.510082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.637252+0000","last_change":"2024-09-21T02:21:11.665529+0000","last_active":"2024-09-21T02:21:13.637252+0000","last_peered":"2024-09-21T02:21:13.637252+0000","last_clean":"2024-09-21T02:21:13.637252+0000","last_became_active":"2024-09-21T02:21:11.664594+0000","last_became_peered":"2024-09-21T02:21:11.664594+0000","last_unstale":"2024-09-21T02:21:13.637252+0000","last_undegraded":"2024-09-21T02:21:13.637252+0000","last_fullsized":"2024-09-21T02:21:13.637252+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:22.224561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864075+0000","last_change":"2024-09-21T02:21:12.684128+0000","last_active":"2024-09-21T02:21:14.864075+0000","last_peered":"2024-09-21T02:21:14.864075+0000","last_clean":"2024-09-21T02:21:14.864075+0000","last_became_active":"2024-09-21T02:21:12.676798+0000","last_became_peered":"2024-09-21T02:21:12.676798+0000","last_unstale":"2024-09-21T02:21:14.864075+0000","last_undegraded":"2024-09-21T02:21:14.864075+0000","last_fullsized":"2024-09-21T02:21:14.864075+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.744261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"142'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.637934+0000","last_change":"2024-09-21T02:21:11.657349+0000","last_active":"2024-09-21T02:21:14.637934+0000","last_peered":"2024-09-21T02:21:14.637934+0000","last_clean":"2024-09-21T02:21:14.637934+0000","last_became_active":"2024-09-21T02:21:11.655754+0000","last_became_peered":"2024-09-21T02:21:11.655754+0000","last_unstale":"2024-09-21T02:21:14.637934+0000","last_undegraded":"2024-09-21T02:21:14.637934+0000","last_fullsized":"2024-09-21T02:21:14.637934+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:25.874920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863352+0000","last_change":"2024-09-21T02:21:12.677929+0000","last_active":"2024-09-21T02:21:14.863352+0000","last_peered":"2024-09-21T02:21:14.863352+0000","last_clean":"2024-09-21T02:21:14.863352+0000","last_became_active":"2024-09-21T02:21:12.677249+0000","last_became_peered":"2024-09-21T02:21:12.677249+0000","last_unstale":"2024-09-21T02:21:14.863352+0000","last_undegraded":"2024-09-21T02:21:14.863352+0000","last_fullsized":"2024-09-21T02:21:14.863352+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:14.377293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640984+0000","last_change":"2024-09-21T02:21:11.675490+0000","last_active":"2024-09-21T02:21:14.640984+0000","last_peered":"2024-09-21T02:21:14.640984+0000","last_clean":"2024-09-21T02:21:14.640984+0000","last_became_active":"2024-09-21T02:21:11.672111+0000","last_became_peered":"2024-09-21T02:21:11.672111+0000","last_unstale":"2024-09-21T02:21:14.640984+0000","last_undegraded":"2024-09-21T02:21:14.640984+0000","last_fullsized":"2024-09-21T02:21:14.640984+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:42.971394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863378+0000","last_change":"2024-09-21T02:21:12.654460+0000","last_active":"2024-09-21T02:21:14.863378+0000","last_peered":"2024-09-21T02:21:14.863378+0000","last_clean":"2024-09-21T02:21:14.863378+0000","last_became_active":"2024-09-21T02:21:12.653559+0000","last_became_peered":"2024-09-21T02:21:12.653559+0000","last_unstale":"2024-09-21T02:21:14.863378+0000","last_undegraded":"2024-09-21T02:21:14.863378+0000","last_fullsized":"2024-09-21T02:21:14.863378+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:56.410047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638384+0000","last_change":"2024-09-21T02:21:11.664402+0000","last_active":"2024-09-21T02:21:13.638384+0000","last_peered":"2024-09-21T02:21:13.638384+0000","last_clean":"2024-09-21T02:21:13.638384+0000","last_became_active":"2024-09-21T02:21:11.661546+0000","last_became_peered":"2024-09-21T02:21:11.661546+0000","last_unstale":"2024-09-21T02:21:13.638384+0000","last_undegraded":"2024-09-21T02:21:13.638384+0000","last_fullsized":"2024-09-21T02:21:13.638384+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.732056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636399+0000","last_change":"2024-09-21T02:21:12.697718+0000","last_active":"2024-09-21T02:21:13.636399+0000","last_peered":"2024-09-21T02:21:13.636399+0000","last_clean":"2024-09-21T02:21:13.636399+0000","last_became_active":"2024-09-21T02:21:12.697241+0000","last_became_peered":"2024-09-21T02:21:12.697241+0000","last_unstale":"2024-09-21T02:21:13.636399+0000","last_undegraded":"2024-09-21T02:21:13.636399+0000","last_fullsized":"2024-09-21T02:21:13.636399+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:23.397612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861831+0000","last_change":"2024-09-21T02:21:11.659379+0000","last_active":"2024-09-21T02:21:14.861831+0000","last_peered":"2024-09-21T02:21:14.861831+0000","last_clean":"2024-09-21T02:21:14.861831+0000","last_became_active":"2024-09-21T02:21:11.658930+0000","last_became_peered":"2024-09-21T02:21:11.658930+0000","last_unstale":"2024-09-21T02:21:14.861831+0000","last_undegraded":"2024-09-21T02:21:14.861831+0000","last_fullsized":"2024-09-21T02:21:14.861831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:59.928726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640027+0000","last_change":"2024-09-21T02:21:12.661266+0000","last_active":"2024-09-21T02:21:14.640027+0000","last_peered":"2024-09-21T02:21:14.640027+0000","last_clean":"2024-09-21T02:21:14.640027+0000","last_became_active":"2024-09-21T02:21:12.660863+0000","last_became_peered":"2024-09-21T02:21:12.660863+0000","last_unstale":"2024-09-21T02:21:14.640027+0000","last_undegraded":"2024-09-21T02:21:14.640027+0000","last_fullsized":"2024-09-21T02:21:14.640027+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:07.663817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864451+0000","last_change":"2024-09-21T02:21:11.660950+0000","last_active":"2024-09-21T02:21:14.864451+0000","last_peered":"2024-09-21T02:21:14.864451+0000","last_clean":"2024-09-21T02:21:14.864451+0000","last_became_active":"2024-09-21T02:21:11.660176+0000","last_became_peered":"2024-09-21T02:21:11.660176+0000","last_unstale":"2024-09-21T02:21:14.864451+0000","last_undegraded":"2024-09-21T02:21:14.864451+0000","last_fullsized":"2024-09-21T02:21:14.864451+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:44.509348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863463+0000","last_change":"2024-09-21T02:21:12.659210+0000","last_active":"2024-09-21T02:21:14.863463+0000","last_peered":"2024-09-21T02:21:14.863463+0000","last_clean":"2024-09-21T02:21:14.863463+0000","last_became_active":"2024-09-21T02:21:12.656404+0000","last_became_peered":"2024-09-21T02:21:12.656404+0000","last_unstale":"2024-09-21T02:21:14.863463+0000","last_undegraded":"2024-09-21T02:21:14.863463+0000","last_fullsized":"2024-09-21T02:21:14.863463+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:58.254733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639927+0000","last_change":"2024-09-21T02:21:11.672994+0000","last_active":"2024-09-21T02:21:14.639927+0000","last_peered":"2024-09-21T02:21:14.639927+0000","last_clean":"2024-09-21T02:21:14.639927+0000","last_became_active":"2024-09-21T02:21:11.672607+0000","last_became_peered":"2024-09-21T02:21:11.672607+0000","last_unstale":"2024-09-21T02:21:14.639927+0000","last_undegraded":"2024-09-21T02:21:14.639927+0000","last_fullsized":"2024-09-21T02:21:14.639927+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:43.506094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639458+0000","last_change":"2024-09-21T02:21:12.674102+0000","last_active":"2024-09-21T02:21:14.639458+0000","last_peered":"2024-09-21T02:21:14.639458+0000","last_clean":"2024-09-21T02:21:14.639458+0000","last_became_active":"2024-09-21T02:21:12.673783+0000","last_became_peered":"2024-09-21T02:21:12.673783+0000","last_unstale":"2024-09-21T02:21:14.639458+0000","last_undegraded":"2024-09-21T02:21:14.639458+0000","last_fullsized":"2024-09-21T02:21:14.639458+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:31.249289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639048+0000","last_change":"2024-09-21T02:21:11.671159+0000","last_active":"2024-09-21T02:21:14.639048+0000","last_peered":"2024-09-21T02:21:14.639048+0000","last_clean":"2024-09-21T02:21:14.639048+0000","last_became_active":"2024-09-21T02:21:11.668339+0000","last_became_peered":"2024-09-21T02:21:11.668339+0000","last_unstale":"2024-09-21T02:21:14.639048+0000","last_undegraded":"2024-09-21T02:21:14.639048+0000","last_fullsized":"2024-09-21T02:21:14.639048+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:50.319183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865328+0000","last_change":"2024-09-21T02:21:12.686555+0000","last_active":"2024-09-21T02:21:14.865328+0000","last_peered":"2024-09-21T02:21:14.865328+0000","last_clean":"2024-09-21T02:21:14.865328+0000","last_became_active":"2024-09-21T02:21:12.685874+0000","last_became_peered":"2024-09-21T02:21:12.685874+0000","last_unstale":"2024-09-21T02:21:14.865328+0000","last_undegraded":"2024-09-21T02:21:14.865328+0000","last_fullsized":"2024-09-21T02:21:14.865328+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:36.084861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862276+0000","last_change":"2024-09-21T02:21:11.668611+0000","last_active":"2024-09-21T02:21:14.862276+0000","last_peered":"2024-09-21T02:21:14.862276+0000","last_clean":"2024-09-21T02:21:14.862276+0000","last_became_active":"2024-09-21T02:21:11.667875+0000","last_became_peered":"2024-09-21T02:21:11.667875+0000","last_unstale":"2024-09-21T02:21:14.862276+0000","last_undegraded":"2024-09-21T02:21:14.862276+0000","last_fullsized":"2024-09-21T02:21:14.862276+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:51.661874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":355,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861570+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:21:14.861570+0000","last_peered":"2024-09-21T02:21:14.861570+0000","last_clean":"2024-09-21T02:21:14.861570+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:21:14.861570+0000","last_undegraded":"2024-09-21T02:21:14.861570+0000","last_fullsized":"2024-09-21T02:21:14.861570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1777664,"data_stored":784451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1769472,"data_stored":778157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509539,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33512,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338328,"statfs":{"total":96636764160,"available":96602447872,"internally_reserved":0,"allocated":2379776,"data_stored":1375491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542244,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":778157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1794048,"data_stored":786085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1781760,"data_stored":785033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41844,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329996,"statfs":{"total":96636764160,"available":96593915904,"internally_reserved":0,"allocated":2383872,"data_stored":1370285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1781760,"data_stored":784521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:21:16.789 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:21:16.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 -- 172.21.15.170:0/2738948558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 msgr2=0x7f5d18003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 --2- 172.21.15.170:0/2738948558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d18003eb0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f5d20006fd0 tx=0x7f5d2001cfa0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 -- 172.21.15.170:0/2738948558 shutdown_connections 2024-09-21T02:21:16.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 --2- 172.21.15.170:0/2738948558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d18003eb0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 -- 172.21.15.170:0/2738948558 >> 172.21.15.170:0/2738948558 conn(0x7f5d300b1c40 msgr2=0x7f5d300b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:16.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 -- 172.21.15.170:0/2738948558 shutdown_connections 2024-09-21T02:21:16.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 -- 172.21.15.170:0/2738948558 wait complete. 2024-09-21T02:21:16.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 Processor -- start 2024-09-21T02:21:16.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.976+0000 7f5d35269640 1 -- start start 2024-09-21T02:21:16.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d301fa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d30156990 0x7f5d3014d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d300d0260 0x7f5d300d0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30070030 con 0x7f5d18003a60 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30074f00 con 0x7f5d30156990 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5d300701b0 con 0x7f5d300d0260 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d301fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d301fa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43144/0 (socket says 172.21.15.170:43144) 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 -- 172.21.15.170:0/1236838455 learned_addr learned my addr 172.21.15.170:0/1236838455 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2e7fc640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d30156990 0x7f5d3014d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 -- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d300d0260 msgr2=0x7f5d300d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d300d0260 0x7f5d300d0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 -- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d30156990 msgr2=0x7f5d3014d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d30156990 0x7f5d3014d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 -- 172.21.15.170:0/1236838455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d300d0bb0 con 0x7f5d18003a60 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d2effd640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d301fa870 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5d2001b6a0 tx=0x7f5d20006cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20001520 con 0x7f5d18003a60 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d200016c0 con 0x7f5d18003a60 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d301fb1c0 con 0x7f5d18003a60 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200064e0 con 0x7f5d18003a60 2024-09-21T02:21:16.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3015d190 con 0x7f5d18003a60 2024-09-21T02:21:16.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d300ca550 con 0x7f5d18003a60 2024-09-21T02:21:16.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.980+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f5d20003620 con 0x7f5d18003a60 2024-09-21T02:21:16.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.984+0000 7f5d177fe640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d1c0738f0 0x7f5d1c075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:16.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.984+0000 7f5d2e7fc640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d1c0738f0 0x7f5d1c075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.984+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f5d2012ed00 con 0x7f5d18003a60 2024-09-21T02:21:16.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.984+0000 7f5d2e7fc640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d1c0738f0 0x7f5d1c075e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005c50 tx=0x7f5d24005be0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:16.988+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5d200be060 con 0x7f5d18003a60 2024-09-21T02:21:17.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.144+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d300706a0 con 0x7f5d1c0738f0 2024-09-21T02:21:17.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.152+0000 7f5d177fe640 1 -- 172.21.15.170:0/1236838455 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f5d300706a0 con 0x7f5d1c0738f0 2024-09-21T02:21:17.156 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:17.158 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:21:17.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d1c0738f0 msgr2=0x7f5d1c075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d1c0738f0 0x7f5d1c075e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005c50 tx=0x7f5d24005be0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 msgr2=0x7f5d301fa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d301fa870 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5d2001b6a0 tx=0x7f5d20006cd0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 shutdown_connections 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5d1c0738f0 0x7f5d1c075e00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d300d0260 0x7f5d300d0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d30156990 0x7f5d3014d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 --2- 172.21.15.170:0/1236838455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d18003a60 0x7f5d301fa870 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 >> 172.21.15.170:0/1236838455 conn(0x7f5d300b1c40 msgr2=0x7f5d300aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 shutdown_connections 2024-09-21T02:21:17.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.156+0000 7f5d35269640 1 -- 172.21.15.170:0/1236838455 wait complete. 2024-09-21T02:21:17.176 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-21T02:21:16.921566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289560,"kb_used_data":15076,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754685160,"statfs":{"total":773094113280,"available":772797603840,"internally_reserved":0,"allocated":15437824,"data_stored":7442180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":280874545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001708"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640934+0000","last_change":"2024-09-21T02:21:12.657140+0000","last_active":"2024-09-21T02:21:14.640934+0000","last_peered":"2024-09-21T02:21:14.640934+0000","last_clean":"2024-09-21T02:21:14.640934+0000","last_became_active":"2024-09-21T02:21:12.656783+0000","last_became_peered":"2024-09-21T02:21:12.656783+0000","last_unstale":"2024-09-21T02:21:14.640934+0000","last_undegraded":"2024-09-21T02:21:14.640934+0000","last_fullsized":"2024-09-21T02:21:14.640934+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:47.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640725+0000","last_change":"2024-09-21T02:21:12.646566+0000","last_active":"2024-09-21T02:21:14.640725+0000","last_peered":"2024-09-21T02:21:14.640725+0000","last_clean":"2024-09-21T02:21:14.640725+0000","last_became_active":"2024-09-21T02:21:12.646230+0000","last_became_peered":"2024-09-21T02:21:12.646230+0000","last_unstale":"2024-09-21T02:21:14.640725+0000","last_undegraded":"2024-09-21T02:21:14.640725+0000","last_fullsized":"2024-09-21T02:21:14.640725+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:22.082763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640246+0000","last_change":"2024-09-21T02:21:12.650516+0000","last_active":"2024-09-21T02:21:14.640246+0000","last_peered":"2024-09-21T02:21:14.640246+0000","last_clean":"2024-09-21T02:21:14.640246+0000","last_became_active":"2024-09-21T02:21:12.650136+0000","last_became_peered":"2024-09-21T02:21:12.650136+0000","last_unstale":"2024-09-21T02:21:14.640246+0000","last_undegraded":"2024-09-21T02:21:14.640246+0000","last_fullsized":"2024-09-21T02:21:14.640246+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:53:01.115540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862522+0000","last_change":"2024-09-21T02:21:12.686783+0000","last_active":"2024-09-21T02:21:14.862522+0000","last_peered":"2024-09-21T02:21:14.862522+0000","last_clean":"2024-09-21T02:21:14.862522+0000","last_became_active":"2024-09-21T02:21:12.686227+0000","last_became_peered":"2024-09-21T02:21:12.686227+0000","last_unstale":"2024-09-21T02:21:14.862522+0000","last_undegraded":"2024-09-21T02:21:14.862522+0000","last_fullsized":"2024-09-21T02:21:14.862522+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:15.635175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861701+0000","last_change":"2024-09-21T02:21:12.648176+0000","last_active":"2024-09-21T02:21:14.861701+0000","last_peered":"2024-09-21T02:21:14.861701+0000","last_clean":"2024-09-21T02:21:14.861701+0000","last_became_active":"2024-09-21T02:21:12.647830+0000","last_became_peered":"2024-09-21T02:21:12.647830+0000","last_unstale":"2024-09-21T02:21:14.861701+0000","last_undegraded":"2024-09-21T02:21:14.861701+0000","last_fullsized":"2024-09-21T02:21:14.861701+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:48.046120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635893+0000","last_change":"2024-09-21T02:21:12.691351+0000","last_active":"2024-09-21T02:21:13.635893+0000","last_peered":"2024-09-21T02:21:13.635893+0000","last_clean":"2024-09-21T02:21:13.635893+0000","last_became_active":"2024-09-21T02:21:12.689586+0000","last_became_peered":"2024-09-21T02:21:12.689586+0000","last_unstale":"2024-09-21T02:21:13.635893+0000","last_undegraded":"2024-09-21T02:21:13.635893+0000","last_fullsized":"2024-09-21T02:21:13.635893+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:30:59.902935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864538+0000","last_change":"2024-09-21T02:21:12.661496+0000","last_active":"2024-09-21T02:21:14.864538+0000","last_peered":"2024-09-21T02:21:14.864538+0000","last_clean":"2024-09-21T02:21:14.864538+0000","last_became_active":"2024-09-21T02:21:12.659318+0000","last_became_peered":"2024-09-21T02:21:12.659318+0000","last_unstale":"2024-09-21T02:21:14.864538+0000","last_undegraded":"2024-09-21T02:21:14.864538+0000","last_fullsized":"2024-09-21T02:21:14.864538+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:52.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517564+0000","last_change":"2024-09-21T02:21:12.649101+0000","last_active":"2024-09-21T02:21:14.517564+0000","last_peered":"2024-09-21T02:21:14.517564+0000","last_clean":"2024-09-21T02:21:14.517564+0000","last_became_active":"2024-09-21T02:21:12.648733+0000","last_became_peered":"2024-09-21T02:21:12.648733+0000","last_unstale":"2024-09-21T02:21:14.517564+0000","last_undegraded":"2024-09-21T02:21:14.517564+0000","last_fullsized":"2024-09-21T02:21:14.517564+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:47.316365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518030+0000","last_change":"2024-09-21T02:21:12.693775+0000","last_active":"2024-09-21T02:21:14.518030+0000","last_peered":"2024-09-21T02:21:14.518030+0000","last_clean":"2024-09-21T02:21:14.518030+0000","last_became_active":"2024-09-21T02:21:12.693465+0000","last_became_peered":"2024-09-21T02:21:12.693465+0000","last_unstale":"2024-09-21T02:21:14.518030+0000","last_undegraded":"2024-09-21T02:21:14.518030+0000","last_fullsized":"2024-09-21T02:21:14.518030+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:26.309641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518033+0000","last_change":"2024-09-21T02:21:12.687333+0000","last_active":"2024-09-21T02:21:14.518033+0000","last_peered":"2024-09-21T02:21:14.518033+0000","last_clean":"2024-09-21T02:21:14.518033+0000","last_became_active":"2024-09-21T02:21:12.687057+0000","last_became_peered":"2024-09-21T02:21:12.687057+0000","last_unstale":"2024-09-21T02:21:14.518033+0000","last_undegraded":"2024-09-21T02:21:14.518033+0000","last_fullsized":"2024-09-21T02:21:14.518033+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:04.829698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861994+0000","last_change":"2024-09-21T02:21:12.684093+0000","last_active":"2024-09-21T02:21:14.861994+0000","last_peered":"2024-09-21T02:21:14.861994+0000","last_clean":"2024-09-21T02:21:14.861994+0000","last_became_active":"2024-09-21T02:21:12.683615+0000","last_became_peered":"2024-09-21T02:21:12.683615+0000","last_unstale":"2024-09-21T02:21:14.861994+0000","last_undegraded":"2024-09-21T02:21:14.861994+0000","last_fullsized":"2024-09-21T02:21:14.861994+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:27.588267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633971+0000","last_change":"2024-09-21T02:21:12.660970+0000","last_active":"2024-09-21T02:21:13.633971+0000","last_peered":"2024-09-21T02:21:13.633971+0000","last_clean":"2024-09-21T02:21:13.633971+0000","last_became_active":"2024-09-21T02:21:12.660621+0000","last_became_peered":"2024-09-21T02:21:12.660621+0000","last_unstale":"2024-09-21T02:21:13.633971+0000","last_undegraded":"2024-09-21T02:21:13.633971+0000","last_fullsized":"2024-09-21T02:21:13.633971+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:33.945102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633523+0000","last_change":"2024-09-21T02:21:12.680944+0000","last_active":"2024-09-21T02:21:13.633523+0000","last_peered":"2024-09-21T02:21:13.633523+0000","last_clean":"2024-09-21T02:21:13.633523+0000","last_became_active":"2024-09-21T02:21:12.680642+0000","last_became_peered":"2024-09-21T02:21:12.680642+0000","last_unstale":"2024-09-21T02:21:13.633523+0000","last_undegraded":"2024-09-21T02:21:13.633523+0000","last_fullsized":"2024-09-21T02:21:13.633523+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:01.368078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518012+0000","last_change":"2024-09-21T02:21:12.645681+0000","last_active":"2024-09-21T02:21:14.518012+0000","last_peered":"2024-09-21T02:21:14.518012+0000","last_clean":"2024-09-21T02:21:14.518012+0000","last_became_active":"2024-09-21T02:21:12.645150+0000","last_became_peered":"2024-09-21T02:21:12.645150+0000","last_unstale":"2024-09-21T02:21:14.518012+0000","last_undegraded":"2024-09-21T02:21:14.518012+0000","last_fullsized":"2024-09-21T02:21:14.518012+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:02.832696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517815+0000","last_change":"2024-09-21T02:21:12.690790+0000","last_active":"2024-09-21T02:21:14.517815+0000","last_peered":"2024-09-21T02:21:14.517815+0000","last_clean":"2024-09-21T02:21:14.517815+0000","last_became_active":"2024-09-21T02:21:12.690538+0000","last_became_peered":"2024-09-21T02:21:12.690538+0000","last_unstale":"2024-09-21T02:21:14.517815+0000","last_undegraded":"2024-09-21T02:21:14.517815+0000","last_fullsized":"2024-09-21T02:21:14.517815+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:48.769696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862691+0000","last_change":"2024-09-21T02:21:12.658615+0000","last_active":"2024-09-21T02:21:14.862691+0000","last_peered":"2024-09-21T02:21:14.862691+0000","last_clean":"2024-09-21T02:21:14.862691+0000","last_became_active":"2024-09-21T02:21:12.657389+0000","last_became_peered":"2024-09-21T02:21:12.657389+0000","last_unstale":"2024-09-21T02:21:14.862691+0000","last_undegraded":"2024-09-21T02:21:14.862691+0000","last_fullsized":"2024-09-21T02:21:14.862691+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:40.353809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.637362+0000","last_change":"2024-09-21T02:21:12.687582+0000","last_active":"2024-09-21T02:21:13.637362+0000","last_peered":"2024-09-21T02:21:13.637362+0000","last_clean":"2024-09-21T02:21:13.637362+0000","last_became_active":"2024-09-21T02:21:12.686744+0000","last_became_peered":"2024-09-21T02:21:12.686744+0000","last_unstale":"2024-09-21T02:21:13.637362+0000","last_undegraded":"2024-09-21T02:21:13.637362+0000","last_fullsized":"2024-09-21T02:21:13.637362+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:19.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862761+0000","last_change":"2024-09-21T02:21:12.678221+0000","last_active":"2024-09-21T02:21:14.862761+0000","last_peered":"2024-09-21T02:21:14.862761+0000","last_clean":"2024-09-21T02:21:14.862761+0000","last_became_active":"2024-09-21T02:21:12.677445+0000","last_became_peered":"2024-09-21T02:21:12.677445+0000","last_unstale":"2024-09-21T02:21:14.862761+0000","last_undegraded":"2024-09-21T02:21:14.862761+0000","last_fullsized":"2024-09-21T02:21:14.862761+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:17.092471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865545+0000","last_change":"2024-09-21T02:21:12.657681+0000","last_active":"2024-09-21T02:21:14.865545+0000","last_peered":"2024-09-21T02:21:14.865545+0000","last_clean":"2024-09-21T02:21:14.865545+0000","last_became_active":"2024-09-21T02:21:12.656726+0000","last_became_peered":"2024-09-21T02:21:12.656726+0000","last_unstale":"2024-09-21T02:21:14.865545+0000","last_undegraded":"2024-09-21T02:21:14.865545+0000","last_fullsized":"2024-09-21T02:21:14.865545+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:04.828657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.634209+0000","last_change":"2024-09-21T02:21:12.654055+0000","last_active":"2024-09-21T02:21:13.634209+0000","last_peered":"2024-09-21T02:21:13.634209+0000","last_clean":"2024-09-21T02:21:13.634209+0000","last_became_active":"2024-09-21T02:21:12.653691+0000","last_became_peered":"2024-09-21T02:21:12.653691+0000","last_unstale":"2024-09-21T02:21:13.634209+0000","last_undegraded":"2024-09-21T02:21:13.634209+0000","last_fullsized":"2024-09-21T02:21:13.634209+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:15.586055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636157+0000","last_change":"2024-09-21T02:21:12.687239+0000","last_active":"2024-09-21T02:21:13.636157+0000","last_peered":"2024-09-21T02:21:13.636157+0000","last_clean":"2024-09-21T02:21:13.636157+0000","last_became_active":"2024-09-21T02:21:12.686136+0000","last_became_peered":"2024-09-21T02:21:12.686136+0000","last_unstale":"2024-09-21T02:21:13.636157+0000","last_undegraded":"2024-09-21T02:21:13.636157+0000","last_fullsized":"2024-09-21T02:21:13.636157+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:39.643004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861651+0000","last_change":"2024-09-21T02:21:12.683850+0000","last_active":"2024-09-21T02:21:14.861651+0000","last_peered":"2024-09-21T02:21:14.861651+0000","last_clean":"2024-09-21T02:21:14.861651+0000","last_became_active":"2024-09-21T02:21:12.683521+0000","last_became_peered":"2024-09-21T02:21:12.683521+0000","last_unstale":"2024-09-21T02:21:14.861651+0000","last_undegraded":"2024-09-21T02:21:14.861651+0000","last_fullsized":"2024-09-21T02:21:14.861651+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:26.955870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638365+0000","last_change":"2024-09-21T02:21:12.644867+0000","last_active":"2024-09-21T02:21:13.638365+0000","last_peered":"2024-09-21T02:21:13.638365+0000","last_clean":"2024-09-21T02:21:13.638365+0000","last_became_active":"2024-09-21T02:21:12.643984+0000","last_became_peered":"2024-09-21T02:21:12.643984+0000","last_unstale":"2024-09-21T02:21:13.638365+0000","last_undegraded":"2024-09-21T02:21:13.638365+0000","last_fullsized":"2024-09-21T02:21:13.638365+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:25.151852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861509+0000","last_change":"2024-09-21T02:21:12.648414+0000","last_active":"2024-09-21T02:21:14.861509+0000","last_peered":"2024-09-21T02:21:14.861509+0000","last_clean":"2024-09-21T02:21:14.861509+0000","last_became_active":"2024-09-21T02:21:12.648066+0000","last_became_peered":"2024-09-21T02:21:12.648066+0000","last_unstale":"2024-09-21T02:21:14.861509+0000","last_undegraded":"2024-09-21T02:21:14.861509+0000","last_fullsized":"2024-09-21T02:21:14.861509+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:15.965614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862207+0000","last_change":"2024-09-21T02:21:12.692205+0000","last_active":"2024-09-21T02:21:14.862207+0000","last_peered":"2024-09-21T02:21:14.862207+0000","last_clean":"2024-09-21T02:21:14.862207+0000","last_became_active":"2024-09-21T02:21:12.691837+0000","last_became_peered":"2024-09-21T02:21:12.691837+0000","last_unstale":"2024-09-21T02:21:14.862207+0000","last_undegraded":"2024-09-21T02:21:14.862207+0000","last_fullsized":"2024-09-21T02:21:14.862207+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:30.686170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638655+0000","last_change":"2024-09-21T02:21:12.678295+0000","last_active":"2024-09-21T02:21:13.638655+0000","last_peered":"2024-09-21T02:21:13.638655+0000","last_clean":"2024-09-21T02:21:13.638655+0000","last_became_active":"2024-09-21T02:21:12.677499+0000","last_became_peered":"2024-09-21T02:21:12.677499+0000","last_unstale":"2024-09-21T02:21:13.638655+0000","last_undegraded":"2024-09-21T02:21:13.638655+0000","last_fullsized":"2024-09-21T02:21:13.638655+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:22:33.803430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862073+0000","last_change":"2024-09-21T02:21:12.657619+0000","last_active":"2024-09-21T02:21:14.862073+0000","last_peered":"2024-09-21T02:21:14.862073+0000","last_clean":"2024-09-21T02:21:14.862073+0000","last_became_active":"2024-09-21T02:21:12.657298+0000","last_became_peered":"2024-09-21T02:21:12.657298+0000","last_unstale":"2024-09-21T02:21:14.862073+0000","last_undegraded":"2024-09-21T02:21:14.862073+0000","last_fullsized":"2024-09-21T02:21:14.862073+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:14.425805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865346+0000","last_change":"2024-09-21T02:21:12.657779+0000","last_active":"2024-09-21T02:21:14.865346+0000","last_peered":"2024-09-21T02:21:14.865346+0000","last_clean":"2024-09-21T02:21:14.865346+0000","last_became_active":"2024-09-21T02:21:12.656796+0000","last_became_peered":"2024-09-21T02:21:12.656796+0000","last_unstale":"2024-09-21T02:21:14.865346+0000","last_undegraded":"2024-09-21T02:21:14.865346+0000","last_fullsized":"2024-09-21T02:21:14.865346+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:20:20.656513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639985+0000","last_change":"2024-09-21T02:21:12.655801+0000","last_active":"2024-09-21T02:21:14.639985+0000","last_peered":"2024-09-21T02:21:14.639985+0000","last_clean":"2024-09-21T02:21:14.639985+0000","last_became_active":"2024-09-21T02:21:12.655459+0000","last_became_peered":"2024-09-21T02:21:12.655459+0000","last_unstale":"2024-09-21T02:21:14.639985+0000","last_undegraded":"2024-09-21T02:21:14.639985+0000","last_fullsized":"2024-09-21T02:21:14.639985+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:42:21.852819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517556+0000","last_change":"2024-09-21T02:21:12.645458+0000","last_active":"2024-09-21T02:21:14.517556+0000","last_peered":"2024-09-21T02:21:14.517556+0000","last_clean":"2024-09-21T02:21:14.517556+0000","last_became_active":"2024-09-21T02:21:12.644827+0000","last_became_peered":"2024-09-21T02:21:12.644827+0000","last_unstale":"2024-09-21T02:21:14.517556+0000","last_undegraded":"2024-09-21T02:21:14.517556+0000","last_fullsized":"2024-09-21T02:21:14.517556+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:10.210563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639685+0000","last_change":"2024-09-21T02:21:12.680088+0000","last_active":"2024-09-21T02:21:14.639685+0000","last_peered":"2024-09-21T02:21:14.639685+0000","last_clean":"2024-09-21T02:21:14.639685+0000","last_became_active":"2024-09-21T02:21:12.679587+0000","last_became_peered":"2024-09-21T02:21:12.679587+0000","last_unstale":"2024-09-21T02:21:14.639685+0000","last_undegraded":"2024-09-21T02:21:14.639685+0000","last_fullsized":"2024-09-21T02:21:14.639685+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:19.183751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635763+0000","last_change":"2024-09-21T02:21:12.688250+0000","last_active":"2024-09-21T02:21:13.635763+0000","last_peered":"2024-09-21T02:21:13.635763+0000","last_clean":"2024-09-21T02:21:13.635763+0000","last_became_active":"2024-09-21T02:21:12.687644+0000","last_became_peered":"2024-09-21T02:21:12.687644+0000","last_unstale":"2024-09-21T02:21:13.635763+0000","last_undegraded":"2024-09-21T02:21:13.635763+0000","last_fullsized":"2024-09-21T02:21:13.635763+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:01.983454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862992+0000","last_change":"2024-09-21T02:21:12.690510+0000","last_active":"2024-09-21T02:21:14.862992+0000","last_peered":"2024-09-21T02:21:14.862992+0000","last_clean":"2024-09-21T02:21:14.862992+0000","last_became_active":"2024-09-21T02:21:12.686384+0000","last_became_peered":"2024-09-21T02:21:12.686384+0000","last_unstale":"2024-09-21T02:21:14.862992+0000","last_undegraded":"2024-09-21T02:21:14.862992+0000","last_fullsized":"2024-09-21T02:21:14.862992+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:01.174469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"141'4","reported_seq":19,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.641134+0000","last_change":"2024-09-21T02:21:11.672513+0000","last_active":"2024-09-21T02:21:14.641134+0000","last_peered":"2024-09-21T02:21:14.641134+0000","last_clean":"2024-09-21T02:21:14.641134+0000","last_became_active":"2024-09-21T02:21:11.672016+0000","last_became_peered":"2024-09-21T02:21:11.672016+0000","last_unstale":"2024-09-21T02:21:14.641134+0000","last_undegraded":"2024-09-21T02:21:14.641134+0000","last_fullsized":"2024-09-21T02:21:14.641134+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:20.387646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.641001+0000","last_change":"2024-09-21T02:21:12.657757+0000","last_active":"2024-09-21T02:21:14.641001+0000","last_peered":"2024-09-21T02:21:14.641001+0000","last_clean":"2024-09-21T02:21:14.641001+0000","last_became_active":"2024-09-21T02:21:12.657198+0000","last_became_peered":"2024-09-21T02:21:12.657198+0000","last_unstale":"2024-09-21T02:21:14.641001+0000","last_undegraded":"2024-09-21T02:21:14.641001+0000","last_fullsized":"2024-09-21T02:21:14.641001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:11:32.705267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518034+0000","last_change":"2024-09-21T02:21:11.649938+0000","last_active":"2024-09-21T02:21:14.518034+0000","last_peered":"2024-09-21T02:21:14.518034+0000","last_clean":"2024-09-21T02:21:14.518034+0000","last_became_active":"2024-09-21T02:21:11.649055+0000","last_became_peered":"2024-09-21T02:21:11.649055+0000","last_unstale":"2024-09-21T02:21:14.518034+0000","last_undegraded":"2024-09-21T02:21:14.518034+0000","last_fullsized":"2024-09-21T02:21:14.518034+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:13.560595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636605+0000","last_change":"2024-09-21T02:21:12.683714+0000","last_active":"2024-09-21T02:21:13.636605+0000","last_peered":"2024-09-21T02:21:13.636605+0000","last_clean":"2024-09-21T02:21:13.636605+0000","last_became_active":"2024-09-21T02:21:12.680809+0000","last_became_peered":"2024-09-21T02:21:12.680809+0000","last_unstale":"2024-09-21T02:21:13.636605+0000","last_undegraded":"2024-09-21T02:21:13.636605+0000","last_fullsized":"2024-09-21T02:21:13.636605+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:02.163749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635997+0000","last_change":"2024-09-21T02:21:11.666571+0000","last_active":"2024-09-21T02:21:13.635997+0000","last_peered":"2024-09-21T02:21:13.635997+0000","last_clean":"2024-09-21T02:21:13.635997+0000","last_became_active":"2024-09-21T02:21:11.665745+0000","last_became_peered":"2024-09-21T02:21:11.665745+0000","last_unstale":"2024-09-21T02:21:13.635997+0000","last_undegraded":"2024-09-21T02:21:13.635997+0000","last_fullsized":"2024-09-21T02:21:13.635997+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:05.063185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518214+0000","last_change":"2024-09-21T02:21:12.683442+0000","last_active":"2024-09-21T02:21:14.518214+0000","last_peered":"2024-09-21T02:21:14.518214+0000","last_clean":"2024-09-21T02:21:14.518214+0000","last_became_active":"2024-09-21T02:21:12.683037+0000","last_became_peered":"2024-09-21T02:21:12.683037+0000","last_unstale":"2024-09-21T02:21:14.518214+0000","last_undegraded":"2024-09-21T02:21:14.518214+0000","last_fullsized":"2024-09-21T02:21:14.518214+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:25.151852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638027+0000","last_change":"2024-09-21T02:21:11.652063+0000","last_active":"2024-09-21T02:21:13.638027+0000","last_peered":"2024-09-21T02:21:13.638027+0000","last_clean":"2024-09-21T02:21:13.638027+0000","last_became_active":"2024-09-21T02:21:11.651183+0000","last_became_peered":"2024-09-21T02:21:11.651183+0000","last_unstale":"2024-09-21T02:21:13.638027+0000","last_undegraded":"2024-09-21T02:21:13.638027+0000","last_fullsized":"2024-09-21T02:21:13.638027+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.390907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863232+0000","last_change":"2024-09-21T02:21:12.691108+0000","last_active":"2024-09-21T02:21:14.863232+0000","last_peered":"2024-09-21T02:21:14.863232+0000","last_clean":"2024-09-21T02:21:14.863232+0000","last_became_active":"2024-09-21T02:21:12.690566+0000","last_became_peered":"2024-09-21T02:21:12.690566+0000","last_unstale":"2024-09-21T02:21:14.863232+0000","last_undegraded":"2024-09-21T02:21:14.863232+0000","last_fullsized":"2024-09-21T02:21:14.863232+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:30.178084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640183+0000","last_change":"2024-09-21T02:21:11.663613+0000","last_active":"2024-09-21T02:21:14.640183+0000","last_peered":"2024-09-21T02:21:14.640183+0000","last_clean":"2024-09-21T02:21:14.640183+0000","last_became_active":"2024-09-21T02:21:11.663194+0000","last_became_peered":"2024-09-21T02:21:11.663194+0000","last_unstale":"2024-09-21T02:21:14.640183+0000","last_undegraded":"2024-09-21T02:21:14.640183+0000","last_fullsized":"2024-09-21T02:21:14.640183+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.094488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636803+0000","last_change":"2024-09-21T02:21:12.694301+0000","last_active":"2024-09-21T02:21:13.636803+0000","last_peered":"2024-09-21T02:21:13.636803+0000","last_clean":"2024-09-21T02:21:13.636803+0000","last_became_active":"2024-09-21T02:21:12.693692+0000","last_became_peered":"2024-09-21T02:21:12.693692+0000","last_unstale":"2024-09-21T02:21:13.636803+0000","last_undegraded":"2024-09-21T02:21:13.636803+0000","last_fullsized":"2024-09-21T02:21:13.636803+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:12:39.725490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862772+0000","last_change":"2024-09-21T02:21:11.655128+0000","last_active":"2024-09-21T02:21:14.862772+0000","last_peered":"2024-09-21T02:21:14.862772+0000","last_clean":"2024-09-21T02:21:14.862772+0000","last_became_active":"2024-09-21T02:21:11.654374+0000","last_became_peered":"2024-09-21T02:21:11.654374+0000","last_unstale":"2024-09-21T02:21:14.862772+0000","last_undegraded":"2024-09-21T02:21:14.862772+0000","last_fullsized":"2024-09-21T02:21:14.862772+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:59.628633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636552+0000","last_change":"2024-09-21T02:21:12.653304+0000","last_active":"2024-09-21T02:21:13.636552+0000","last_peered":"2024-09-21T02:21:13.636552+0000","last_clean":"2024-09-21T02:21:13.636552+0000","last_became_active":"2024-09-21T02:21:12.649190+0000","last_became_peered":"2024-09-21T02:21:12.649190+0000","last_unstale":"2024-09-21T02:21:13.636552+0000","last_undegraded":"2024-09-21T02:21:13.636552+0000","last_fullsized":"2024-09-21T02:21:13.636552+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:04.829698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639713+0000","last_change":"2024-09-21T02:21:11.673015+0000","last_active":"2024-09-21T02:21:14.639713+0000","last_peered":"2024-09-21T02:21:14.639713+0000","last_clean":"2024-09-21T02:21:14.639713+0000","last_became_active":"2024-09-21T02:21:11.671967+0000","last_became_peered":"2024-09-21T02:21:11.671967+0000","last_unstale":"2024-09-21T02:21:14.639713+0000","last_undegraded":"2024-09-21T02:21:14.639713+0000","last_fullsized":"2024-09-21T02:21:14.639713+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:23.690995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862420+0000","last_change":"2024-09-21T02:21:12.692476+0000","last_active":"2024-09-21T02:21:14.862420+0000","last_peered":"2024-09-21T02:21:14.862420+0000","last_clean":"2024-09-21T02:21:14.862420+0000","last_became_active":"2024-09-21T02:21:12.691881+0000","last_became_peered":"2024-09-21T02:21:12.691881+0000","last_unstale":"2024-09-21T02:21:14.862420+0000","last_undegraded":"2024-09-21T02:21:14.862420+0000","last_fullsized":"2024-09-21T02:21:14.862420+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:36.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636753+0000","last_change":"2024-09-21T02:21:11.668220+0000","last_active":"2024-09-21T02:21:13.636753+0000","last_peered":"2024-09-21T02:21:13.636753+0000","last_clean":"2024-09-21T02:21:13.636753+0000","last_became_active":"2024-09-21T02:21:11.664451+0000","last_became_peered":"2024-09-21T02:21:11.664451+0000","last_unstale":"2024-09-21T02:21:13.636753+0000","last_undegraded":"2024-09-21T02:21:13.636753+0000","last_fullsized":"2024-09-21T02:21:13.636753+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:30.242334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865538+0000","last_change":"2024-09-21T02:21:12.676467+0000","last_active":"2024-09-21T02:21:14.865538+0000","last_peered":"2024-09-21T02:21:14.865538+0000","last_clean":"2024-09-21T02:21:14.865538+0000","last_became_active":"2024-09-21T02:21:12.674933+0000","last_became_peered":"2024-09-21T02:21:12.674933+0000","last_unstale":"2024-09-21T02:21:14.865538+0000","last_undegraded":"2024-09-21T02:21:14.865538+0000","last_fullsized":"2024-09-21T02:21:14.865538+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:10.947619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.637030+0000","last_change":"2024-09-21T02:21:11.673836+0000","last_active":"2024-09-21T02:21:13.637030+0000","last_peered":"2024-09-21T02:21:13.637030+0000","last_clean":"2024-09-21T02:21:13.637030+0000","last_became_active":"2024-09-21T02:21:11.673065+0000","last_became_peered":"2024-09-21T02:21:11.673065+0000","last_unstale":"2024-09-21T02:21:13.637030+0000","last_undegraded":"2024-09-21T02:21:13.637030+0000","last_fullsized":"2024-09-21T02:21:13.637030+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:08.601736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865713+0000","last_change":"2024-09-21T02:21:12.675693+0000","last_active":"2024-09-21T02:21:14.865713+0000","last_peered":"2024-09-21T02:21:14.865713+0000","last_clean":"2024-09-21T02:21:14.865713+0000","last_became_active":"2024-09-21T02:21:12.674822+0000","last_became_peered":"2024-09-21T02:21:12.674822+0000","last_unstale":"2024-09-21T02:21:14.865713+0000","last_undegraded":"2024-09-21T02:21:14.865713+0000","last_fullsized":"2024-09-21T02:21:14.865713+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:08.443398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"141'3","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638286+0000","last_change":"2024-09-21T02:21:11.671036+0000","last_active":"2024-09-21T02:21:13.638286+0000","last_peered":"2024-09-21T02:21:13.638286+0000","last_clean":"2024-09-21T02:21:13.638286+0000","last_became_active":"2024-09-21T02:21:11.670106+0000","last_became_peered":"2024-09-21T02:21:11.670106+0000","last_unstale":"2024-09-21T02:21:13.638286+0000","last_undegraded":"2024-09-21T02:21:13.638286+0000","last_fullsized":"2024-09-21T02:21:13.638286+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:27.959717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640842+0000","last_change":"2024-09-21T02:21:12.652675+0000","last_active":"2024-09-21T02:21:14.640842+0000","last_peered":"2024-09-21T02:21:14.640842+0000","last_clean":"2024-09-21T02:21:14.640842+0000","last_became_active":"2024-09-21T02:21:12.652287+0000","last_became_peered":"2024-09-21T02:21:12.652287+0000","last_unstale":"2024-09-21T02:21:14.640842+0000","last_undegraded":"2024-09-21T02:21:14.640842+0000","last_fullsized":"2024-09-21T02:21:14.640842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:19.836373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"141'6","reported_seq":21,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861863+0000","last_change":"2024-09-21T02:21:11.668119+0000","last_active":"2024-09-21T02:21:14.861863+0000","last_peered":"2024-09-21T02:21:14.861863+0000","last_clean":"2024-09-21T02:21:14.861863+0000","last_became_active":"2024-09-21T02:21:11.665334+0000","last_became_peered":"2024-09-21T02:21:11.665334+0000","last_unstale":"2024-09-21T02:21:14.861863+0000","last_undegraded":"2024-09-21T02:21:14.861863+0000","last_fullsized":"2024-09-21T02:21:14.861863+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:00.112531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864412+0000","last_change":"2024-09-21T02:21:12.657519+0000","last_active":"2024-09-21T02:21:14.864412+0000","last_peered":"2024-09-21T02:21:14.864412+0000","last_clean":"2024-09-21T02:21:14.864412+0000","last_became_active":"2024-09-21T02:21:12.656650+0000","last_became_peered":"2024-09-21T02:21:12.656650+0000","last_unstale":"2024-09-21T02:21:14.864412+0000","last_undegraded":"2024-09-21T02:21:14.864412+0000","last_fullsized":"2024-09-21T02:21:14.864412+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:23.619967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862470+0000","last_change":"2024-09-21T02:21:11.664189+0000","last_active":"2024-09-21T02:21:14.862470+0000","last_peered":"2024-09-21T02:21:14.862470+0000","last_clean":"2024-09-21T02:21:14.862470+0000","last_became_active":"2024-09-21T02:21:11.663121+0000","last_became_peered":"2024-09-21T02:21:11.663121+0000","last_unstale":"2024-09-21T02:21:14.862470+0000","last_undegraded":"2024-09-21T02:21:14.862470+0000","last_fullsized":"2024-09-21T02:21:14.862470+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:46.443462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638939+0000","last_change":"2024-09-21T02:21:12.661821+0000","last_active":"2024-09-21T02:21:13.638939+0000","last_peered":"2024-09-21T02:21:13.638939+0000","last_clean":"2024-09-21T02:21:13.638939+0000","last_became_active":"2024-09-21T02:21:12.654243+0000","last_became_peered":"2024-09-21T02:21:12.654243+0000","last_unstale":"2024-09-21T02:21:13.638939+0000","last_undegraded":"2024-09-21T02:21:13.638939+0000","last_fullsized":"2024-09-21T02:21:13.638939+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:08:31.011889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636465+0000","last_change":"2024-09-21T02:21:11.668855+0000","last_active":"2024-09-21T02:21:13.636465+0000","last_peered":"2024-09-21T02:21:13.636465+0000","last_clean":"2024-09-21T02:21:13.636465+0000","last_became_active":"2024-09-21T02:21:11.668145+0000","last_became_peered":"2024-09-21T02:21:11.668145+0000","last_unstale":"2024-09-21T02:21:13.636465+0000","last_undegraded":"2024-09-21T02:21:13.636465+0000","last_fullsized":"2024-09-21T02:21:13.636465+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:54.680181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.635850+0000","last_change":"2024-09-21T02:21:12.681547+0000","last_active":"2024-09-21T02:21:13.635850+0000","last_peered":"2024-09-21T02:21:13.635850+0000","last_clean":"2024-09-21T02:21:13.635850+0000","last_became_active":"2024-09-21T02:21:12.680756+0000","last_became_peered":"2024-09-21T02:21:12.680756+0000","last_unstale":"2024-09-21T02:21:13.635850+0000","last_undegraded":"2024-09-21T02:21:13.635850+0000","last_fullsized":"2024-09-21T02:21:13.635850+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:35.062212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865126+0000","last_change":"2024-09-21T02:21:11.670891+0000","last_active":"2024-09-21T02:21:14.865126+0000","last_peered":"2024-09-21T02:21:14.865126+0000","last_clean":"2024-09-21T02:21:14.865126+0000","last_became_active":"2024-09-21T02:21:11.670034+0000","last_became_peered":"2024-09-21T02:21:11.670034+0000","last_unstale":"2024-09-21T02:21:14.865126+0000","last_undegraded":"2024-09-21T02:21:14.865126+0000","last_fullsized":"2024-09-21T02:21:14.865126+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:07:16.771406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.517663+0000","last_change":"2024-09-21T02:21:12.667739+0000","last_active":"2024-09-21T02:21:14.517663+0000","last_peered":"2024-09-21T02:21:14.517663+0000","last_clean":"2024-09-21T02:21:14.517663+0000","last_became_active":"2024-09-21T02:21:12.667455+0000","last_became_peered":"2024-09-21T02:21:12.667455+0000","last_unstale":"2024-09-21T02:21:14.517663+0000","last_undegraded":"2024-09-21T02:21:14.517663+0000","last_fullsized":"2024-09-21T02:21:14.517663+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:53:38.161318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"141'8","reported_seq":23,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861492+0000","last_change":"2024-09-21T02:21:11.669023+0000","last_active":"2024-09-21T02:21:14.861492+0000","last_peered":"2024-09-21T02:21:14.861492+0000","last_clean":"2024-09-21T02:21:14.861492+0000","last_became_active":"2024-09-21T02:21:11.665723+0000","last_became_peered":"2024-09-21T02:21:11.665723+0000","last_unstale":"2024-09-21T02:21:14.861492+0000","last_undegraded":"2024-09-21T02:21:14.861492+0000","last_fullsized":"2024-09-21T02:21:14.861492+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:46:48.330139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865414+0000","last_change":"2024-09-21T02:21:12.684723+0000","last_active":"2024-09-21T02:21:14.865414+0000","last_peered":"2024-09-21T02:21:14.865414+0000","last_clean":"2024-09-21T02:21:14.865414+0000","last_became_active":"2024-09-21T02:21:12.684059+0000","last_became_peered":"2024-09-21T02:21:12.684059+0000","last_unstale":"2024-09-21T02:21:14.865414+0000","last_undegraded":"2024-09-21T02:21:14.865414+0000","last_fullsized":"2024-09-21T02:21:14.865414+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:07.221606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862355+0000","last_change":"2024-09-21T02:21:11.668215+0000","last_active":"2024-09-21T02:21:14.862355+0000","last_peered":"2024-09-21T02:21:14.862355+0000","last_clean":"2024-09-21T02:21:14.862355+0000","last_became_active":"2024-09-21T02:21:11.667087+0000","last_became_peered":"2024-09-21T02:21:11.667087+0000","last_unstale":"2024-09-21T02:21:14.862355+0000","last_undegraded":"2024-09-21T02:21:14.862355+0000","last_fullsized":"2024-09-21T02:21:14.862355+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:21.695038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864027+0000","last_change":"2024-09-21T02:21:12.685002+0000","last_active":"2024-09-21T02:21:14.864027+0000","last_peered":"2024-09-21T02:21:14.864027+0000","last_clean":"2024-09-21T02:21:14.864027+0000","last_became_active":"2024-09-21T02:21:12.678126+0000","last_became_peered":"2024-09-21T02:21:12.678126+0000","last_unstale":"2024-09-21T02:21:14.864027+0000","last_undegraded":"2024-09-21T02:21:14.864027+0000","last_fullsized":"2024-09-21T02:21:14.864027+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:28.280903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.518260+0000","last_change":"2024-09-21T02:21:11.669801+0000","last_active":"2024-09-21T02:21:14.518260+0000","last_peered":"2024-09-21T02:21:14.518260+0000","last_clean":"2024-09-21T02:21:14.518260+0000","last_became_active":"2024-09-21T02:21:11.669043+0000","last_became_peered":"2024-09-21T02:21:11.669043+0000","last_unstale":"2024-09-21T02:21:14.518260+0000","last_undegraded":"2024-09-21T02:21:14.518260+0000","last_fullsized":"2024-09-21T02:21:14.518260+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:49.124823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863520+0000","last_change":"2024-09-21T02:21:12.691615+0000","last_active":"2024-09-21T02:21:14.863520+0000","last_peered":"2024-09-21T02:21:14.863520+0000","last_clean":"2024-09-21T02:21:14.863520+0000","last_became_active":"2024-09-21T02:21:12.690630+0000","last_became_peered":"2024-09-21T02:21:12.690630+0000","last_unstale":"2024-09-21T02:21:14.863520+0000","last_undegraded":"2024-09-21T02:21:14.863520+0000","last_fullsized":"2024-09-21T02:21:14.863520+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:20:55.636074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"141'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862070+0000","last_change":"2024-09-21T02:21:11.666644+0000","last_active":"2024-09-21T02:21:14.862070+0000","last_peered":"2024-09-21T02:21:14.862070+0000","last_clean":"2024-09-21T02:21:14.862070+0000","last_became_active":"2024-09-21T02:21:11.665755+0000","last_became_peered":"2024-09-21T02:21:11.665755+0000","last_unstale":"2024-09-21T02:21:14.862070+0000","last_undegraded":"2024-09-21T02:21:14.862070+0000","last_fullsized":"2024-09-21T02:21:14.862070+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:30.663362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638797+0000","last_change":"2024-09-21T02:21:12.669916+0000","last_active":"2024-09-21T02:21:13.638797+0000","last_peered":"2024-09-21T02:21:13.638797+0000","last_clean":"2024-09-21T02:21:13.638797+0000","last_became_active":"2024-09-21T02:21:12.668825+0000","last_became_peered":"2024-09-21T02:21:12.668825+0000","last_unstale":"2024-09-21T02:21:13.638797+0000","last_undegraded":"2024-09-21T02:21:13.638797+0000","last_fullsized":"2024-09-21T02:21:13.638797+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:16.073590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.634281+0000","last_change":"2024-09-21T02:21:11.648087+0000","last_active":"2024-09-21T02:21:13.634281+0000","last_peered":"2024-09-21T02:21:13.634281+0000","last_clean":"2024-09-21T02:21:13.634281+0000","last_became_active":"2024-09-21T02:21:11.647677+0000","last_became_peered":"2024-09-21T02:21:11.647677+0000","last_unstale":"2024-09-21T02:21:13.634281+0000","last_undegraded":"2024-09-21T02:21:13.634281+0000","last_fullsized":"2024-09-21T02:21:13.634281+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:18.412204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639759+0000","last_change":"2024-09-21T02:21:12.694126+0000","last_active":"2024-09-21T02:21:14.639759+0000","last_peered":"2024-09-21T02:21:14.639759+0000","last_clean":"2024-09-21T02:21:14.639759+0000","last_became_active":"2024-09-21T02:21:12.693728+0000","last_became_peered":"2024-09-21T02:21:12.693728+0000","last_unstale":"2024-09-21T02:21:14.639759+0000","last_undegraded":"2024-09-21T02:21:14.639759+0000","last_fullsized":"2024-09-21T02:21:14.639759+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:05.491955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.639300+0000","last_change":"2024-09-21T02:21:11.658742+0000","last_active":"2024-09-21T02:21:13.639300+0000","last_peered":"2024-09-21T02:21:13.639300+0000","last_clean":"2024-09-21T02:21:13.639300+0000","last_became_active":"2024-09-21T02:21:11.657887+0000","last_became_peered":"2024-09-21T02:21:11.657887+0000","last_unstale":"2024-09-21T02:21:13.639300+0000","last_undegraded":"2024-09-21T02:21:13.639300+0000","last_fullsized":"2024-09-21T02:21:13.639300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:06.656862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638986+0000","last_change":"2024-09-21T02:21:12.667823+0000","last_active":"2024-09-21T02:21:13.638986+0000","last_peered":"2024-09-21T02:21:13.638986+0000","last_clean":"2024-09-21T02:21:13.638986+0000","last_became_active":"2024-09-21T02:21:12.666931+0000","last_became_peered":"2024-09-21T02:21:12.666931+0000","last_unstale":"2024-09-21T02:21:13.638986+0000","last_undegraded":"2024-09-21T02:21:13.638986+0000","last_fullsized":"2024-09-21T02:21:13.638986+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:17:08.441996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633574+0000","last_change":"2024-09-21T02:21:11.660664+0000","last_active":"2024-09-21T02:21:13.633574+0000","last_peered":"2024-09-21T02:21:13.633574+0000","last_clean":"2024-09-21T02:21:13.633574+0000","last_became_active":"2024-09-21T02:21:11.659805+0000","last_became_peered":"2024-09-21T02:21:11.659805+0000","last_unstale":"2024-09-21T02:21:13.633574+0000","last_undegraded":"2024-09-21T02:21:13.633574+0000","last_fullsized":"2024-09-21T02:21:13.633574+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:20:52.021191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639426+0000","last_change":"2024-09-21T02:21:12.650755+0000","last_active":"2024-09-21T02:21:14.639426+0000","last_peered":"2024-09-21T02:21:14.639426+0000","last_clean":"2024-09-21T02:21:14.639426+0000","last_became_active":"2024-09-21T02:21:12.650491+0000","last_became_peered":"2024-09-21T02:21:12.650491+0000","last_unstale":"2024-09-21T02:21:14.639426+0000","last_undegraded":"2024-09-21T02:21:14.639426+0000","last_fullsized":"2024-09-21T02:21:14.639426+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:42.573239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862778+0000","last_change":"2024-09-21T02:21:11.674291+0000","last_active":"2024-09-21T02:21:14.862778+0000","last_peered":"2024-09-21T02:21:14.862778+0000","last_clean":"2024-09-21T02:21:14.862778+0000","last_became_active":"2024-09-21T02:21:11.673583+0000","last_became_peered":"2024-09-21T02:21:11.673583+0000","last_unstale":"2024-09-21T02:21:14.862778+0000","last_undegraded":"2024-09-21T02:21:14.862778+0000","last_fullsized":"2024-09-21T02:21:14.862778+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:32.119498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864588+0000","last_change":"2024-09-21T02:21:12.663749+0000","last_active":"2024-09-21T02:21:14.864588+0000","last_peered":"2024-09-21T02:21:14.864588+0000","last_clean":"2024-09-21T02:21:14.864588+0000","last_became_active":"2024-09-21T02:21:12.662912+0000","last_became_peered":"2024-09-21T02:21:12.662912+0000","last_unstale":"2024-09-21T02:21:14.864588+0000","last_undegraded":"2024-09-21T02:21:14.864588+0000","last_fullsized":"2024-09-21T02:21:14.864588+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:28.936634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.633657+0000","last_change":"2024-09-21T02:21:11.647147+0000","last_active":"2024-09-21T02:21:13.633657+0000","last_peered":"2024-09-21T02:21:13.633657+0000","last_clean":"2024-09-21T02:21:13.633657+0000","last_became_active":"2024-09-21T02:21:11.646267+0000","last_became_peered":"2024-09-21T02:21:11.646267+0000","last_unstale":"2024-09-21T02:21:13.633657+0000","last_undegraded":"2024-09-21T02:21:13.633657+0000","last_fullsized":"2024-09-21T02:21:13.633657+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:35.455966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638834+0000","last_change":"2024-09-21T02:21:12.680202+0000","last_active":"2024-09-21T02:21:13.638834+0000","last_peered":"2024-09-21T02:21:13.638834+0000","last_clean":"2024-09-21T02:21:13.638834+0000","last_became_active":"2024-09-21T02:21:12.679372+0000","last_became_peered":"2024-09-21T02:21:12.679372+0000","last_unstale":"2024-09-21T02:21:13.638834+0000","last_undegraded":"2024-09-21T02:21:13.638834+0000","last_fullsized":"2024-09-21T02:21:13.638834+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:28.510082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"141'2","reported_seq":17,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.637252+0000","last_change":"2024-09-21T02:21:11.665529+0000","last_active":"2024-09-21T02:21:13.637252+0000","last_peered":"2024-09-21T02:21:13.637252+0000","last_clean":"2024-09-21T02:21:13.637252+0000","last_became_active":"2024-09-21T02:21:11.664594+0000","last_became_peered":"2024-09-21T02:21:11.664594+0000","last_unstale":"2024-09-21T02:21:13.637252+0000","last_undegraded":"2024-09-21T02:21:13.637252+0000","last_fullsized":"2024-09-21T02:21:13.637252+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:22.224561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864075+0000","last_change":"2024-09-21T02:21:12.684128+0000","last_active":"2024-09-21T02:21:14.864075+0000","last_peered":"2024-09-21T02:21:14.864075+0000","last_clean":"2024-09-21T02:21:14.864075+0000","last_became_active":"2024-09-21T02:21:12.676798+0000","last_became_peered":"2024-09-21T02:21:12.676798+0000","last_unstale":"2024-09-21T02:21:14.864075+0000","last_undegraded":"2024-09-21T02:21:14.864075+0000","last_fullsized":"2024-09-21T02:21:14.864075+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:09.744261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"142'1","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.637934+0000","last_change":"2024-09-21T02:21:11.657349+0000","last_active":"2024-09-21T02:21:14.637934+0000","last_peered":"2024-09-21T02:21:14.637934+0000","last_clean":"2024-09-21T02:21:14.637934+0000","last_became_active":"2024-09-21T02:21:11.655754+0000","last_became_peered":"2024-09-21T02:21:11.655754+0000","last_unstale":"2024-09-21T02:21:14.637934+0000","last_undegraded":"2024-09-21T02:21:14.637934+0000","last_fullsized":"2024-09-21T02:21:14.637934+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:25.874920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863352+0000","last_change":"2024-09-21T02:21:12.677929+0000","last_active":"2024-09-21T02:21:14.863352+0000","last_peered":"2024-09-21T02:21:14.863352+0000","last_clean":"2024-09-21T02:21:14.863352+0000","last_became_active":"2024-09-21T02:21:12.677249+0000","last_became_peered":"2024-09-21T02:21:12.677249+0000","last_unstale":"2024-09-21T02:21:14.863352+0000","last_undegraded":"2024-09-21T02:21:14.863352+0000","last_fullsized":"2024-09-21T02:21:14.863352+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:14.377293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640984+0000","last_change":"2024-09-21T02:21:11.675490+0000","last_active":"2024-09-21T02:21:14.640984+0000","last_peered":"2024-09-21T02:21:14.640984+0000","last_clean":"2024-09-21T02:21:14.640984+0000","last_became_active":"2024-09-21T02:21:11.672111+0000","last_became_peered":"2024-09-21T02:21:11.672111+0000","last_unstale":"2024-09-21T02:21:14.640984+0000","last_undegraded":"2024-09-21T02:21:14.640984+0000","last_fullsized":"2024-09-21T02:21:14.640984+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:42.971394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863378+0000","last_change":"2024-09-21T02:21:12.654460+0000","last_active":"2024-09-21T02:21:14.863378+0000","last_peered":"2024-09-21T02:21:14.863378+0000","last_clean":"2024-09-21T02:21:14.863378+0000","last_became_active":"2024-09-21T02:21:12.653559+0000","last_became_peered":"2024-09-21T02:21:12.653559+0000","last_unstale":"2024-09-21T02:21:14.863378+0000","last_undegraded":"2024-09-21T02:21:14.863378+0000","last_fullsized":"2024-09-21T02:21:14.863378+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:56.410047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.638384+0000","last_change":"2024-09-21T02:21:11.664402+0000","last_active":"2024-09-21T02:21:13.638384+0000","last_peered":"2024-09-21T02:21:13.638384+0000","last_clean":"2024-09-21T02:21:13.638384+0000","last_became_active":"2024-09-21T02:21:11.661546+0000","last_became_peered":"2024-09-21T02:21:11.661546+0000","last_unstale":"2024-09-21T02:21:13.638384+0000","last_undegraded":"2024-09-21T02:21:13.638384+0000","last_fullsized":"2024-09-21T02:21:13.638384+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:50:52.732056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:13.636399+0000","last_change":"2024-09-21T02:21:12.697718+0000","last_active":"2024-09-21T02:21:13.636399+0000","last_peered":"2024-09-21T02:21:13.636399+0000","last_clean":"2024-09-21T02:21:13.636399+0000","last_became_active":"2024-09-21T02:21:12.697241+0000","last_became_peered":"2024-09-21T02:21:12.697241+0000","last_unstale":"2024-09-21T02:21:13.636399+0000","last_undegraded":"2024-09-21T02:21:13.636399+0000","last_fullsized":"2024-09-21T02:21:13.636399+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:23.397612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861831+0000","last_change":"2024-09-21T02:21:11.659379+0000","last_active":"2024-09-21T02:21:14.861831+0000","last_peered":"2024-09-21T02:21:14.861831+0000","last_clean":"2024-09-21T02:21:14.861831+0000","last_became_active":"2024-09-21T02:21:11.658930+0000","last_became_peered":"2024-09-21T02:21:11.658930+0000","last_unstale":"2024-09-21T02:21:14.861831+0000","last_undegraded":"2024-09-21T02:21:14.861831+0000","last_fullsized":"2024-09-21T02:21:14.861831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:59.928726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.640027+0000","last_change":"2024-09-21T02:21:12.661266+0000","last_active":"2024-09-21T02:21:14.640027+0000","last_peered":"2024-09-21T02:21:14.640027+0000","last_clean":"2024-09-21T02:21:14.640027+0000","last_became_active":"2024-09-21T02:21:12.660863+0000","last_became_peered":"2024-09-21T02:21:12.660863+0000","last_unstale":"2024-09-21T02:21:14.640027+0000","last_undegraded":"2024-09-21T02:21:14.640027+0000","last_fullsized":"2024-09-21T02:21:14.640027+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:07.663817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.864451+0000","last_change":"2024-09-21T02:21:11.660950+0000","last_active":"2024-09-21T02:21:14.864451+0000","last_peered":"2024-09-21T02:21:14.864451+0000","last_clean":"2024-09-21T02:21:14.864451+0000","last_became_active":"2024-09-21T02:21:11.660176+0000","last_became_peered":"2024-09-21T02:21:11.660176+0000","last_unstale":"2024-09-21T02:21:14.864451+0000","last_undegraded":"2024-09-21T02:21:14.864451+0000","last_fullsized":"2024-09-21T02:21:14.864451+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:44.509348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.863463+0000","last_change":"2024-09-21T02:21:12.659210+0000","last_active":"2024-09-21T02:21:14.863463+0000","last_peered":"2024-09-21T02:21:14.863463+0000","last_clean":"2024-09-21T02:21:14.863463+0000","last_became_active":"2024-09-21T02:21:12.656404+0000","last_became_peered":"2024-09-21T02:21:12.656404+0000","last_unstale":"2024-09-21T02:21:14.863463+0000","last_undegraded":"2024-09-21T02:21:14.863463+0000","last_fullsized":"2024-09-21T02:21:14.863463+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:58.254733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639927+0000","last_change":"2024-09-21T02:21:11.672994+0000","last_active":"2024-09-21T02:21:14.639927+0000","last_peered":"2024-09-21T02:21:14.639927+0000","last_clean":"2024-09-21T02:21:14.639927+0000","last_became_active":"2024-09-21T02:21:11.672607+0000","last_became_peered":"2024-09-21T02:21:11.672607+0000","last_unstale":"2024-09-21T02:21:14.639927+0000","last_undegraded":"2024-09-21T02:21:14.639927+0000","last_fullsized":"2024-09-21T02:21:14.639927+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:43.506094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639458+0000","last_change":"2024-09-21T02:21:12.674102+0000","last_active":"2024-09-21T02:21:14.639458+0000","last_peered":"2024-09-21T02:21:14.639458+0000","last_clean":"2024-09-21T02:21:14.639458+0000","last_became_active":"2024-09-21T02:21:12.673783+0000","last_became_peered":"2024-09-21T02:21:12.673783+0000","last_unstale":"2024-09-21T02:21:14.639458+0000","last_undegraded":"2024-09-21T02:21:14.639458+0000","last_fullsized":"2024-09-21T02:21:14.639458+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:44:31.249289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.639048+0000","last_change":"2024-09-21T02:21:11.671159+0000","last_active":"2024-09-21T02:21:14.639048+0000","last_peered":"2024-09-21T02:21:14.639048+0000","last_clean":"2024-09-21T02:21:14.639048+0000","last_became_active":"2024-09-21T02:21:11.668339+0000","last_became_peered":"2024-09-21T02:21:11.668339+0000","last_unstale":"2024-09-21T02:21:14.639048+0000","last_undegraded":"2024-09-21T02:21:14.639048+0000","last_fullsized":"2024-09-21T02:21:14.639048+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:32:50.319183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.865328+0000","last_change":"2024-09-21T02:21:12.686555+0000","last_active":"2024-09-21T02:21:14.865328+0000","last_peered":"2024-09-21T02:21:14.865328+0000","last_clean":"2024-09-21T02:21:14.865328+0000","last_became_active":"2024-09-21T02:21:12.685874+0000","last_became_peered":"2024-09-21T02:21:12.685874+0000","last_unstale":"2024-09-21T02:21:14.865328+0000","last_undegraded":"2024-09-21T02:21:14.865328+0000","last_fullsized":"2024-09-21T02:21:14.865328+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:11.611701+0000","last_clean_scrub_stamp":"2024-09-21T02:21:11.611701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:36.084861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.862276+0000","last_change":"2024-09-21T02:21:11.668611+0000","last_active":"2024-09-21T02:21:14.862276+0000","last_peered":"2024-09-21T02:21:14.862276+0000","last_clean":"2024-09-21T02:21:14.862276+0000","last_became_active":"2024-09-21T02:21:11.667875+0000","last_became_peered":"2024-09-21T02:21:11.667875+0000","last_unstale":"2024-09-21T02:21:14.862276+0000","last_undegraded":"2024-09-21T02:21:14.862276+0000","last_fullsized":"2024-09-21T02:21:14.862276+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:10.604747+0000","last_clean_scrub_stamp":"2024-09-21T02:21:10.604747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:51.661874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":355,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T02:21:14.861570+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:21:14.861570+0000","last_peered":"2024-09-21T02:21:14.861570+0000","last_clean":"2024-09-21T02:21:14.861570+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:21:14.861570+0000","last_undegraded":"2024-09-21T02:21:14.861570+0000","last_fullsized":"2024-09-21T02:21:14.861570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1777664,"data_stored":784451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1769472,"data_stored":778157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509539,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33512,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338328,"statfs":{"total":96636764160,"available":96602447872,"internally_reserved":0,"allocated":2379776,"data_stored":1375491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542244,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1769472,"data_stored":778157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1794048,"data_stored":786085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1781760,"data_stored":785033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41844,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329996,"statfs":{"total":96636764160,"available":96593915904,"internally_reserved":0,"allocated":2383872,"data_stored":1370285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1781760,"data_stored":784521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:21:17.180 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:21:17.180 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 -- 172.21.15.170:0/2595347286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300ce070 msgr2=0x7f7b300ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 --2- 172.21.15.170:0/2595347286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300ce070 0x7f7b300ce480 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7b300726b0 tx=0x7f7b2801cd80 comp rx=0 tx=0).stop 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 -- 172.21.15.170:0/2595347286 shutdown_connections 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 --2- 172.21.15.170:0/2595347286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300ce070 0x7f7b300ce480 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 --2- 172.21.15.170:0/2595347286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300740a0 0x7f7b300744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 --2- 172.21.15.170:0/2595347286 >> v2:172.21.15.191:3300/0 conn(0x7f7b300cf7d0 0x7f7b30073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 -- 172.21.15.170:0/2595347286 >> 172.21.15.170:0/2595347286 conn(0x7f7b300d0170 msgr2=0x7f7b300d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 -- 172.21.15.170:0/2595347286 shutdown_connections 2024-09-21T02:21:17.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 -- 172.21.15.170:0/2595347286 wait complete. 2024-09-21T02:21:17.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 Processor -- start 2024-09-21T02:21:17.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.384+0000 7f7b36157640 1 -- start start 2024-09-21T02:21:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 0x7f7b300c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300ce070 0x7f7b30152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b300cf7d0 0x7f7b30153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7b300cbe20 con 0x7f7b300740a0 2024-09-21T02:21:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b300cbca0 con 0x7f7b300cf7d0 2024-09-21T02:21:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7b300cbfa0 con 0x7f7b300ce070 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 0x7f7b300c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 0x7f7b300c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43164/0 (socket says 172.21.15.170:43164) 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 -- 172.21.15.170:0/3581058998 learned_addr learned my addr 172.21.15.170:0/3581058998 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2effd640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300ce070 0x7f7b30152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2ffff640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b300cf7d0 0x7f7b30153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 -- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300ce070 msgr2=0x7f7b30152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300ce070 0x7f7b30152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 -- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b300cf7d0 msgr2=0x7f7b30153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b300cf7d0 0x7f7b30153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 -- 172.21.15.170:0/3581058998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b300c5d40 con 0x7f7b300740a0 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2effd640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300ce070 0x7f7b30152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2ffff640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b300cf7d0 0x7f7b30153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b2f7fe640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 0x7f7b300c3c20 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800d950 tx=0x7f7b1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18017070 con 0x7f7b300740a0 2024-09-21T02:21:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b18009d20 con 0x7f7b300740a0 2024-09-21T02:21:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b300c6020 con 0x7f7b300740a0 2024-09-21T02:21:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180106c0 con 0x7f7b300740a0 2024-09-21T02:21:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b300c65e0 con 0x7f7b300740a0 2024-09-21T02:21:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.388+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b300ce480 con 0x7f7b300740a0 2024-09-21T02:21:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.392+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f7b1801a070 con 0x7f7b300740a0 2024-09-21T02:21:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.392+0000 7f7b27fff640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b100738c0 0x7f7b10075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.392+0000 7f7b2effd640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b100738c0 0x7f7b10075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.392+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f7b180951a0 con 0x7f7b300740a0 2024-09-21T02:21:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.392+0000 7f7b2effd640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b100738c0 0x7f7b10075dd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005fd0 tx=0x7f7b20005d60 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.396+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7b1805f670 con 0x7f7b300740a0 2024-09-21T02:21:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.624+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7b300b36b0 con 0x7f7b300740a0 2024-09-21T02:21:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b27fff640 1 -- 172.21.15.170:0/3581058998 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) ==== 190+0+147 (secure 0 0 0) 0x7f7b1805ef70 con 0x7f7b300740a0 2024-09-21T02:21:17.628 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:21:17.629 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:21:17.629 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:21:17.629 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:21:17.629 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:21:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b100738c0 msgr2=0x7f7b10075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b100738c0 0x7f7b10075dd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005fd0 tx=0x7f7b20005d60 comp rx=0 tx=0).stop 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 msgr2=0x7f7b300c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 0x7f7b300c3c20 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800d950 tx=0x7f7b1800de10 comp rx=0 tx=0).stop 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 shutdown_connections 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7b100738c0 0x7f7b10075dd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b300ce070 0x7f7b30152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b300cf7d0 0x7f7b30153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 --2- 172.21.15.170:0/3581058998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b300740a0 0x7f7b300c3c20 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 >> 172.21.15.170:0/3581058998 conn(0x7f7b300d0170 msgr2=0x7f7b30156360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 shutdown_connections 2024-09-21T02:21:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.628+0000 7f7b36157640 1 -- 172.21.15.170:0/3581058998 wait complete. 2024-09-21T02:21:17.649 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 -- 172.21.15.170:0/1990218581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 msgr2=0x7f09d406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 --2- 172.21.15.170:0/1990218581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d406fa30 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009840 tx=0x7f09d001c820 comp rx=0 tx=0).stop 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 -- 172.21.15.170:0/1990218581 shutdown_connections 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 --2- 172.21.15.170:0/1990218581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d406fa30 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 -- 172.21.15.170:0/1990218581 >> 172.21.15.170:0/1990218581 conn(0x7f09d40b1c40 msgr2=0x7f09d40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 -- 172.21.15.170:0/1990218581 shutdown_connections 2024-09-21T02:21:17.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 -- 172.21.15.170:0/1990218581 wait complete. 2024-09-21T02:21:17.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 Processor -- start 2024-09-21T02:21:17.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.804+0000 7f09dbc24640 1 -- start start 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d40b2f40 0x7f09d40ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d4156fd0 0x7f09d40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09d416b570 con 0x7f09d40b5410 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09d4070180 con 0x7f09d40b2f40 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f09d416b6f0 con 0x7f09d4156fd0 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d4143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43180/0 (socket says 172.21.15.170:43180) 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 -- 172.21.15.170:0/3963845890 learned_addr learned my addr 172.21.15.170:0/3963845890 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d99c0640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d40b2f40 0x7f09d40ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 -- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d4156fd0 msgr2=0x7f09d40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09da1c1640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d4156fd0 0x7f09d40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d4156fd0 0x7f09d40ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 -- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d40b2f40 msgr2=0x7f09d40ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d40b2f40 0x7f09d40ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 -- 172.21.15.170:0/3963845890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41f9f50 con 0x7f09d40b5410 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d91bf640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d4143ab0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b440 tx=0x7f09c400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4015020 con 0x7f09d40b5410 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09c4011980 con 0x7f09d40b5410 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4011dd0 con 0x7f09d40b5410 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d41fa230 con 0x7f09d40b5410 2024-09-21T02:21:17.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d41fa7f0 con 0x7f09d40b5410 2024-09-21T02:21:17.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d406fa30 con 0x7f09d40b5410 2024-09-21T02:21:17.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f09c4018020 con 0x7f09d40b5410 2024-09-21T02:21:17.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09ca7fc640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09b80738c0 0x7f09b8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:17.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09d99c0640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09b80738c0 0x7f09b8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:17.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.808+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f09c4094ec0 con 0x7f09d40b5410 2024-09-21T02:21:17.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.812+0000 7f09d99c0640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09b80738c0 0x7f09b8075dd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f09d00af000 tx=0x7f09d001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:17.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:17.816+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f09c401f030 con 0x7f09d40b5410 2024-09-21T02:21:18.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.012+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d4158720 con 0x7f09d40b5410 2024-09-21T02:21:18.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.012+0000 7f09ca7fc640 1 -- 172.21.15.170:0/3963845890 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5238 (secure 0 0 0) 0x7f09c400d040 con 0x7f09d40b5410 2024-09-21T02:21:18.013 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:18.013 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":100,"btime":"2024-09-21T02:21:15:682112+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:12.056653+0000","modified":"2024-09-21T02:21:15.085591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2032631050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2032631050},{"type":"v1","addr":"172.21.15.170:6835","nonce":2032631050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":11}]} 2024-09-21T02:21:18.014 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 100 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.012+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09b80738c0 msgr2=0x7f09b8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.012+0000 7f09dbc24640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09b80738c0 0x7f09b8075dd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f09d00af000 tx=0x7f09d001b060 comp rx=0 tx=0).stop 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.012+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 msgr2=0x7f09d4143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.012+0000 7f09dbc24640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d4143ab0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b440 tx=0x7f09c400b900 comp rx=0 tx=0).stop 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 shutdown_connections 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09b80738c0 0x7f09b8075dd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d4156fd0 0x7f09d40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d40b5410 0x7f09d4143ab0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 --2- 172.21.15.170:0/3963845890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d40b2f40 0x7f09d40ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 >> 172.21.15.170:0/3963845890 conn(0x7f09d40b1c40 msgr2=0x7f09d40c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 shutdown_connections 2024-09-21T02:21:18.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.016+0000 7f09dbc24640 1 -- 172.21.15.170:0/3963845890 wait complete. 2024-09-21T02:21:18.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:21:12.056653+0000', 'modified': '2024-09-21T02:21:15.085591+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8535': {'gid': 8535, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/2032631050', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2032631050}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2032631050}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8535, 'qdb_cluster': [8535]} 2024-09-21T02:21:18.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:21:18.437 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:21:18.578 INFO:tasks.ceph.mds.c.smithi170.stderr:2024-09-21T02:21:18.576+0000 7f6989975640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:21:18.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 -- 172.21.15.170:0/988617672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 msgr2=0x7f3c9c0b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/988617672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c0b44a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009840 tx=0x7f3c8c01c820 comp rx=0 tx=0).stop 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 -- 172.21.15.170:0/988617672 shutdown_connections 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/988617672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c0b44a0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/988617672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c0b25f0 0x7f3c9c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 -- 172.21.15.170:0/988617672 >> 172.21.15.170:0/988617672 conn(0x7f3c9c09a9c0 msgr2=0x7f3c9c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 -- 172.21.15.170:0/988617672 shutdown_connections 2024-09-21T02:21:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.604+0000 7f3ca2f66640 1 -- 172.21.15.170:0/988617672 wait complete. 2024-09-21T02:21:18.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 Processor -- start 2024-09-21T02:21:18.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 -- start start 2024-09-21T02:21:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c0b25f0 0x7f3c9c14e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c15faf0 0x7f3c9c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c166db0 con 0x7f3c9c160610 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c0b57c0 con 0x7f3c9c0b25f0 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c166f30 con 0x7f3c9c15faf0 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c1501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43200/0 (socket says 172.21.15.170:43200) 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 -- 172.21.15.170:0/2149178375 learned_addr learned my addr 172.21.15.170:0/2149178375 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c15faf0 0x7f3c9c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c0b25f0 0x7f3c9c14e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 -- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c15faf0 msgr2=0x7f3c9c14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c15faf0 0x7f3c9c14fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 -- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c0b25f0 msgr2=0x7f3c9c14e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c0b25f0 0x7f3c9c14e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 -- 172.21.15.170:0/2149178375 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c14f200 con 0x7f3c9c160610 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c15faf0 0x7f3c9c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca1503640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c1501e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0b0040 tx=0x7f3c8c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3c997fa640 1 -- 172.21.15.170:0/2149178375 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0b2040 con 0x7f3c9c160610 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3c997fa640 1 -- 172.21.15.170:0/2149178375 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c8c0bb070 con 0x7f3c9c160610 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c168010 con 0x7f3c9c160610 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3c997fa640 1 -- 172.21.15.170:0/2149178375 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c003600 con 0x7f3c9c160610 2024-09-21T02:21:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.608+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1682f0 con 0x7f3c9c160610 2024-09-21T02:21:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3c997fa640 1 -- 172.21.15.170:0/2149178375 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f3c8c0037a0 con 0x7f3c9c160610 2024-09-21T02:21:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3c997fa640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c7c0738c0 0x7f3c7c075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3c997fa640 1 -- 172.21.15.170:0/2149178375 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f3c8c12f270 con 0x7f3c9c160610 2024-09-21T02:21:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c7c0738c0 0x7f3c7c075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c7c0738c0 0x7f3c7c075dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009e00 tx=0x7f3c94009390 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 Processor -- start 2024-09-21T02:21:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 -- start start 2024-09-21T02:21:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6629460 0x5605f6b6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6b6b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605f6a3f410 con 0x5605f6a02ca0 2024-09-21T02:21:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5605f697cb40 con 0x5605f6629460 2024-09-21T02:21:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5605f696bf70 con 0x5605f6a19950 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3ca1503640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6b6b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3ca1503640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6b6b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43206/0 (socket says 172.21.15.170:43206) 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.612+0000 7f3ca1503640 1 -- 172.21.15.170:0/4170024309 learned_addr learned my addr 172.21.15.170:0/4170024309 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6629460 0x5605f6b6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca1503640 1 -- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 msgr2=0x5605f6b6abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca1503640 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b6abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca1503640 1 -- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6629460 msgr2=0x5605f6b6a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca1503640 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6629460 0x5605f6b6a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca1503640 1 -- 172.21.15.170:0/4170024309 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f69335b0 con 0x5605f6a02ca0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca1503640 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6b6b120 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c01cd00 tx=0x7f3c8c0f90c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37acaeec756e5245 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3c927fc640 1 -- 172.21.15.170:0/4170024309 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c12f0c0 con 0x5605f6a02ca0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3c927fc640 1 -- 172.21.15.170:0/4170024309 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c8c0f8e00 con 0x5605f6a02ca0 2024-09-21T02:21:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3c927fc640 1 -- 172.21.15.170:0/4170024309 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c0f99f0 con 0x5605f6a02ca0 2024-09-21T02:21:18.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- 172.21.15.170:0/4170024309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 msgr2=0x5605f6b6b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6b6b120 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c01cd00 tx=0x7f3c8c0f90c0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- 172.21.15.170:0/4170024309 shutdown_connections 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6b6b120 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- 172.21.15.170:0/4170024309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6629460 0x5605f6b6a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- 172.21.15.170:0/4170024309 >> 172.21.15.170:0/4170024309 conn(0x5605f6aefd90 msgr2=0x5605f6b1f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- 172.21.15.170:0/4170024309 shutdown_connections 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- 172.21.15.170:0/4170024309 wait complete. 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- start start 2024-09-21T02:21:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 Processor -- start 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6b71920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6b71e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605f69065f0 con 0x5605f6629460 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5605f696fd80 con 0x5605f6a02ca0 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5605f6912520 con 0x5605f6a19950 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3caa231000 10 client.0 _ll_register_callbacks cb 0x5605f6916830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.616+0000 7f3ca0d02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6b71920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6b71920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43210/0 (socket says 172.21.15.170:43210) 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 -- 172.21.15.170:0/1064726540 learned_addr learned my addr 172.21.15.170:0/1064726540 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca1503640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6b71e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 -- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 msgr2=0x5605f6b723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 -- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 msgr2=0x5605f6b71e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6b71e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 -- 172.21.15.170:0/1064726540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f690a190 con 0x5605f6629460 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca1503640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6b71920 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f3c94011790 tx=0x7f3c94004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea46bbe9eaa53b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/1064726540 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94010ad0 con 0x5605f6629460 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/1064726540 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f3c94010c70 con 0x5605f6629460 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605f6904110 con 0x5605f6629460 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/1064726540 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3c9402f050 con 0x5605f6629460 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 10 client.8658 fetch_fsmap learned FSMap version 100 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5605f69f0f10 con 0x5605f6629460 2024-09-21T02:21:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/1064726540 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5605f6904110 con 0x5605f6629460 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/1064726540 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 100) ==== 3502+0+0 (secure 0 0 0) 0x7f3c9401f430 con 0x5605f6629460 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 10 client.8658 fetch_fsmap finished waiting for FSMap version 100 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 10 client.8658 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 20 client.8658 populate_metadata read hostname 'smithi170' 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f660e7d0 0x5605f6b7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 4 client.8658 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 --> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f697ea90 con 0x5605f660e7d0 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f660e7d0 0x5605f6b7bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 10 client.8658 ms_handle_connect on v2:172.21.15.170:6834/2032631050 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f660e7d0 0x5605f6b7bd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a7cf81c3b3e38e server_cookie=4ee31efc0683405c in_seq=0 out_seq=0 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/1064726540 <== mds.0 v2:172.21.15.170:6834/2032631050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605f697ea90 con 0x5605f660e7d0 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 10 client.8658 handle_command_reply: tid=0 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 client.8658 shutdown 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 msgr2=0x5605f6b71920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6b71920 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f3c94011790 tx=0x7f3c94004a50 comp rx=0 tx=0).stop 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 shutdown_connections 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6b723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f660e7d0 0x5605f6b7bd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6b71e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 --2- 172.21.15.170:0/1064726540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6b71920 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3c7b7fe640 0 client.8658 ms_handle_reset on v2:172.21.15.170:6834/2032631050 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 >> 172.21.15.170:0/1064726540 conn(0x5605f6aefd90 msgr2=0x5605f6b26440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 shutdown_connections 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 1 -- 172.21.15.170:0/1064726540 wait complete. 2024-09-21T02:21:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.620+0000 7f3caa231000 20 client.8658 trim_cache size 0 max 16384 2024-09-21T02:21:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 Processor -- start 2024-09-21T02:21:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 -- start start 2024-09-21T02:21:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6629460 0x5605f6bdc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6bdcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a19950 0x5605f6bdd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c205260 con 0x5605f6a02ca0 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c2043e0 con 0x5605f6a19950 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c165d90 con 0x5605f6629460 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6bdcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6bdcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43220/0 (socket says 172.21.15.170:43220) 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 -- 172.21.15.170:0/84797620 learned_addr learned my addr 172.21.15.170:0/84797620 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6629460 0x5605f6bdc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 -- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6629460 msgr2=0x5605f6bdc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3ca1503640 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a19950 0x5605f6bdd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6629460 0x5605f6bdc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 -- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a19950 msgr2=0x5605f6bdd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a19950 0x5605f6bdd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 -- 172.21.15.170:0/84797620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f69c7db0 con 0x5605f6a02ca0 2024-09-21T02:21:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6bdcb60 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3c84007bb0 tx=0x7f3c84009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6430f33a058f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c91ffb640 1 -- 172.21.15.170:0/84797620 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84010040 con 0x5605f6a02ca0 2024-09-21T02:21:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.640+0000 7f3c91ffb640 1 -- 172.21.15.170:0/84797620 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c84022070 con 0x5605f6a02ca0 2024-09-21T02:21:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3c91ffb640 1 -- 172.21.15.170:0/84797620 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84017b00 con 0x5605f6a02ca0 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- 172.21.15.170:0/84797620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 msgr2=0x5605f6bdcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6bdcb60 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3c84007bb0 tx=0x7f3c84009040 comp rx=0 tx=0).stop 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- 172.21.15.170:0/84797620 shutdown_connections 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a19950 0x5605f6bdd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6a02ca0 0x5605f6bdcb60 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- 172.21.15.170:0/84797620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6629460 0x5605f6bdc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- 172.21.15.170:0/84797620 >> 172.21.15.170:0/84797620 conn(0x5605f6aefd90 msgr2=0x5605f6b68620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- 172.21.15.170:0/84797620 shutdown_connections 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- 172.21.15.170:0/84797620 wait complete. 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- start start 2024-09-21T02:21:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 Processor -- start 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6be0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6be08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6be0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605f6a2b750 con 0x5605f6629460 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5605f697b910 con 0x5605f6a02ca0 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x5605f69f0a80 con 0x5605f6a19950 2024-09-21T02:21:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3caa231000 10 client.0 _ll_register_callbacks cb 0x7f3c94015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6be0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6be0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43232/0 (socket says 172.21.15.170:43232) 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 -- 172.21.15.170:0/436895642 learned_addr learned my addr 172.21.15.170:0/436895642 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6be08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca1503640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6be0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 -- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 msgr2=0x5605f6be0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6be0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 -- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 msgr2=0x5605f6be08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6be08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 -- 172.21.15.170:0/436895642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f6932ab0 con 0x5605f6629460 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6be08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3ca0d02640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6be0390 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009340 tx=0x7f3c9401f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e411aba8fb9c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3c7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:21:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/436895642 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c940109a0 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.644+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/436895642 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f3c94010b40 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c84022070 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/436895642 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3c9401fbc0 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 10 client.8664 fetch_fsmap learned FSMap version 100 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5605f6911070 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/436895642 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c84022070 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/436895642 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 100) ==== 3502+0+0 (secure 0 0 0) 0x7f3c94043050 con 0x5605f6629460 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 10 client.8664 fetch_fsmap finished waiting for FSMap version 100 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 10 client.8664 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 20 client.8664 populate_metadata read hostname 'smithi170' 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f65fc830 0x5605f6b9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 4 client.8664 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 --> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] -- command(tid 0: {"prefix": "status"}) -- 0x5605f697a540 con 0x5605f65fc830 2024-09-21T02:21:18.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f65fc830 0x5605f6b9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 10 client.8664 ms_handle_connect on v2:172.21.15.170:6834/2032631050 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c9bfff640 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f65fc830 0x5605f6b9c760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12119c61bc1dae84 server_cookie=1a8577c976e1199a in_seq=0 out_seq=0 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 1 -- 172.21.15.170:0/436895642 <== mds.0 v2:172.21.15.170:6834/2032631050 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5605f697a540 con 0x5605f65fc830 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 10 client.8664 handle_command_reply: tid=0 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 client.8664 shutdown 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 msgr2=0x5605f6be0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6be0390 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009340 tx=0x7f3c9401f6e0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 shutdown_connections 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5605f6a19950 0x5605f6be0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:6834/2032631050,v1:172.21.15.170:6835/2032631050] conn(0x5605f65fc830 0x5605f6b9c760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5605f6a02ca0 0x5605f6be08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 --2- 172.21.15.170:0/436895642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605f6629460 0x5605f6be0390 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3c7b7fe640 0 client.8664 ms_handle_reset on v2:172.21.15.170:6834/2032631050 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 >> 172.21.15.170:0/436895642 conn(0x5605f6aefd90 msgr2=0x5605f6b68fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 shutdown_connections 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 1 -- 172.21.15.170:0/436895642 wait complete. 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3caa231000 20 client.8664 trim_cache size 0 max 16384 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8535, 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 6.0231735710000001, 2024-09-21T02:21:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 100, 2024-09-21T02:21:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 142, 2024-09-21T02:21:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 142, 2024-09-21T02:21:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.50261809 2024-09-21T02:21:18.655 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c7c0738c0 msgr2=0x7f3c7c075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c7c0738c0 0x7f3c7c075dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009e00 tx=0x7f3c94009390 comp rx=0 tx=0).stop 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 msgr2=0x7f3c9c1501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c1501e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0b0040 tx=0x7f3c8c005d40 comp rx=0 tx=0).stop 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 shutdown_connections 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c7c0738c0 0x7f3c7c075dd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c9c15faf0 0x7f3c9c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c9c160610 0x7f3c9c1501e0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 --2- 172.21.15.170:0/2149178375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c9c0b25f0 0x7f3c9c14e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 >> 172.21.15.170:0/2149178375 conn(0x7f3c9c09a9c0 msgr2=0x7f3c9c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 shutdown_connections 2024-09-21T02:21:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.648+0000 7f3ca2f66640 1 -- 172.21.15.170:0/2149178375 wait complete. 2024-09-21T02:21:18.668 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 -- 172.21.15.170:0/3185185864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cf7e0 msgr2=0x7fac9c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 --2- 172.21.15.170:0/3185185864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cf7e0 0x7fac9c070340 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009900 tx=0x7fac8c01c900 comp rx=0 tx=0).stop 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 -- 172.21.15.170:0/3185185864 shutdown_connections 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 --2- 172.21.15.170:0/3185185864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac9c0ce1c0 0x7fac9c0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 --2- 172.21.15.170:0/3185185864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cf7e0 0x7fac9c070340 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 -- 172.21.15.170:0/3185185864 >> 172.21.15.170:0/3185185864 conn(0x7fac9c0b1c40 msgr2=0x7fac9c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 -- 172.21.15.170:0/3185185864 shutdown_connections 2024-09-21T02:21:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 -- 172.21.15.170:0/3185185864 wait complete. 2024-09-21T02:21:18.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 Processor -- start 2024-09-21T02:21:18.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.804+0000 7faca1e7e640 1 -- start start 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 0x7fac9c1659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9c0ce1c0 0x7fac9c165f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac9c0cf7e0 0x7fac9c1497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c1690d0 con 0x7fac9c0cb460 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c168f50 con 0x7fac9c0ce1c0 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c169250 con 0x7fac9c0cf7e0 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 0x7fac9c1659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 0x7fac9c1659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43254/0 (socket says 172.21.15.170:43254) 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 -- 172.21.15.170:0/4287159045 learned_addr learned my addr 172.21.15.170:0/4287159045 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 -- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac9c0cf7e0 msgr2=0x7fac9c1497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9bfff640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac9c0cf7e0 0x7fac9c1497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9affd640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9c0ce1c0 0x7fac9c165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac9c0cf7e0 0x7fac9c1497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 -- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9c0ce1c0 msgr2=0x7fac9c165f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9c0ce1c0 0x7fac9c165f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 -- 172.21.15.170:0/4287159045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c147460 con 0x7fac9c0cb460 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9affd640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9c0ce1c0 0x7fac9c165f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9b7fe640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 0x7fac9c1659f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0b0040 tx=0x7fac8c0a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c0b2040 con 0x7fac9c0cb460 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac8c003770 con 0x7fac9c0cb460 2024-09-21T02:21:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c14adf0 con 0x7fac9c0cb460 2024-09-21T02:21:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c003b60 con 0x7fac9c0cb460 2024-09-21T02:21:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c14b090 con 0x7fac9c0cb460 2024-09-21T02:21:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c0ce5d0 con 0x7fac9c0cb460 2024-09-21T02:21:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fac8c0bf020 con 0x7fac9c0cb460 2024-09-21T02:21:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac83fff640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac880738c0 0x7fac88075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fac8c12fca0 con 0x7fac9c0cb460 2024-09-21T02:21:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.808+0000 7fac9affd640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac880738c0 0x7fac88075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.812+0000 7fac9affd640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac880738c0 0x7fac88075dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fac9c14de70 tx=0x7fac90008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:18.812+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fac8c0c6020 con 0x7fac9c0cb460 2024-09-21T02:21:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.016+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fac9c06fc30 con 0x7fac9c0cb460 2024-09-21T02:21:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.016+0000 7fac83fff640 1 -- 172.21.15.170:0/4287159045 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7fac8c0164f0 con 0x7fac9c0cb460 2024-09-21T02:21:19.017 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac880738c0 msgr2=0x7fac88075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac880738c0 0x7fac88075dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fac9c14de70 tx=0x7fac90008040 comp rx=0 tx=0).stop 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 msgr2=0x7fac9c1659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 0x7fac9c1659f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0b0040 tx=0x7fac8c0a5760 comp rx=0 tx=0).stop 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 shutdown_connections 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac880738c0 0x7fac88075dd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac9c0cf7e0 0x7fac9c1497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9c0ce1c0 0x7fac9c165f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 --2- 172.21.15.170:0/4287159045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac9c0cb460 0x7fac9c1659f0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 >> 172.21.15.170:0/4287159045 conn(0x7fac9c0b1c40 msgr2=0x7fac9c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 shutdown_connections 2024-09-21T02:21:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19.020+0000 7faca1e7e640 1 -- 172.21.15.170:0/4287159045 wait complete. 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:21:19.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:21:19.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:21:19.040 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:21:19.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.040 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:21:19.046 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:21:19.046 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:19.047 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42036sec preferred_lft 42036sec 2024-09-21T02:21:19.048 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:21:19.049 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:19.050 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:21:19.051 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:19.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.051 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:21:19.096 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:21:19.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.098 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:19.147 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:21:19.148 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:21:19.148 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:21:19.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:21:19.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:21:19.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:21:19.148 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:21:19.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:21:19.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:19.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.260 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:21:19.307 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:21:19.307 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:21:19.308 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:21:19.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:21:19.362 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:21:19.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:21:19.363 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:19.408 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:21:19.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:21:19.408 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:19.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:21:19.513 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:19.562 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:21:19.562 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:21:19.563 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:19.564 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:19.565 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:19.565 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:19.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.566 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:21:19.612 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:21:19.613 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:21:19.657 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:21:19.673 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:19.724 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:21:19.724 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:21:19.725 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:21:19.726 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:21:19.727 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:21:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:21:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:21:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:19.728 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:19.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:19.729 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:21:19.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:21:19.764+0000 7f7eda621580 -1 init, newargv = 0x7f7ec8004370 newargc=15 2024-09-21T02:21:19.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[50716]: starting ceph client 2024-09-21T02:21:19.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[50716]: starting fuse 2024-09-21T02:21:20.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:21:20.888 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:20.937 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:21:20.937 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:21:20.937 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:21:20.937 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:21:20.938 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:21:20.939 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:20.940 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:20.940 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:20.940 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:20.940 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:20.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:20.941 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:21:20.988 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:21:20.989 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:21:20.989 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:21:20.990 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:21:21.113 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.50716.asok 2024-09-21T02:21:21.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:21.125 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50716.asok 2024-09-21T02:21:21.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:21.126 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50716.asok status 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "50716", 2024-09-21T02:21:21.310 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8676, 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 8676 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3769426209 2024-09-21T02:21:21.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3769426209 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.8676 192.168.0.1:0/3769426209", 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/3769426209", 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 100, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 142, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:21:21.312 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:21:21.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:21.326 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:21:21.326 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:21.332 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:21:21.332 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:21.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:21.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:21.389 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 -- 172.21.15.170:0/1997864587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a380cc1b0 msgr2=0x7f4a380cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/1997864587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a380cc1b0 0x7f4a380cc5c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009840 tx=0x7f4a2001d1f0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 -- 172.21.15.170:0/1997864587 shutdown_connections 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/1997864587 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a380cc1b0 0x7f4a380cc5c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/1997864587 >> v2:172.21.15.191:3300/0 conn(0x7f4a38071bd0 0x7f4a3815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 -- 172.21.15.170:0/1997864587 >> 172.21.15.170:0/1997864587 conn(0x7f4a380cd3c0 msgr2=0x7f4a380cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 -- 172.21.15.170:0/1997864587 shutdown_connections 2024-09-21T02:21:21.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.592+0000 7f4a3ec29640 1 -- 172.21.15.170:0/1997864587 wait complete. 2024-09-21T02:21:21.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 Processor -- start 2024-09-21T02:21:21.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 -- start start 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a38071bd0 0x7f4a381505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 0x7f4a3814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a380ce510 0x7f4a38157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38168de0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a38168c60 con 0x7f4a380ce510 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4a38168f60 con 0x7f4a38071bd0 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 0x7f4a3814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:21.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 0x7f4a3814aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43268/0 (socket says 172.21.15.170:43268) 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 -- 172.21.15.170:0/3067787904 learned_addr learned my addr 172.21.15.170:0/3067787904 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3c9c5640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a38071bd0 0x7f4a381505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3d1c6640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a380ce510 0x7f4a38157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 -- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a38071bd0 msgr2=0x7f4a381505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a38071bd0 0x7f4a381505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 -- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a380ce510 msgr2=0x7f4a38157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a380ce510 0x7f4a38157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 -- 172.21.15.170:0/3067787904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a38150b10 con 0x7f4a380cc1b0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a37fff640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 0x7f4a3814aab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800c730 tx=0x7f4a2800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28003b80 con 0x7f4a380cc1b0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3814f530 con 0x7f4a380cc1b0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a28003d40 con 0x7f4a380cc1b0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800aa50 con 0x7f4a380cc1b0 2024-09-21T02:21:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.596+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3814faf0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.600+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3815b5e0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.600+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f4a2800abf0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.600+0000 7f4a357fa640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a180738c0 0x7f4a18075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.600+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f4a28095690 con 0x7f4a380cc1b0 2024-09-21T02:21:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.600+0000 7f4a3c9c5640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a180738c0 0x7f4a18075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:21.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.600+0000 7f4a3c9c5640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a180738c0 0x7f4a18075dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009ad0 tx=0x7f4a30009260 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:21.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.604+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4a2805fdf0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.812+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4a3814aff0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.812+0000 7f4a357fa640 1 -- 172.21.15.170:0/3067787904 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f4a2805f6f0 con 0x7f4a380cc1b0 2024-09-21T02:21:21.813 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:21:21.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a180738c0 msgr2=0x7f4a18075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:21.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a180738c0 0x7f4a18075dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009ad0 tx=0x7f4a30009260 comp rx=0 tx=0).stop 2024-09-21T02:21:21.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 msgr2=0x7f4a3814aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 0x7f4a3814aab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800c730 tx=0x7f4a2800cbf0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 shutdown_connections 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4a180738c0 0x7f4a18075dd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a380ce510 0x7f4a38157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a380cc1b0 0x7f4a3814aab0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 --2- 172.21.15.170:0/3067787904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a38071bd0 0x7f4a381505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 >> 172.21.15.170:0/3067787904 conn(0x7f4a380cd3c0 msgr2=0x7f4a380cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 shutdown_connections 2024-09-21T02:21:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:21.816+0000 7f4a3ec29640 1 -- 172.21.15.170:0/3067787904 wait complete. 2024-09-21T02:21:21.834 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:21:21.834 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:21:21.834 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:21:21.834 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:21:21.835 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:21.835 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:21:21.835 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:21:21.835 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:21:21.835 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:21:21.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:21.835 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:21:21.842 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:21.843 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42027sec preferred_lft 42027sec 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:21.844 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:21.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:21.845 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:21:21.902 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:21:21.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:21.903 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:21.954 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:21.955 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:21:21.955 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:21:21.955 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:21:21.955 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:21:21.955 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:21:21.955 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:21:21.955 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:21:22.046 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:21:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:21:22.073 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:21:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:22.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:22.080 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:21:22.085 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:21:22.085 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:21:22.085 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:21:22.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:22.086 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:21:22.142 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:21:22.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:21:22.143 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:22.190 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:21:22.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:21:22.190 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:22.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:21:22.296 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:22.343 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:21:22.343 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:21:22.344 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:21:22.345 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:22.346 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:22.346 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:22.346 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:22.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:22.347 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:21:22.394 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:21:22.394 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:21:22.439 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:21:22.453 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:22.502 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:22.503 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:21:22.504 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:22.505 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:22.505 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:22.505 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:22.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:22.506 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:21:22.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:21:22.548+0000 7f041b4e0580 -1 init, newargv = 0x7f040c00a3f0 newargc=15 2024-09-21T02:21:22.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35714]: starting ceph client 2024-09-21T02:21:22.560 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35714]: starting fuse 2024-09-21T02:21:23.654 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:21:23.671 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:23.719 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:21:23.719 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:21:23.719 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:21:23.719 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:21:23.719 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:21:23.720 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:23.721 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:21:23.722 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:23.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:23.722 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:21:23.770 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:21:23.770 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:21:23.771 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:21:23.772 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:21:23.896 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.35714.asok 2024-09-21T02:21:23.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:21:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:23.908 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35714.asok 2024-09-21T02:21:23.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:23.908 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35714.asok status 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "35714", 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:21:24.090 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6386, 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6386 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 631227079 2024-09-21T02:21:24.091 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 631227079 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6386 192.168.0.1:0/631227079", 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/631227079", 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 100, 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 142, 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:21:24.092 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:21:24.093 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:21:24.093 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:21:24.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:24.106 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:21:24.106 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:24.112 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:21:24.112 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:24.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:24.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:24.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2662014612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40cd1f0 msgr2=0x7f1ba406ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2662014612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40cd1f0 0x7f1ba406ef40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009bc0 tx=0x7f1b9c01cb10 comp rx=0 tx=0).stop 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2662014612 shutdown_connections 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2662014612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40cd1f0 0x7f1ba406ef40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2662014612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ba40b52d0 0x7f1ba40af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2662014612 >> 172.21.15.170:0/2662014612 conn(0x7f1ba40b1c40 msgr2=0x7f1ba40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2662014612 shutdown_connections 2024-09-21T02:21:24.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.336+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2662014612 wait complete. 2024-09-21T02:21:24.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 Processor -- start 2024-09-21T02:21:24.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- start start 2024-09-21T02:21:24.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40b4d80 0x7f1ba41553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:24.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ba40b52d0 0x7f1ba415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:24.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 0x7f1ba40bec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:24.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4169f20 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4169da0 con 0x7f1ba40b4d80 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba416a0a0 con 0x7f1ba40b52d0 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 0x7f1ba40bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 0x7f1ba40bec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43284/0 (socket says 172.21.15.170:43284) 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 -- 172.21.15.170:0/2741236726 learned_addr learned my addr 172.21.15.170:0/2741236726 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba8f65640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40b4d80 0x7f1ba41553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba3fff640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ba40b52d0 0x7f1ba415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 -- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ba40b52d0 msgr2=0x7f1ba415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ba40b52d0 0x7f1ba415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 -- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40b4d80 msgr2=0x7f1ba41553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40b4d80 0x7f1ba41553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 -- 172.21.15.170:0/2741236726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba4155d10 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba8f65640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40b4d80 0x7f1ba41553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba9766640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 0x7f1ba40bec50 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c01a740 tx=0x7f1b9c009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0bb070 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b9c004d40 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba415a9f0 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0b2040 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba415afb0 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.340+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba40af440 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.344+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f1b9c0be070 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.344+0000 7f1ba17fa640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b900738c0 0x7f1b90075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:24.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.344+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f1b9c12ed90 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.344+0000 7f1ba8f65640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b900738c0 0x7f1b90075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:24.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.344+0000 7f1ba8f65640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b900738c0 0x7f1b90075dd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009b90 tx=0x7f1b980091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:24.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.348+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1b9c016790 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.552+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f1ba4156cc0 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.934 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:21:24.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:21:24.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.988+0000 7f1ba17fa640 1 -- 172.21.15.170:0/2741236726 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v101) ==== 136+0+0 (secure 0 0 0) 0x7f1b9c0f9610 con 0x7f1ba40cbfd0 2024-09-21T02:21:24.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b900738c0 msgr2=0x7f1b90075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b900738c0 0x7f1b90075dd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009b90 tx=0x7f1b980091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 msgr2=0x7f1ba40bec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 0x7f1ba40bec50 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c01a740 tx=0x7f1b9c009280 comp rx=0 tx=0).stop 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 shutdown_connections 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1b900738c0 0x7f1b90075dd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ba40b52d0 0x7f1ba415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ba40cbfd0 0x7f1ba40bec50 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 --2- 172.21.15.170:0/2741236726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ba40b4d80 0x7f1ba41553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 >> 172.21.15.170:0/2741236726 conn(0x7f1ba40b1c40 msgr2=0x7f1ba40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 shutdown_connections 2024-09-21T02:21:24.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:24.992+0000 7f1bab1c9640 1 -- 172.21.15.170:0/2741236726 wait complete. 2024-09-21T02:21:25.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:21:25.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 -- 172.21.15.170:0/349558354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b406d340 msgr2=0x7f65b4160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 --2- 172.21.15.170:0/349558354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b406d340 0x7f65b4160390 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f65b4073a30 tx=0x7f65ac01cd80 comp rx=0 tx=0).stop 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 -- 172.21.15.170:0/349558354 shutdown_connections 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 --2- 172.21.15.170:0/349558354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b406d340 0x7f65b4160390 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 --2- 172.21.15.170:0/349558354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 0x7f65b406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 --2- 172.21.15.170:0/349558354 >> v2:172.21.15.191:3300/0 conn(0x7f65b40b22b0 0x7f65b40b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 -- 172.21.15.170:0/349558354 >> 172.21.15.170:0/349558354 conn(0x7f65b409a9c0 msgr2=0x7f65b40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 -- 172.21.15.170:0/349558354 shutdown_connections 2024-09-21T02:21:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 -- 172.21.15.170:0/349558354 wait complete. 2024-09-21T02:21:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 Processor -- start 2024-09-21T02:21:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.224+0000 7f65b992f640 1 -- start start 2024-09-21T02:21:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b406d340 0x7f65b41474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 0x7f65b41441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 0x7f65b40cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415c420 con 0x7f65b40b22b0 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415c2a0 con 0x7f65b406d340 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f65b415c5a0 con 0x7f65b40b6930 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 0x7f65b41441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 0x7f65b41441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43302/0 (socket says 172.21.15.170:43302) 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 -- 172.21.15.170:0/2959923256 learned_addr learned my addr 172.21.15.170:0/2959923256 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b37fe640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 0x7f65b40cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b2ffd640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b406d340 0x7f65b41474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 -- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 msgr2=0x7f65b40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 0x7f65b40cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 -- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b406d340 msgr2=0x7f65b41474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b406d340 0x7f65b41474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 -- 172.21.15.170:0/2959923256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b41fb120 con 0x7f65b40b22b0 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b37fe640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 0x7f65b40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65abfff640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 0x7f65b41441f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d720 tx=0x7f659c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c017070 con 0x7f65b40b22b0 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f659c009d20 con 0x7f65b40b22b0 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b4147a20 con 0x7f65b40b22b0 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0106c0 con 0x7f65b40b22b0 2024-09-21T02:21:25.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b41fa5a0 con 0x7f65b40b22b0 2024-09-21T02:21:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.228+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b415cb30 con 0x7f65b40b22b0 2024-09-21T02:21:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.232+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f659c01a070 con 0x7f65b40b22b0 2024-09-21T02:21:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.232+0000 7f65b0ff9640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65940738c0 0x7f6594075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.232+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f659c0954f0 con 0x7f65b40b22b0 2024-09-21T02:21:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.232+0000 7f65b2ffd640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65940738c0 0x7f6594075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.232+0000 7f65b2ffd640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65940738c0 0x7f6594075dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65a400daa0 tx=0x7f65a4005db0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:25.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.236+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f65b4147a20 con 0x7f65b40b22b0 2024-09-21T02:21:25.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.428+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f65b41fa330 con 0x7f65940738c0 2024-09-21T02:21:25.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.432+0000 7f65b0ff9640 1 -- 172.21.15.170:0/2959923256 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f65b41fa330 con 0x7f65940738c0 2024-09-21T02:21:25.433 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:21:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:21:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:25.434 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65940738c0 msgr2=0x7f6594075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65940738c0 0x7f6594075dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65a400daa0 tx=0x7f65a4005db0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 msgr2=0x7f65b41441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 0x7f65b41441f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d720 tx=0x7f659c00dbe0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 shutdown_connections 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f65940738c0 0x7f6594075dd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65b40b6930 0x7f65b40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65b40b22b0 0x7f65b41441f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 --2- 172.21.15.170:0/2959923256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65b406d340 0x7f65b41474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 >> 172.21.15.170:0/2959923256 conn(0x7f65b409a9c0 msgr2=0x7f65b4071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:25.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 shutdown_connections 2024-09-21T02:21:25.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.436+0000 7f65b992f640 1 -- 172.21.15.170:0/2959923256 wait complete. 2024-09-21T02:21:25.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:21:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 -- 172.21.15.170:0/1926028537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 msgr2=0x7f2d0c003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 --2- 172.21.15.170:0/1926028537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d0c003f20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009b60 tx=0x7f2d1401d2b0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d22d9d640 1 -- 172.21.15.170:0/1926028537 reap_dead start 2024-09-21T02:21:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 -- 172.21.15.170:0/1926028537 shutdown_connections 2024-09-21T02:21:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 -- 172.21.15.170:0/1926028537 >> 172.21.15.170:0/1926028537 conn(0x7f2d2409d8c0 msgr2=0x7f2d240d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 -- 172.21.15.170:0/1926028537 shutdown_connections 2024-09-21T02:21:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 -- 172.21.15.170:0/1926028537 wait complete. 2024-09-21T02:21:25.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 Processor -- start 2024-09-21T02:21:25.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.612+0000 7f2d28b85640 1 -- start start 2024-09-21T02:21:25.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d24165200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d240b8fc0 0x7f2d240b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d240b4a40 0x7f2d241f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2416a4c0 con 0x7f2d0c003ab0 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2416a340 con 0x7f2d240b4a40 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2416a640 con 0x7f2d240b8fc0 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d24165200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d24165200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43226/0 (socket says 172.21.15.170:43226) 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 -- 172.21.15.170:0/2954401974 learned_addr learned my addr 172.21.15.170:0/2954401974 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d22d9d640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d240b4a40 0x7f2d241f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d21d9b640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d240b8fc0 0x7f2d240b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 -- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d240b8fc0 msgr2=0x7f2d240b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d240b8fc0 0x7f2d240b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 -- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d240b4a40 msgr2=0x7f2d241f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d240b4a40 0x7f2d241f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 -- 172.21.15.170:0/2954401974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d240b4e50 con 0x7f2d0c003ab0 2024-09-21T02:21:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d2259c640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d24165200 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2d140af040 tx=0x7f2d140055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140b1040 con 0x7f2d0c003ab0 2024-09-21T02:21:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d140bb070 con 0x7f2d0c003ab0 2024-09-21T02:21:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004270 con 0x7f2d0c003ab0 2024-09-21T02:21:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241fcbf0 con 0x7f2d0c003ab0 2024-09-21T02:21:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241fd1b0 con 0x7f2d0c003ab0 2024-09-21T02:21:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d240d2c10 con 0x7f2d0c003ab0 2024-09-21T02:21:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f2d140046d0 con 0x7f2d0c003ab0 2024-09-21T02:21:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d12ffd640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2d040738f0 0x7f2d04075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d21d9b640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2d040738f0 0x7f2d04075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f2d14130750 con 0x7f2d0c003ab0 2024-09-21T02:21:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.616+0000 7f2d21d9b640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2d040738f0 0x7f2d04075e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2d180058d0 tx=0x7f2d18005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:25.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.620+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2d240b4e50 con 0x7f2d0c003ab0 2024-09-21T02:21:25.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.780+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2d240b6e80 con 0x7f2d0c003ab0 2024-09-21T02:21:25.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.780+0000 7f2d12ffd640 1 -- 172.21.15.170:0/2954401974 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f2d14016fc0 con 0x7f2d0c003ab0 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2d040738f0 msgr2=0x7f2d04075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2d040738f0 0x7f2d04075e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2d180058d0 tx=0x7f2d18005860 comp rx=0 tx=0).stop 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 msgr2=0x7f2d24165200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d24165200 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2d140af040 tx=0x7f2d140055d0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 shutdown_connections 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2d040738f0 0x7f2d04075e00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d240b8fc0 0x7f2d240b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d240b4a40 0x7f2d241f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 --2- 172.21.15.170:0/2954401974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d0c003ab0 0x7f2d24165200 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 >> 172.21.15.170:0/2954401974 conn(0x7f2d2409d8c0 msgr2=0x7f2d24160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 shutdown_connections 2024-09-21T02:21:25.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:25.784+0000 7f2d28b85640 1 -- 172.21.15.170:0/2954401974 wait complete. 2024-09-21T02:21:25.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:25.802 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol19990) 2024-09-21T02:21:25.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:25.824 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp12690/subvol71491) 2024-09-21T02:21:25.886 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol19990 2024-09-21T02:21:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 -- 172.21.15.170:0/3607424850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0744b0 msgr2=0x7f442c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 --2- 172.21.15.170:0/3607424850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0744b0 0x7f442c0ce780 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4424009c80 tx=0x7f442401cd40 comp rx=0 tx=0).stop 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 -- 172.21.15.170:0/3607424850 shutdown_connections 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 --2- 172.21.15.170:0/3607424850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0744b0 0x7f442c0ce780 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 --2- 172.21.15.170:0/3607424850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f442c0cf7d0 0x7f442c073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 -- 172.21.15.170:0/3607424850 >> 172.21.15.170:0/3607424850 conn(0x7f442c0d0170 msgr2=0x7f442c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 -- 172.21.15.170:0/3607424850 shutdown_connections 2024-09-21T02:21:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 -- 172.21.15.170:0/3607424850 wait complete. 2024-09-21T02:21:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 Processor -- start 2024-09-21T02:21:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.088+0000 7f44334ab640 1 -- start start 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c0740a0 0x7f442c0b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f442c0744b0 0x7f442c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 0x7f442c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0b5cd0 con 0x7f442c0cf7d0 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f442c0ceed0 con 0x7f442c0740a0 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f442c168cb0 con 0x7f442c0744b0 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 0x7f442c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 0x7f442c06fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43240/0 (socket says 172.21.15.170:43240) 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 -- 172.21.15.170:0/3842668931 learned_addr learned my addr 172.21.15.170:0/3842668931 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431247640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c0740a0 0x7f442c0b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4430a46640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f442c0744b0 0x7f442c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 -- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f442c0744b0 msgr2=0x7f442c06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f442c0744b0 0x7f442c06f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 -- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c0740a0 msgr2=0x7f442c0b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c0740a0 0x7f442c0b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 -- 172.21.15.170:0/3842668931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c1fbe40 con 0x7f442c0cf7d0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431247640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c0740a0 0x7f442c0b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4431a48640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 0x7f442c06fd10 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f442401c7b0 tx=0x7f4424004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240bb070 con 0x7f442c0cf7d0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4424004a50 con 0x7f442c0cf7d0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c1fc0b0 con 0x7f442c0cf7d0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240b2040 con 0x7f442c0cf7d0 2024-09-21T02:21:26.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1fc670 con 0x7f442c0cf7d0 2024-09-21T02:21:26.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c073b60 con 0x7f442c0cf7d0 2024-09-21T02:21:26.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.092+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f44240be070 con 0x7f442c0cf7d0 2024-09-21T02:21:26.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.096+0000 7f4421ffb640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f44100738c0 0x7f4410075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.096+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f442412f5a0 con 0x7f442c0cf7d0 2024-09-21T02:21:26.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.096+0000 7f4431247640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f44100738c0 0x7f4410075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.096+0000 7f4431247640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f44100738c0 0x7f4410075dd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4428005fd0 tx=0x7f4428005dc0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:26.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.100+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f44240c5070 con 0x7f442c0cf7d0 2024-09-21T02:21:26.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.292+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol19990", "target": ["mon-mgr", ""]}) -- 0x7f442c0ae2a0 con 0x7f44100738c0 2024-09-21T02:21:26.301 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:26.300+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:26.301 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:26.300+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:26.301 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:26.300+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:26.301 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:26.300+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:26.301 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:26.300+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:26.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.328+0000 7f4421ffb640 1 -- 172.21.15.170:0/3842668931 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f442c0ae2a0 con 0x7f44100738c0 2024-09-21T02:21:26.330 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol19990 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f44100738c0 msgr2=0x7f4410075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f44100738c0 0x7f4410075dd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4428005fd0 tx=0x7f4428005dc0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 msgr2=0x7f442c06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 0x7f442c06fd10 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f442401c7b0 tx=0x7f4424004270 comp rx=0 tx=0).stop 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 shutdown_connections 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f44100738c0 0x7f4410075dd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f442c0744b0 0x7f442c06f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f442c0cf7d0 0x7f442c06fd10 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 --2- 172.21.15.170:0/3842668931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c0740a0 0x7f442c0b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 >> 172.21.15.170:0/3842668931 conn(0x7f442c0d0170 msgr2=0x7f442c0ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:26.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 shutdown_connections 2024-09-21T02:21:26.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.332+0000 7f44334ab640 1 -- 172.21.15.170:0/3842668931 wait complete. 2024-09-21T02:21:26.350 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol71491 subvol_grp12690 2024-09-21T02:21:26.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 -- 172.21.15.170:0/1912315597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e080ca7b0 msgr2=0x7f2e080cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 --2- 172.21.15.170:0/1912315597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e080ca7b0 0x7f2e080cabc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2df8005fd0 tx=0x7f2df801c820 comp rx=0 tx=0).stop 2024-09-21T02:21:26.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 -- 172.21.15.170:0/1912315597 shutdown_connections 2024-09-21T02:21:26.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 --2- 172.21.15.170:0/1912315597 >> v2:172.21.15.191:3300/0 conn(0x7f2e0806ffc0 0x7f2e080703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 --2- 172.21.15.170:0/1912315597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e080ca7b0 0x7f2e080cabc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 -- 172.21.15.170:0/1912315597 >> 172.21.15.170:0/1912315597 conn(0x7f2e080b1c40 msgr2=0x7f2e080b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:26.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 -- 172.21.15.170:0/1912315597 shutdown_connections 2024-09-21T02:21:26.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.492+0000 7f2e0def7640 1 -- 172.21.15.170:0/1912315597 wait complete. 2024-09-21T02:21:26.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 Processor -- start 2024-09-21T02:21:26.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 -- start start 2024-09-21T02:21:26.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e0806ffc0 0x7f2e0806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 0x7f2e0806f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e080cc7e0 0x7f2e081fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2e08169220 con 0x7f2e080ca7b0 2024-09-21T02:21:26.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2e081690a0 con 0x7f2e080cc7e0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2e081693a0 con 0x7f2e0806ffc0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 0x7f2e0806f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 0x7f2e0806f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43272/0 (socket says 172.21.15.170:43272) 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 -- 172.21.15.170:0/462034962 learned_addr learned my addr 172.21.15.170:0/462034962 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e077fe640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e0806ffc0 0x7f2e0806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e07fff640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e080cc7e0 0x7f2e081fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 -- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e0806ffc0 msgr2=0x7f2e0806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e0806ffc0 0x7f2e0806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 -- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e080cc7e0 msgr2=0x7f2e081fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e080cc7e0 0x7f2e081fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 -- 172.21.15.170:0/462034962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0815ec40 con 0x7f2e080ca7b0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2dfffff640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 0x7f2e0806f3f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f2df000e9f0 tx=0x7f2df000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df000cde0 con 0x7f2e080ca7b0 2024-09-21T02:21:26.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2df0004020 con 0x7f2e080ca7b0 2024-09-21T02:21:26.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081637f0 con 0x7f2e080ca7b0 2024-09-21T02:21:26.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0004470 con 0x7f2e080ca7b0 2024-09-21T02:21:26.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.496+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e08163db0 con 0x7f2e080ca7b0 2024-09-21T02:21:26.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.500+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e080703d0 con 0x7f2e080ca7b0 2024-09-21T02:21:26.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.500+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7f2df0003600 con 0x7f2e080ca7b0 2024-09-21T02:21:26.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.500+0000 7f2e04ff9640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2de80738f0 0x7f2de8075e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.500+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f2df00958b0 con 0x7f2e080ca7b0 2024-09-21T02:21:26.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.500+0000 7f2e077fe640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2de80738f0 0x7f2de8075e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.500+0000 7f2e077fe640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2de80738f0 0x7f2de8075e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2df80ae000 tx=0x7f2df801b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:26.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.504+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2df005fe90 con 0x7f2e080ca7b0 2024-09-21T02:21:26.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.708+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol71491", "group_name": "subvol_grp12690", "target": ["mon-mgr", ""]}) -- 0x7f2e080b3190 con 0x7f2de80738f0 2024-09-21T02:21:26.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.732+0000 7f2e04ff9640 1 -- 172.21.15.170:0/462034962 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2e080b3190 con 0x7f2de80738f0 2024-09-21T02:21:26.735 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp12690/subvol71491 2024-09-21T02:21:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2de80738f0 msgr2=0x7f2de8075e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2de80738f0 0x7f2de8075e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2df80ae000 tx=0x7f2df801b060 comp rx=0 tx=0).stop 2024-09-21T02:21:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 msgr2=0x7f2e0806f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 0x7f2e0806f3f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f2df000e9f0 tx=0x7f2df000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 shutdown_connections 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2de80738f0 0x7f2de8075e00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e080cc7e0 0x7f2e081fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e080ca7b0 0x7f2e0806f3f0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 --2- 172.21.15.170:0/462034962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e0806ffc0 0x7f2e0806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 >> 172.21.15.170:0/462034962 conn(0x7f2e080b1c40 msgr2=0x7f2e080ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 shutdown_connections 2024-09-21T02:21:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.736+0000 7f2e0def7640 1 -- 172.21.15.170:0/462034962 wait complete. 2024-09-21T02:21:26.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol19990 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 -- 172.21.15.170:0/2498674023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cec50 msgr2=0x7fa3fc06ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 --2- 172.21.15.170:0/2498674023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc06ff50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc072940 tx=0x7fa3ec01cd80 comp rx=0 tx=0).stop 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 -- 172.21.15.170:0/2498674023 shutdown_connections 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 --2- 172.21.15.170:0/2498674023 >> v2:172.21.15.191:3300/0 conn(0x7fa3fc0cf060 0x7fa3fc070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 --2- 172.21.15.170:0/2498674023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc06ff50 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 -- 172.21.15.170:0/2498674023 >> 172.21.15.170:0/2498674023 conn(0x7fa3fc0d0170 msgr2=0x7fa3fc0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 -- 172.21.15.170:0/2498674023 shutdown_connections 2024-09-21T02:21:26.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.896+0000 7fa403a6c640 1 -- 172.21.15.170:0/2498674023 wait complete. 2024-09-21T02:21:26.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 Processor -- start 2024-09-21T02:21:26.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 -- start start 2024-09-21T02:21:26.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3fc0b2fc0 0x7fa3fc09d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc0afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 0x7fa3fc0b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc169fe0 con 0x7fa3fc0cf060 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc0b35c0 con 0x7fa3fc0cec50 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc16a160 con 0x7fa3fc0b2fc0 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 0x7fa3fc0b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 0x7fa3fc0b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43300/0 (socket says 172.21.15.170:43300) 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 -- 172.21.15.170:0/1450932148 learned_addr learned my addr 172.21.15.170:0/1450932148 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa401808640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3fc0b2fc0 0x7fa3fc09d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa401007640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc0afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 -- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3fc0b2fc0 msgr2=0x7fa3fc09d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3fc0b2fc0 0x7fa3fc09d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 -- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3fc0cec50 msgr2=0x7fa3fc0afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc0afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 -- 172.21.15.170:0/1450932148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc1f9540 con 0x7fa3fc0cf060 2024-09-21T02:21:26.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa401007640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc0afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa402009640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 0x7fa3fc0b0110 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400c970 tx=0x7fa3f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004460 con 0x7fa3fc0cf060 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3f40077a0 con 0x7fa3fc0cf060 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3fc1f9820 con 0x7fa3fc0cf060 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4007bf0 con 0x7fa3fc0cf060 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.900+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3fc0cbf00 con 0x7fa3fc0cf060 2024-09-21T02:21:26.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.904+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3fc1f9d40 con 0x7fa3fc0cf060 2024-09-21T02:21:26.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.904+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 97846+0+0 (secure 0 0 0) 0x7fa3f4005020 con 0x7fa3fc0cf060 2024-09-21T02:21:26.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.904+0000 7fa3fa7fc640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3e80738c0 0x7fa3e8075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:26.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.904+0000 7fa401808640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3e80738c0 0x7fa3e8075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:26.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.904+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fa3f4095760 con 0x7fa3fc0cf060 2024-09-21T02:21:26.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.904+0000 7fa401808640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3e80738c0 0x7fa3e8075dd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b810 tx=0x7fa3f0008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:26.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:26.908+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa3fc1f9d40 con 0x7fa3fc0cf060 2024-09-21T02:21:27.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.016+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 24) ==== 97897+0+0 (secure 0 0 0) 0x7fa3f405bb50 con 0x7fa3fc0cf060 2024-09-21T02:21:27.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.104+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol19990", "target": ["mon-mgr", ""]}) -- 0x7fa3fc0cc160 con 0x7fa3e80738c0 2024-09-21T02:21:27.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.108+0000 7fa3fa7fc640 1 -- 172.21.15.170:0/1450932148 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fa3fc0cc160 con 0x7fa3e80738c0 2024-09-21T02:21:27.109 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol19990 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3e80738c0 msgr2=0x7fa3e8075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3e80738c0 0x7fa3e8075dd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b810 tx=0x7fa3f0008040 comp rx=0 tx=0).stop 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 msgr2=0x7fa3fc0b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 0x7fa3fc0b0110 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400c970 tx=0x7fa3f400ce30 comp rx=0 tx=0).stop 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 shutdown_connections 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3e80738c0 0x7fa3e8075dd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3fc0cf060 0x7fa3fc0b0110 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3fc0cec50 0x7fa3fc0afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 --2- 172.21.15.170:0/1450932148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3fc0b2fc0 0x7fa3fc09d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 >> 172.21.15.170:0/1450932148 conn(0x7fa3fc0d0170 msgr2=0x7fa3fc1598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:27.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 shutdown_connections 2024-09-21T02:21:27.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:27.112+0000 7fa403a6c640 1 -- 172.21.15.170:0/1450932148 wait complete. 2024-09-21T02:21:27.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:27.129 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/a58128e4893d8bcc46e08e1e24503c26.meta) 2024-09-21T02:21:28.162 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:21:28.162 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:21:28.163 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:21:28.163 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol19990 2024-09-21T02:21:28.163 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:21:28.163 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:28.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol71491 subvol_grp12690 2024-09-21T02:21:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 -- 172.21.15.170:0/2767615817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340ca7b0 msgr2=0x7f81340cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 --2- 172.21.15.170:0/2767615817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340ca7b0 0x7f81340cabc0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f81340ce380 tx=0x7f812c01cce0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 -- 172.21.15.170:0/2767615817 shutdown_connections 2024-09-21T02:21:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 --2- 172.21.15.170:0/2767615817 >> v2:172.21.15.191:3300/0 conn(0x7f81340735d0 0x7f81340739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 --2- 172.21.15.170:0/2767615817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340ca7b0 0x7f81340cabc0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 -- 172.21.15.170:0/2767615817 >> 172.21.15.170:0/2767615817 conn(0x7f81340b1c40 msgr2=0x7f81340b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 -- 172.21.15.170:0/2767615817 shutdown_connections 2024-09-21T02:21:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 -- 172.21.15.170:0/2767615817 wait complete. 2024-09-21T02:21:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 Processor -- start 2024-09-21T02:21:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.328+0000 7f813c089640 1 -- start start 2024-09-21T02:21:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f813406ffc0 0x7f81341503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 0x7f813406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81340ca7b0 0x7f813406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8134168cc0 con 0x7f81340735d0 2024-09-21T02:21:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8134168b40 con 0x7f81340ca7b0 2024-09-21T02:21:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8134168e40 con 0x7f813406ffc0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 0x7f813406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 0x7f813406eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43310/0 (socket says 172.21.15.170:43310) 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 -- 172.21.15.170:0/1155089811 learned_addr learned my addr 172.21.15.170:0/1155089811 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 -- 172.21.15.170:0/1155089811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f813406ffc0 msgr2=0x7f81341503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f813406ffc0 0x7f81341503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 -- 172.21.15.170:0/1155089811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81340ca7b0 msgr2=0x7f813406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81340ca7b0 0x7f813406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 -- 172.21.15.170:0/1155089811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8134150910 con 0x7f81340735d0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f8139624640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 0x7f813406eeb0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f812800c400 tx=0x7f812800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128017070 con 0x7f81340735d0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8128005400 con 0x7f81340735d0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813415d140 con 0x7f81340735d0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128003860 con 0x7f81340735d0 2024-09-21T02:21:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81341f6cc0 con 0x7f81340735d0 2024-09-21T02:21:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81340739e0 con 0x7f81340735d0 2024-09-21T02:21:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.332+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 97897+0+0 (secure 0 0 0) 0x7f812801a070 con 0x7f81340735d0 2024-09-21T02:21:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.336+0000 7f81327fc640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8120073980 0x7f8120075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.336+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f8128094ee0 con 0x7f81340735d0 2024-09-21T02:21:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.336+0000 7f8139e25640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8120073980 0x7f8120075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.336+0000 7f8139e25640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8120073980 0x7f8120075e90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f812c0af000 tx=0x7f812c019a80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.340+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f812805e980 con 0x7f81340735d0 2024-09-21T02:21:28.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.524+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol71491", "group_name": "subvol_grp12690", "target": ["mon-mgr", ""]}) -- 0x7f81340c9100 con 0x7f8120073980 2024-09-21T02:21:28.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.532+0000 7f81327fc640 1 -- 172.21.15.170:0/1155089811 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81340c9100 con 0x7f8120073980 2024-09-21T02:21:28.534 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp12690/subvol71491 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8120073980 msgr2=0x7f8120075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8120073980 0x7f8120075e90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f812c0af000 tx=0x7f812c019a80 comp rx=0 tx=0).stop 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 msgr2=0x7f813406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 0x7f813406eeb0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f812800c400 tx=0x7f812800c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 shutdown_connections 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8120073980 0x7f8120075e90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81340ca7b0 0x7f813406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81340735d0 0x7f813406eeb0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 --2- 172.21.15.170:0/1155089811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f813406ffc0 0x7f81341503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:28.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 >> 172.21.15.170:0/1155089811 conn(0x7f81340b1c40 msgr2=0x7f813415e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:28.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 shutdown_connections 2024-09-21T02:21:28.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:28.536+0000 7f813c089640 1 -- 172.21.15.170:0/1155089811 wait complete. 2024-09-21T02:21:28.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:28.554 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/06b7f889f9957617dc74010fa6d2f6bd.meta) 2024-09-21T02:21:33.160 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:21:33.160 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:21:33.160 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:21:33.160 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/subvol_grp12690/subvol71491 2024-09-21T02:21:33.161 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:21:33.161 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:33.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol19990 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb64a043640 1 -- 172.21.15.170:0/2801101951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb62c004d50 msgr2=0x7fb62c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb64a043640 1 --2- 172.21.15.170:0/2801101951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb62c004d50 0x7fb62c005160 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0013e0 tx=0x7fb63c01d260 comp rx=0 tx=0).stop 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb643fff640 1 -- 172.21.15.170:0/2801101951 reap_dead start 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb64a043640 1 -- 172.21.15.170:0/2801101951 shutdown_connections 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb64a043640 1 -- 172.21.15.170:0/2801101951 >> 172.21.15.170:0/2801101951 conn(0x7fb6440d0170 msgr2=0x7fb6440d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb64a043640 1 -- 172.21.15.170:0/2801101951 shutdown_connections 2024-09-21T02:21:33.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.328+0000 7fb64a043640 1 -- 172.21.15.170:0/2801101951 wait complete. 2024-09-21T02:21:33.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 Processor -- start 2024-09-21T02:21:33.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 -- start start 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62c004d50 0x7fb6441536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb644153d40 0x7fb644157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 0x7fb6440b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440b33d0 con 0x7fb6440b1d60 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440afa10 con 0x7fb62c004d50 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440b3550 con 0x7fb644153d40 2024-09-21T02:21:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 0x7fb6440b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 0x7fb6440b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43330/0 (socket says 172.21.15.170:43330) 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 -- 172.21.15.170:0/4074321775 learned_addr learned my addr 172.21.15.170:0/4074321775 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb642ffd640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb644153d40 0x7fb644157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb6437fe640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62c004d50 0x7fb6441536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 -- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb644153d40 msgr2=0x7fb644157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb644153d40 0x7fb644157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 -- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62c004d50 msgr2=0x7fb6441536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62c004d50 0x7fb6441536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 -- 172.21.15.170:0/4074321775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6440b26b0 con 0x7fb6440b1d60 2024-09-21T02:21:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb643fff640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 0x7fb6440b2170 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb63c098e30 tx=0x7fb63c001d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c004e40 con 0x7fb6440b1d60 2024-09-21T02:21:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb63c003b90 con 0x7fb6440b1d60 2024-09-21T02:21:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64415a3e0 con 0x7fb6440b1d60 2024-09-21T02:21:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c006030 con 0x7fb6440b1d60 2024-09-21T02:21:33.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.332+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64415a9a0 con 0x7fb6440b1d60 2024-09-21T02:21:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.336+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6440cf180 con 0x7fb6440b1d60 2024-09-21T02:21:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.336+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 97897+0+0 (secure 0 0 0) 0x7fb63c0061d0 con 0x7fb6440b1d60 2024-09-21T02:21:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.336+0000 7fb62bfff640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb630073950 0x7fb630075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.336+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fb63c12fde0 con 0x7fb6440b1d60 2024-09-21T02:21:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.336+0000 7fb6437fe640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb630073950 0x7fb630075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.336+0000 7fb6437fe640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb630073950 0x7fb630075e60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb634009b30 tx=0x7fb6340091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:33.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.340+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb63c136080 con 0x7fb6440b1d60 2024-09-21T02:21:33.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.532+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol19990", "target": ["mon-mgr", ""]}) -- 0x7fb64415a160 con 0x7fb630073950 2024-09-21T02:21:33.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.560+0000 7fb62bfff640 1 -- 172.21.15.170:0/4074321775 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb64415a160 con 0x7fb630073950 2024-09-21T02:21:33.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb630073950 msgr2=0x7fb630075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb630073950 0x7fb630075e60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb634009b30 tx=0x7fb6340091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 msgr2=0x7fb6440b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 0x7fb6440b2170 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb63c098e30 tx=0x7fb63c001d30 comp rx=0 tx=0).stop 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 shutdown_connections 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb630073950 0x7fb630075e60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb644153d40 0x7fb644157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6440b1d60 0x7fb6440b2170 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 --2- 172.21.15.170:0/4074321775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb62c004d50 0x7fb6441536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 >> 172.21.15.170:0/4074321775 conn(0x7fb6440d0170 msgr2=0x7fb6440b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 shutdown_connections 2024-09-21T02:21:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.564+0000 7fb64a043640 1 -- 172.21.15.170:0/4074321775 wait complete. 2024-09-21T02:21:33.566 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.564+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.566 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.564+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.566 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.564+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.566 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.564+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.566 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.564+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.580 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.580+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.581 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.580+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.581 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.580+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.581 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.580+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.581 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:21:33.580+0000 7efba0ac7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:21:33.587 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol71491 subvol_grp12690 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 -- 172.21.15.170:0/1868909146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cd1f0 msgr2=0x7f4ec006ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/1868909146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cd1f0 0x7f4ec006ef40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009c80 tx=0x7f4eb801cd10 comp rx=0 tx=0).stop 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 -- 172.21.15.170:0/1868909146 shutdown_connections 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/1868909146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cd1f0 0x7f4ec006ef40 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/1868909146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ec00b52d0 0x7f4ec00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 -- 172.21.15.170:0/1868909146 >> 172.21.15.170:0/1868909146 conn(0x7f4ec00b1c40 msgr2=0x7f4ec00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:33.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 -- 172.21.15.170:0/1868909146 shutdown_connections 2024-09-21T02:21:33.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.736+0000 7f4ec63a6640 1 -- 172.21.15.170:0/1868909146 wait complete. 2024-09-21T02:21:33.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 Processor -- start 2024-09-21T02:21:33.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- start start 2024-09-21T02:21:33.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ec00b4d80 0x7f4ec01553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ec00b52d0 0x7f4ec015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 0x7f4ec00bec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0169f20 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0169da0 con 0x7f4ec00b4d80 2024-09-21T02:21:33.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec016a0a0 con 0x7f4ec00b52d0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 0x7f4ec00bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 0x7f4ec00bec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43386/0 (socket says 172.21.15.170:43386) 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 -- 172.21.15.170:0/775422107 learned_addr learned my addr 172.21.15.170:0/775422107 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ebffff640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ec00b4d80 0x7f4ec01553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ebf7fe640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ec00b52d0 0x7f4ec015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 -- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ec00b52d0 msgr2=0x7f4ec015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ec00b52d0 0x7f4ec015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 -- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ec00b4d80 msgr2=0x7f4ec01553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ec00b4d80 0x7f4ec01553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 -- 172.21.15.170:0/775422107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec0155d10 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ebffff640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ec00b4d80 0x7f4ec01553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec4943640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 0x7f4ec00bec50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009bd0 tx=0x7f4eb80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80bb070 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eb8004e00 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec015a9f0 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80b2040 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec015afb0 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.740+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec00af440 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.744+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 97897+0+0 (secure 0 0 0) 0x7f4eb80be070 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.744+0000 7f4ebcff9640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4eac073950 0x7f4eac075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:33.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.744+0000 7f4ebffff640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4eac073950 0x7f4eac075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:33.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.744+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f4eb812edf0 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.744+0000 7f4ebffff640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4eac073950 0x7f4eac075e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005fd0 tx=0x7f4eb0005e20 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:33.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.748+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4eb8016990 con 0x7f4ec00cbfd0 2024-09-21T02:21:33.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.944+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol71491", "group_name": "subvol_grp12690", "target": ["mon-mgr", ""]}) -- 0x7f4ec0156cc0 con 0x7f4eac073950 2024-09-21T02:21:33.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ebcff9640 1 -- 172.21.15.170:0/775422107 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ec0156cc0 con 0x7f4eac073950 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4eac073950 msgr2=0x7f4eac075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4eac073950 0x7f4eac075e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005fd0 tx=0x7f4eb0005e20 comp rx=0 tx=0).stop 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 msgr2=0x7f4ec00bec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 0x7f4ec00bec50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009bd0 tx=0x7f4eb80093d0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 shutdown_connections 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4eac073950 0x7f4eac075e60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ec00b52d0 0x7f4ec015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec00cbfd0 0x7f4ec00bec50 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 --2- 172.21.15.170:0/775422107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ec00b4d80 0x7f4ec01553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:33.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.972+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 >> 172.21.15.170:0/775422107 conn(0x7f4ec00b1c40 msgr2=0x7f4ec00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:33.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.976+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 shutdown_connections 2024-09-21T02:21:33.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:33.976+0000 7f4ec63a6640 1 -- 172.21.15.170:0/775422107 wait complete. 2024-09-21T02:21:33.996 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:21:34.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:21:34.045 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:21:34.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:34.052 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp12690 2024-09-21T02:21:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 -- 172.21.15.170:0/786098621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340cf7d0 msgr2=0x7efd34073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/786098621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340cf7d0 0x7efd34073b60 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7efd30009900 tx=0x7efd3001c880 comp rx=0 tx=0).stop 2024-09-21T02:21:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 -- 172.21.15.170:0/786098621 shutdown_connections 2024-09-21T02:21:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/786098621 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd340744b0 0x7efd340ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/786098621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340cf7d0 0x7efd34073b60 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 -- 172.21.15.170:0/786098621 >> 172.21.15.170:0/786098621 conn(0x7efd340d0170 msgr2=0x7efd340d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 -- 172.21.15.170:0/786098621 shutdown_connections 2024-09-21T02:21:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 -- 172.21.15.170:0/786098621 wait complete. 2024-09-21T02:21:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 Processor -- start 2024-09-21T02:21:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.252+0000 7efd3b6a6640 1 -- start start 2024-09-21T02:21:34.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 0x7efd3415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd340744b0 0x7efd3409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd340cf7d0 0x7efd3414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd340ceed0 con 0x7efd340740a0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efd34168cb0 con 0x7efd340cf7d0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7efd3409b000 con 0x7efd340744b0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 0x7efd3415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 0x7efd3415e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43400/0 (socket says 172.21.15.170:43400) 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 -- 172.21.15.170:0/4110960528 learned_addr learned my addr 172.21.15.170:0/4110960528 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 -- 172.21.15.170:0/4110960528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd340744b0 msgr2=0x7efd3409a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd340744b0 0x7efd3409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 -- 172.21.15.170:0/4110960528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd340cf7d0 msgr2=0x7efd3414a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd340cf7d0 0x7efd3414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 -- 172.21.15.170:0/4110960528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3415ed60 con 0x7efd340740a0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd39442640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 0x7efd3415e460 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7efd30009850 tx=0x7efd300a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300b2040 con 0x7efd340740a0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd300bd070 con 0x7efd340740a0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3415c080 con 0x7efd340740a0 2024-09-21T02:21:34.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30003600 con 0x7efd340740a0 2024-09-21T02:21:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3415c600 con 0x7efd340740a0 2024-09-21T02:21:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd34073b60 con 0x7efd340740a0 2024-09-21T02:21:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.256+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 97897+0+0 (secure 0 0 0) 0x7efd300037a0 con 0x7efd340740a0 2024-09-21T02:21:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.260+0000 7efd29ffb640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd18073950 0x7efd18075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.260+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7efd3012fc40 con 0x7efd340740a0 2024-09-21T02:21:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.260+0000 7efd38c41640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd18073950 0x7efd18075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.260+0000 7efd38c41640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd18073950 0x7efd18075e60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd24007fd0 tx=0x7efd24007940 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:34.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.264+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7efd300164d0 con 0x7efd340740a0 2024-09-21T02:21:34.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.444+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp12690", "target": ["mon-mgr", ""]}) -- 0x7efd340b5cd0 con 0x7efd18073950 2024-09-21T02:21:34.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd29ffb640 1 -- 172.21.15.170:0/4110960528 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd240050d0 con 0x7efd18073950 2024-09-21T02:21:34.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd18073950 msgr2=0x7efd18075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd18073950 0x7efd18075e60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd24007fd0 tx=0x7efd24007940 comp rx=0 tx=0).stop 2024-09-21T02:21:34.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 msgr2=0x7efd3415e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 0x7efd3415e460 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7efd30009850 tx=0x7efd300a5630 comp rx=0 tx=0).stop 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 shutdown_connections 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7efd18073950 0x7efd18075e60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd340744b0 0x7efd3409a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd340cf7d0 0x7efd3414a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 --2- 172.21.15.170:0/4110960528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd340740a0 0x7efd3415e460 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 >> 172.21.15.170:0/4110960528 conn(0x7efd340d0170 msgr2=0x7efd340b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 shutdown_connections 2024-09-21T02:21:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.456+0000 7efd3b6a6640 1 -- 172.21.15.170:0/4110960528 wait complete. 2024-09-21T02:21:34.476 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:21:34.476 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:21:34.549 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:21:34.627 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:21:34.627 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:21:34.627 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:34.628 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:34.633 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:34.633 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:34.680 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:21:34.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 -- 172.21.15.170:0/3202331354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a8163850 msgr2=0x7fe0a8165c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/3202331354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a8163850 0x7fe0a8165c70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000ad90 tx=0x7fe0a001dcb0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 -- 172.21.15.170:0/3202331354 shutdown_connections 2024-09-21T02:21:34.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/3202331354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a8163850 0x7fe0a8165c70 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 -- 172.21.15.170:0/3202331354 >> 172.21.15.170:0/3202331354 conn(0x7fe0a80b1c40 msgr2=0x7fe0a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:34.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 -- 172.21.15.170:0/3202331354 shutdown_connections 2024-09-21T02:21:34.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 -- 172.21.15.170:0/3202331354 wait complete. 2024-09-21T02:21:34.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 Processor -- start 2024-09-21T02:21:34.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.884+0000 7fe0acd2b640 1 -- start start 2024-09-21T02:21:34.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0a8070470 0x7fe0a8155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 0x7fe0a809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0a8163850 0x7fe0a8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8169280 con 0x7fe0a815c5e0 2024-09-21T02:21:34.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8169100 con 0x7fe0a8070470 2024-09-21T02:21:34.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a8169400 con 0x7fe0a8163850 2024-09-21T02:21:34.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 0x7fe0a809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:34.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 0x7fe0a809a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43418/0 (socket says 172.21.15.170:43418) 2024-09-21T02:21:34.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 -- 172.21.15.170:0/4157426342 learned_addr learned my addr 172.21.15.170:0/4157426342 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:34.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0a659c640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0a8070470 0x7fe0a8155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:34.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0a6d9d640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0a8163850 0x7fe0a8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:34.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 -- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0a8163850 msgr2=0x7fe0a8073cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0a8163850 0x7fe0a8073cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 -- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0a8070470 msgr2=0x7fe0a8155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0a8070470 0x7fe0a8155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 -- 172.21.15.170:0/4157426342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a809d360 con 0x7fe0a815c5e0 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0a659c640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0a8070470 0x7fe0a8155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09ffff640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 0x7fe0a809a9c0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe09000c430 tx=0x7fe09000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090017070 con 0x7fe0a815c5e0 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0900053c0 con 0x7fe0a815c5e0 2024-09-21T02:21:34.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090003860 con 0x7fe0a815c5e0 2024-09-21T02:21:34.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a809d640 con 0x7fe0a815c5e0 2024-09-21T02:21:34.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a809dc00 con 0x7fe0a815c5e0 2024-09-21T02:21:34.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.888+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a80cd0f0 con 0x7fe0a815c5e0 2024-09-21T02:21:34.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.892+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 97897+0+0 (secure 0 0 0) 0x7fe09001a070 con 0x7fe0a815c5e0 2024-09-21T02:21:34.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.892+0000 7fe09effd640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe088073980 0x7fe088075e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:34.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.892+0000 7fe0a659c640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe088073980 0x7fe088075e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:34.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.892+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fe090094830 con 0x7fe0a815c5e0 2024-09-21T02:21:34.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.892+0000 7fe0a659c640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe088073980 0x7fe088075e90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0980098b0 tx=0x7fe098008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:34.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:34.896+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe09005e8b0 con 0x7fe0a815c5e0 2024-09-21T02:21:35.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.040+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fe09005a420 con 0x7fe0a815c5e0 2024-09-21T02:21:35.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.072+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe0a8156710 con 0x7fe0a815c5e0 2024-09-21T02:21:35.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.072+0000 7fe09effd640 1 -- 172.21.15.170:0/4157426342 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7fe090019e90 con 0x7fe0a815c5e0 2024-09-21T02:21:35.074 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:21:35.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe088073980 msgr2=0x7fe088075e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe088073980 0x7fe088075e90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0980098b0 tx=0x7fe098008040 comp rx=0 tx=0).stop 2024-09-21T02:21:35.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 msgr2=0x7fe0a809a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 0x7fe0a809a9c0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe09000c430 tx=0x7fe09000c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 shutdown_connections 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe088073980 0x7fe088075e90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0a8163850 0x7fe0a8073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0a815c5e0 0x7fe0a809a9c0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 --2- 172.21.15.170:0/4157426342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0a8070470 0x7fe0a8155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 >> 172.21.15.170:0/4157426342 conn(0x7fe0a80b1c40 msgr2=0x7fe0a815c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 shutdown_connections 2024-09-21T02:21:35.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.076+0000 7fe0acd2b640 1 -- 172.21.15.170:0/4157426342 wait complete. 2024-09-21T02:21:35.094 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:21:35.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:35.095 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:35.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[50716]: fuse finished with error 0 and tester_r 0 2024-09-21T02:21:35.105 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:21:35.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:35.105 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:21:35.191 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:21:35.191 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:35.191 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:35.198 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:35.198 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:35.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:21:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 -- 172.21.15.170:0/3410195699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a9c005360 msgr2=0x7f1a9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 --2- 172.21.15.170:0/3410195699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a9c005360 0x7f1a9c00b9d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009e10 tx=0x7f1aac01d190 comp rx=0 tx=0).stop 2024-09-21T02:21:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 -- 172.21.15.170:0/3410195699 shutdown_connections 2024-09-21T02:21:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 --2- 172.21.15.170:0/3410195699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a9c005360 0x7f1a9c00b9d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 -- 172.21.15.170:0/3410195699 >> 172.21.15.170:0/3410195699 conn(0x7f1ab40b1c40 msgr2=0x7f1ab40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 -- 172.21.15.170:0/3410195699 shutdown_connections 2024-09-21T02:21:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 -- 172.21.15.170:0/3410195699 wait complete. 2024-09-21T02:21:35.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 Processor -- start 2024-09-21T02:21:35.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.408+0000 7f1abab05640 1 -- start start 2024-09-21T02:21:35.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a9c005360 0x7f1ab415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ab415afc0 0x7f1ab41496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 0x7f1ab414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40cf760 con 0x7f1ab414a2c0 2024-09-21T02:21:35.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40c3a10 con 0x7f1a9c005360 2024-09-21T02:21:35.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40cf8e0 con 0x7f1ab415afc0 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 0x7f1ab414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 0x7f1ab414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36042/0 (socket says 172.21.15.170:36042) 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 -- 172.21.15.170:0/671168187 learned_addr learned my addr 172.21.15.170:0/671168187 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab88a1640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a9c005360 0x7f1ab415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab3fff640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ab415afc0 0x7f1ab41496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 -- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ab415afc0 msgr2=0x7f1ab41496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ab415afc0 0x7f1ab41496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 -- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a9c005360 msgr2=0x7f1ab415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a9c005360 0x7f1ab415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 -- 172.21.15.170:0/671168187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab41602f0 con 0x7f1ab414a2c0 2024-09-21T02:21:35.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab90a2640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 0x7f1ab414a6d0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009d30 tx=0x7f1aac004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0b1040 con 0x7f1ab414a2c0 2024-09-21T02:21:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aac0ac7b0 con 0x7f1ab414a2c0 2024-09-21T02:21:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab415f770 con 0x7f1ab414a2c0 2024-09-21T02:21:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0acc00 con 0x7f1ab414a2c0 2024-09-21T02:21:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab415fd30 con 0x7f1ab414a2c0 2024-09-21T02:21:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.412+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab415f4e0 con 0x7f1ab414a2c0 2024-09-21T02:21:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.416+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f1aac0be020 con 0x7f1ab414a2c0 2024-09-21T02:21:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.416+0000 7f1ab17fa640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa0073a70 0x7f1aa0075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.416+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f1aac12d2b0 con 0x7f1ab414a2c0 2024-09-21T02:21:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.416+0000 7f1ab88a1640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa0073a70 0x7f1aa0075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.416+0000 7f1ab88a1640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa0073a70 0x7f1aa0075f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4006690 tx=0x7f1aa4006b30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:35.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.420+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1aac0c5020 con 0x7f1ab414a2c0 2024-09-21T02:21:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.588+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1ab40b3510 con 0x7f1ab414a2c0 2024-09-21T02:21:35.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.588+0000 7f1ab17fa640 1 -- 172.21.15.170:0/671168187 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7f1aac016ea0 con 0x7f1ab414a2c0 2024-09-21T02:21:35.591 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa0073a70 msgr2=0x7f1aa0075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa0073a70 0x7f1aa0075f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4006690 tx=0x7f1aa4006b30 comp rx=0 tx=0).stop 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 msgr2=0x7f1ab414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 0x7f1ab414a6d0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009d30 tx=0x7f1aac004f80 comp rx=0 tx=0).stop 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 shutdown_connections 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f1aa0073a70 0x7f1aa0075f80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ab415afc0 0x7f1ab41496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab414a2c0 0x7f1ab414a6d0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 --2- 172.21.15.170:0/671168187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a9c005360 0x7f1ab415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 >> 172.21.15.170:0/671168187 conn(0x7f1ab40b1c40 msgr2=0x7f1ab40cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 shutdown_connections 2024-09-21T02:21:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.592+0000 7f1abab05640 1 -- 172.21.15.170:0/671168187 wait complete. 2024-09-21T02:21:35.613 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:21:35.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:35.614 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:21:35.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35714]: fuse finished with error 0 and tester_r 0 2024-09-21T02:21:35.629 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:21:35.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:35.629 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:21:35.727 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:21:35.728 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:35.728 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:35.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:21:35.735 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:21:35.735 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:35.735 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:35.742 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:21:35.743 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:35.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 -- 172.21.15.170:0/3824904105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67815f990 msgr2=0x7fb678070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/3824904105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67815f990 0x7fb678070030 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb678074c60 tx=0x7fb66801c890 comp rx=0 tx=0).stop 2024-09-21T02:21:35.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 -- 172.21.15.170:0/3824904105 shutdown_connections 2024-09-21T02:21:35.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/3824904105 >> v2:172.21.15.191:3300/0 conn(0x7fb6780cea00 0x7fb6780cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/3824904105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb678070570 0x7fb6780ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/3824904105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb67815f990 0x7fb678070030 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 -- 172.21.15.170:0/3824904105 >> 172.21.15.170:0/3824904105 conn(0x7fb6780b1c40 msgr2=0x7fb6780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:35.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 -- 172.21.15.170:0/3824904105 shutdown_connections 2024-09-21T02:21:35.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 -- 172.21.15.170:0/3824904105 wait complete. 2024-09-21T02:21:35.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 Processor -- start 2024-09-21T02:21:35.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.932+0000 7fb67ec5b640 1 -- start start 2024-09-21T02:21:35.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb678070570 0x7fb678155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 0x7fb6781f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb67815f990 0x7fb67809bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb678073ae0 con 0x7fb6780cea00 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb678073960 con 0x7fb678070570 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb678073c60 con 0x7fb67815f990 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 0x7fb6781f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 0x7fb6781f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36056/0 (socket says 172.21.15.170:36056) 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 -- 172.21.15.170:0/4132238911 learned_addr learned my addr 172.21.15.170:0/4132238911 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67d1f8640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb67815f990 0x7fb67809bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67c9f7640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb678070570 0x7fb678155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 -- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb67815f990 msgr2=0x7fb67809bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb67815f990 0x7fb67809bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 -- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb678070570 msgr2=0x7fb678155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb678070570 0x7fb678155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 -- 172.21.15.170:0/4132238911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67809ab60 con 0x7fb6780cea00 2024-09-21T02:21:35.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67c9f7640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb678070570 0x7fb678155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:35.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb677fff640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 0x7fb6781f6380 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00b420 tx=0x7fb66c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:35.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c011830 con 0x7fb6780cea00 2024-09-21T02:21:35.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb66c016070 con 0x7fb6780cea00 2024-09-21T02:21:35.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb678159220 con 0x7fb6780cea00 2024-09-21T02:21:35.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c011d60 con 0x7fb6780cea00 2024-09-21T02:21:35.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.936+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67809ae00 con 0x7fb6780cea00 2024-09-21T02:21:35.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.940+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6780cadf0 con 0x7fb6780cea00 2024-09-21T02:21:35.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.940+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fb66c01b460 con 0x7fb6780cea00 2024-09-21T02:21:35.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.940+0000 7fb6757fa640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb664073aa0 0x7fb664075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:35.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.940+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fb66c0927a0 con 0x7fb6780cea00 2024-09-21T02:21:35.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.940+0000 7fb67c9f7640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb664073aa0 0x7fb664075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:35.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.940+0000 7fb67c9f7640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb664073aa0 0x7fb664075fb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb6680af000 tx=0x7fb66801b0d0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:35.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:35.944+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb66c05f300 con 0x7fb6780cea00 2024-09-21T02:21:36.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.136+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6780ce2a0 con 0x7fb6780cea00 2024-09-21T02:21:36.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.136+0000 7fb6757fa640 1 -- 172.21.15.170:0/4132238911 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5238 (secure 0 0 0) 0x7fb66c05ec00 con 0x7fb6780cea00 2024-09-21T02:21:36.139 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:36.139 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"btime":"2024-09-21T02:21:24:984660+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:12.056653+0000","modified":"2024-09-21T02:21:15.085591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2032631050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2032631050},{"type":"v1","addr":"172.21.15.170:6835","nonce":2032631050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":11}]} 2024-09-21T02:21:36.139 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2024-09-21T02:21:36.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb664073aa0 msgr2=0x7fb664075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb664073aa0 0x7fb664075fb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb6680af000 tx=0x7fb66801b0d0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 msgr2=0x7fb6781f6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 0x7fb6781f6380 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00b420 tx=0x7fb66c00b8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 shutdown_connections 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb664073aa0 0x7fb664075fb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb67815f990 0x7fb67809bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6780cea00 0x7fb6781f6380 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 --2- 172.21.15.170:0/4132238911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb678070570 0x7fb678155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 >> 172.21.15.170:0/4132238911 conn(0x7fb6780b1c40 msgr2=0x7fb6780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 shutdown_connections 2024-09-21T02:21:36.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.140+0000 7fb67ec5b640 1 -- 172.21.15.170:0/4132238911 wait complete. 2024-09-21T02:21:36.161 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 -- 172.21.15.170:0/790466110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc00744a0 msgr2=0x7f2dc00cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/790466110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc00744a0 0x7f2dc00cc740 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009bc0 tx=0x7f2db801cba0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 -- 172.21.15.170:0/790466110 shutdown_connections 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/790466110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc00744a0 0x7f2dc00cc740 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 -- 172.21.15.170:0/790466110 >> 172.21.15.170:0/790466110 conn(0x7f2dc00cd3c0 msgr2=0x7f2dc00cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 -- 172.21.15.170:0/790466110 shutdown_connections 2024-09-21T02:21:36.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.308+0000 7f2dc52bd640 1 -- 172.21.15.170:0/790466110 wait complete. 2024-09-21T02:21:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 Processor -- start 2024-09-21T02:21:36.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 -- start start 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 0x7f2dc0158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc00744a0 0x7f2dc014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc015b9c0 0x7f2dc01591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc016b210 con 0x7f2dc0074090 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc016b090 con 0x7f2dc015b9c0 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dc52bd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc016b390 con 0x7f2dc00744a0 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 0x7f2dc0158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 0x7f2dc0158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36086/0 (socket says 172.21.15.170:36086) 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 -- 172.21.15.170:0/2570908688 learned_addr learned my addr 172.21.15.170:0/2570908688 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbe7fc640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc00744a0 0x7f2dc014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbf7fe640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc015b9c0 0x7f2dc01591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 -- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc00744a0 msgr2=0x7f2dc014bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc00744a0 0x7f2dc014bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 -- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc015b9c0 msgr2=0x7f2dc01591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc015b9c0 0x7f2dc01591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbeffd640 1 -- 172.21.15.170:0/2570908688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc0159fc0 con 0x7f2dc0074090 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.312+0000 7f2dbf7fe640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc015b9c0 0x7f2dc01591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2dbeffd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 0x7f2dc0158590 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c400 tx=0x7f2db400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4011790 con 0x7f2dc0074090 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2db4017070 con 0x7f2dc0074090 2024-09-21T02:21:36.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4011cc0 con 0x7f2dc0074090 2024-09-21T02:21:36.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc015a2a0 con 0x7f2dc0074090 2024-09-21T02:21:36.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc014e000 con 0x7f2dc0074090 2024-09-21T02:21:36.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc0070130 con 0x7f2dc0074090 2024-09-21T02:21:36.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f2db401c460 con 0x7f2dc0074090 2024-09-21T02:21:36.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2da77fe640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dac073aa0 0x7f2dac075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f2db4092820 con 0x7f2dc0074090 2024-09-21T02:21:36.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2dbe7fc640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dac073aa0 0x7f2dac075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.316+0000 7f2dbe7fc640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dac073aa0 0x7f2dac075fb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2dc014dc50 tx=0x7f2db0008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:36.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.324+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2db405f2f0 con 0x7f2dc0074090 2024-09-21T02:21:36.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.532+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dc00c72a0 con 0x7f2dc0074090 2024-09-21T02:21:36.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.532+0000 7f2da77fe640 1 -- 172.21.15.170:0/2570908688 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5238 (secure 0 0 0) 0x7f2db405ebf0 con 0x7f2dc0074090 2024-09-21T02:21:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:36.535 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"btime":"2024-09-21T02:21:24:984660+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:12.056653+0000","modified":"2024-09-21T02:21:15.085591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2032631050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2032631050},{"type":"v1","addr":"172.21.15.170:6835","nonce":2032631050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":11}]} 2024-09-21T02:21:36.535 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2024-09-21T02:21:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dac073aa0 msgr2=0x7f2dac075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dac073aa0 0x7f2dac075fb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2dc014dc50 tx=0x7f2db0008040 comp rx=0 tx=0).stop 2024-09-21T02:21:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 msgr2=0x7f2dc0158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 0x7f2dc0158590 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c400 tx=0x7f2db400c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 shutdown_connections 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2dac073aa0 0x7f2dac075fb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dc00744a0 0x7f2dc014bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc015b9c0 0x7f2dc01591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 --2- 172.21.15.170:0/2570908688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dc0074090 0x7f2dc0158590 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 >> 172.21.15.170:0/2570908688 conn(0x7f2dc00cd3c0 msgr2=0x7f2dc00b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 shutdown_connections 2024-09-21T02:21:36.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.536+0000 7f2dc52bd640 1 -- 172.21.15.170:0/2570908688 wait complete. 2024-09-21T02:21:36.558 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 -- 172.21.15.170:0/2358055512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500d0aa0 msgr2=0x7fc1500d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 --2- 172.21.15.170:0/2358055512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500d0aa0 0x7fc1500d0eb0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fc140009bc0 tx=0x7fc14001cba0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 -- 172.21.15.170:0/2358055512 shutdown_connections 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 --2- 172.21.15.170:0/2358055512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500d0aa0 0x7fc1500d0eb0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 --2- 172.21.15.170:0/2358055512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500b4540 0x7fc15015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 -- 172.21.15.170:0/2358055512 >> 172.21.15.170:0/2358055512 conn(0x7fc1500cee30 msgr2=0x7fc1500cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 -- 172.21.15.170:0/2358055512 shutdown_connections 2024-09-21T02:21:36.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.712+0000 7fc156867640 1 -- 172.21.15.170:0/2358055512 wait complete. 2024-09-21T02:21:36.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 Processor -- start 2024-09-21T02:21:36.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 -- start start 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 0x7fc1501fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500d0aa0 0x7fc1501529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc15015ca60 0x7fc150157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc15016a560 con 0x7fc1500b7ab0 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc15016a3e0 con 0x7fc15015ca60 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc15016a6e0 con 0x7fc1500d0aa0 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 0x7fc1501fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 0x7fc1501fa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36096/0 (socket says 172.21.15.170:36096) 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 -- 172.21.15.170:0/783621318 learned_addr learned my addr 172.21.15.170:0/783621318 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14f7fe640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500d0aa0 0x7fc1501529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc154e04640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc15015ca60 0x7fc150157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 -- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500d0aa0 msgr2=0x7fc1501529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500d0aa0 0x7fc1501529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 -- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc15015ca60 msgr2=0x7fc150157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc15015ca60 0x7fc150157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 -- 172.21.15.170:0/783621318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15015efd0 con 0x7fc1500b7ab0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14f7fe640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500d0aa0 0x7fc1501529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14ffff640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 0x7fc1501fa7f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c430 tx=0x7fc14800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148011790 con 0x7fc1500b7ab0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc148017070 con 0x7fc1500b7ab0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15015f590 con 0x7fc1500b7ab0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148011ce0 con 0x7fc1500b7ab0 2024-09-21T02:21:36.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.716+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15015f2b0 con 0x7fc1500b7ab0 2024-09-21T02:21:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.720+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15015c850 con 0x7fc1500b7ab0 2024-09-21T02:21:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.720+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fc14801c460 con 0x7fc1500b7ab0 2024-09-21T02:21:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.720+0000 7fc14cff9640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc130073a70 0x7fc130075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.720+0000 7fc14f7fe640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc130073a70 0x7fc130075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.720+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fc1480927f0 con 0x7fc1500b7ab0 2024-09-21T02:21:36.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.720+0000 7fc14f7fe640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc130073a70 0x7fc130075f80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc138005fd0 tx=0x7fc138005de0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:36.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.724+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc14805f410 con 0x7fc1500b7ab0 2024-09-21T02:21:36.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.896+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc150152f10 con 0x7fc1500b7ab0 2024-09-21T02:21:36.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.896+0000 7fc14cff9640 1 -- 172.21.15.170:0/783621318 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16831 (secure 0 0 0) 0x7fc14805ed10 con 0x7fc1500b7ab0 2024-09-21T02:21:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:36.900 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:13.627328+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:09.751144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:10.994981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc130073a70 msgr2=0x7fc130075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc130073a70 0x7fc130075f80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc138005fd0 tx=0x7fc138005de0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 msgr2=0x7fc1501fa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 0x7fc1501fa7f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c430 tx=0x7fc14800c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 shutdown_connections 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc130073a70 0x7fc130075f80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1500d0aa0 0x7fc1501529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc15015ca60 0x7fc150157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 --2- 172.21.15.170:0/783621318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1500b7ab0 0x7fc1501fa7f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 >> 172.21.15.170:0/783621318 conn(0x7fc1500cee30 msgr2=0x7fc150072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:36.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 shutdown_connections 2024-09-21T02:21:36.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:36.900+0000 7fc156867640 1 -- 172.21.15.170:0/783621318 wait complete. 2024-09-21T02:21:36.921 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:21:36.921 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:37.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 -- 172.21.15.170:0/2321618861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 msgr2=0x7f77ac003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/2321618861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77ac003e60 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f77b4005fd0 tx=0x7f77b401cd60 comp rx=0 tx=0).stop 2024-09-21T02:21:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 -- 172.21.15.170:0/2321618861 shutdown_connections 2024-09-21T02:21:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/2321618861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77ac003e60 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 -- 172.21.15.170:0/2321618861 >> 172.21.15.170:0/2321618861 conn(0x7f77c40b1c40 msgr2=0x7f77c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 -- 172.21.15.170:0/2321618861 shutdown_connections 2024-09-21T02:21:37.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.056+0000 7f77ca8ce640 1 -- 172.21.15.170:0/2321618861 wait complete. 2024-09-21T02:21:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 Processor -- start 2024-09-21T02:21:37.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 -- start start 2024-09-21T02:21:37.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77c409d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c4074a50 0x7f77c4073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c40c3060 0x7f77c4072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f77c40cce40 con 0x7f77ac003a10 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77c40cccc0 con 0x7f77c40c3060 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f77c40ccfc0 con 0x7f77c4074a50 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77c409d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77c409d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36126/0 (socket says 172.21.15.170:36126) 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 -- 172.21.15.170:0/313299032 learned_addr learned my addr 172.21.15.170:0/313299032 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 -- 172.21.15.170:0/313299032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c4074a50 msgr2=0x7f77c4073ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c4074a50 0x7f77c4073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 -- 172.21.15.170:0/313299032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c40c3060 msgr2=0x7f77c4072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c40c3060 0x7f77c4072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 -- 172.21.15.170:0/313299032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c409ddc0 con 0x7f77ac003a10 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c3fff640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77c409d880 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f77b401cb90 tx=0x7f77b400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40be070 con 0x7f77ac003a10 2024-09-21T02:21:37.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77b4005350 con 0x7f77ac003a10 2024-09-21T02:21:37.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c41f9270 con 0x7f77ac003a10 2024-09-21T02:21:37.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40ab050 con 0x7f77ac003a10 2024-09-21T02:21:37.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.060+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c409cda0 con 0x7f77ac003a10 2024-09-21T02:21:37.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.064+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c40ce7b0 con 0x7f77ac003a10 2024-09-21T02:21:37.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.064+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f77b40c1070 con 0x7f77ac003a10 2024-09-21T02:21:37.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.064+0000 7f77c0ff9640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77b0073a70 0x7f77b0075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.064+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f77b412cd50 con 0x7f77ac003a10 2024-09-21T02:21:37.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.064+0000 7f77c37fe640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77b0073a70 0x7f77b0075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.064+0000 7f77c37fe640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77b0073a70 0x7f77b0075f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77b8007fd0 tx=0x7f77b80079c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:37.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.068+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f77b4016a70 con 0x7f77ac003a10 2024-09-21T02:21:37.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.268+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c406de10 con 0x7f77ac003a10 2024-09-21T02:21:37.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77c0ff9640 1 -- 172.21.15.170:0/313299032 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5238 (secure 0 0 0) 0x7f77b40f9940 con 0x7f77ac003a10 2024-09-21T02:21:37.272 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:37.272 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"btime":"2024-09-21T02:21:24:984660+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:12.056653+0000","modified":"2024-09-21T02:21:15.085591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2032631050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2032631050},{"type":"v1","addr":"172.21.15.170:6835","nonce":2032631050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":11}]} 2024-09-21T02:21:37.273 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2024-09-21T02:21:37.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77b0073a70 msgr2=0x7f77b0075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77b0073a70 0x7f77b0075f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77b8007fd0 tx=0x7f77b80079c0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 msgr2=0x7f77c409d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77c409d880 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f77b401cb90 tx=0x7f77b400b6f0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 shutdown_connections 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f77b0073a70 0x7f77b0075f80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77c4074a50 0x7f77c4073ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c40c3060 0x7f77c4072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 --2- 172.21.15.170:0/313299032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77ac003a10 0x7f77c409d880 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 >> 172.21.15.170:0/313299032 conn(0x7f77c40b1c40 msgr2=0x7f77c40c7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 shutdown_connections 2024-09-21T02:21:37.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.272+0000 7f77ca8ce640 1 -- 172.21.15.170:0/313299032 wait complete. 2024-09-21T02:21:37.293 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:37.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 -- 172.21.15.170:0/909260183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 msgr2=0x7f17080b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 --2- 172.21.15.170:0/909260183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17080b0a80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f1708073450 tx=0x7f170401c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 -- 172.21.15.170:0/909260183 shutdown_connections 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 --2- 172.21.15.170:0/909260183 >> v2:172.21.15.191:3300/0 conn(0x7f17080cc470 0x7f17080cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 --2- 172.21.15.170:0/909260183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17080b0a80 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 -- 172.21.15.170:0/909260183 >> 172.21.15.170:0/909260183 conn(0x7f170809a9c0 msgr2=0x7f17080b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 -- 172.21.15.170:0/909260183 shutdown_connections 2024-09-21T02:21:37.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.444+0000 7f170f350640 1 -- 172.21.15.170:0/909260183 wait complete. 2024-09-21T02:21:37.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 Processor -- start 2024-09-21T02:21:37.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- start start 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17081513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17080cc470 0x7f1708155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17080ccdc0 0x7f17080afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1708168d20 con 0x7f17080b65a0 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1708168ba0 con 0x7f17080cc470 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f1708168ea0 con 0x7f17080ccdc0 2024-09-21T02:21:37.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17081513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17081513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36144/0 (socket says 172.21.15.170:36144) 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 -- 172.21.15.170:0/2101906865 learned_addr learned my addr 172.21.15.170:0/2101906865 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170c8eb640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17080cc470 0x7f1708155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 -- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17080ccdc0 msgr2=0x7f17080afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d8ed640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17080ccdc0 0x7f17080afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17080ccdc0 0x7f17080afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 -- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17080cc470 msgr2=0x7f1708155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17080cc470 0x7f1708155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 -- 172.21.15.170:0/2101906865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1708154740 con 0x7f17080b65a0 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170c8eb640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17080cc470 0x7f1708155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170d0ec640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17081513b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1704009850 tx=0x7f17040a6470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:37.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17040ac070 con 0x7f17080b65a0 2024-09-21T02:21:37.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1704004030 con 0x7f17080b65a0 2024-09-21T02:21:37.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704004480 con 0x7f17080b65a0 2024-09-21T02:21:37.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1708154a20 con 0x7f17080b65a0 2024-09-21T02:21:37.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1708154fe0 con 0x7f17080b65a0 2024-09-21T02:21:37.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.448+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1708154260 con 0x7f17080b65a0 2024-09-21T02:21:37.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.452+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f17040b9030 con 0x7f17080b65a0 2024-09-21T02:21:37.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.452+0000 7f16fdffb640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f16ec073aa0 0x7f16ec075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.452+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7f170412efb0 con 0x7f17080b65a0 2024-09-21T02:21:37.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.452+0000 7f170c8eb640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f16ec073aa0 0x7f16ec075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.452+0000 7f170c8eb640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f16ec073aa0 0x7f16ec075fb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005fd0 tx=0x7f16f8005de0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:37.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.456+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f1708154260 con 0x7f17080b65a0 2024-09-21T02:21:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.648+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17080d1f60 con 0x7f17080b65a0 2024-09-21T02:21:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.648+0000 7f16fdffb640 1 -- 172.21.15.170:0/2101906865 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5238 (secure 0 0 0) 0x7f17040164b0 con 0x7f17080b65a0 2024-09-21T02:21:37.651 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:37.651 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"btime":"2024-09-21T02:21:24:984660+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:12.056653+0000","modified":"2024-09-21T02:21:15.085591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2032631050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2032631050},{"type":"v1","addr":"172.21.15.170:6835","nonce":2032631050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":11}]} 2024-09-21T02:21:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2024-09-21T02:21:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f16ec073aa0 msgr2=0x7f16ec075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f16ec073aa0 0x7f16ec075fb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005fd0 tx=0x7f16f8005de0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 msgr2=0x7f17081513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17081513b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1704009850 tx=0x7f17040a6470 comp rx=0 tx=0).stop 2024-09-21T02:21:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 shutdown_connections 2024-09-21T02:21:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f16ec073aa0 0x7f16ec075fb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17080ccdc0 0x7f17080afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17080cc470 0x7f1708155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 --2- 172.21.15.170:0/2101906865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17080b65a0 0x7f17081513b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 >> 172.21.15.170:0/2101906865 conn(0x7f170809a9c0 msgr2=0x7f1708070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 shutdown_connections 2024-09-21T02:21:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.652+0000 7f170f350640 1 -- 172.21.15.170:0/2101906865 wait complete. 2024-09-21T02:21:37.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 -- 172.21.15.170:0/478572267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b5410 msgr2=0x7fd6d806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/478572267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b5410 0x7fd6d806fa30 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009840 tx=0x7fd6d401c820 comp rx=0 tx=0).stop 2024-09-21T02:21:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 -- 172.21.15.170:0/478572267 shutdown_connections 2024-09-21T02:21:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/478572267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b5410 0x7fd6d806fa30 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 -- 172.21.15.170:0/478572267 >> 172.21.15.170:0/478572267 conn(0x7fd6d80b1c40 msgr2=0x7fd6d80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:37.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 -- 172.21.15.170:0/478572267 shutdown_connections 2024-09-21T02:21:37.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.824+0000 7fd6dfab2640 1 -- 172.21.15.170:0/478572267 wait complete. 2024-09-21T02:21:37.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 Processor -- start 2024-09-21T02:21:37.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- start start 2024-09-21T02:21:37.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 0x7fd6d814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6d80b5410 0x7fd6d80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6d81589b0 0x7fd6d80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d816b570 con 0x7fd6d80b2f40 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8070180 con 0x7fd6d81589b0 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d816b6f0 con 0x7fd6d80b5410 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 0x7fd6d814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 0x7fd6d814a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36166/0 (socket says 172.21.15.170:36166) 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 -- 172.21.15.170:0/2528175121 learned_addr learned my addr 172.21.15.170:0/2528175121 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6de04f640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6d81589b0 0x7fd6d80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd04d640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6d80b5410 0x7fd6d80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 -- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6d80b5410 msgr2=0x7fd6d80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6d80b5410 0x7fd6d80b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 -- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6d81589b0 msgr2=0x7fd6d80b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6d81589b0 0x7fd6d80b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:37.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 -- 172.21.15.170:0/2528175121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d80c9f80 con 0x7fd6d80b2f40 2024-09-21T02:21:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dd84e640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 0x7fd6d814a5b0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009790 tx=0x7fd6d40a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4004040 con 0x7fd6d80b2f40 2024-09-21T02:21:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6d40be070 con 0x7fd6d80b2f40 2024-09-21T02:21:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4005050 con 0x7fd6d80b2f40 2024-09-21T02:21:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d81591f0 con 0x7fd6d80b2f40 2024-09-21T02:21:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d8074c60 con 0x7fd6d80b2f40 2024-09-21T02:21:37.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.828+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d8158fe0 con 0x7fd6d80b2f40 2024-09-21T02:21:37.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.832+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fd6d40051f0 con 0x7fd6d80b2f40 2024-09-21T02:21:37.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.832+0000 7fd6ce7fc640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6bc073aa0 0x7fd6bc075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:37.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.832+0000 7fd6dd04d640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6bc073aa0 0x7fd6bc075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:37.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.832+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7fd6d412d3a0 con 0x7fd6d80b2f40 2024-09-21T02:21:37.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.832+0000 7fd6dd04d640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6bc073aa0 0x7fd6bc075fb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8158410 tx=0x7fd6c8009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:37.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:37.836+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6d4016410 con 0x7fd6d80b2f40 2024-09-21T02:21:38.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.012+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6d806fa30 con 0x7fd6d80b2f40 2024-09-21T02:21:38.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.012+0000 7fd6ce7fc640 1 -- 172.21.15.170:0/2528175121 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16831 (secure 0 0 0) 0x7fd6d40f9f00 con 0x7fd6d80b2f40 2024-09-21T02:21:38.015 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:38.015 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:13.627328+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:09.751144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:10.994981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6bc073aa0 msgr2=0x7fd6bc075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6bc073aa0 0x7fd6bc075fb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8158410 tx=0x7fd6c8009040 comp rx=0 tx=0).stop 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 msgr2=0x7fd6d814a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 0x7fd6d814a5b0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009790 tx=0x7fd6d40a65a0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 shutdown_connections 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6bc073aa0 0x7fd6bc075fb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6d80b5410 0x7fd6d80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6d81589b0 0x7fd6d80b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 --2- 172.21.15.170:0/2528175121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6d80b2f40 0x7fd6d814a5b0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 >> 172.21.15.170:0/2528175121 conn(0x7fd6d80b1c40 msgr2=0x7fd6d809dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 shutdown_connections 2024-09-21T02:21:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.016+0000 7fd6dfab2640 1 -- 172.21.15.170:0/2528175121 wait complete. 2024-09-21T02:21:38.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:21:38.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 -- 172.21.15.170:0/675181212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b5410 msgr2=0x7ff8dc06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/675181212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc06fa30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc070e90 tx=0x7ff8d401cd80 comp rx=0 tx=0).stop 2024-09-21T02:21:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 -- 172.21.15.170:0/675181212 shutdown_connections 2024-09-21T02:21:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/675181212 >> v2:172.21.15.191:3300/0 conn(0x7ff8dc0b2f40 0x7ff8dc0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/675181212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc06fa30 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 -- 172.21.15.170:0/675181212 >> 172.21.15.170:0/675181212 conn(0x7ff8dc0b1c40 msgr2=0x7ff8dc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 -- 172.21.15.170:0/675181212 shutdown_connections 2024-09-21T02:21:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.176+0000 7ff8e1c87640 1 -- 172.21.15.170:0/675181212 wait complete. 2024-09-21T02:21:38.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 Processor -- start 2024-09-21T02:21:38.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 -- start start 2024-09-21T02:21:38.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 0x7ff8dc09d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc0ccb00 0x7ff8dc0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc169fd0 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc169e50 con 0x7ff8dc0b5410 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8e1c87640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc16a150 con 0x7ff8dc0ccb00 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 0x7ff8dc09d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 0x7ff8dc09d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36190/0 (socket says 172.21.15.170:36190) 2024-09-21T02:21:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 -- 172.21.15.170:0/881378500 learned_addr learned my addr 172.21.15.170:0/881378500 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 -- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc0ccb00 msgr2=0x7ff8dc0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8dbfff640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc0ccb00 0x7ff8dc0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8daffd640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc0ccb00 0x7ff8dc0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 -- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8dc0b5410 msgr2=0x7ff8dc0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 -- 172.21.15.170:0/881378500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc0b0fe0 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8daffd640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8db7fe640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 0x7ff8dc09d1d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7ff8d401b430 tx=0x7ff8d40a5c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.180+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40bc070 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc0ad590 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8d40044b0 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40b2040 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc159b50 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc0b3350 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7ff8d40bf070 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8c3fff640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8c8073aa0 0x7ff8c8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8daffd640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8c8073aa0 0x7ff8c8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5737+0+0 (secure 0 0 0) 0x7ff8d412ce30 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.184+0000 7ff8daffd640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8c8073aa0 0x7ff8c8075fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc1595e0 tx=0x7ff8d0005cc0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:38.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.188+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff8d4016970 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.380+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff8dc0b4550 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.384 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:21:38.384+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:21:38.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.384+0000 7ff8c3fff640 1 -- 172.21.15.170:0/881378500 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 161+0+0 (secure 0 0 0) 0x7ff8d40f9990 con 0x7ff8dc0b2f40 2024-09-21T02:21:38.387 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8c8073aa0 msgr2=0x7ff8c8075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8c8073aa0 0x7ff8c8075fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc1595e0 tx=0x7ff8d0005cc0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 msgr2=0x7ff8dc09d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 0x7ff8dc09d1d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7ff8d401b430 tx=0x7ff8d40a5c80 comp rx=0 tx=0).stop 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 shutdown_connections 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff8c8073aa0 0x7ff8c8075fb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8dc0ccb00 0x7ff8dc0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8dc0b5410 0x7ff8dc0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 --2- 172.21.15.170:0/881378500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8dc0b2f40 0x7ff8dc09d1d0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 >> 172.21.15.170:0/881378500 conn(0x7ff8dc0b1c40 msgr2=0x7ff8dc0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:38.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 shutdown_connections 2024-09-21T02:21:38.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.388+0000 7ff8e1c87640 1 -- 172.21.15.170:0/881378500 wait complete. 2024-09-21T02:21:38.412 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:21:38.463 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:21:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 -- 172.21.15.170:0/3597712195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 msgr2=0x7fbc04070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 --2- 172.21.15.170:0/3597712195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 0x7fbc04070490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009c80 tx=0x7fbbfc01cda0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 -- 172.21.15.170:0/3597712195 shutdown_connections 2024-09-21T02:21:38.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 --2- 172.21.15.170:0/3597712195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 0x7fbc04070490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 --2- 172.21.15.170:0/3597712195 >> v2:172.21.15.191:3300/0 conn(0x7fbc040cec50 0x7fbc0406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 -- 172.21.15.170:0/3597712195 >> 172.21.15.170:0/3597712195 conn(0x7fbc040d0170 msgr2=0x7fbc040d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:38.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 -- 172.21.15.170:0/3597712195 shutdown_connections 2024-09-21T02:21:38.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 -- 172.21.15.170:0/3597712195 wait complete. 2024-09-21T02:21:38.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 Processor -- start 2024-09-21T02:21:38.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.564+0000 7fbc0af73640 1 -- start start 2024-09-21T02:21:38.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc040b2fc0 0x7fbc041fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 0x7fbc040b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 0x7fbc040b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04169fe0 con 0x7fbc040cec50 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc040b35c0 con 0x7fbc040b2fc0 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0416a160 con 0x7fbc040cf060 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 0x7fbc040b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 0x7fbc040b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36216/0 (socket says 172.21.15.170:36216) 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 -- 172.21.15.170:0/4161003028 learned_addr learned my addr 172.21.15.170:0/4161003028 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc08d0f640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc040b2fc0 0x7fbc041fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc09510640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 0x7fbc040b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 -- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 msgr2=0x7fbc040b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 0x7fbc040b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 -- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc040b2fc0 msgr2=0x7fbc041fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc040b2fc0 0x7fbc041fcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 -- 172.21.15.170:0/4161003028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc04158bf0 con 0x7fbc040cec50 2024-09-21T02:21:38.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc08d0f640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc040b2fc0 0x7fbc041fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:38.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc03fff640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 0x7fbc040b23b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800e7a0 tx=0x7fbbf800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:38.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800ccc0 con 0x7fbc040cec50 2024-09-21T02:21:38.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf8010430 con 0x7fbc040cec50 2024-09-21T02:21:38.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc040b28f0 con 0x7fbc040cec50 2024-09-21T02:21:38.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8010860 con 0x7fbc040cec50 2024-09-21T02:21:38.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc041660a0 con 0x7fbc040cec50 2024-09-21T02:21:38.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc040ae3e0 con 0x7fbc040cec50 2024-09-21T02:21:38.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.568+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fbbf8010a00 con 0x7fbc040cec50 2024-09-21T02:21:38.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.572+0000 7fbc017fa640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbbf0073aa0 0x7fbbf0075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:38.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.572+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5823+0+0 (secure 0 0 0) 0x7fbbf8093510 con 0x7fbc040cec50 2024-09-21T02:21:38.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.572+0000 7fbc08d0f640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbbf0073aa0 0x7fbbf0075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:38.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.572+0000 7fbc08d0f640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbbf0073aa0 0x7fbbf0075fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4001630 tx=0x7fbbf4009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:38.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.576+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbc04158bf0 con 0x7fbc040cec50 2024-09-21T02:21:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:38.780+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbc040ade20 con 0x7fbc040cec50 2024-09-21T02:21:39.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.388+0000 7fbc017fa640 1 -- 172.21.15.170:0/4161003028 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) ==== 104+0+0 (secure 0 0 0) 0x7fbbf805ffe0 con 0x7fbc040cec50 2024-09-21T02:21:39.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbbf0073aa0 msgr2=0x7fbbf0075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:39.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbbf0073aa0 0x7fbbf0075fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4001630 tx=0x7fbbf4009040 comp rx=0 tx=0).stop 2024-09-21T02:21:39.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 msgr2=0x7fbc040b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:39.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 0x7fbc040b23b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800e7a0 tx=0x7fbbf800ec60 comp rx=0 tx=0).stop 2024-09-21T02:21:39.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 shutdown_connections 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbbf0073aa0 0x7fbbf0075fb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc040cf060 0x7fbc040b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc040cec50 0x7fbc040b23b0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 --2- 172.21.15.170:0/4161003028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc040b2fc0 0x7fbc041fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 >> 172.21.15.170:0/4161003028 conn(0x7fbc040d0170 msgr2=0x7fbc040acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 shutdown_connections 2024-09-21T02:21:39.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.392+0000 7fbc0af73640 1 -- 172.21.15.170:0/4161003028 wait complete. 2024-09-21T02:21:39.419 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- 172.21.15.170:0/1752530473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 msgr2=0x7f5fdc0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 --2- 172.21.15.170:0/1752530473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 0x7f5fdc0b3f40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009c80 tx=0x7f5fd401cda0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- 172.21.15.170:0/1752530473 shutdown_connections 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 --2- 172.21.15.170:0/1752530473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 0x7f5fdc0b3f40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 --2- 172.21.15.170:0/1752530473 >> v2:172.21.15.191:3300/0 conn(0x7f5fdc0cdb20 0x7f5fdc15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- 172.21.15.170:0/1752530473 >> 172.21.15.170:0/1752530473 conn(0x7f5fdc0b2720 msgr2=0x7f5fdc0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- 172.21.15.170:0/1752530473 shutdown_connections 2024-09-21T02:21:39.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- 172.21.15.170:0/1752530473 wait complete. 2024-09-21T02:21:39.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 Processor -- start 2024-09-21T02:21:39.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- start start 2024-09-21T02:21:39.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fdc0690d0 0x7f5fdc157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:39.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 0x7f5fdc1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:39.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 0x7f5fdc1fa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:39.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc169c90 con 0x7f5fdc1fa460 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc169b10 con 0x7f5fdc0690d0 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.580+0000 7f5fe452b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5fdc169e10 con 0x7f5fdc0b3b30 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 0x7f5fdc1fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 0x7f5fdc1fa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36224/0 (socket says 172.21.15.170:36224) 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 -- 172.21.15.170:0/2998145816 learned_addr learned my addr 172.21.15.170:0/2998145816 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe1ac6640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 0x7f5fdc1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe22c7640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fdc0690d0 0x7f5fdc157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 -- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 msgr2=0x7f5fdc1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 0x7f5fdc1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 -- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fdc0690d0 msgr2=0x7f5fdc157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fdc0690d0 0x7f5fdc157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 -- 172.21.15.170:0/2998145816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc14b1f0 con 0x7f5fdc1fa460 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe2ac8640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 0x7f5fdc1fa870 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f5fd401b450 tx=0x7f5fd4005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:39.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40bc070 con 0x7f5fdc1fa460 2024-09-21T02:21:39.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fd40040c0 con 0x7f5fdc1fa460 2024-09-21T02:21:39.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc0c7190 con 0x7f5fdc1fa460 2024-09-21T02:21:39.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40b2040 con 0x7f5fdc1fa460 2024-09-21T02:21:39.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1f7370 con 0x7f5fdc1fa460 2024-09-21T02:21:39.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc0b3f40 con 0x7f5fdc1fa460 2024-09-21T02:21:39.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f5fd40bf070 con 0x7f5fdc1fa460 2024-09-21T02:21:39.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fd2ffd640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fc0073aa0 0x7f5fc0075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:39.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5823+0+0 (secure 0 0 0) 0x7f5fd412c8b0 con 0x7f5fdc1fa460 2024-09-21T02:21:39.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe22c7640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fc0073aa0 0x7f5fc0075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:39.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.584+0000 7f5fe22c7640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fc0073aa0 0x7f5fc0075fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc001590 tx=0x7f5fcc009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:39.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.588+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5fd4016990 con 0x7f5fdc1fa460 2024-09-21T02:21:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:39.760+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5fdc0d1340 con 0x7f5fdc1fa460 2024-09-21T02:21:40.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.400+0000 7f5fd2ffd640 1 -- 172.21.15.170:0/2998145816 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) ==== 181+0+0 (secure 0 0 0) 0x7f5fd40f93f0 con 0x7f5fdc1fa460 2024-09-21T02:21:40.403 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:21:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.408+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fc0073aa0 msgr2=0x7f5fc0075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.408+0000 7f5fe452b640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fc0073aa0 0x7f5fc0075fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc001590 tx=0x7f5fcc009040 comp rx=0 tx=0).stop 2024-09-21T02:21:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.408+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 msgr2=0x7f5fdc1fa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.408+0000 7f5fe452b640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 0x7f5fdc1fa870 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f5fd401b450 tx=0x7f5fd4005ae0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 shutdown_connections 2024-09-21T02:21:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5fc0073aa0 0x7f5fc0075fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fdc0b3b30 0x7f5fdc1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fdc1fa460 0x7f5fdc1fa870 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 --2- 172.21.15.170:0/2998145816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fdc0690d0 0x7f5fdc157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 >> 172.21.15.170:0/2998145816 conn(0x7f5fdc0b2720 msgr2=0x7f5fdc0735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 shutdown_connections 2024-09-21T02:21:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.412+0000 7f5fe452b640 1 -- 172.21.15.170:0/2998145816 wait complete. 2024-09-21T02:21:40.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 -- 172.21.15.170:0/3672671351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900cd1f0 msgr2=0x7f799006ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 --2- 172.21.15.170:0/3672671351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900cd1f0 0x7f799006ef40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f7988009bc0 tx=0x7f798801cb10 comp rx=0 tx=0).stop 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 -- 172.21.15.170:0/3672671351 shutdown_connections 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 --2- 172.21.15.170:0/3672671351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900cd1f0 0x7f799006ef40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 --2- 172.21.15.170:0/3672671351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79900b52d0 0x7f79900af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 -- 172.21.15.170:0/3672671351 >> 172.21.15.170:0/3672671351 conn(0x7f79900b1c40 msgr2=0x7f79900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 -- 172.21.15.170:0/3672671351 shutdown_connections 2024-09-21T02:21:40.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 -- 172.21.15.170:0/3672671351 wait complete. 2024-09-21T02:21:40.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 Processor -- start 2024-09-21T02:21:40.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.600+0000 7f7994998640 1 -- start start 2024-09-21T02:21:40.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 0x7f799014a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79900b52d0 0x7f799015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f799015bcc0 0x7f7990155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7990169f20 con 0x7f79900b4d80 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7990169da0 con 0x7f799015bcc0 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f799016a0a0 con 0x7f79900b52d0 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 0x7f799014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 0x7f799014a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36236/0 (socket says 172.21.15.170:36236) 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 -- 172.21.15.170:0/3100924290 learned_addr learned my addr 172.21.15.170:0/3100924290 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 -- 172.21.15.170:0/3100924290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79900b52d0 msgr2=0x7f799015b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79900b52d0 0x7f799015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 -- 172.21.15.170:0/3100924290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f799015bcc0 msgr2=0x7f7990155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f799015bcc0 0x7f7990155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:40.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 -- 172.21.15.170:0/3100924290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7990153cd0 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f798e59c640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 0x7f799014a530 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f798000c9a0 tx=0x7f798000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980007bb0 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7980007e40 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7990153fb0 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980005290 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7990154570 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79900af440 con 0x7f79900b4d80 2024-09-21T02:21:40.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.604+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f7980005430 con 0x7f79900b4d80 2024-09-21T02:21:40.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.608+0000 7f797effd640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7970073a70 0x7f7970075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:40.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.608+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5385+0+0 (secure 0 0 0) 0x7f7980093340 con 0x7f79900b4d80 2024-09-21T02:21:40.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.608+0000 7f798dd9b640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7970073a70 0x7f7970075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:40.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.608+0000 7f798dd9b640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7970073a70 0x7f7970075f80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f79840046c0 tx=0x7f7984007a80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:40.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.608+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7980014070 con 0x7f79900b4d80 2024-09-21T02:21:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:40.780+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f79900707b0 con 0x7f79900b4d80 2024-09-21T02:21:41.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.412+0000 7f797effd640 1 -- 172.21.15.170:0/3100924290 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) ==== 169+0+0 (secure 0 0 0) 0x7f798005fe80 con 0x7f79900b4d80 2024-09-21T02:21:41.415 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7970073a70 msgr2=0x7f7970075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7970073a70 0x7f7970075f80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f79840046c0 tx=0x7f7984007a80 comp rx=0 tx=0).stop 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 msgr2=0x7f799014a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 0x7f799014a530 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f798000c9a0 tx=0x7f798000ce60 comp rx=0 tx=0).stop 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 shutdown_connections 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7970073a70 0x7f7970075f80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79900b52d0 0x7f799015b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f799015bcc0 0x7f7990155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 --2- 172.21.15.170:0/3100924290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79900b4d80 0x7f799014a530 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 >> 172.21.15.170:0/3100924290 conn(0x7f79900b1c40 msgr2=0x7f79900b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:41.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.416+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 shutdown_connections 2024-09-21T02:21:41.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.424+0000 7f7994998640 1 -- 172.21.15.170:0/3100924290 wait complete. 2024-09-21T02:21:41.452 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 -- 172.21.15.170:0/3108567266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 msgr2=0x7f716c0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 --2- 172.21.15.170:0/3108567266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c0ce5d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f716c0d1160 tx=0x7f716401cb80 comp rx=0 tx=0).stop 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 -- 172.21.15.170:0/3108567266 shutdown_connections 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 --2- 172.21.15.170:0/3108567266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c0ce5d0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 --2- 172.21.15.170:0/3108567266 >> v2:172.21.15.191:3300/0 conn(0x7f716c0cf7e0 0x7f716c070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 -- 172.21.15.170:0/3108567266 >> 172.21.15.170:0/3108567266 conn(0x7f716c0b1c40 msgr2=0x7f716c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 -- 172.21.15.170:0/3108567266 shutdown_connections 2024-09-21T02:21:41.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.596+0000 7f7173d79640 1 -- 172.21.15.170:0/3108567266 wait complete. 2024-09-21T02:21:41.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 Processor -- start 2024-09-21T02:21:41.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- start start 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f716c0cb460 0x7f716c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716c0cf7e0 0x7f716c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1670c0 con 0x7f716c0ce1c0 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f716c166f40 con 0x7f716c0cf7e0 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f716c167240 con 0x7f716c0cb460 2024-09-21T02:21:41.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c1532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36256/0 (socket says 172.21.15.170:36256) 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 -- 172.21.15.170:0/2174088937 learned_addr learned my addr 172.21.15.170:0/2174088937 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7172316640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716c0cf7e0 0x7f716c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171b15640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f716c0cb460 0x7f716c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 -- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f716c0cb460 msgr2=0x7f716c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f716c0cb460 0x7f716c15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 -- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716c0cf7e0 msgr2=0x7f716c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716c0cf7e0 0x7f716c15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 -- 172.21.15.170:0/2174088937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c15db90 con 0x7f716c0ce1c0 2024-09-21T02:21:41.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7172316640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716c0cf7e0 0x7f716c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171b15640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f716c0cb460 0x7f716c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7171314640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c1532f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f716800c400 tx=0x7f716800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168017070 con 0x7f716c0ce1c0 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7168003600 con 0x7f716c0ce1c0 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c15d2f0 con 0x7f716c0ce1c0 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168003a50 con 0x7f716c0ce1c0 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c15d8b0 con 0x7f716c0ce1c0 2024-09-21T02:21:41.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.600+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716c0ce5d0 con 0x7f716c0ce1c0 2024-09-21T02:21:41.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.604+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f716801a070 con 0x7f716c0ce1c0 2024-09-21T02:21:41.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.604+0000 7f71627fc640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7150073a70 0x7f7150075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.604+0000 7f7171b15640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7150073a70 0x7f7150075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.604+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7f7168094590 con 0x7f716c0ce1c0 2024-09-21T02:21:41.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.604+0000 7f7171b15640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7150073a70 0x7f7150075f80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f715c00a870 tx=0x7f715c008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:41.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.608+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f716805e860 con 0x7f716c0ce1c0 2024-09-21T02:21:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.784+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f716c15d080 con 0x7f716c0ce1c0 2024-09-21T02:21:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f71627fc640 1 -- 172.21.15.170:0/2174088937 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v49) ==== 71+0+0 (secure 0 0 0) 0x7f716805e160 con 0x7f716c0ce1c0 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7150073a70 msgr2=0x7f7150075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7150073a70 0x7f7150075f80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f715c00a870 tx=0x7f715c008040 comp rx=0 tx=0).stop 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 msgr2=0x7f716c1532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c1532f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f716800c400 tx=0x7f716800c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 shutdown_connections 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7150073a70 0x7f7150075f80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716c0cf7e0 0x7f716c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f716c0ce1c0 0x7f716c1532f0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 --2- 172.21.15.170:0/2174088937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f716c0cb460 0x7f716c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 >> 172.21.15.170:0/2174088937 conn(0x7f716c0b1c40 msgr2=0x7f716c0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 shutdown_connections 2024-09-21T02:21:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.792+0000 7f7173d79640 1 -- 172.21.15.170:0/2174088937 wait complete. 2024-09-21T02:21:41.814 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-21T02:21:41.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 -- 172.21.15.170:0/3641010499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af8164510 msgr2=0x7f2af8166930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 --2- 172.21.15.170:0/3641010499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af8164510 0x7f2af8166930 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009bc0 tx=0x7f2af001cba0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 -- 172.21.15.170:0/3641010499 shutdown_connections 2024-09-21T02:21:41.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 --2- 172.21.15.170:0/3641010499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af8164510 0x7f2af8166930 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 -- 172.21.15.170:0/3641010499 >> 172.21.15.170:0/3641010499 conn(0x7f2af80b1c40 msgr2=0x7f2af80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:41.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 -- 172.21.15.170:0/3641010499 shutdown_connections 2024-09-21T02:21:41.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 -- 172.21.15.170:0/3641010499 wait complete. 2024-09-21T02:21:41.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 Processor -- start 2024-09-21T02:21:41.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 -- start start 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.960+0000 7f2afcd27640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2af80cb460 0x7f2af815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 0x7f2af809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2af8164510 0x7f2af809af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8169e60 con 0x7f2af80cf7e0 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8169ce0 con 0x7f2af8164510 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2af8169fe0 con 0x7f2af80cb460 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 0x7f2af809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 0x7f2af809a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36278/0 (socket says 172.21.15.170:36278) 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 -- 172.21.15.170:0/1982463392 learned_addr learned my addr 172.21.15.170:0/1982463392 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af6d9d640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2af8164510 0x7f2af809af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 -- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2af80cb460 msgr2=0x7f2af815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af659c640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2af80cb460 0x7f2af815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2af80cb460 0x7f2af815dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 -- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2af8164510 msgr2=0x7f2af809af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2af8164510 0x7f2af809af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 -- 172.21.15.170:0/1982463392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af81f70d0 con 0x7f2af80cf7e0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af6d9d640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2af8164510 0x7f2af809af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af5d9b640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 0x7f2af809a9c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00c430 tx=0x7f2aec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec011790 con 0x7f2af80cf7e0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aec017070 con 0x7f2af80cf7e0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af81f73b0 con 0x7f2af80cf7e0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec011cc0 con 0x7f2af80cf7e0 2024-09-21T02:21:41.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81f7970 con 0x7f2af80cf7e0 2024-09-21T02:21:41.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af80cb870 con 0x7f2af80cf7e0 2024-09-21T02:21:41.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f2aec01c460 con 0x7f2af80cf7e0 2024-09-21T02:21:41.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2ae6ffd640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ad8073aa0 0x7f2ad8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:41.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2af659c640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ad8073aa0 0x7f2ad8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:41.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.964+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7f2aec0927a0 con 0x7f2af80cf7e0 2024-09-21T02:21:41.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.968+0000 7f2af659c640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ad8073aa0 0x7f2ad8075fb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800a750 tx=0x7f2ae8008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:41.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:41.972+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2aec05f300 con 0x7f2af80cf7e0 2024-09-21T02:21:42.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.204+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f2af806fc00 con 0x7f2af80cf7e0 2024-09-21T02:21:42.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.416+0000 7f2ae6ffd640 1 -- 172.21.15.170:0/1982463392 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v438) ==== 149+0+0 (secure 0 0 0) 0x7f2aec05ec00 con 0x7f2af80cf7e0 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ad8073aa0 msgr2=0x7f2ad8075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ad8073aa0 0x7f2ad8075fb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800a750 tx=0x7f2ae8008040 comp rx=0 tx=0).stop 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 msgr2=0x7f2af809a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 0x7f2af809a9c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00c430 tx=0x7f2aec00c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 shutdown_connections 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ad8073aa0 0x7f2ad8075fb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2af8164510 0x7f2af809af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2af80cf7e0 0x7f2af809a9c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 --2- 172.21.15.170:0/1982463392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2af80cb460 0x7f2af815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 >> 172.21.15.170:0/1982463392 conn(0x7f2af80b1c40 msgr2=0x7f2af806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:42.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 shutdown_connections 2024-09-21T02:21:42.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.420+0000 7f2afcd27640 1 -- 172.21.15.170:0/1982463392 wait complete. 2024-09-21T02:21:42.443 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-21T02:21:42.443 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-21T02:21:42.444 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:21:42.444 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 -- 172.21.15.170:0/663957272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 msgr2=0x7fa35015ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 --2- 172.21.15.170:0/663957272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa35015ff00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa344009900 tx=0x7fa34401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 -- 172.21.15.170:0/663957272 shutdown_connections 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 --2- 172.21.15.170:0/663957272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa35015ff00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 --2- 172.21.15.170:0/663957272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3500b25f0 0x7fa3500cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 -- 172.21.15.170:0/663957272 >> 172.21.15.170:0/663957272 conn(0x7fa35009a9c0 msgr2=0x7fa3500d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 -- 172.21.15.170:0/663957272 shutdown_connections 2024-09-21T02:21:42.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.600+0000 7fa35561c640 1 -- 172.21.15.170:0/663957272 wait complete. 2024-09-21T02:21:42.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 Processor -- start 2024-09-21T02:21:42.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 -- start start 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 0x7fa35015c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa350153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa350160440 0x7fa3500711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501693f0 con 0x7fa3500b25f0 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa350169270 con 0x7fa35015faf0 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa350169570 con 0x7fa350160440 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 0x7fa35015c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 0x7fa35015c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36292/0 (socket says 172.21.15.170:36292) 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 -- 172.21.15.170:0/1310880123 learned_addr learned my addr 172.21.15.170:0/1310880123 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34e7fc640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa350153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34f7fe640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa350160440 0x7fa3500711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 -- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa350160440 msgr2=0x7fa3500711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa350160440 0x7fa3500711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 -- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 msgr2=0x7fa350153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa350153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 -- 172.21.15.170:0/1310880123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501fd250 con 0x7fa3500b25f0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34e7fc640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa350153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa34effd640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 0x7fa35015c580 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa34000ea10 tx=0x7fa34000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34000cde0 con 0x7fa3500b25f0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa350153d20 con 0x7fa3500b25f0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa340004020 con 0x7fa3500b25f0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340004470 con 0x7fa3500b25f0 2024-09-21T02:21:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.604+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501fc920 con 0x7fa3500b25f0 2024-09-21T02:21:42.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.608+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3500cf950 con 0x7fa3500b25f0 2024-09-21T02:21:42.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.608+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fa340005030 con 0x7fa3500b25f0 2024-09-21T02:21:42.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.608+0000 7fa3377fe640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa33c073aa0 0x7fa33c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.608+0000 7fa34e7fc640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa33c073aa0 0x7fa33c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.608+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7fa340093570 con 0x7fa3500b25f0 2024-09-21T02:21:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.608+0000 7fa34e7fc640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa33c073aa0 0x7fa33c075fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa3440af000 tx=0x7fa34401b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:42.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.612+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa34005ffc0 con 0x7fa3500b25f0 2024-09-21T02:21:42.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.780+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3500ce9d0 con 0x7fa3500b25f0 2024-09-21T02:21:42.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa3377fe640 1 -- 172.21.15.170:0/1310880123 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+13136 (secure 0 0 0) 0x7fa34005f8c0 con 0x7fa3500b25f0 2024-09-21T02:21:42.785 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:42.785 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":145,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:41.406200+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6835/2032631050":"2024-09-22T02:21:38.387468+0000","172.21.15.170:6834/2032631050":"2024-09-22T02:21:38.387468+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:42.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa33c073aa0 msgr2=0x7fa33c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa33c073aa0 0x7fa33c075fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa3440af000 tx=0x7fa34401b120 comp rx=0 tx=0).stop 2024-09-21T02:21:42.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 msgr2=0x7fa35015c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 0x7fa35015c580 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa34000ea10 tx=0x7fa34000eed0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 shutdown_connections 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa33c073aa0 0x7fa33c075fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa350160440 0x7fa3500711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35015faf0 0x7fa350153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 --2- 172.21.15.170:0/1310880123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500b25f0 0x7fa35015c580 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 >> 172.21.15.170:0/1310880123 conn(0x7fa35009a9c0 msgr2=0x7fa35015e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 shutdown_connections 2024-09-21T02:21:42.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.784+0000 7fa35561c640 1 -- 172.21.15.170:0/1310880123 wait complete. 2024-09-21T02:21:42.806 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:21:42.806 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 -- 172.21.15.170:0/1470300057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d80cb790 msgr2=0x7fa3d80ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/1470300057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d80cb790 0x7fa3d80ce5b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009840 tx=0x7fa3cc01c790 comp rx=0 tx=0).stop 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 -- 172.21.15.170:0/1470300057 shutdown_connections 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/1470300057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d80cb790 0x7fa3d80ce5b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/1470300057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3d80cb380 0x7fa3d80ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 -- 172.21.15.170:0/1470300057 >> 172.21.15.170:0/1470300057 conn(0x7fa3d80d0170 msgr2=0x7fa3d80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 -- 172.21.15.170:0/1470300057 shutdown_connections 2024-09-21T02:21:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.948+0000 7fa3dcca3640 1 -- 172.21.15.170:0/1470300057 wait complete. 2024-09-21T02:21:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 Processor -- start 2024-09-21T02:21:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 -- start start 2024-09-21T02:21:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 0x7fa3d815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3d80cb380 0x7fa3d809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3d80cb790 0x7fa3d8166bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d80746e0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d8074560 con 0x7fa3d80cb380 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3dcca3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d81695e0 con 0x7fa3d80cb790 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 0x7fa3d815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 0x7fa3d815a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36320/0 (socket says 172.21.15.170:36320) 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 -- 172.21.15.170:0/3826064415 learned_addr learned my addr 172.21.15.170:0/3826064415 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 -- 172.21.15.170:0/3826064415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3d80cb790 msgr2=0x7fa3d8166bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3d80cb790 0x7fa3d8166bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 -- 172.21.15.170:0/3826064415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3d80cb380 msgr2=0x7fa3d809bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3d80cb380 0x7fa3d809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 -- 172.21.15.170:0/3826064415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d80b22c0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.952+0000 7fa3d659c640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 0x7fa3d815a450 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80058d0 tx=0x7fa3c800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8017070 con 0x7fa3d806f9a0 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3c8009d20 con 0x7fa3d806f9a0 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d80b25a0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80106c0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81666d0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d81664c0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fa3c801a070 con 0x7fa3d806f9a0 2024-09-21T02:21:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3c6ffd640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3b8073a70 0x7fa3b8075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7fa3c80952f0 con 0x7fa3d806f9a0 2024-09-21T02:21:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3d5d9b640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3b8073a70 0x7fa3b8075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.956+0000 7fa3d5d9b640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3b8073a70 0x7fa3b8075f80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0af000 tx=0x7fa3cc019530 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:42.960+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa3c805f700 con 0x7fa3d806f9a0 2024-09-21T02:21:43.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.120+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3d806fdb0 con 0x7fa3d806f9a0 2024-09-21T02:21:43.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.120+0000 7fa3c6ffd640 1 -- 172.21.15.170:0/3826064415 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fa3c805f000 con 0x7fa3d806f9a0 2024-09-21T02:21:43.123 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:21:43.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3b8073a70 msgr2=0x7fa3b8075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:43.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3b8073a70 0x7fa3b8075f80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0af000 tx=0x7fa3cc019530 comp rx=0 tx=0).stop 2024-09-21T02:21:43.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 msgr2=0x7fa3d815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:43.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 0x7fa3d815a450 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80058d0 tx=0x7fa3c800dde0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 shutdown_connections 2024-09-21T02:21:43.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa3b8073a70 0x7fa3b8075f80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3d80cb790 0x7fa3d8166bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3d80cb380 0x7fa3d809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 --2- 172.21.15.170:0/3826064415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d806f9a0 0x7fa3d815a450 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 >> 172.21.15.170:0/3826064415 conn(0x7fa3d80d0170 msgr2=0x7fa3d80ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:43.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 shutdown_connections 2024-09-21T02:21:43.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.124+0000 7fa3dcca3640 1 -- 172.21.15.170:0/3826064415 wait complete. 2024-09-21T02:21:43.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/577632670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cb180 msgr2=0x7f0b2806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/577632670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cb180 0x7f0b2806eeb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000acd0 tx=0x7f0b2001de60 comp rx=0 tx=0).stop 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/577632670 shutdown_connections 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/577632670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cb180 0x7f0b2806eeb0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/577632670 >> 172.21.15.170:0/577632670 conn(0x7f0b280b1c40 msgr2=0x7f0b280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/577632670 shutdown_connections 2024-09-21T02:21:43.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/577632670 wait complete. 2024-09-21T02:21:43.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 Processor -- start 2024-09-21T02:21:43.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- start start 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b280701a0 0x7f0b28164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 0x7f0b28166170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b280c9fb0 0x7f0b28162920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28169560 con 0x7f0b280cfbf0 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281693e0 con 0x7f0b280c9fb0 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2fdc1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0b281696e0 con 0x7f0b280701a0 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2d35c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 0x7f0b28166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2d35c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 0x7f0b28166170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36336/0 (socket says 172.21.15.170:36336) 2024-09-21T02:21:43.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.284+0000 7f0b2d35c640 1 -- 172.21.15.170:0/2463706624 learned_addr learned my addr 172.21.15.170:0/2463706624 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2e35e640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b280c9fb0 0x7f0b28162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2db5d640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b280701a0 0x7f0b28164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2d35c640 1 -- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b280701a0 msgr2=0x7f0b28164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2d35c640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b280701a0 0x7f0b28164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2d35c640 1 -- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b280c9fb0 msgr2=0x7f0b28162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2d35c640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b280c9fb0 0x7f0b28162920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2d35c640 1 -- 172.21.15.170:0/2463706624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b281faea0 con 0x7f0b280cfbf0 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2d35c640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 0x7f0b28166170 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800c430 tx=0x7f0b1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18011790 con 0x7f0b280cfbf0 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b18017070 con 0x7f0b280cfbf0 2024-09-21T02:21:43.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18011ce0 con 0x7f0b280cfbf0 2024-09-21T02:21:43.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b281fb180 con 0x7f0b280cfbf0 2024-09-21T02:21:43.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b28164760 con 0x7f0b280cfbf0 2024-09-21T02:21:43.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b281608c0 con 0x7f0b280cfbf0 2024-09-21T02:21:43.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f0b1801c460 con 0x7f0b280cfbf0 2024-09-21T02:21:43.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.288+0000 7f0b1e7fc640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b0c073aa0 0x7f0b0c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:43.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.292+0000 7f0b2db5d640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b0c073aa0 0x7f0b0c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:43.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.292+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7f0b18093350 con 0x7f0b280cfbf0 2024-09-21T02:21:43.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.292+0000 7f0b2db5d640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b0c073aa0 0x7f0b0c075fb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b810 tx=0x7f0b24008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:43.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.292+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0b1805fec0 con 0x7f0b280cfbf0 2024-09-21T02:21:43.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:43.516+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f0b280b46e0 con 0x7f0b280cfbf0 2024-09-21T02:21:44.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.428+0000 7f0b1e7fc640 1 -- 172.21.15.170:0/2463706624 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v440) ==== 148+0+0 (secure 0 0 0) 0x7f0b1805f7c0 con 0x7f0b280cfbf0 2024-09-21T02:21:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b0c073aa0 msgr2=0x7f0b0c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b0c073aa0 0x7f0b0c075fb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400b810 tx=0x7f0b24008040 comp rx=0 tx=0).stop 2024-09-21T02:21:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 msgr2=0x7f0b28166170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 0x7f0b28166170 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800c430 tx=0x7f0b1800c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 shutdown_connections 2024-09-21T02:21:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0b0c073aa0 0x7f0b0c075fb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b280c9fb0 0x7f0b28162920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b280cfbf0 0x7f0b28166170 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 --2- 172.21.15.170:0/2463706624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b280701a0 0x7f0b28164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 >> 172.21.15.170:0/2463706624 conn(0x7f0b280b1c40 msgr2=0x7f0b280b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 shutdown_connections 2024-09-21T02:21:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.432+0000 7f0b2fdc1640 1 -- 172.21.15.170:0/2463706624 wait complete. 2024-09-21T02:21:44.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 -- 172.21.15.170:0/1528838602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 msgr2=0x7f15140d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 --2- 172.21.15.170:0/1528838602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f15140d1260 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f15141625c0 tx=0x7f150401c770 comp rx=0 tx=0).stop 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 -- 172.21.15.170:0/1528838602 shutdown_connections 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 --2- 172.21.15.170:0/1528838602 >> v2:172.21.15.191:3300/0 conn(0x7f15140b1230 0x7f15140b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 --2- 172.21.15.170:0/1528838602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f15140d1260 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 -- 172.21.15.170:0/1528838602 >> 172.21.15.170:0/1528838602 conn(0x7f151408bf50 msgr2=0x7f151409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 -- 172.21.15.170:0/1528838602 shutdown_connections 2024-09-21T02:21:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 -- 172.21.15.170:0/1528838602 wait complete. 2024-09-21T02:21:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 Processor -- start 2024-09-21T02:21:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.612+0000 7f151bee6640 1 -- start start 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15140b1230 0x7f15140b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15140b3fa0 0x7f15140ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f151409e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f151416bfd0 con 0x7f15140d0e50 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f151416be50 con 0x7f15140b3fa0 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f151416c150 con 0x7f15140b1230 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f151409e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f151409e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36348/0 (socket says 172.21.15.170:36348) 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 -- 172.21.15.170:0/2012629249 learned_addr learned my addr 172.21.15.170:0/2012629249 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f1519481640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15140b3fa0 0x7f15140ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 -- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15140b1230 msgr2=0x7f15140b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f1519c82640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15140b1230 0x7f15140b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15140b1230 0x7f15140b65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 -- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15140b3fa0 msgr2=0x7f15140ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15140b3fa0 0x7f15140ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 -- 172.21.15.170:0/2012629249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151415a2b0 con 0x7f15140d0e50 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f1519481640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15140b3fa0 0x7f15140ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151a483640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f151409e380 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f150c00e940 tx=0x7f150c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c004d30 con 0x7f15140d0e50 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f150c00cb20 con 0x7f15140d0e50 2024-09-21T02:21:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c005020 con 0x7f15140d0e50 2024-09-21T02:21:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151415a590 con 0x7f15140d0e50 2024-09-21T02:21:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151415ab50 con 0x7f15140d0e50 2024-09-21T02:21:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.616+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15140b1640 con 0x7f15140d0e50 2024-09-21T02:21:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.620+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f150c0051c0 con 0x7f15140d0e50 2024-09-21T02:21:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.620+0000 7f150a7fc640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14f8073a70 0x7f14f8075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.620+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7f150c093820 con 0x7f15140d0e50 2024-09-21T02:21:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.620+0000 7f1519c82640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14f8073a70 0x7f14f8075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.620+0000 7f1519c82640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14f8073a70 0x7f14f8075f80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15140b3bc0 tx=0x7f1510009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.624+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f150c016030 con 0x7f15140d0e50 2024-09-21T02:21:44.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.796+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f151415a040 con 0x7f15140d0e50 2024-09-21T02:21:44.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.800+0000 7f150a7fc640 1 -- 172.21.15.170:0/2012629249 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v49) ==== 59+0+2294 (secure 0 0 0) 0x7f150c060360 con 0x7f15140d0e50 2024-09-21T02:21:44.802 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:21:44.802 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:21:44.803 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:21:44.804 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:21:44.805 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:21:44.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14f8073a70 msgr2=0x7f14f8075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14f8073a70 0x7f14f8075f80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15140b3bc0 tx=0x7f1510009040 comp rx=0 tx=0).stop 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 msgr2=0x7f151409e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f151409e380 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f150c00e940 tx=0x7f150c00ee00 comp rx=0 tx=0).stop 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 shutdown_connections 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f14f8073a70 0x7f14f8075f80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15140d0e50 0x7f151409e380 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15140b3fa0 0x7f15140ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 --2- 172.21.15.170:0/2012629249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15140b1230 0x7f15140b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 >> 172.21.15.170:0/2012629249 conn(0x7f151408bf50 msgr2=0x7f15140d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 shutdown_connections 2024-09-21T02:21:44.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.804+0000 7f151bee6640 1 -- 172.21.15.170:0/2012629249 wait complete. 2024-09-21T02:21:44.827 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:21:44.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 -- 172.21.15.170:0/4160091125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49580b2fc0 msgr2=0x7f49580ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 --2- 172.21.15.170:0/4160091125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49580b2fc0 0x7f49580ce540 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f494c009840 tx=0x7f494c01c820 comp rx=0 tx=0).stop 2024-09-21T02:21:44.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 -- 172.21.15.170:0/4160091125 shutdown_connections 2024-09-21T02:21:44.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 --2- 172.21.15.170:0/4160091125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49580b2fc0 0x7f49580ce540 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 -- 172.21.15.170:0/4160091125 >> 172.21.15.170:0/4160091125 conn(0x7f49580d0170 msgr2=0x7f49580d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:44.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 -- 172.21.15.170:0/4160091125 shutdown_connections 2024-09-21T02:21:44.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.972+0000 7f495e1be640 1 -- 172.21.15.170:0/4160091125 wait complete. 2024-09-21T02:21:44.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 Processor -- start 2024-09-21T02:21:44.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- start start 2024-09-21T02:21:44.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 0x7f49581f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49580b2fc0 0x7f49581f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49580cea80 0x7f49581fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f495816b670 con 0x7f4958070660 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f495816b4f0 con 0x7f49580cea80 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f495816b7f0 con 0x7f49580b2fc0 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 0x7f49581f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 0x7f49581f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36370/0 (socket says 172.21.15.170:36370) 2024-09-21T02:21:44.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 -- 172.21.15.170:0/1971527849 learned_addr learned my addr 172.21.15.170:0/1971527849 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:44.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 -- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49580b2fc0 msgr2=0x7f49581f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f4956ffd640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49580b2fc0 0x7f49581f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f4957fff640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49580cea80 0x7f49581fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49580b2fc0 0x7f49581f90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 -- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49580cea80 msgr2=0x7f49581fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49580cea80 0x7f49581fcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 -- 172.21.15.170:0/1971527849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495815d310 con 0x7f4958070660 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f49577fe640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 0x7f49581f8b60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f494c009790 tx=0x7f494c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c004230 con 0x7f4958070660 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f494c0be070 con 0x7f4958070660 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0052b0 con 0x7f4958070660 2024-09-21T02:21:44.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4958155dc0 con 0x7f4958070660 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4958144c40 con 0x7f4958070660 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.976+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4958155bf0 con 0x7f4958070660 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.980+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f494c005450 con 0x7f4958070660 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.980+0000 7f493ffff640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4944073a70 0x7f4944075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.980+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7f494c12e1f0 con 0x7f4958070660 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.980+0000 7f4956ffd640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4944073a70 0x7f4944075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:44.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.980+0000 7f4956ffd640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4944073a70 0x7f4944075f80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4948007fd0 tx=0x7f4948009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:44.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:44.984+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f494c0c7020 con 0x7f4958070660 2024-09-21T02:21:45.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.144+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4958144e70 con 0x7f4958070660 2024-09-21T02:21:45.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.148+0000 7f493ffff640 1 -- 172.21.15.170:0/1971527849 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f494c016410 con 0x7f4958070660 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4944073a70 msgr2=0x7f4944075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4944073a70 0x7f4944075f80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4948007fd0 tx=0x7f4948009040 comp rx=0 tx=0).stop 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 msgr2=0x7f49581f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 0x7f49581f8b60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f494c009790 tx=0x7f494c004040 comp rx=0 tx=0).stop 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 shutdown_connections 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4944073a70 0x7f4944075f80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49580b2fc0 0x7f49581f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49580cea80 0x7f49581fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 --2- 172.21.15.170:0/1971527849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4958070660 0x7f49581f8b60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 >> 172.21.15.170:0/1971527849 conn(0x7f49580d0170 msgr2=0x7f49580b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:45.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 shutdown_connections 2024-09-21T02:21:45.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.152+0000 7f495e1be640 1 -- 172.21.15.170:0/1971527849 wait complete. 2024-09-21T02:21:45.172 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:21:45.172 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:21:45.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:21:45.347 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:21:45.347 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:45.347 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:45.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:21:45.353 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:45.353 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:45.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:21:45.359 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:21:45.359 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:45.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:21:45.400 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:21:45.400 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:21:45.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:21:45.406 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:21:45.406 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 -- 172.21.15.170:0/156678222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 msgr2=0x7f24ac0cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 --2- 172.21.15.170:0/156678222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0cb200 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009bc0 tx=0x7f24a401cba0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 -- 172.21.15.170:0/156678222 shutdown_connections 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 --2- 172.21.15.170:0/156678222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0cb200 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 --2- 172.21.15.170:0/156678222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24ac0ce4c0 0x7f24ac0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 -- 172.21.15.170:0/156678222 >> 172.21.15.170:0/156678222 conn(0x7f24ac0b1c40 msgr2=0x7f24ac0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 -- 172.21.15.170:0/156678222 shutdown_connections 2024-09-21T02:21:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 -- 172.21.15.170:0/156678222 wait complete. 2024-09-21T02:21:45.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 Processor -- start 2024-09-21T02:21:45.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.604+0000 7f24b2b89640 1 -- start start 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24ac0ce4c0 0x7f24ac146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac15f990 0x7f24ac147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac166160 con 0x7f24ac0cadf0 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac165fe0 con 0x7f24ac15f990 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac1662e0 con 0x7f24ac0ce4c0 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54422/0 (socket says 172.21.15.170:54422) 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 -- 172.21.15.170:0/1790473881 learned_addr learned my addr 172.21.15.170:0/1790473881 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24abfff640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24ac0ce4c0 0x7f24ac146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b1126640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac15f990 0x7f24ac147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:45.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 -- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24ac0ce4c0 msgr2=0x7f24ac146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24ac0ce4c0 0x7f24ac146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 -- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac15f990 msgr2=0x7f24ac147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac15f990 0x7f24ac147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 -- 172.21.15.170:0/1790473881 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1fb240 con 0x7f24ac0cadf0 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b1126640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac15f990 0x7f24ac147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b0925640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0c9d60 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f249c00c430 tx=0x7f249c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c011790 con 0x7f24ac0cadf0 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f249c017070 con 0x7f24ac0cadf0 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac15db70 con 0x7f24ac0cadf0 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c011cc0 con 0x7f24ac0cadf0 2024-09-21T02:21:45.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1faf20 con 0x7f24ac0cadf0 2024-09-21T02:21:45.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac0cb200 con 0x7f24ac0cadf0 2024-09-21T02:21:45.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f249c01c460 con 0x7f24ac0cadf0 2024-09-21T02:21:45.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24a97fa640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2498073aa0 0x7f2498075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:45.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7f249c093300 con 0x7f24ac0cadf0 2024-09-21T02:21:45.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.608+0000 7f24abfff640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2498073aa0 0x7f2498075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:45.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.612+0000 7f24abfff640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2498073aa0 0x7f2498075fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24ac165fb0 tx=0x7f24a0008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:45.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.616+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f24ac0cb740 con 0x7f24ac0cadf0 2024-09-21T02:21:45.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.828+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24ac1fb410 con 0x7f24ac0cadf0 2024-09-21T02:21:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.828+0000 7f24a97fa640 1 -- 172.21.15.170:0/1790473881 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+3922 (secure 0 0 0) 0x7f249c05fdb0 con 0x7f24ac0cadf0 2024-09-21T02:21:45.829 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:45.829 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":103,"btime":"2024-09-21T02:21:39:390329+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/921294667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":921294667},{"type":"v1","addr":"172.21.15.191:6801","nonce":921294667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":6838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3092832680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3092832680},{"type":"v1","addr":"172.21.15.191:6835","nonce":3092832680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2584608514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2584608514},{"type":"v1","addr":"172.21.15.170:6833","nonce":2584608514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8835,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2571618170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2571618170},{"type":"v1","addr":"172.21.15.170:6835","nonce":2571618170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[]} 2024-09-21T02:21:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 103 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2498073aa0 msgr2=0x7f2498075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2498073aa0 0x7f2498075fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f24ac165fb0 tx=0x7f24a0008040 comp rx=0 tx=0).stop 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 msgr2=0x7f24ac0c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0c9d60 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f249c00c430 tx=0x7f249c00c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 shutdown_connections 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2498073aa0 0x7f2498075fb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24ac0ce4c0 0x7f24ac146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac15f990 0x7f24ac147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 --2- 172.21.15.170:0/1790473881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24ac0cadf0 0x7f24ac0c9d60 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:45.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 >> 172.21.15.170:0/1790473881 conn(0x7f24ac0b1c40 msgr2=0x7f24ac0aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:45.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 shutdown_connections 2024-09-21T02:21:45.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:45.832+0000 7f24b2b89640 1 -- 172.21.15.170:0/1790473881 wait complete. 2024-09-21T02:21:45.852 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:21:45.852 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:21:45.852 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:21:45.852 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:45.852 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:21:45.852 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:21:45.853 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:21:45.853 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:45.853 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:21:45.853 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:21:45.853 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:21:45.853 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:45.853 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:21:45.853 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:21:45.854 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:21:45.854 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:21:45.865 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:21:45.866 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:21:45.866 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:21:45.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:21:45.872 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:21:45.872 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:21:45.877 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:21:45.877 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:21:45.909 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:21:45.909 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:45.909 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:21:45.910 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:45.922 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:21:45.923 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:45.924 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:21:45.924 DEBUG:teuthology.parallel:result is None 2024-09-21T02:21:45.924 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:21:45.995 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:21:45.999 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:21:46.021 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:21:46.026 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 -- 172.21.15.170:0/2844124869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 msgr2=0x7fa6280cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 --2- 172.21.15.170:0/2844124869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa6280cf970 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fa618009900 tx=0x7fa61801c670 comp rx=0 tx=0).stop 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 -- 172.21.15.170:0/2844124869 shutdown_connections 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 --2- 172.21.15.170:0/2844124869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6280b1230 0x7fa6280b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 --2- 172.21.15.170:0/2844124869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa6280cf970 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 -- 172.21.15.170:0/2844124869 >> 172.21.15.170:0/2844124869 conn(0x7fa62808bf50 msgr2=0x7fa62809d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 -- 172.21.15.170:0/2844124869 shutdown_connections 2024-09-21T02:21:46.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 -- 172.21.15.170:0/2844124869 wait complete. 2024-09-21T02:21:46.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 Processor -- start 2024-09-21T02:21:46.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.068+0000 7fa62ed55640 1 -- start start 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6280b1230 0x7fa628157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa62815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6280d0e50 0x7fa62815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa62816c060 con 0x7fa6280b3fa0 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa62816bee0 con 0x7fa6280d0e50 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa62816c1e0 con 0x7fa6280b1230 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa62815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa62815a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54472/0 (socket says 172.21.15.170:54472) 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 -- 172.21.15.170:0/4158361154 learned_addr learned my addr 172.21.15.170:0/4158361154 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:46.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62caf1640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6280b1230 0x7fa628157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62d2f2640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6280d0e50 0x7fa62815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 -- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6280b1230 msgr2=0x7fa628157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6280b1230 0x7fa628157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 -- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6280d0e50 msgr2=0x7fa62815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6280d0e50 0x7fa62815ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 -- 172.21.15.170:0/4158361154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6281580f0 con 0x7fa6280b3fa0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa627fff640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa62815a8e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00b410 tx=0x7fa61c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c015030 con 0x7fa6280b3fa0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa61c004020 con 0x7fa6280b3fa0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6280ce580 con 0x7fa6280b3fa0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004470 con 0x7fa6280b3fa0 2024-09-21T02:21:46.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62815e8b0 con 0x7fa6280b3fa0 2024-09-21T02:21:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6280b1640 con 0x7fa6280b3fa0 2024-09-21T02:21:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fa61c018070 con 0x7fa6280b3fa0 2024-09-21T02:21:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa6257fa640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa614073a70 0x7fa614075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa62caf1640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa614073a70 0x7fa614075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.072+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4971+0+0 (secure 0 0 0) 0x7fa61c092550 con 0x7fa6280b3fa0 2024-09-21T02:21:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.076+0000 7fa62caf1640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa614073a70 0x7fa614075f80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa6180af000 tx=0x7fa61801aeb0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:46.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.080+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa61c05e950 con 0x7fa6280b3fa0 2024-09-21T02:21:46.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.260+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa6280b9c80 con 0x7fa6280b3fa0 2024-09-21T02:21:46.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.828+0000 7fa6257fa640 1 -- 172.21.15.170:0/4158361154 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v147) ==== 97+0+0 (secure 0 0 0) 0x7fa61c063380 con 0x7fa6280b3fa0 2024-09-21T02:21:46.828 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:21:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.828+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa614073a70 msgr2=0x7fa614075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.828+0000 7fa62ed55640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa614073a70 0x7fa614075f80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa6180af000 tx=0x7fa61801aeb0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.828+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 msgr2=0x7fa62815a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.828+0000 7fa62ed55640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa62815a8e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00b410 tx=0x7fa61c00b8d0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 shutdown_connections 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa614073a70 0x7fa614075f80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6280d0e50 0x7fa62815ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6280b3fa0 0x7fa62815a8e0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 --2- 172.21.15.170:0/4158361154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6280b1230 0x7fa628157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 >> 172.21.15.170:0/4158361154 conn(0x7fa62808bf50 msgr2=0x7fa62815f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 shutdown_connections 2024-09-21T02:21:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:46.832+0000 7fa62ed55640 1 -- 172.21.15.170:0/4158361154 wait complete. 2024-09-21T02:21:46.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e8b64640 1 -- 172.21.15.170:0/60009887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0004810 msgr2=0x7f30d0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e8b64640 1 --2- 172.21.15.170:0/60009887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0004810 0x7f30d0004c20 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f30e4160550 tx=0x7f30cc01c750 comp rx=0 tx=0).stop 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e2d9d640 1 -- 172.21.15.170:0/60009887 reap_dead start 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e8b64640 1 -- 172.21.15.170:0/60009887 shutdown_connections 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e8b64640 1 -- 172.21.15.170:0/60009887 >> 172.21.15.170:0/60009887 conn(0x7f30e40b1c40 msgr2=0x7f30e40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e8b64640 1 -- 172.21.15.170:0/60009887 shutdown_connections 2024-09-21T02:21:47.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.016+0000 7f30e8b64640 1 -- 172.21.15.170:0/60009887 wait complete. 2024-09-21T02:21:47.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 Processor -- start 2024-09-21T02:21:47.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- start start 2024-09-21T02:21:47.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0004810 0x7f30e41e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 0x7f30e409dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e414aff0 0x7f30e414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f30e40b34d0 con 0x7f30e409d8d0 2024-09-21T02:21:47.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30e40b3350 con 0x7f30e414aff0 2024-09-21T02:21:47.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f30e40b3650 con 0x7f30d0004810 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 0x7f30e409dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 0x7f30e409dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54496/0 (socket says 172.21.15.170:54496) 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 -- 172.21.15.170:0/830505108 learned_addr learned my addr 172.21.15.170:0/830505108 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e259c640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0004810 0x7f30e41e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 -- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0004810 msgr2=0x7f30e41e5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e2d9d640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e414aff0 0x7f30e414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0004810 0x7f30e41e5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 -- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e414aff0 msgr2=0x7f30e414b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e414aff0 0x7f30e414b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 -- 172.21.15.170:0/830505108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e409e220 con 0x7f30e409d8d0 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e259c640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0004810 0x7f30e41e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e1d9b640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 0x7f30e409dce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01cc30 tx=0x7f30cc001390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc003600 con 0x7f30e409d8d0 2024-09-21T02:21:47.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30cc003890 con 0x7f30e409d8d0 2024-09-21T02:21:47.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e41e6610 con 0x7f30e409d8d0 2024-09-21T02:21:47.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc003ca0 con 0x7f30e409d8d0 2024-09-21T02:21:47.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e41e6bd0 con 0x7f30e409d8d0 2024-09-21T02:21:47.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.020+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e41e6380 con 0x7f30e409d8d0 2024-09-21T02:21:47.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.024+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f30cc0a55b0 con 0x7f30e409d8d0 2024-09-21T02:21:47.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.024+0000 7f30daffd640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f30c4073a70 0x7f30c4075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.024+0000 7f30e259c640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f30c4073a70 0x7f30c4075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.024+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4885+0+0 (secure 0 0 0) 0x7f30cc12c780 con 0x7f30e409d8d0 2024-09-21T02:21:47.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.024+0000 7f30e259c640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f30c4073a70 0x7f30c4075f80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009b40 tx=0x7f30dc0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:47.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.028+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f30cc0164b0 con 0x7f30e409d8d0 2024-09-21T02:21:47.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.248+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f30e40ce400 con 0x7f30e409d8d0 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30daffd640 1 -- 172.21.15.170:0/830505108 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) ==== 80+0+5304 (secure 0 0 0) 0x7f30cc0f9250 con 0x7f30e409d8d0 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:21:47.254 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:21:47.256 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.258 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:21:47.259 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.261 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.263 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:21:47.264 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.265 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.265 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:21:47.265 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:21:47.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f30c4073a70 msgr2=0x7f30c4075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f30c4073a70 0x7f30c4075f80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009b40 tx=0x7f30dc0091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 msgr2=0x7f30e409dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 0x7f30e409dce0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f30cc01cc30 tx=0x7f30cc001390 comp rx=0 tx=0).stop 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 shutdown_connections 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f30c4073a70 0x7f30c4075f80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e414aff0 0x7f30e414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30e409d8d0 0x7f30e409dce0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 --2- 172.21.15.170:0/830505108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0004810 0x7f30e41e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 >> 172.21.15.170:0/830505108 conn(0x7f30e40b1c40 msgr2=0x7f30e415c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 shutdown_connections 2024-09-21T02:21:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.252+0000 7f30e8b64640 1 -- 172.21.15.170:0/830505108 wait complete. 2024-09-21T02:21:47.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:21:47.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 -- 172.21.15.170:0/1616187612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 msgr2=0x7f91ec0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 --2- 172.21.15.170:0/1616187612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec0b3350 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f91ec071d80 tx=0x7f91e001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 -- 172.21.15.170:0/1616187612 shutdown_connections 2024-09-21T02:21:47.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 --2- 172.21.15.170:0/1616187612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec0b3350 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 --2- 172.21.15.170:0/1616187612 >> v2:172.21.15.191:3300/0 conn(0x7f91ec0b5410 0x7f91ec06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 -- 172.21.15.170:0/1616187612 >> 172.21.15.170:0/1616187612 conn(0x7f91ec0b1c40 msgr2=0x7f91ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:47.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 -- 172.21.15.170:0/1616187612 shutdown_connections 2024-09-21T02:21:47.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 -- 172.21.15.170:0/1616187612 wait complete. 2024-09-21T02:21:47.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 Processor -- start 2024-09-21T02:21:47.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.428+0000 7f91f276c640 1 -- start start 2024-09-21T02:21:47.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91ec0b5410 0x7f91ec155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91ec0ccb00 0x7f91ec156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec169fd0 con 0x7f91ec0b2f40 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec169e50 con 0x7f91ec0ccb00 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec16a150 con 0x7f91ec0b5410 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54504/0 (socket says 172.21.15.170:54504) 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 -- 172.21.15.170:0/2155309286 learned_addr learned my addr 172.21.15.170:0/2155309286 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91eb7fe640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91ec0b5410 0x7f91ec155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f0d09640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91ec0ccb00 0x7f91ec156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 -- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91ec0b5410 msgr2=0x7f91ec155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91ec0b5410 0x7f91ec155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 -- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91ec0ccb00 msgr2=0x7f91ec156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91ec0ccb00 0x7f91ec156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 -- 172.21.15.170:0/2155309286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec155340 con 0x7f91ec0b2f40 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91ebfff640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec154e00 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00d720 tx=0x7f91dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc017070 con 0x7f91ec0b2f40 2024-09-21T02:21:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91dc009d20 con 0x7f91ec0b2f40 2024-09-21T02:21:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec151450 con 0x7f91ec0b2f40 2024-09-21T02:21:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc005280 con 0x7f91ec0b2f40 2024-09-21T02:21:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec151a10 con 0x7f91ec0b2f40 2024-09-21T02:21:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec151c50 con 0x7f91ec0b2f40 2024-09-21T02:21:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.432+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f91dc005420 con 0x7f91ec0b2f40 2024-09-21T02:21:47.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.436+0000 7f91e8ff9640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91d8073aa0 0x7f91d8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.436+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4885+0+0 (secure 0 0 0) 0x7f91dc095200 con 0x7f91ec0b2f40 2024-09-21T02:21:47.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.436+0000 7f91eb7fe640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91d8073aa0 0x7f91d8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.436+0000 7f91eb7fe640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91d8073aa0 0x7f91d8075fb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91e00af000 tx=0x7f91e001b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:47.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.440+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f91ec155340 con 0x7f91ec0b2f40 2024-09-21T02:21:47.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.600+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f91ec0b3350 con 0x7f91ec0b2f40 2024-09-21T02:21:47.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.600+0000 7f91e8ff9640 1 -- 172.21.15.170:0/2155309286 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) ==== 84+0+65 (secure 0 0 0) 0x7f91dc05f640 con 0x7f91ec0b2f40 2024-09-21T02:21:47.603 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:47.603 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:21:47.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:21:47.604 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:21:47.604 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:21:47.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:47.604 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91d8073aa0 msgr2=0x7f91d8075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91d8073aa0 0x7f91d8075fb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91e00af000 tx=0x7f91e001b100 comp rx=0 tx=0).stop 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 msgr2=0x7f91ec154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec154e00 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00d720 tx=0x7f91dc00dbe0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 shutdown_connections 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f91d8073aa0 0x7f91d8075fb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91ec0b5410 0x7f91ec155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91ec0ccb00 0x7f91ec156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 --2- 172.21.15.170:0/2155309286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91ec0b2f40 0x7f91ec154e00 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 >> 172.21.15.170:0/2155309286 conn(0x7f91ec0b1c40 msgr2=0x7f91ec0b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:47.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 shutdown_connections 2024-09-21T02:21:47.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.604+0000 7f91f276c640 1 -- 172.21.15.170:0/2155309286 wait complete. 2024-09-21T02:21:47.625 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:21:47.626 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.780+0000 7f59d1f45640 1 -- 172.21.15.170:0/3628137188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59b4004d50 msgr2=0x7f59b4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.780+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3628137188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59b4004d50 0x7f59b4005220 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009eb0 tx=0x7f59c401d5d0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.780+0000 7f59cbfff640 1 -- 172.21.15.170:0/3628137188 reap_dead start 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.780+0000 7f59d1f45640 1 -- 172.21.15.170:0/3628137188 shutdown_connections 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- 172.21.15.170:0/3628137188 >> 172.21.15.170:0/3628137188 conn(0x7f59cc0b1c40 msgr2=0x7f59cc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- 172.21.15.170:0/3628137188 shutdown_connections 2024-09-21T02:21:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- 172.21.15.170:0/3628137188 wait complete. 2024-09-21T02:21:47.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 Processor -- start 2024-09-21T02:21:47.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- start start 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59cc1e5270 0x7f59cc0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59cc1f30e0 0x7f59cc1f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 0x7f59cc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0af880 con 0x7f59cc15e080 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0af700 con 0x7f59cc1e5270 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59d1f45640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc06f650 con 0x7f59cc1f30e0 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 0x7f59cc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 0x7f59cc15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54518/0 (socket says 172.21.15.170:54518) 2024-09-21T02:21:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 -- 172.21.15.170:0/3197336152 learned_addr learned my addr 172.21.15.170:0/3197336152 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59caffd640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59cc1f30e0 0x7f59cc1f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cb7fe640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59cc1e5270 0x7f59cc0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 -- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59cc1f30e0 msgr2=0x7f59cc1f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59cc1f30e0 0x7f59cc1f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 -- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59cc1e5270 msgr2=0x7f59cc0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59cc1e5270 0x7f59cc0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 -- 172.21.15.170:0/3197336152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc15e9d0 con 0x7f59cc15e080 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cb7fe640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59cc1e5270 0x7f59cc0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.784+0000 7f59cbfff640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 0x7f59cc15e490 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009eb0 tx=0x7f59c4009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4009ab0 con 0x7f59cc15e080 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59c40a9510 con 0x7f59cc15e080 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4009200 con 0x7f59cc15e080 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc0c9570 con 0x7f59cc15e080 2024-09-21T02:21:47.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1f75f0 con 0x7f59cc15e080 2024-09-21T02:21:47.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc06fb40 con 0x7f59cc15e080 2024-09-21T02:21:47.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f59c40b9030 con 0x7f59cc15e080 2024-09-21T02:21:47.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59b3fff640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59b8073a70 0x7f59b8075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:47.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59cb7fe640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59b8073a70 0x7f59b8075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:47.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4885+0+0 (secure 0 0 0) 0x7f59c412d3e0 con 0x7f59cc15e080 2024-09-21T02:21:47.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.788+0000 7f59cb7fe640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59b8073a70 0x7f59b8075f80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000c10 tx=0x7f59bc006b70 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:47.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.792+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f59cc15e9d0 con 0x7f59cc15e080 2024-09-21T02:21:47.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:47.960+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f59cc1f35f0 con 0x7f59cc15e080 2024-09-21T02:21:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.848+0000 7f59b3fff640 1 -- 172.21.15.170:0/3197336152 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) ==== 138+0+0 (secure 0 0 0) 0x7f59c40ad030 con 0x7f59cc15e080 2024-09-21T02:21:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:21:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59b8073a70 msgr2=0x7f59b8075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59b8073a70 0x7f59b8075f80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000c10 tx=0x7f59bc006b70 comp rx=0 tx=0).stop 2024-09-21T02:21:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 msgr2=0x7f59cc15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 0x7f59cc15e490 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009eb0 tx=0x7f59c4009480 comp rx=0 tx=0).stop 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 shutdown_connections 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59b8073a70 0x7f59b8075f80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59cc1f30e0 0x7f59cc1f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59cc15e080 0x7f59cc15e490 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 --2- 172.21.15.170:0/3197336152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59cc1e5270 0x7f59cc0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 >> 172.21.15.170:0/3197336152 conn(0x7f59cc0b1c40 msgr2=0x7f59cc1e90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 shutdown_connections 2024-09-21T02:21:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:48.852+0000 7f59d1f45640 1 -- 172.21.15.170:0/3197336152 wait complete. 2024-09-21T02:21:48.886 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:21:49.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 -- 172.21.15.170:0/733469252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf4003a10 msgr2=0x7fdaf4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:49.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 --2- 172.21.15.170:0/733469252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf4003a10 0x7fdaf4003e80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009b60 tx=0x7fdafc01d2b0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb0b7fe640 1 -- 172.21.15.170:0/733469252 reap_dead start 2024-09-21T02:21:49.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 -- 172.21.15.170:0/733469252 shutdown_connections 2024-09-21T02:21:49.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 -- 172.21.15.170:0/733469252 >> 172.21.15.170:0/733469252 conn(0x7fdb0c0d0170 msgr2=0x7fdb0c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:49.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 -- 172.21.15.170:0/733469252 shutdown_connections 2024-09-21T02:21:49.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 -- 172.21.15.170:0/733469252 wait complete. 2024-09-21T02:21:49.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 Processor -- start 2024-09-21T02:21:49.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.056+0000 7fdb113d0640 1 -- start start 2024-09-21T02:21:49.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c150890 0x7fdb0c1fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:49.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 0x7fdb0c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:49.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c92d0 0x7fdb0c1f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:49.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c073f80 con 0x7fdb0c1fac20 2024-09-21T02:21:49.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c073e00 con 0x7fdb0c150890 2024-09-21T02:21:49.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c074100 con 0x7fdb0c0c92d0 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 0x7fdb0c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 0x7fdb0c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54534/0 (socket says 172.21.15.170:54534) 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 -- 172.21.15.170:0/4078362451 learned_addr learned my addr 172.21.15.170:0/4078362451 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0affd640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c150890 0x7fdb0c1fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0b7fe640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c92d0 0x7fdb0c1f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 -- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c92d0 msgr2=0x7fdb0c1f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c92d0 0x7fdb0c1f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 -- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c150890 msgr2=0x7fdb0c1fa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c150890 0x7fdb0c1fa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 -- 172.21.15.170:0/4078362451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c0c3a70 con 0x7fdb0c1fac20 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb0a7fc640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 0x7fdb0c0c8d90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c600 tx=0x7fdb0000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00017070 con 0x7fdb0c1fac20 2024-09-21T02:21:49.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb000036e0 con 0x7fdb0c1fac20 2024-09-21T02:21:49.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00003b10 con 0x7fdb0c1fac20 2024-09-21T02:21:49.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c1fb030 con 0x7fdb0c1fac20 2024-09-21T02:21:49.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c09c0f0 con 0x7fdb0c1fac20 2024-09-21T02:21:49.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0c074690 con 0x7fdb0c1fac20 2024-09-21T02:21:49.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fdb0001a070 con 0x7fdb0c1fac20 2024-09-21T02:21:49.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.060+0000 7fdaf37fe640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdaf8073aa0 0x7fdaf8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:49.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.064+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5255+0+0 (secure 0 0 0) 0x7fdb00092420 con 0x7fdb0c1fac20 2024-09-21T02:21:49.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.064+0000 7fdb0affd640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdaf8073aa0 0x7fdaf8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:49.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.064+0000 7fdb0affd640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdaf8073aa0 0x7fdaf8075fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0ae000 tx=0x7fdafc01a050 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:49.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.068+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdb00021020 con 0x7fdb0c1fac20 2024-09-21T02:21:49.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.240+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdb0c0addd0 con 0x7fdb0c1fac20 2024-09-21T02:21:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.860+0000 7fdaf37fe640 1 -- 172.21.15.170:0/4078362451 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) ==== 170+0+0 (secure 0 0 0) 0x7fdb0c1fb030 con 0x7fdb0c1fac20 2024-09-21T02:21:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdaf8073aa0 msgr2=0x7fdaf8075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdaf8073aa0 0x7fdaf8075fb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0ae000 tx=0x7fdafc01a050 comp rx=0 tx=0).stop 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 msgr2=0x7fdb0c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 0x7fdb0c0c8d90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c600 tx=0x7fdb0000cac0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 shutdown_connections 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdaf8073aa0 0x7fdaf8075fb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c92d0 0x7fdb0c1f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb0c1fac20 0x7fdb0c0c8d90 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 --2- 172.21.15.170:0/4078362451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c150890 0x7fdb0c1fa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:49.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 >> 172.21.15.170:0/4078362451 conn(0x7fdb0c0d0170 msgr2=0x7fdb0c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:49.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 shutdown_connections 2024-09-21T02:21:49.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:49.868+0000 7fdb113d0640 1 -- 172.21.15.170:0/4078362451 wait complete. 2024-09-21T02:21:49.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:21:50.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.068+0000 7f727be6b640 1 -- 172.21.15.170:0/655157178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 msgr2=0x7f7260004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:50.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.068+0000 7f727be6b640 1 --2- 172.21.15.170:0/655157178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f7260004c20 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f7264009b60 tx=0x7f726401cff0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.068+0000 7f727a408640 1 -- 172.21.15.170:0/655157178 reap_dead start 2024-09-21T02:21:50.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- 172.21.15.170:0/655157178 shutdown_connections 2024-09-21T02:21:50.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- 172.21.15.170:0/655157178 >> 172.21.15.170:0/655157178 conn(0x7f72740cd5d0 msgr2=0x7f72740cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:50.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- 172.21.15.170:0/655157178 shutdown_connections 2024-09-21T02:21:50.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- 172.21.15.170:0/655157178 wait complete. 2024-09-21T02:21:50.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 Processor -- start 2024-09-21T02:21:50.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- start start 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f727414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72741483d0 0x7f727415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72741487e0 0x7f727415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72740b6740 con 0x7f7260004810 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72740b65c0 con 0x7f72741483d0 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727be6b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f72740b68c0 con 0x7f72741487e0 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f7279c07640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f727414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f7279c07640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f727414b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54558/0 (socket says 172.21.15.170:54558) 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f7279c07640 1 -- 172.21.15.170:0/32267048 learned_addr learned my addr 172.21.15.170:0/32267048 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:50.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f727a408640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72741487e0 0x7f727415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.072+0000 7f7279406640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72741483d0 0x7f727415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279c07640 1 -- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72741487e0 msgr2=0x7f727415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279c07640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72741487e0 0x7f727415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279c07640 1 -- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72741483d0 msgr2=0x7f727415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279c07640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72741483d0 0x7f727415deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279c07640 1 -- 172.21.15.170:0/32267048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727414bd30 con 0x7f7260004810 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279c07640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f727414b7f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f7270004f10 tx=0x7f7270001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270015020 con 0x7f7260004810 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7270011960 con 0x7f7260004810 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7270011d90 con 0x7f7260004810 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741590c0 con 0x7f7260004810 2024-09-21T02:21:50.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72740b0b20 con 0x7f7260004810 2024-09-21T02:21:50.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72740b0910 con 0x7f7260004810 2024-09-21T02:21:50.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f7270018020 con 0x7f7260004810 2024-09-21T02:21:50.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f726a7fc640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7258073a70 0x7f7258075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:50.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f7279406640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7258073a70 0x7f7258075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:50.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.076+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5637+0+0 (secure 0 0 0) 0x7f72700939d0 con 0x7f7260004810 2024-09-21T02:21:50.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.080+0000 7f7279406640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7258073a70 0x7f7258075f80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f72640ae000 tx=0x7f7264019d90 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:50.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.080+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f727001f020 con 0x7f7260004810 2024-09-21T02:21:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.292+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f72740cf3a0 con 0x7f7260004810 2024-09-21T02:21:50.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.864+0000 7f726a7fc640 1 -- 172.21.15.170:0/32267048 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v106) ==== 174+0+0 (secure 0 0 0) 0x7f727414bd30 con 0x7f7260004810 2024-09-21T02:21:50.866 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 24 and data pool 25 2024-09-21T02:21:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7258073a70 msgr2=0x7f7258075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:50.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7258073a70 0x7f7258075f80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f72640ae000 tx=0x7f7264019d90 comp rx=0 tx=0).stop 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 msgr2=0x7f727414b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f727414b7f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f7270004f10 tx=0x7f7270001640 comp rx=0 tx=0).stop 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 shutdown_connections 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7258073a70 0x7f7258075f80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72741487e0 0x7f727415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72741483d0 0x7f727415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 --2- 172.21.15.170:0/32267048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260004810 0x7f727414b7f0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 >> 172.21.15.170:0/32267048 conn(0x7f72740cd5d0 msgr2=0x7f72740b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 shutdown_connections 2024-09-21T02:21:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:50.872+0000 7f727be6b640 1 -- 172.21.15.170:0/32267048 wait complete. 2024-09-21T02:21:50.913 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:51.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 -- 172.21.15.170:0/4077139032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 msgr2=0x7f38780cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 --2- 172.21.15.170:0/4077139032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 0x7f38780cb200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f387000acd0 tx=0x7f387001dec0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 -- 172.21.15.170:0/4077139032 shutdown_connections 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 --2- 172.21.15.170:0/4077139032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 0x7f38780cb200 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 --2- 172.21.15.170:0/4077139032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f387815f990 0x7f3878070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 -- 172.21.15.170:0/4077139032 >> 172.21.15.170:0/4077139032 conn(0x7f38780b1c40 msgr2=0x7f38780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 -- 172.21.15.170:0/4077139032 shutdown_connections 2024-09-21T02:21:51.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 -- 172.21.15.170:0/4077139032 wait complete. 2024-09-21T02:21:51.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 Processor -- start 2024-09-21T02:21:51.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.068+0000 7f387d658640 1 -- start start 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 0x7f38780d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 0x7f3878160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f387815f990 0x7f387809cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3878167ff0 con 0x7f3878070570 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38780cede0 con 0x7f38780cadf0 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3878168170 con 0x7f387815f990 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 0x7f38780d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 0x7f38780d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54570/0 (socket says 172.21.15.170:54570) 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 -- 172.21.15.170:0/3525072775 learned_addr learned my addr 172.21.15.170:0/3525072775 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f38777fe640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f387815f990 0x7f387809cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f38767fc640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 0x7f3878160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 -- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f387815f990 msgr2=0x7f387809cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f387815f990 0x7f387809cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 -- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 msgr2=0x7f3878160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 0x7f3878160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 -- 172.21.15.170:0/3525072775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387809b8b0 con 0x7f3878070570 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f3876ffd640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 0x7f38780d0210 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e7d0 tx=0x7f386c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c00ccc0 con 0x7f3878070570 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f386c010430 con 0x7f3878070570 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c010860 con 0x7f3878070570 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387809aaf0 con 0x7f3878070570 2024-09-21T02:21:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387809b0b0 con 0x7f3878070570 2024-09-21T02:21:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.072+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38780cb200 con 0x7f3878070570 2024-09-21T02:21:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.076+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f386c010a00 con 0x7f3878070570 2024-09-21T02:21:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.076+0000 7f385f7fe640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3864073a70 0x7f3864075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.076+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5737+0+0 (secure 0 0 0) 0x7f386c0934c0 con 0x7f3878070570 2024-09-21T02:21:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.076+0000 7f38767fc640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3864073a70 0x7f3864075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.076+0000 7f38767fc640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3864073a70 0x7f3864075f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3878165fb0 tx=0x7f3868009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.080+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f386c05ffa0 con 0x7f3878070570 2024-09-21T02:21:51.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.256+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38780ce2a0 con 0x7f3878070570 2024-09-21T02:21:51.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f385f7fe640 1 -- 172.21.15.170:0/3525072775 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16788 (secure 0 0 0) 0x7f386c05f8a0 con 0x7f3878070570 2024-09-21T02:21:51.260 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:51.260 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":152,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:50.297168+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:47.975445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:49.257998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:51.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3864073a70 msgr2=0x7f3864075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3864073a70 0x7f3864075f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3878165fb0 tx=0x7f3868009040 comp rx=0 tx=0).stop 2024-09-21T02:21:51.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 msgr2=0x7f38780d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 0x7f38780d0210 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e7d0 tx=0x7f386c00ec90 comp rx=0 tx=0).stop 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 shutdown_connections 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3864073a70 0x7f3864075f80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f387815f990 0x7f387809cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38780cadf0 0x7f3878160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 --2- 172.21.15.170:0/3525072775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3878070570 0x7f38780d0210 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 >> 172.21.15.170:0/3525072775 conn(0x7f38780b1c40 msgr2=0x7f38780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 shutdown_connections 2024-09-21T02:21:51.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.260+0000 7f387d658640 1 -- 172.21.15.170:0/3525072775 wait complete. 2024-09-21T02:21:51.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:51.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 -- 172.21.15.170:0/2659230741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b4540 msgr2=0x7f839015c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 --2- 172.21.15.170:0/2659230741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b4540 0x7f839015c850 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8378009c80 tx=0x7f837801cda0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 -- 172.21.15.170:0/2659230741 shutdown_connections 2024-09-21T02:21:51.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 --2- 172.21.15.170:0/2659230741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83900d0aa0 0x7f83900d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 --2- 172.21.15.170:0/2659230741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b4540 0x7f839015c850 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 -- 172.21.15.170:0/2659230741 >> 172.21.15.170:0/2659230741 conn(0x7f83900cee30 msgr2=0x7f83900cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:51.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 -- 172.21.15.170:0/2659230741 shutdown_connections 2024-09-21T02:21:51.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.436+0000 7f83960fb640 1 -- 172.21.15.170:0/2659230741 wait complete. 2024-09-21T02:21:51.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 Processor -- start 2024-09-21T02:21:51.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- start start 2024-09-21T02:21:51.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 0x7f839015a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83900d0aa0 0x7f83901fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f839015f750 0x7f83901faa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f839016a590 con 0x7f83900b7ab0 2024-09-21T02:21:51.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f839016a410 con 0x7f83900d0aa0 2024-09-21T02:21:51.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f839016a710 con 0x7f839015f750 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 0x7f839015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 0x7f839015a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54592/0 (socket says 172.21.15.170:54592) 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 -- 172.21.15.170:0/570731242 learned_addr learned my addr 172.21.15.170:0/570731242 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838ffff640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f839015f750 0x7f83901faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838effd640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83900d0aa0 0x7f83901fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 -- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f839015f750 msgr2=0x7f83901faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f839015f750 0x7f83901faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 -- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83900d0aa0 msgr2=0x7f83901fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83900d0aa0 0x7f83901fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 -- 172.21.15.170:0/570731242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839015e0d0 con 0x7f83900b7ab0 2024-09-21T02:21:51.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838effd640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83900d0aa0 0x7f83901fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:51.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838ffff640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f839015f750 0x7f83901faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:51.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f838f7fe640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 0x7f839015a820 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f838000b440 tx=0x7f838000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:51.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380015020 con 0x7f83900b7ab0 2024-09-21T02:21:51.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8380011980 con 0x7f83900b7ab0 2024-09-21T02:21:51.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380011dd0 con 0x7f83900b7ab0 2024-09-21T02:21:51.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83901fb250 con 0x7f83900b7ab0 2024-09-21T02:21:51.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839015b4d0 con 0x7f83900b7ab0 2024-09-21T02:21:51.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.440+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839015c850 con 0x7f83900b7ab0 2024-09-21T02:21:51.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.444+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f8380018020 con 0x7f83900b7ab0 2024-09-21T02:21:51.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.444+0000 7f8387fff640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8370073a70 0x7f8370075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.444+0000 7f838effd640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8370073a70 0x7f8370075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.444+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5737+0+0 (secure 0 0 0) 0x7f8380092dc0 con 0x7f83900b7ab0 2024-09-21T02:21:51.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.444+0000 7f838effd640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8370073a70 0x7f8370075f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f83780af000 tx=0x7f837801b5e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:51.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.448+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f838001f070 con 0x7f83900b7ab0 2024-09-21T02:21:51.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.624+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83901faf50 con 0x7f83900b7ab0 2024-09-21T02:21:51.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.624+0000 7f8387fff640 1 -- 172.21.15.170:0/570731242 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16788 (secure 0 0 0) 0x7f838000d040 con 0x7f83900b7ab0 2024-09-21T02:21:51.627 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:51.627 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":152,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:50.297168+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:47.975445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:49.257998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:51.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8370073a70 msgr2=0x7f8370075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8370073a70 0x7f8370075f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f83780af000 tx=0x7f837801b5e0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 msgr2=0x7f839015a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 0x7f839015a820 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f838000b440 tx=0x7f838000b900 comp rx=0 tx=0).stop 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 shutdown_connections 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8370073a70 0x7f8370075f80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f839015f750 0x7f83901faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83900d0aa0 0x7f83901fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 --2- 172.21.15.170:0/570731242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83900b7ab0 0x7f839015a820 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 >> 172.21.15.170:0/570731242 conn(0x7f83900cee30 msgr2=0x7f83900b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 shutdown_connections 2024-09-21T02:21:51.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.628+0000 7f83960fb640 1 -- 172.21.15.170:0/570731242 wait complete. 2024-09-21T02:21:51.649 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:21:51.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb0a826640 1 -- 172.21.15.170:0/2410400480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feaec003a40 msgr2=0x7feaec003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb0a826640 1 --2- 172.21.15.170:0/2410400480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feaec003a40 0x7feaec003eb0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7feb0406de50 tx=0x7feaf801cc60 comp rx=0 tx=0).stop 2024-09-21T02:21:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb08dc3640 1 -- 172.21.15.170:0/2410400480 reap_dead start 2024-09-21T02:21:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb0a826640 1 -- 172.21.15.170:0/2410400480 shutdown_connections 2024-09-21T02:21:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb0a826640 1 -- 172.21.15.170:0/2410400480 >> 172.21.15.170:0/2410400480 conn(0x7feb040d0170 msgr2=0x7feb040d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb0a826640 1 -- 172.21.15.170:0/2410400480 shutdown_connections 2024-09-21T02:21:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.792+0000 7feb0a826640 1 -- 172.21.15.170:0/2410400480 wait complete. 2024-09-21T02:21:51.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 Processor -- start 2024-09-21T02:21:51.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- start start 2024-09-21T02:21:51.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec003a40 0x7feb0415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb041fcfc0 0x7feb04153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 0x7feb04166a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb04167dc0 con 0x7feb04166650 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feb040b35e0 con 0x7feaec003a40 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7feb04167f40 con 0x7feb041fcfc0 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 0x7feb04166a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 0x7feb04166a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54612/0 (socket says 172.21.15.170:54612) 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 -- 172.21.15.170:0/3717268937 learned_addr learned my addr 172.21.15.170:0/3717268937 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb03fff640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec003a40 0x7feb0415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb037fe640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb041fcfc0 0x7feb04153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 -- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb041fcfc0 msgr2=0x7feb04153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb041fcfc0 0x7feb04153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 -- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec003a40 msgr2=0x7feb0415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec003a40 0x7feb0415e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 -- 172.21.15.170:0/3717268937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb04166fa0 con 0x7feb04166650 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb037fe640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb041fcfc0 0x7feb04153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb08dc3640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 0x7feb04166a60 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7feafc0020d0 tx=0x7feafc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cc20 con 0x7feb04166650 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feafc018070 con 0x7feb04166650 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0415f280 con 0x7feb04166650 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0116c0 con 0x7feb04166650 2024-09-21T02:21:51.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb041fc7c0 con 0x7feb04166650 2024-09-21T02:21:51.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.796+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb040cc7d0 con 0x7feb04166650 2024-09-21T02:21:51.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.800+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7feafc011860 con 0x7feb04166650 2024-09-21T02:21:51.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.800+0000 7feb00ff9640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feaf0073a70 0x7feaf0075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:51.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.800+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5737+0+0 (secure 0 0 0) 0x7feafc092e40 con 0x7feb04166650 2024-09-21T02:21:51.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.800+0000 7feb03fff640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feaf0073a70 0x7feaf0075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:51.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.800+0000 7feb03fff640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feaf0073a70 0x7feaf0075f80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feaf80b0000 tx=0x7feaf80a7040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:51.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:51.804+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feb040cc7d0 con 0x7feb04166650 2024-09-21T02:21:52.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.020+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feb0415f010 con 0x7feb04166650 2024-09-21T02:21:52.834 INFO:tasks.ceph.mds.c.smithi170.stderr:2024-09-21T02:21:52.836+0000 7fa09fea7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:21:52.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.920+0000 7feb00ff9640 1 -- 172.21.15.170:0/3717268937 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v109) ==== 118+0+0 (secure 0 0 0) 0x7feafc093e20 con 0x7feb04166650 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feaf0073a70 msgr2=0x7feaf0075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feaf0073a70 0x7feaf0075f80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feaf80b0000 tx=0x7feaf80a7040 comp rx=0 tx=0).stop 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 msgr2=0x7feb04166a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 0x7feb04166a60 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7feafc0020d0 tx=0x7feafc00ed10 comp rx=0 tx=0).stop 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 shutdown_connections 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7feaf0073a70 0x7feaf0075f80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb041fcfc0 0x7feb04153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb04166650 0x7feb04166a60 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:52.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 --2- 172.21.15.170:0/3717268937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaec003a40 0x7feb0415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:52.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 >> 172.21.15.170:0/3717268937 conn(0x7feb040d0170 msgr2=0x7feb040ade80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:52.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 shutdown_connections 2024-09-21T02:21:52.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:52.928+0000 7feb0a826640 1 -- 172.21.15.170:0/3717268937 wait complete. 2024-09-21T02:21:52.951 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:21:52.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 -- 172.21.15.170:0/4203140997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf970 msgr2=0x7f23e40d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 --2- 172.21.15.170:0/4203140997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf970 0x7f23e40d12c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f23e40b53c0 tx=0x7f23dc01ccb0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 -- 172.21.15.170:0/4203140997 shutdown_connections 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 --2- 172.21.15.170:0/4203140997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf970 0x7f23e40d12c0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 --2- 172.21.15.170:0/4203140997 >> v2:172.21.15.191:3300/0 conn(0x7f23e40d3020 0x7f23e40cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 -- 172.21.15.170:0/4203140997 >> 172.21.15.170:0/4203140997 conn(0x7f23e409d8b0 msgr2=0x7f23e40d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 -- 172.21.15.170:0/4203140997 shutdown_connections 2024-09-21T02:21:53.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.104+0000 7f23e9994640 1 -- 172.21.15.170:0/4203140997 wait complete. 2024-09-21T02:21:53.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 Processor -- start 2024-09-21T02:21:53.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- start start 2024-09-21T02:21:53.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 0x7f23e4161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:53.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23e40cf970 0x7f23e41fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e40d3020 0x7f23e41fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f23e416be90 con 0x7f23e40cf560 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23e416bd10 con 0x7f23e40d3020 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f23e416c010 con 0x7f23e40cf970 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 0x7f23e4161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 0x7f23e4161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54624/0 (socket says 172.21.15.170:54624) 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 -- 172.21.15.170:0/482100656 learned_addr learned my addr 172.21.15.170:0/482100656 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e37fe640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e40d3020 0x7f23e41fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 -- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23e40cf970 msgr2=0x7f23e41fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23e40cf970 0x7f23e41fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 -- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e40d3020 msgr2=0x7f23e41fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e40d3020 0x7f23e41fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 -- 172.21.15.170:0/482100656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e41f8b40 con 0x7f23e40cf560 2024-09-21T02:21:53.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e2ffd640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 0x7f23e4161bb0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c970 tx=0x7f23d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:53.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004220 con 0x7f23e40cf560 2024-09-21T02:21:53.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23d8007950 con 0x7f23e40cf560 2024-09-21T02:21:53.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e41f8e20 con 0x7f23e40cf560 2024-09-21T02:21:53.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d80044d0 con 0x7f23e40cf560 2024-09-21T02:21:53.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e41fcee0 con 0x7f23e40cf560 2024-09-21T02:21:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e40cf020 con 0x7f23e40cf560 2024-09-21T02:21:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.108+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f23d8003620 con 0x7f23e40cf560 2024-09-21T02:21:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.112+0000 7f23cbfff640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f23d0073aa0 0x7f23d0075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.112+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f23d8092ac0 con 0x7f23e40cf560 2024-09-21T02:21:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.112+0000 7f23e27fc640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f23d0073aa0 0x7f23d0075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.112+0000 7f23e27fc640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f23d0073aa0 0x7f23d0075fb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f23e4162690 tx=0x7f23d4006bd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.116+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f23d8014030 con 0x7f23e40cf560 2024-09-21T02:21:53.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.308+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f23e40b6c40 con 0x7f23e40cf560 2024-09-21T02:21:53.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.932+0000 7f23cbfff640 1 -- 172.21.15.170:0/482100656 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v110) ==== 122+0+0 (secure 0 0 0) 0x7f23d805f620 con 0x7f23e40cf560 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f23d0073aa0 msgr2=0x7f23d0075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f23d0073aa0 0x7f23d0075fb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f23e4162690 tx=0x7f23d4006bd0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 msgr2=0x7f23e4161bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 0x7f23e4161bb0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c970 tx=0x7f23d800ce30 comp rx=0 tx=0).stop 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 shutdown_connections 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f23d0073aa0 0x7f23d0075fb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23e40cf970 0x7f23e41fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e40d3020 0x7f23e41fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 --2- 172.21.15.170:0/482100656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23e40cf560 0x7f23e4161bb0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:53.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 >> 172.21.15.170:0/482100656 conn(0x7f23e409d8b0 msgr2=0x7f23e40b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:53.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 shutdown_connections 2024-09-21T02:21:53.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:53.936+0000 7f23e9994640 1 -- 172.21.15.170:0/482100656 wait complete. 2024-09-21T02:21:53.965 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:21:53.965 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:54.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 -- 172.21.15.170:0/1072850899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cb460 msgr2=0x7fbe480cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/1072850899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cb460 0x7fbe480cb870 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fbe480d1160 tx=0x7fbe3001c800 comp rx=0 tx=0).stop 2024-09-21T02:21:54.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 -- 172.21.15.170:0/1072850899 shutdown_connections 2024-09-21T02:21:54.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/1072850899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cb460 0x7fbe480cb870 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/1072850899 >> v2:172.21.15.191:3300/0 conn(0x7fbe480cf7e0 0x7fbe48070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 -- 172.21.15.170:0/1072850899 >> 172.21.15.170:0/1072850899 conn(0x7fbe480b1c40 msgr2=0x7fbe480b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:54.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 -- 172.21.15.170:0/1072850899 shutdown_connections 2024-09-21T02:21:54.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 -- 172.21.15.170:0/1072850899 wait complete. 2024-09-21T02:21:54.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 Processor -- start 2024-09-21T02:21:54.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.124+0000 7fbe4d17d640 1 -- start start 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe480cb460 0x7fbe4815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 0x7fbe480b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe480cf7e0 0x7fbe480b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481690d0 con 0x7fbe480cc8c0 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe48168f50 con 0x7fbe480cf7e0 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbe48169250 con 0x7fbe480cb460 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 0x7fbe480b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 0x7fbe480b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54650/0 (socket says 172.21.15.170:54650) 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 -- 172.21.15.170:0/823037847 learned_addr learned my addr 172.21.15.170:0/823037847 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 -- 172.21.15.170:0/823037847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe480cb460 msgr2=0x7fbe4815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe480cb460 0x7fbe4815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 -- 172.21.15.170:0/823037847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe480cf7e0 msgr2=0x7fbe480b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe480cf7e0 0x7fbe480b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 -- 172.21.15.170:0/823037847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe48156330 con 0x7fbe480cc8c0 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3ffff640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 0x7fbe480b2f40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbe30000c10 tx=0x7fbe300a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300bb070 con 0x7fbe480cc8c0 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe30005030 con 0x7fbe480cc8c0 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe48156610 con 0x7fbe480cc8c0 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300b2040 con 0x7fbe480cc8c0 2024-09-21T02:21:54.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe48147ab0 con 0x7fbe480cc8c0 2024-09-21T02:21:54.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe480cb870 con 0x7fbe480cc8c0 2024-09-21T02:21:54.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.128+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fbe300be070 con 0x7fbe480cc8c0 2024-09-21T02:21:54.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.132+0000 7fbe3f7fe640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbe28073aa0 0x7fbe28075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.132+0000 7fbe46d9d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbe28073aa0 0x7fbe28075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.132+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7fbe3012c800 con 0x7fbe480cc8c0 2024-09-21T02:21:54.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.132+0000 7fbe46d9d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbe28073aa0 0x7fbe28075fb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe380058d0 tx=0x7fbe38005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:54.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.136+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbe300c5020 con 0x7fbe480cc8c0 2024-09-21T02:21:54.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe480b44b0 con 0x7fbe480cc8c0 2024-09-21T02:21:54.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe3f7fe640 1 -- 172.21.15.170:0/823037847 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5245 (secure 0 0 0) 0x7fbe300163f0 con 0x7fbe480cc8c0 2024-09-21T02:21:54.341 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:54.341 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":110,"btime":"2024-09-21T02:21:53:926764+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:50.297124+0000","modified":"2024-09-21T02:21:53.314129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8934":{"gid":8934,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2398609544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2398609544},{"type":"v1","addr":"172.21.15.170:6835","nonce":2398609544}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8934,"qdb_cluster":[8934]},"id":12}]} 2024-09-21T02:21:54.341 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 110 2024-09-21T02:21:54.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbe28073aa0 msgr2=0x7fbe28075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbe28073aa0 0x7fbe28075fb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe380058d0 tx=0x7fbe38005860 comp rx=0 tx=0).stop 2024-09-21T02:21:54.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 msgr2=0x7fbe480b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 0x7fbe480b2f40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbe30000c10 tx=0x7fbe300a56c0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 shutdown_connections 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbe28073aa0 0x7fbe28075fb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe480cf7e0 0x7fbe480b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe480cc8c0 0x7fbe480b2f40 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 --2- 172.21.15.170:0/823037847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe480cb460 0x7fbe4815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.340+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 >> 172.21.15.170:0/823037847 conn(0x7fbe480b1c40 msgr2=0x7fbe480acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.344+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 shutdown_connections 2024-09-21T02:21:54.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.344+0000 7fbe4d17d640 1 -- 172.21.15.170:0/823037847 wait complete. 2024-09-21T02:21:54.362 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:21:54.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 -- 172.21.15.170:0/2227183418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 msgr2=0x7f11d806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 --2- 172.21.15.170:0/2227183418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 0x7f11d806fa30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f11d80b3a10 tx=0x7f11c801c820 comp rx=0 tx=0).stop 2024-09-21T02:21:54.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 -- 172.21.15.170:0/2227183418 shutdown_connections 2024-09-21T02:21:54.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 --2- 172.21.15.170:0/2227183418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11d80b2f40 0x7f11d80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 --2- 172.21.15.170:0/2227183418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 0x7f11d806fa30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 -- 172.21.15.170:0/2227183418 >> 172.21.15.170:0/2227183418 conn(0x7f11d80b1c40 msgr2=0x7f11d80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:54.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 -- 172.21.15.170:0/2227183418 shutdown_connections 2024-09-21T02:21:54.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.516+0000 7f11de190640 1 -- 172.21.15.170:0/2227183418 wait complete. 2024-09-21T02:21:54.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 Processor -- start 2024-09-21T02:21:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 -- start start 2024-09-21T02:21:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 0x7f11d815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 0x7f11d815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11d80ccb00 0x7f11d815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8169fd0 con 0x7f11d80b2f40 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8169e50 con 0x7f11d80b5410 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f11d816a150 con 0x7f11d80ccb00 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 0x7f11d815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 0x7f11d815c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54658/0 (socket says 172.21.15.170:54658) 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 -- 172.21.15.170:0/1742136518 learned_addr learned my addr 172.21.15.170:0/1742136518 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d7fff640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11d80ccb00 0x7f11d815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11ceffd640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 0x7f11d815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 -- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11d80ccb00 msgr2=0x7f11d815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11d80ccb00 0x7f11d815e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 -- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 msgr2=0x7f11d815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 0x7f11d815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 -- 172.21.15.170:0/1742136518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d8160070 con 0x7f11d80b2f40 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d77fe640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 0x7f11d815c010 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c10 tx=0x7f11c8005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c80bb070 con 0x7f11d80b2f40 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11c80a5bd0 con 0x7f11d80b2f40 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d8164a30 con 0x7f11d80b2f40 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c80b2040 con 0x7f11d80b2f40 2024-09-21T02:21:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.520+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d8164ff0 con 0x7f11d80b2f40 2024-09-21T02:21:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.524+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d80b3350 con 0x7f11d80b2f40 2024-09-21T02:21:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.524+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f11c80be070 con 0x7f11d80b2f40 2024-09-21T02:21:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.524+0000 7f11d4ff9640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f11b8073a70 0x7f11b8075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.524+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f11c812c530 con 0x7f11d80b2f40 2024-09-21T02:21:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.524+0000 7f11ceffd640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f11b8073a70 0x7f11b8075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.524+0000 7f11ceffd640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f11b8073a70 0x7f11b8075f80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f11d815ce40 tx=0x7f11c00091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:54.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.528+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f11c8016410 con 0x7f11d80b2f40 2024-09-21T02:21:54.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.688+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11d8074e40 con 0x7f11d80b2f40 2024-09-21T02:21:54.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11d4ff9640 1 -- 172.21.15.170:0/1742136518 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16775 (secure 0 0 0) 0x7f11c80f8a40 con 0x7f11d80b2f40 2024-09-21T02:21:54.693 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:54.693 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":153,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:21:51.869264+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:47.975445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:49.257998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:21:54.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f11b8073a70 msgr2=0x7f11b8075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f11b8073a70 0x7f11b8075f80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f11d815ce40 tx=0x7f11c00091e0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 msgr2=0x7f11d815c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 0x7f11d815c010 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c10 tx=0x7f11c8005d40 comp rx=0 tx=0).stop 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 shutdown_connections 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f11b8073a70 0x7f11b8075f80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.692+0000 7f11de190640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f11d80ccb00 0x7f11d815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.696+0000 7f11de190640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11d80b5410 0x7f11d815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.696+0000 7f11de190640 1 --2- 172.21.15.170:0/1742136518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11d80b2f40 0x7f11d815c010 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.696+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 >> 172.21.15.170:0/1742136518 conn(0x7f11d80b1c40 msgr2=0x7f11d80c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.696+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 shutdown_connections 2024-09-21T02:21:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.696+0000 7f11de190640 1 -- 172.21.15.170:0/1742136518 wait complete. 2024-09-21T02:21:54.715 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:21:54.715 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 -- 172.21.15.170:0/1837296929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 msgr2=0x7f2570070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 --2- 172.21.15.170:0/1837296929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f2570070340 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f2560009840 tx=0x7f256001c820 comp rx=0 tx=0).stop 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 -- 172.21.15.170:0/1837296929 shutdown_connections 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 --2- 172.21.15.170:0/1837296929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f2570070340 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 -- 172.21.15.170:0/1837296929 >> 172.21.15.170:0/1837296929 conn(0x7f25700b1c40 msgr2=0x7f25700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 -- 172.21.15.170:0/1837296929 shutdown_connections 2024-09-21T02:21:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 -- 172.21.15.170:0/1837296929 wait complete. 2024-09-21T02:21:54.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 Processor -- start 2024-09-21T02:21:54.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.860+0000 7f2575693640 1 -- start start 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25700cb460 0x7f25701544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25700ce1c0 0x7f257009cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f257009d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f257016a2a0 con 0x7f25700cf7e0 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f257016a120 con 0x7f25700cb460 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f257016a420 con 0x7f25700ce1c0 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f257009d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f257009d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54670/0 (socket says 172.21.15.170:54670) 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 -- 172.21.15.170:0/3673165761 learned_addr learned my addr 172.21.15.170:0/3673165761 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 -- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25700ce1c0 msgr2=0x7f257009cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25700ce1c0 0x7f257009cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 -- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25700cb460 msgr2=0x7f25701544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256effd640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25700cb460 0x7f25701544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25700cb460 0x7f25701544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 -- 172.21.15.170:0/3673165761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25700af390 con 0x7f25700cf7e0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256f7fe640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f257009d4e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f256800ba10 tx=0x7f256800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568019070 con 0x7f25700cf7e0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f256800cc30 con 0x7f25700cf7e0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680146a0 con 0x7f25700cf7e0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25700af650 con 0x7f25700cf7e0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25700afc30 con 0x7f25700cf7e0 2024-09-21T02:21:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25701f9be0 con 0x7f25700cf7e0 2024-09-21T02:21:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f256801c070 con 0x7f25700cf7e0 2024-09-21T02:21:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2567fff640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2550073aa0 0x7f2550075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f25680925c0 con 0x7f25700cf7e0 2024-09-21T02:21:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.864+0000 7f256effd640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2550073aa0 0x7f2550075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:54.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.868+0000 7f256effd640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2550073aa0 0x7f2550075fb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f25600af000 tx=0x7f256001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:54.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:54.872+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f256800ce20 con 0x7f25700cf7e0 2024-09-21T02:21:54.999 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:21:55.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:21:55.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.028+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2570070340 con 0x7f2550073aa0 2024-09-21T02:21:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.036+0000 7f2567fff640 1 -- 172.21.15.170:0/3673165761 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f2570070340 con 0x7f2550073aa0 2024-09-21T02:21:55.039 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:55.041 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2550073aa0 msgr2=0x7f2550075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2550073aa0 0x7f2550075fb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f25600af000 tx=0x7f256001b060 comp rx=0 tx=0).stop 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 msgr2=0x7f257009d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f257009d4e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f256800ba10 tx=0x7f256800bed0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 shutdown_connections 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2550073aa0 0x7f2550075fb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25700ce1c0 0x7f257009cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25700cf7e0 0x7f257009d4e0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 --2- 172.21.15.170:0/3673165761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25700cb460 0x7f25701544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 >> 172.21.15.170:0/3673165761 conn(0x7f25700b1c40 msgr2=0x7f25700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 shutdown_connections 2024-09-21T02:21:55.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.040+0000 7f2575693640 1 -- 172.21.15.170:0/3673165761 wait complete. 2024-09-21T02:21:55.059 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-21T02:21:54.933315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294880,"kb_used_data":16232,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754679840,"statfs":{"total":773094113280,"available":772792156160,"internally_reserved":0,"allocated":16621568,"data_stored":7969228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773054+0000","last_change":"2024-09-21T02:21:50.931420+0000","last_active":"2024-09-21T02:21:52.773054+0000","last_peered":"2024-09-21T02:21:52.773054+0000","last_clean":"2024-09-21T02:21:52.773054+0000","last_became_active":"2024-09-21T02:21:50.931011+0000","last_became_peered":"2024-09-21T02:21:50.931011+0000","last_unstale":"2024-09-21T02:21:52.773054+0000","last_undegraded":"2024-09-21T02:21:52.773054+0000","last_fullsized":"2024-09-21T02:21:52.773054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:29.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879756+0000","last_change":"2024-09-21T02:21:50.886516+0000","last_active":"2024-09-21T02:21:51.879756+0000","last_peered":"2024-09-21T02:21:51.879756+0000","last_clean":"2024-09-21T02:21:51.879756+0000","last_became_active":"2024-09-21T02:21:50.885997+0000","last_became_peered":"2024-09-21T02:21:50.885997+0000","last_unstale":"2024-09-21T02:21:51.879756+0000","last_undegraded":"2024-09-21T02:21:51.879756+0000","last_fullsized":"2024-09-21T02:21:51.879756+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:36.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878602+0000","last_change":"2024-09-21T02:21:50.888430+0000","last_active":"2024-09-21T02:21:51.878602+0000","last_peered":"2024-09-21T02:21:51.878602+0000","last_clean":"2024-09-21T02:21:51.878602+0000","last_became_active":"2024-09-21T02:21:50.887325+0000","last_became_peered":"2024-09-21T02:21:50.887325+0000","last_unstale":"2024-09-21T02:21:51.878602+0000","last_undegraded":"2024-09-21T02:21:51.878602+0000","last_fullsized":"2024-09-21T02:21:51.878602+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:20.391904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879291+0000","last_change":"2024-09-21T02:21:50.909802+0000","last_active":"2024-09-21T02:21:51.879291+0000","last_peered":"2024-09-21T02:21:51.879291+0000","last_clean":"2024-09-21T02:21:51.879291+0000","last_became_active":"2024-09-21T02:21:50.909496+0000","last_became_peered":"2024-09-21T02:21:50.909496+0000","last_unstale":"2024-09-21T02:21:51.879291+0000","last_undegraded":"2024-09-21T02:21:51.879291+0000","last_fullsized":"2024-09-21T02:21:51.879291+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:53.617495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774632+0000","last_change":"2024-09-21T02:21:50.945220+0000","last_active":"2024-09-21T02:21:52.774632+0000","last_peered":"2024-09-21T02:21:52.774632+0000","last_clean":"2024-09-21T02:21:52.774632+0000","last_became_active":"2024-09-21T02:21:50.943890+0000","last_became_peered":"2024-09-21T02:21:50.943890+0000","last_unstale":"2024-09-21T02:21:52.774632+0000","last_undegraded":"2024-09-21T02:21:52.774632+0000","last_fullsized":"2024-09-21T02:21:52.774632+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:18.011149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772367+0000","last_change":"2024-09-21T02:21:50.952232+0000","last_active":"2024-09-21T02:21:52.772367+0000","last_peered":"2024-09-21T02:21:52.772367+0000","last_clean":"2024-09-21T02:21:52.772367+0000","last_became_active":"2024-09-21T02:21:50.951638+0000","last_became_peered":"2024-09-21T02:21:50.951638+0000","last_unstale":"2024-09-21T02:21:52.772367+0000","last_undegraded":"2024-09-21T02:21:52.772367+0000","last_fullsized":"2024-09-21T02:21:52.772367+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:27.875060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771595+0000","last_change":"2024-09-21T02:21:50.908683+0000","last_active":"2024-09-21T02:21:52.771595+0000","last_peered":"2024-09-21T02:21:52.771595+0000","last_clean":"2024-09-21T02:21:52.771595+0000","last_became_active":"2024-09-21T02:21:50.908270+0000","last_became_peered":"2024-09-21T02:21:50.908270+0000","last_unstale":"2024-09-21T02:21:52.771595+0000","last_undegraded":"2024-09-21T02:21:52.771595+0000","last_fullsized":"2024-09-21T02:21:52.771595+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:25:05.158877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772134+0000","last_change":"2024-09-21T02:21:50.926342+0000","last_active":"2024-09-21T02:21:52.772134+0000","last_peered":"2024-09-21T02:21:52.772134+0000","last_clean":"2024-09-21T02:21:52.772134+0000","last_became_active":"2024-09-21T02:21:50.925616+0000","last_became_peered":"2024-09-21T02:21:50.925616+0000","last_unstale":"2024-09-21T02:21:52.772134+0000","last_undegraded":"2024-09-21T02:21:52.772134+0000","last_fullsized":"2024-09-21T02:21:52.772134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:04.024540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878115+0000","last_change":"2024-09-21T02:21:50.881337+0000","last_active":"2024-09-21T02:21:51.878115+0000","last_peered":"2024-09-21T02:21:51.878115+0000","last_clean":"2024-09-21T02:21:51.878115+0000","last_became_active":"2024-09-21T02:21:50.880993+0000","last_became_peered":"2024-09-21T02:21:50.880993+0000","last_unstale":"2024-09-21T02:21:51.878115+0000","last_undegraded":"2024-09-21T02:21:51.878115+0000","last_fullsized":"2024-09-21T02:21:51.878115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:03.407808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775280+0000","last_change":"2024-09-21T02:21:50.932172+0000","last_active":"2024-09-21T02:21:52.775280+0000","last_peered":"2024-09-21T02:21:52.775280+0000","last_clean":"2024-09-21T02:21:52.775280+0000","last_became_active":"2024-09-21T02:21:50.931495+0000","last_became_peered":"2024-09-21T02:21:50.931495+0000","last_unstale":"2024-09-21T02:21:52.775280+0000","last_undegraded":"2024-09-21T02:21:52.775280+0000","last_fullsized":"2024-09-21T02:21:52.775280+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:30.787407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771530+0000","last_change":"2024-09-21T02:21:50.890482+0000","last_active":"2024-09-21T02:21:52.771530+0000","last_peered":"2024-09-21T02:21:52.771530+0000","last_clean":"2024-09-21T02:21:52.771530+0000","last_became_active":"2024-09-21T02:21:50.890057+0000","last_became_peered":"2024-09-21T02:21:50.890057+0000","last_unstale":"2024-09-21T02:21:52.771530+0000","last_undegraded":"2024-09-21T02:21:52.771530+0000","last_fullsized":"2024-09-21T02:21:52.771530+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:45.772234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876884+0000","last_change":"2024-09-21T02:21:50.892927+0000","last_active":"2024-09-21T02:21:51.876884+0000","last_peered":"2024-09-21T02:21:51.876884+0000","last_clean":"2024-09-21T02:21:51.876884+0000","last_became_active":"2024-09-21T02:21:50.892579+0000","last_became_peered":"2024-09-21T02:21:50.892579+0000","last_unstale":"2024-09-21T02:21:51.876884+0000","last_undegraded":"2024-09-21T02:21:51.876884+0000","last_fullsized":"2024-09-21T02:21:51.876884+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:43.485832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773222+0000","last_change":"2024-09-21T02:21:50.936793+0000","last_active":"2024-09-21T02:21:52.773222+0000","last_peered":"2024-09-21T02:21:52.773222+0000","last_clean":"2024-09-21T02:21:52.773222+0000","last_became_active":"2024-09-21T02:21:50.936482+0000","last_became_peered":"2024-09-21T02:21:50.936482+0000","last_unstale":"2024-09-21T02:21:52.773222+0000","last_undegraded":"2024-09-21T02:21:52.773222+0000","last_fullsized":"2024-09-21T02:21:52.773222+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:56.547832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878114+0000","last_change":"2024-09-21T02:21:50.931201+0000","last_active":"2024-09-21T02:21:51.878114+0000","last_peered":"2024-09-21T02:21:51.878114+0000","last_clean":"2024-09-21T02:21:51.878114+0000","last_became_active":"2024-09-21T02:21:50.930823+0000","last_became_peered":"2024-09-21T02:21:50.930823+0000","last_unstale":"2024-09-21T02:21:51.878114+0000","last_undegraded":"2024-09-21T02:21:51.878114+0000","last_fullsized":"2024-09-21T02:21:51.878114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:02.137050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772127+0000","last_change":"2024-09-21T02:21:50.891151+0000","last_active":"2024-09-21T02:21:52.772127+0000","last_peered":"2024-09-21T02:21:52.772127+0000","last_clean":"2024-09-21T02:21:52.772127+0000","last_became_active":"2024-09-21T02:21:50.890219+0000","last_became_peered":"2024-09-21T02:21:50.890219+0000","last_unstale":"2024-09-21T02:21:52.772127+0000","last_undegraded":"2024-09-21T02:21:52.772127+0000","last_fullsized":"2024-09-21T02:21:52.772127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:37.038252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877966+0000","last_change":"2024-09-21T02:21:50.885425+0000","last_active":"2024-09-21T02:21:51.877966+0000","last_peered":"2024-09-21T02:21:51.877966+0000","last_clean":"2024-09-21T02:21:51.877966+0000","last_became_active":"2024-09-21T02:21:50.884764+0000","last_became_peered":"2024-09-21T02:21:50.884764+0000","last_unstale":"2024-09-21T02:21:51.877966+0000","last_undegraded":"2024-09-21T02:21:51.877966+0000","last_fullsized":"2024-09-21T02:21:51.877966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:15.636349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773690+0000","last_change":"2024-09-21T02:21:50.943513+0000","last_active":"2024-09-21T02:21:52.773690+0000","last_peered":"2024-09-21T02:21:52.773690+0000","last_clean":"2024-09-21T02:21:52.773690+0000","last_became_active":"2024-09-21T02:21:50.942861+0000","last_became_peered":"2024-09-21T02:21:50.942861+0000","last_unstale":"2024-09-21T02:21:52.773690+0000","last_undegraded":"2024-09-21T02:21:52.773690+0000","last_fullsized":"2024-09-21T02:21:52.773690+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:04.889121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773481+0000","last_change":"2024-09-21T02:21:50.891329+0000","last_active":"2024-09-21T02:21:52.773481+0000","last_peered":"2024-09-21T02:21:52.773481+0000","last_clean":"2024-09-21T02:21:52.773481+0000","last_became_active":"2024-09-21T02:21:50.890996+0000","last_became_peered":"2024-09-21T02:21:50.890996+0000","last_unstale":"2024-09-21T02:21:52.773481+0000","last_undegraded":"2024-09-21T02:21:52.773481+0000","last_fullsized":"2024-09-21T02:21:52.773481+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:47.035210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878722+0000","last_change":"2024-09-21T02:21:50.882529+0000","last_active":"2024-09-21T02:21:51.878722+0000","last_peered":"2024-09-21T02:21:51.878722+0000","last_clean":"2024-09-21T02:21:51.878722+0000","last_became_active":"2024-09-21T02:21:50.881737+0000","last_became_peered":"2024-09-21T02:21:50.881737+0000","last_unstale":"2024-09-21T02:21:51.878722+0000","last_undegraded":"2024-09-21T02:21:51.878722+0000","last_fullsized":"2024-09-21T02:21:51.878722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:06.987021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775798+0000","last_change":"2024-09-21T02:21:50.948599+0000","last_active":"2024-09-21T02:21:52.775798+0000","last_peered":"2024-09-21T02:21:52.775798+0000","last_clean":"2024-09-21T02:21:52.775798+0000","last_became_active":"2024-09-21T02:21:50.944949+0000","last_became_peered":"2024-09-21T02:21:50.944949+0000","last_unstale":"2024-09-21T02:21:52.775798+0000","last_undegraded":"2024-09-21T02:21:52.775798+0000","last_fullsized":"2024-09-21T02:21:52.775798+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:26.378832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878370+0000","last_change":"2024-09-21T02:21:50.882144+0000","last_active":"2024-09-21T02:21:51.878370+0000","last_peered":"2024-09-21T02:21:51.878370+0000","last_clean":"2024-09-21T02:21:51.878370+0000","last_became_active":"2024-09-21T02:21:50.881100+0000","last_became_peered":"2024-09-21T02:21:50.881100+0000","last_unstale":"2024-09-21T02:21:51.878370+0000","last_undegraded":"2024-09-21T02:21:51.878370+0000","last_fullsized":"2024-09-21T02:21:51.878370+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.650025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774501+0000","last_change":"2024-09-21T02:21:50.948792+0000","last_active":"2024-09-21T02:21:52.774501+0000","last_peered":"2024-09-21T02:21:52.774501+0000","last_clean":"2024-09-21T02:21:52.774501+0000","last_became_active":"2024-09-21T02:21:50.944850+0000","last_became_peered":"2024-09-21T02:21:50.944850+0000","last_unstale":"2024-09-21T02:21:52.774501+0000","last_undegraded":"2024-09-21T02:21:52.774501+0000","last_fullsized":"2024-09-21T02:21:52.774501+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:09.901651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771134+0000","last_change":"2024-09-21T02:21:50.950690+0000","last_active":"2024-09-21T02:21:52.771134+0000","last_peered":"2024-09-21T02:21:52.771134+0000","last_clean":"2024-09-21T02:21:52.771134+0000","last_became_active":"2024-09-21T02:21:50.950185+0000","last_became_peered":"2024-09-21T02:21:50.950185+0000","last_unstale":"2024-09-21T02:21:52.771134+0000","last_undegraded":"2024-09-21T02:21:52.771134+0000","last_fullsized":"2024-09-21T02:21:52.771134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:39.110347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772391+0000","last_change":"2024-09-21T02:21:50.932062+0000","last_active":"2024-09-21T02:21:52.772391+0000","last_peered":"2024-09-21T02:21:52.772391+0000","last_clean":"2024-09-21T02:21:52.772391+0000","last_became_active":"2024-09-21T02:21:50.931337+0000","last_became_peered":"2024-09-21T02:21:50.931337+0000","last_unstale":"2024-09-21T02:21:52.772391+0000","last_undegraded":"2024-09-21T02:21:52.772391+0000","last_fullsized":"2024-09-21T02:21:52.772391+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:48.917825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.880475+0000","last_change":"2024-09-21T02:21:50.887104+0000","last_active":"2024-09-21T02:21:51.880475+0000","last_peered":"2024-09-21T02:21:51.880475+0000","last_clean":"2024-09-21T02:21:51.880475+0000","last_became_active":"2024-09-21T02:21:50.886773+0000","last_became_peered":"2024-09-21T02:21:50.886773+0000","last_unstale":"2024-09-21T02:21:51.880475+0000","last_undegraded":"2024-09-21T02:21:51.880475+0000","last_fullsized":"2024-09-21T02:21:51.880475+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:07.176400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876585+0000","last_change":"2024-09-21T02:21:50.891281+0000","last_active":"2024-09-21T02:21:51.876585+0000","last_peered":"2024-09-21T02:21:51.876585+0000","last_clean":"2024-09-21T02:21:51.876585+0000","last_became_active":"2024-09-21T02:21:50.890926+0000","last_became_peered":"2024-09-21T02:21:50.890926+0000","last_unstale":"2024-09-21T02:21:51.876585+0000","last_undegraded":"2024-09-21T02:21:51.876585+0000","last_fullsized":"2024-09-21T02:21:51.876585+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:49.814230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879260+0000","last_change":"2024-09-21T02:21:50.882309+0000","last_active":"2024-09-21T02:21:51.879260+0000","last_peered":"2024-09-21T02:21:51.879260+0000","last_clean":"2024-09-21T02:21:51.879260+0000","last_became_active":"2024-09-21T02:21:50.881367+0000","last_became_peered":"2024-09-21T02:21:50.881367+0000","last_unstale":"2024-09-21T02:21:51.879260+0000","last_undegraded":"2024-09-21T02:21:51.879260+0000","last_fullsized":"2024-09-21T02:21:51.879260+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:25.666140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774607+0000","last_change":"2024-09-21T02:21:50.936522+0000","last_active":"2024-09-21T02:21:52.774607+0000","last_peered":"2024-09-21T02:21:52.774607+0000","last_clean":"2024-09-21T02:21:52.774607+0000","last_became_active":"2024-09-21T02:21:50.934679+0000","last_became_peered":"2024-09-21T02:21:50.934679+0000","last_unstale":"2024-09-21T02:21:52.774607+0000","last_undegraded":"2024-09-21T02:21:52.774607+0000","last_fullsized":"2024-09-21T02:21:52.774607+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:47.336663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876096+0000","last_change":"2024-09-21T02:21:50.934226+0000","last_active":"2024-09-21T02:21:51.876096+0000","last_peered":"2024-09-21T02:21:51.876096+0000","last_clean":"2024-09-21T02:21:51.876096+0000","last_became_active":"2024-09-21T02:21:50.933960+0000","last_became_peered":"2024-09-21T02:21:50.933960+0000","last_unstale":"2024-09-21T02:21:51.876096+0000","last_undegraded":"2024-09-21T02:21:51.876096+0000","last_fullsized":"2024-09-21T02:21:51.876096+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:59.985384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876353+0000","last_change":"2024-09-21T02:21:50.887932+0000","last_active":"2024-09-21T02:21:51.876353+0000","last_peered":"2024-09-21T02:21:51.876353+0000","last_clean":"2024-09-21T02:21:51.876353+0000","last_became_active":"2024-09-21T02:21:50.887600+0000","last_became_peered":"2024-09-21T02:21:50.887600+0000","last_unstale":"2024-09-21T02:21:51.876353+0000","last_undegraded":"2024-09-21T02:21:51.876353+0000","last_fullsized":"2024-09-21T02:21:51.876353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:13.449111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773186+0000","last_change":"2024-09-21T02:21:50.952058+0000","last_active":"2024-09-21T02:21:52.773186+0000","last_peered":"2024-09-21T02:21:52.773186+0000","last_clean":"2024-09-21T02:21:52.773186+0000","last_became_active":"2024-09-21T02:21:50.948466+0000","last_became_peered":"2024-09-21T02:21:50.948466+0000","last_unstale":"2024-09-21T02:21:52.773186+0000","last_undegraded":"2024-09-21T02:21:52.773186+0000","last_fullsized":"2024-09-21T02:21:52.773186+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:03.814658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772785+0000","last_change":"2024-09-21T02:21:50.939975+0000","last_active":"2024-09-21T02:21:52.772785+0000","last_peered":"2024-09-21T02:21:52.772785+0000","last_clean":"2024-09-21T02:21:52.772785+0000","last_became_active":"2024-09-21T02:21:50.939313+0000","last_became_peered":"2024-09-21T02:21:50.939313+0000","last_unstale":"2024-09-21T02:21:52.772785+0000","last_undegraded":"2024-09-21T02:21:52.772785+0000","last_fullsized":"2024-09-21T02:21:52.772785+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:47.035210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776982+0000","last_change":"2024-09-21T02:21:50.943255+0000","last_active":"2024-09-21T02:21:52.776982+0000","last_peered":"2024-09-21T02:21:52.776982+0000","last_clean":"2024-09-21T02:21:52.776982+0000","last_became_active":"2024-09-21T02:21:50.940007+0000","last_became_peered":"2024-09-21T02:21:50.940007+0000","last_unstale":"2024-09-21T02:21:52.776982+0000","last_undegraded":"2024-09-21T02:21:52.776982+0000","last_fullsized":"2024-09-21T02:21:52.776982+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.808475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"152'4","reported_seq":19,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879535+0000","last_change":"2024-09-21T02:21:49.890527+0000","last_active":"2024-09-21T02:21:51.879535+0000","last_peered":"2024-09-21T02:21:51.879535+0000","last_clean":"2024-09-21T02:21:51.879535+0000","last_became_active":"2024-09-21T02:21:49.889683+0000","last_became_peered":"2024-09-21T02:21:49.889683+0000","last_unstale":"2024-09-21T02:21:51.879535+0000","last_undegraded":"2024-09-21T02:21:51.879535+0000","last_fullsized":"2024-09-21T02:21:51.879535+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:08.897765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775823+0000","last_change":"2024-09-21T02:21:50.947004+0000","last_active":"2024-09-21T02:21:52.775823+0000","last_peered":"2024-09-21T02:21:52.775823+0000","last_clean":"2024-09-21T02:21:52.775823+0000","last_became_active":"2024-09-21T02:21:50.946289+0000","last_became_peered":"2024-09-21T02:21:50.946289+0000","last_unstale":"2024-09-21T02:21:52.775823+0000","last_undegraded":"2024-09-21T02:21:52.775823+0000","last_fullsized":"2024-09-21T02:21:52.775823+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:40.626169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876857+0000","last_change":"2024-09-21T02:21:49.899999+0000","last_active":"2024-09-21T02:21:51.876857+0000","last_peered":"2024-09-21T02:21:51.876857+0000","last_clean":"2024-09-21T02:21:51.876857+0000","last_became_active":"2024-09-21T02:21:49.899531+0000","last_became_peered":"2024-09-21T02:21:49.899531+0000","last_unstale":"2024-09-21T02:21:51.876857+0000","last_undegraded":"2024-09-21T02:21:51.876857+0000","last_fullsized":"2024-09-21T02:21:51.876857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:06.666382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879900+0000","last_change":"2024-09-21T02:21:50.888279+0000","last_active":"2024-09-21T02:21:51.879900+0000","last_peered":"2024-09-21T02:21:51.879900+0000","last_clean":"2024-09-21T02:21:51.879900+0000","last_became_active":"2024-09-21T02:21:50.887854+0000","last_became_peered":"2024-09-21T02:21:50.887854+0000","last_unstale":"2024-09-21T02:21:51.879900+0000","last_undegraded":"2024-09-21T02:21:51.879900+0000","last_fullsized":"2024-09-21T02:21:51.879900+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:01.859733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776588+0000","last_change":"2024-09-21T02:21:49.923894+0000","last_active":"2024-09-21T02:21:52.776588+0000","last_peered":"2024-09-21T02:21:52.776588+0000","last_clean":"2024-09-21T02:21:52.776588+0000","last_became_active":"2024-09-21T02:21:49.922968+0000","last_became_peered":"2024-09-21T02:21:49.922968+0000","last_unstale":"2024-09-21T02:21:52.776588+0000","last_undegraded":"2024-09-21T02:21:52.776588+0000","last_fullsized":"2024-09-21T02:21:52.776588+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:57.407489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879547+0000","last_change":"2024-09-21T02:21:50.923063+0000","last_active":"2024-09-21T02:21:51.879547+0000","last_peered":"2024-09-21T02:21:51.879547+0000","last_clean":"2024-09-21T02:21:51.879547+0000","last_became_active":"2024-09-21T02:21:50.922685+0000","last_became_peered":"2024-09-21T02:21:50.922685+0000","last_unstale":"2024-09-21T02:21:51.879547+0000","last_undegraded":"2024-09-21T02:21:51.879547+0000","last_fullsized":"2024-09-21T02:21:51.879547+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:18.593576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877367+0000","last_change":"2024-09-21T02:21:49.900228+0000","last_active":"2024-09-21T02:21:51.877367+0000","last_peered":"2024-09-21T02:21:51.877367+0000","last_clean":"2024-09-21T02:21:51.877367+0000","last_became_active":"2024-09-21T02:21:49.899791+0000","last_became_peered":"2024-09-21T02:21:49.899791+0000","last_unstale":"2024-09-21T02:21:51.877367+0000","last_undegraded":"2024-09-21T02:21:51.877367+0000","last_fullsized":"2024-09-21T02:21:51.877367+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:03.020654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772960+0000","last_change":"2024-09-21T02:21:50.949467+0000","last_active":"2024-09-21T02:21:52.772960+0000","last_peered":"2024-09-21T02:21:52.772960+0000","last_clean":"2024-09-21T02:21:52.772960+0000","last_became_active":"2024-09-21T02:21:50.948824+0000","last_became_peered":"2024-09-21T02:21:50.948824+0000","last_unstale":"2024-09-21T02:21:52.772960+0000","last_undegraded":"2024-09-21T02:21:52.772960+0000","last_fullsized":"2024-09-21T02:21:52.772960+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:30.375733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775216+0000","last_change":"2024-09-21T02:21:49.918189+0000","last_active":"2024-09-21T02:21:52.775216+0000","last_peered":"2024-09-21T02:21:52.775216+0000","last_clean":"2024-09-21T02:21:52.775216+0000","last_became_active":"2024-09-21T02:21:49.917375+0000","last_became_peered":"2024-09-21T02:21:49.917375+0000","last_unstale":"2024-09-21T02:21:52.775216+0000","last_undegraded":"2024-09-21T02:21:52.775216+0000","last_fullsized":"2024-09-21T02:21:52.775216+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:31.859793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775337+0000","last_change":"2024-09-21T02:21:50.947633+0000","last_active":"2024-09-21T02:21:52.775337+0000","last_peered":"2024-09-21T02:21:52.775337+0000","last_clean":"2024-09-21T02:21:52.775337+0000","last_became_active":"2024-09-21T02:21:50.944395+0000","last_became_peered":"2024-09-21T02:21:50.944395+0000","last_unstale":"2024-09-21T02:21:52.775337+0000","last_undegraded":"2024-09-21T02:21:52.775337+0000","last_fullsized":"2024-09-21T02:21:52.775337+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:01.053341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774970+0000","last_change":"2024-09-21T02:21:49.914067+0000","last_active":"2024-09-21T02:21:52.774970+0000","last_peered":"2024-09-21T02:21:52.774970+0000","last_clean":"2024-09-21T02:21:52.774970+0000","last_became_active":"2024-09-21T02:21:49.913102+0000","last_became_peered":"2024-09-21T02:21:49.913102+0000","last_unstale":"2024-09-21T02:21:52.774970+0000","last_undegraded":"2024-09-21T02:21:52.774970+0000","last_fullsized":"2024-09-21T02:21:52.774970+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:37.161206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772564+0000","last_change":"2024-09-21T02:21:50.900120+0000","last_active":"2024-09-21T02:21:52.772564+0000","last_peered":"2024-09-21T02:21:52.772564+0000","last_clean":"2024-09-21T02:21:52.772564+0000","last_became_active":"2024-09-21T02:21:50.899723+0000","last_became_peered":"2024-09-21T02:21:50.899723+0000","last_unstale":"2024-09-21T02:21:52.772564+0000","last_undegraded":"2024-09-21T02:21:52.772564+0000","last_fullsized":"2024-09-21T02:21:52.772564+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:35.407486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771325+0000","last_change":"2024-09-21T02:21:49.915755+0000","last_active":"2024-09-21T02:21:52.771325+0000","last_peered":"2024-09-21T02:21:52.771325+0000","last_clean":"2024-09-21T02:21:52.771325+0000","last_became_active":"2024-09-21T02:21:49.915293+0000","last_became_peered":"2024-09-21T02:21:49.915293+0000","last_unstale":"2024-09-21T02:21:52.771325+0000","last_undegraded":"2024-09-21T02:21:52.771325+0000","last_fullsized":"2024-09-21T02:21:52.771325+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:23.100912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879427+0000","last_change":"2024-09-21T02:21:50.922402+0000","last_active":"2024-09-21T02:21:51.879427+0000","last_peered":"2024-09-21T02:21:51.879427+0000","last_clean":"2024-09-21T02:21:51.879427+0000","last_became_active":"2024-09-21T02:21:50.921527+0000","last_became_peered":"2024-09-21T02:21:50.921527+0000","last_unstale":"2024-09-21T02:21:51.879427+0000","last_undegraded":"2024-09-21T02:21:51.879427+0000","last_fullsized":"2024-09-21T02:21:51.879427+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:16.556388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773059+0000","last_change":"2024-09-21T02:21:49.895696+0000","last_active":"2024-09-21T02:21:52.773059+0000","last_peered":"2024-09-21T02:21:52.773059+0000","last_clean":"2024-09-21T02:21:52.773059+0000","last_became_active":"2024-09-21T02:21:49.894887+0000","last_became_peered":"2024-09-21T02:21:49.894887+0000","last_unstale":"2024-09-21T02:21:52.773059+0000","last_undegraded":"2024-09-21T02:21:52.773059+0000","last_fullsized":"2024-09-21T02:21:52.773059+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:16.327882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776030+0000","last_change":"2024-09-21T02:21:50.932260+0000","last_active":"2024-09-21T02:21:52.776030+0000","last_peered":"2024-09-21T02:21:52.776030+0000","last_clean":"2024-09-21T02:21:52.776030+0000","last_became_active":"2024-09-21T02:21:50.929555+0000","last_became_peered":"2024-09-21T02:21:50.929555+0000","last_unstale":"2024-09-21T02:21:52.776030+0000","last_undegraded":"2024-09-21T02:21:52.776030+0000","last_fullsized":"2024-09-21T02:21:52.776030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:44.390759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776104+0000","last_change":"2024-09-21T02:21:49.902786+0000","last_active":"2024-09-21T02:21:52.776104+0000","last_peered":"2024-09-21T02:21:52.776104+0000","last_clean":"2024-09-21T02:21:52.776104+0000","last_became_active":"2024-09-21T02:21:49.901771+0000","last_became_peered":"2024-09-21T02:21:49.901771+0000","last_unstale":"2024-09-21T02:21:52.776104+0000","last_undegraded":"2024-09-21T02:21:52.776104+0000","last_fullsized":"2024-09-21T02:21:52.776104+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:06.829878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877327+0000","last_change":"2024-09-21T02:21:50.892428+0000","last_active":"2024-09-21T02:21:51.877327+0000","last_peered":"2024-09-21T02:21:51.877327+0000","last_clean":"2024-09-21T02:21:51.877327+0000","last_became_active":"2024-09-21T02:21:50.892019+0000","last_became_peered":"2024-09-21T02:21:50.892019+0000","last_unstale":"2024-09-21T02:21:51.877327+0000","last_undegraded":"2024-09-21T02:21:51.877327+0000","last_fullsized":"2024-09-21T02:21:51.877327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.650025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"152'3","reported_seq":18,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776788+0000","last_change":"2024-09-21T02:21:49.922091+0000","last_active":"2024-09-21T02:21:52.776788+0000","last_peered":"2024-09-21T02:21:52.776788+0000","last_clean":"2024-09-21T02:21:52.776788+0000","last_became_active":"2024-09-21T02:21:49.921264+0000","last_became_peered":"2024-09-21T02:21:49.921264+0000","last_unstale":"2024-09-21T02:21:52.776788+0000","last_undegraded":"2024-09-21T02:21:52.776788+0000","last_fullsized":"2024-09-21T02:21:52.776788+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:40.077793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775378+0000","last_change":"2024-09-21T02:21:50.931450+0000","last_active":"2024-09-21T02:21:52.775378+0000","last_peered":"2024-09-21T02:21:52.775378+0000","last_clean":"2024-09-21T02:21:52.775378+0000","last_became_active":"2024-09-21T02:21:50.930828+0000","last_became_peered":"2024-09-21T02:21:50.930828+0000","last_unstale":"2024-09-21T02:21:52.775378+0000","last_undegraded":"2024-09-21T02:21:52.775378+0000","last_fullsized":"2024-09-21T02:21:52.775378+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.562930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"152'6","reported_seq":21,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776277+0000","last_change":"2024-09-21T02:21:49.918733+0000","last_active":"2024-09-21T02:21:52.776277+0000","last_peered":"2024-09-21T02:21:52.776277+0000","last_clean":"2024-09-21T02:21:52.776277+0000","last_became_active":"2024-09-21T02:21:49.918026+0000","last_became_peered":"2024-09-21T02:21:49.918026+0000","last_unstale":"2024-09-21T02:21:52.776277+0000","last_undegraded":"2024-09-21T02:21:52.776277+0000","last_fullsized":"2024-09-21T02:21:52.776277+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:26.005577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772141+0000","last_change":"2024-09-21T02:21:50.954218+0000","last_active":"2024-09-21T02:21:52.772141+0000","last_peered":"2024-09-21T02:21:52.772141+0000","last_clean":"2024-09-21T02:21:52.772141+0000","last_became_active":"2024-09-21T02:21:50.953549+0000","last_became_peered":"2024-09-21T02:21:50.953549+0000","last_unstale":"2024-09-21T02:21:52.772141+0000","last_undegraded":"2024-09-21T02:21:52.772141+0000","last_fullsized":"2024-09-21T02:21:52.772141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:38.135408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876836+0000","last_change":"2024-09-21T02:21:49.894254+0000","last_active":"2024-09-21T02:21:51.876836+0000","last_peered":"2024-09-21T02:21:51.876836+0000","last_clean":"2024-09-21T02:21:51.876836+0000","last_became_active":"2024-09-21T02:21:49.893836+0000","last_became_peered":"2024-09-21T02:21:49.893836+0000","last_unstale":"2024-09-21T02:21:51.876836+0000","last_undegraded":"2024-09-21T02:21:51.876836+0000","last_fullsized":"2024-09-21T02:21:51.876836+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:25:04.154991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772187+0000","last_change":"2024-09-21T02:21:50.922390+0000","last_active":"2024-09-21T02:21:52.772187+0000","last_peered":"2024-09-21T02:21:52.772187+0000","last_clean":"2024-09-21T02:21:52.772187+0000","last_became_active":"2024-09-21T02:21:50.922104+0000","last_became_peered":"2024-09-21T02:21:50.922104+0000","last_unstale":"2024-09-21T02:21:52.772187+0000","last_undegraded":"2024-09-21T02:21:52.772187+0000","last_fullsized":"2024-09-21T02:21:52.772187+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:04.024540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772004+0000","last_change":"2024-09-21T02:21:49.920371+0000","last_active":"2024-09-21T02:21:52.772004+0000","last_peered":"2024-09-21T02:21:52.772004+0000","last_clean":"2024-09-21T02:21:52.772004+0000","last_became_active":"2024-09-21T02:21:49.919578+0000","last_became_peered":"2024-09-21T02:21:49.919578+0000","last_unstale":"2024-09-21T02:21:52.772004+0000","last_undegraded":"2024-09-21T02:21:52.772004+0000","last_fullsized":"2024-09-21T02:21:52.772004+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:49.573706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876108+0000","last_change":"2024-09-21T02:21:50.910973+0000","last_active":"2024-09-21T02:21:51.876108+0000","last_peered":"2024-09-21T02:21:51.876108+0000","last_clean":"2024-09-21T02:21:51.876108+0000","last_became_active":"2024-09-21T02:21:50.910642+0000","last_became_peered":"2024-09-21T02:21:50.910642+0000","last_unstale":"2024-09-21T02:21:51.876108+0000","last_undegraded":"2024-09-21T02:21:51.876108+0000","last_fullsized":"2024-09-21T02:21:51.876108+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.808475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776659+0000","last_change":"2024-09-21T02:21:49.922108+0000","last_active":"2024-09-21T02:21:52.776659+0000","last_peered":"2024-09-21T02:21:52.776659+0000","last_clean":"2024-09-21T02:21:52.776659+0000","last_became_active":"2024-09-21T02:21:49.921355+0000","last_became_peered":"2024-09-21T02:21:49.921355+0000","last_unstale":"2024-09-21T02:21:52.776659+0000","last_undegraded":"2024-09-21T02:21:52.776659+0000","last_fullsized":"2024-09-21T02:21:52.776659+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:40.068577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773875+0000","last_change":"2024-09-21T02:21:50.949081+0000","last_active":"2024-09-21T02:21:52.773875+0000","last_peered":"2024-09-21T02:21:52.773875+0000","last_clean":"2024-09-21T02:21:52.773875+0000","last_became_active":"2024-09-21T02:21:50.943163+0000","last_became_peered":"2024-09-21T02:21:50.943163+0000","last_unstale":"2024-09-21T02:21:52.773875+0000","last_undegraded":"2024-09-21T02:21:52.773875+0000","last_fullsized":"2024-09-21T02:21:52.773875+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:26.960103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"152'8","reported_seq":23,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771647+0000","last_change":"2024-09-21T02:21:49.891653+0000","last_active":"2024-09-21T02:21:52.771647+0000","last_peered":"2024-09-21T02:21:52.771647+0000","last_clean":"2024-09-21T02:21:52.771647+0000","last_became_active":"2024-09-21T02:21:49.890965+0000","last_became_peered":"2024-09-21T02:21:49.890965+0000","last_unstale":"2024-09-21T02:21:52.771647+0000","last_undegraded":"2024-09-21T02:21:52.771647+0000","last_fullsized":"2024-09-21T02:21:52.771647+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:04.299884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775901+0000","last_change":"2024-09-21T02:21:50.931980+0000","last_active":"2024-09-21T02:21:52.775901+0000","last_peered":"2024-09-21T02:21:52.775901+0000","last_clean":"2024-09-21T02:21:52.775901+0000","last_became_active":"2024-09-21T02:21:50.931192+0000","last_became_peered":"2024-09-21T02:21:50.931192+0000","last_unstale":"2024-09-21T02:21:52.775901+0000","last_undegraded":"2024-09-21T02:21:52.775901+0000","last_fullsized":"2024-09-21T02:21:52.775901+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.867901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774717+0000","last_change":"2024-09-21T02:21:49.909749+0000","last_active":"2024-09-21T02:21:52.774717+0000","last_peered":"2024-09-21T02:21:52.774717+0000","last_clean":"2024-09-21T02:21:52.774717+0000","last_became_active":"2024-09-21T02:21:49.908867+0000","last_became_peered":"2024-09-21T02:21:49.908867+0000","last_unstale":"2024-09-21T02:21:52.774717+0000","last_undegraded":"2024-09-21T02:21:52.774717+0000","last_fullsized":"2024-09-21T02:21:52.774717+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:15.397198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878458+0000","last_change":"2024-09-21T02:21:50.896972+0000","last_active":"2024-09-21T02:21:51.878458+0000","last_peered":"2024-09-21T02:21:51.878458+0000","last_clean":"2024-09-21T02:21:51.878458+0000","last_became_active":"2024-09-21T02:21:50.896517+0000","last_became_peered":"2024-09-21T02:21:50.896517+0000","last_unstale":"2024-09-21T02:21:51.878458+0000","last_undegraded":"2024-09-21T02:21:51.878458+0000","last_fullsized":"2024-09-21T02:21:51.878458+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:20.391904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773827+0000","last_change":"2024-09-21T02:21:49.889518+0000","last_active":"2024-09-21T02:21:52.773827+0000","last_peered":"2024-09-21T02:21:52.773827+0000","last_clean":"2024-09-21T02:21:52.773827+0000","last_became_active":"2024-09-21T02:21:49.888574+0000","last_became_peered":"2024-09-21T02:21:49.888574+0000","last_unstale":"2024-09-21T02:21:52.773827+0000","last_undegraded":"2024-09-21T02:21:52.773827+0000","last_fullsized":"2024-09-21T02:21:52.773827+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:46.093955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.880023+0000","last_change":"2024-09-21T02:21:50.881983+0000","last_active":"2024-09-21T02:21:51.880023+0000","last_peered":"2024-09-21T02:21:51.880023+0000","last_clean":"2024-09-21T02:21:51.880023+0000","last_became_active":"2024-09-21T02:21:50.881541+0000","last_became_peered":"2024-09-21T02:21:50.881541+0000","last_unstale":"2024-09-21T02:21:51.880023+0000","last_undegraded":"2024-09-21T02:21:51.880023+0000","last_fullsized":"2024-09-21T02:21:51.880023+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:30.876175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772019+0000","last_change":"2024-09-21T02:21:49.892079+0000","last_active":"2024-09-21T02:21:52.772019+0000","last_peered":"2024-09-21T02:21:52.772019+0000","last_clean":"2024-09-21T02:21:52.772019+0000","last_became_active":"2024-09-21T02:21:49.891687+0000","last_became_peered":"2024-09-21T02:21:49.891687+0000","last_unstale":"2024-09-21T02:21:52.772019+0000","last_undegraded":"2024-09-21T02:21:52.772019+0000","last_fullsized":"2024-09-21T02:21:52.772019+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:53.782747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876884+0000","last_change":"2024-09-21T02:21:50.887537+0000","last_active":"2024-09-21T02:21:51.876884+0000","last_peered":"2024-09-21T02:21:51.876884+0000","last_clean":"2024-09-21T02:21:51.876884+0000","last_became_active":"2024-09-21T02:21:50.887213+0000","last_became_peered":"2024-09-21T02:21:50.887213+0000","last_unstale":"2024-09-21T02:21:51.876884+0000","last_undegraded":"2024-09-21T02:21:51.876884+0000","last_fullsized":"2024-09-21T02:21:51.876884+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:15.636349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771498+0000","last_change":"2024-09-21T02:21:49.898568+0000","last_active":"2024-09-21T02:21:52.771498+0000","last_peered":"2024-09-21T02:21:52.771498+0000","last_clean":"2024-09-21T02:21:52.771498+0000","last_became_active":"2024-09-21T02:21:49.897645+0000","last_became_peered":"2024-09-21T02:21:49.897645+0000","last_unstale":"2024-09-21T02:21:52.771498+0000","last_undegraded":"2024-09-21T02:21:52.771498+0000","last_fullsized":"2024-09-21T02:21:52.771498+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:49.171053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773033+0000","last_change":"2024-09-21T02:21:50.920935+0000","last_active":"2024-09-21T02:21:52.773033+0000","last_peered":"2024-09-21T02:21:52.773033+0000","last_clean":"2024-09-21T02:21:52.773033+0000","last_became_active":"2024-09-21T02:21:50.920016+0000","last_became_peered":"2024-09-21T02:21:50.920016+0000","last_unstale":"2024-09-21T02:21:52.773033+0000","last_undegraded":"2024-09-21T02:21:52.773033+0000","last_fullsized":"2024-09-21T02:21:52.773033+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:11.851393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775570+0000","last_change":"2024-09-21T02:21:49.918276+0000","last_active":"2024-09-21T02:21:52.775570+0000","last_peered":"2024-09-21T02:21:52.775570+0000","last_clean":"2024-09-21T02:21:52.775570+0000","last_became_active":"2024-09-21T02:21:49.917548+0000","last_became_peered":"2024-09-21T02:21:49.917548+0000","last_unstale":"2024-09-21T02:21:52.775570+0000","last_undegraded":"2024-09-21T02:21:52.775570+0000","last_fullsized":"2024-09-21T02:21:52.775570+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:06.308291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772548+0000","last_change":"2024-09-21T02:21:50.936808+0000","last_active":"2024-09-21T02:21:52.772548+0000","last_peered":"2024-09-21T02:21:52.772548+0000","last_clean":"2024-09-21T02:21:52.772548+0000","last_became_active":"2024-09-21T02:21:50.936471+0000","last_became_peered":"2024-09-21T02:21:50.936471+0000","last_unstale":"2024-09-21T02:21:52.772548+0000","last_undegraded":"2024-09-21T02:21:52.772548+0000","last_fullsized":"2024-09-21T02:21:52.772548+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:48.917825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773465+0000","last_change":"2024-09-21T02:21:49.894700+0000","last_active":"2024-09-21T02:21:52.773465+0000","last_peered":"2024-09-21T02:21:52.773465+0000","last_clean":"2024-09-21T02:21:52.773465+0000","last_became_active":"2024-09-21T02:21:49.893021+0000","last_became_peered":"2024-09-21T02:21:49.893021+0000","last_unstale":"2024-09-21T02:21:52.773465+0000","last_undegraded":"2024-09-21T02:21:52.773465+0000","last_fullsized":"2024-09-21T02:21:52.773465+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.556540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771082+0000","last_change":"2024-09-21T02:21:50.893600+0000","last_active":"2024-09-21T02:21:52.771082+0000","last_peered":"2024-09-21T02:21:52.771082+0000","last_clean":"2024-09-21T02:21:52.771082+0000","last_became_active":"2024-09-21T02:21:50.893232+0000","last_became_peered":"2024-09-21T02:21:50.893232+0000","last_unstale":"2024-09-21T02:21:52.771082+0000","last_undegraded":"2024-09-21T02:21:52.771082+0000","last_fullsized":"2024-09-21T02:21:52.771082+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:37.767974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772378+0000","last_change":"2024-09-21T02:21:49.903683+0000","last_active":"2024-09-21T02:21:52.772378+0000","last_peered":"2024-09-21T02:21:52.772378+0000","last_clean":"2024-09-21T02:21:52.772378+0000","last_became_active":"2024-09-21T02:21:49.902756+0000","last_became_peered":"2024-09-21T02:21:49.902756+0000","last_unstale":"2024-09-21T02:21:52.772378+0000","last_undegraded":"2024-09-21T02:21:52.772378+0000","last_fullsized":"2024-09-21T02:21:52.772378+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:40.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771149+0000","last_change":"2024-09-21T02:21:50.920837+0000","last_active":"2024-09-21T02:21:52.771149+0000","last_peered":"2024-09-21T02:21:52.771149+0000","last_clean":"2024-09-21T02:21:52.771149+0000","last_became_active":"2024-09-21T02:21:50.920076+0000","last_became_peered":"2024-09-21T02:21:50.920076+0000","last_unstale":"2024-09-21T02:21:52.771149+0000","last_undegraded":"2024-09-21T02:21:52.771149+0000","last_fullsized":"2024-09-21T02:21:52.771149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.466439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771599+0000","last_change":"2024-09-21T02:21:49.893349+0000","last_active":"2024-09-21T02:21:52.771599+0000","last_peered":"2024-09-21T02:21:52.771599+0000","last_clean":"2024-09-21T02:21:52.771599+0000","last_became_active":"2024-09-21T02:21:49.892960+0000","last_became_peered":"2024-09-21T02:21:49.892960+0000","last_unstale":"2024-09-21T02:21:52.771599+0000","last_undegraded":"2024-09-21T02:21:52.771599+0000","last_fullsized":"2024-09-21T02:21:52.771599+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:57.296742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879710+0000","last_change":"2024-09-21T02:21:50.891057+0000","last_active":"2024-09-21T02:21:51.879710+0000","last_peered":"2024-09-21T02:21:51.879710+0000","last_clean":"2024-09-21T02:21:51.879710+0000","last_became_active":"2024-09-21T02:21:50.890394+0000","last_became_peered":"2024-09-21T02:21:50.890394+0000","last_unstale":"2024-09-21T02:21:51.879710+0000","last_undegraded":"2024-09-21T02:21:51.879710+0000","last_fullsized":"2024-09-21T02:21:51.879710+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:33.259929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773362+0000","last_change":"2024-09-21T02:21:49.896095+0000","last_active":"2024-09-21T02:21:52.773362+0000","last_peered":"2024-09-21T02:21:52.773362+0000","last_clean":"2024-09-21T02:21:52.773362+0000","last_became_active":"2024-09-21T02:21:49.895284+0000","last_became_peered":"2024-09-21T02:21:49.895284+0000","last_unstale":"2024-09-21T02:21:52.773362+0000","last_undegraded":"2024-09-21T02:21:52.773362+0000","last_fullsized":"2024-09-21T02:21:52.773362+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:37.085768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878919+0000","last_change":"2024-09-21T02:21:50.893236+0000","last_active":"2024-09-21T02:21:51.878919+0000","last_peered":"2024-09-21T02:21:51.878919+0000","last_clean":"2024-09-21T02:21:51.878919+0000","last_became_active":"2024-09-21T02:21:50.889612+0000","last_became_peered":"2024-09-21T02:21:50.889612+0000","last_unstale":"2024-09-21T02:21:51.878919+0000","last_undegraded":"2024-09-21T02:21:51.878919+0000","last_fullsized":"2024-09-21T02:21:51.878919+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:19.557641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"153'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879995+0000","last_change":"2024-09-21T02:21:49.903670+0000","last_active":"2024-09-21T02:21:51.879995+0000","last_peered":"2024-09-21T02:21:51.879995+0000","last_clean":"2024-09-21T02:21:51.879995+0000","last_became_active":"2024-09-21T02:21:49.902913+0000","last_became_peered":"2024-09-21T02:21:49.902913+0000","last_unstale":"2024-09-21T02:21:51.879995+0000","last_undegraded":"2024-09-21T02:21:51.879995+0000","last_fullsized":"2024-09-21T02:21:51.879995+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:02.810772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878954+0000","last_change":"2024-09-21T02:21:50.896611+0000","last_active":"2024-09-21T02:21:51.878954+0000","last_peered":"2024-09-21T02:21:51.878954+0000","last_clean":"2024-09-21T02:21:51.878954+0000","last_became_active":"2024-09-21T02:21:50.895606+0000","last_became_peered":"2024-09-21T02:21:50.895606+0000","last_unstale":"2024-09-21T02:21:51.878954+0000","last_undegraded":"2024-09-21T02:21:51.878954+0000","last_fullsized":"2024-09-21T02:21:51.878954+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:10.510035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879289+0000","last_change":"2024-09-21T02:21:49.905594+0000","last_active":"2024-09-21T02:21:51.879289+0000","last_peered":"2024-09-21T02:21:51.879289+0000","last_clean":"2024-09-21T02:21:51.879289+0000","last_became_active":"2024-09-21T02:21:49.904826+0000","last_became_peered":"2024-09-21T02:21:49.904826+0000","last_unstale":"2024-09-21T02:21:51.879289+0000","last_undegraded":"2024-09-21T02:21:51.879289+0000","last_fullsized":"2024-09-21T02:21:51.879289+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:25.956217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776100+0000","last_change":"2024-09-21T02:21:50.944530+0000","last_active":"2024-09-21T02:21:52.776100+0000","last_peered":"2024-09-21T02:21:52.776100+0000","last_clean":"2024-09-21T02:21:52.776100+0000","last_became_active":"2024-09-21T02:21:50.943461+0000","last_became_peered":"2024-09-21T02:21:50.943461+0000","last_unstale":"2024-09-21T02:21:52.776100+0000","last_undegraded":"2024-09-21T02:21:52.776100+0000","last_fullsized":"2024-09-21T02:21:52.776100+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:59.985384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776511+0000","last_change":"2024-09-21T02:21:49.918888+0000","last_active":"2024-09-21T02:21:52.776511+0000","last_peered":"2024-09-21T02:21:52.776511+0000","last_clean":"2024-09-21T02:21:52.776511+0000","last_became_active":"2024-09-21T02:21:49.918074+0000","last_became_peered":"2024-09-21T02:21:49.918074+0000","last_unstale":"2024-09-21T02:21:52.776511+0000","last_undegraded":"2024-09-21T02:21:52.776511+0000","last_fullsized":"2024-09-21T02:21:52.776511+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:47.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878729+0000","last_change":"2024-09-21T02:21:50.915579+0000","last_active":"2024-09-21T02:21:51.878729+0000","last_peered":"2024-09-21T02:21:51.878729+0000","last_clean":"2024-09-21T02:21:51.878729+0000","last_became_active":"2024-09-21T02:21:50.915236+0000","last_became_peered":"2024-09-21T02:21:50.915236+0000","last_unstale":"2024-09-21T02:21:51.878729+0000","last_undegraded":"2024-09-21T02:21:51.878729+0000","last_fullsized":"2024-09-21T02:21:51.878729+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:38.508821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879602+0000","last_change":"2024-09-21T02:21:49.921193+0000","last_active":"2024-09-21T02:21:51.879602+0000","last_peered":"2024-09-21T02:21:51.879602+0000","last_clean":"2024-09-21T02:21:51.879602+0000","last_became_active":"2024-09-21T02:21:49.920790+0000","last_became_peered":"2024-09-21T02:21:49.920790+0000","last_unstale":"2024-09-21T02:21:51.879602+0000","last_undegraded":"2024-09-21T02:21:51.879602+0000","last_fullsized":"2024-09-21T02:21:51.879602+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:00.398615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772173+0000","last_change":"2024-09-21T02:21:50.888693+0000","last_active":"2024-09-21T02:21:52.772173+0000","last_peered":"2024-09-21T02:21:52.772173+0000","last_clean":"2024-09-21T02:21:52.772173+0000","last_became_active":"2024-09-21T02:21:50.888356+0000","last_became_peered":"2024-09-21T02:21:50.888356+0000","last_unstale":"2024-09-21T02:21:52.772173+0000","last_undegraded":"2024-09-21T02:21:52.772173+0000","last_fullsized":"2024-09-21T02:21:52.772173+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:30.260587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878880+0000","last_change":"2024-09-21T02:21:49.902562+0000","last_active":"2024-09-21T02:21:51.878880+0000","last_peered":"2024-09-21T02:21:51.878880+0000","last_clean":"2024-09-21T02:21:51.878880+0000","last_became_active":"2024-09-21T02:21:49.901933+0000","last_became_peered":"2024-09-21T02:21:49.901933+0000","last_unstale":"2024-09-21T02:21:51.878880+0000","last_undegraded":"2024-09-21T02:21:51.878880+0000","last_fullsized":"2024-09-21T02:21:51.878880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:15.742764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877695+0000","last_change":"2024-09-21T02:21:50.935783+0000","last_active":"2024-09-21T02:21:51.877695+0000","last_peered":"2024-09-21T02:21:51.877695+0000","last_clean":"2024-09-21T02:21:51.877695+0000","last_became_active":"2024-09-21T02:21:50.935462+0000","last_became_peered":"2024-09-21T02:21:50.935462+0000","last_unstale":"2024-09-21T02:21:51.877695+0000","last_undegraded":"2024-09-21T02:21:51.877695+0000","last_fullsized":"2024-09-21T02:21:51.877695+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:09.741008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878553+0000","last_change":"2024-09-21T02:21:49.899956+0000","last_active":"2024-09-21T02:21:51.878553+0000","last_peered":"2024-09-21T02:21:51.878553+0000","last_clean":"2024-09-21T02:21:51.878553+0000","last_became_active":"2024-09-21T02:21:49.899423+0000","last_became_peered":"2024-09-21T02:21:49.899423+0000","last_unstale":"2024-09-21T02:21:51.878553+0000","last_undegraded":"2024-09-21T02:21:51.878553+0000","last_fullsized":"2024-09-21T02:21:51.878553+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:54.341679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774580+0000","last_change":"2024-09-21T02:21:50.954014+0000","last_active":"2024-09-21T02:21:52.774580+0000","last_peered":"2024-09-21T02:21:52.774580+0000","last_clean":"2024-09-21T02:21:52.774580+0000","last_became_active":"2024-09-21T02:21:50.952065+0000","last_became_peered":"2024-09-21T02:21:50.952065+0000","last_unstale":"2024-09-21T02:21:52.774580+0000","last_undegraded":"2024-09-21T02:21:52.774580+0000","last_fullsized":"2024-09-21T02:21:52.774580+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:51.526321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772430+0000","last_change":"2024-09-21T02:21:49.904831+0000","last_active":"2024-09-21T02:21:52.772430+0000","last_peered":"2024-09-21T02:21:52.772430+0000","last_clean":"2024-09-21T02:21:52.772430+0000","last_became_active":"2024-09-21T02:21:49.904017+0000","last_became_peered":"2024-09-21T02:21:49.904017+0000","last_unstale":"2024-09-21T02:21:52.772430+0000","last_undegraded":"2024-09-21T02:21:52.772430+0000","last_fullsized":"2024-09-21T02:21:52.772430+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:08.357080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772529+0000","last_change":"2024-09-21T02:21:50.911284+0000","last_active":"2024-09-21T02:21:52.772529+0000","last_peered":"2024-09-21T02:21:52.772529+0000","last_clean":"2024-09-21T02:21:52.772529+0000","last_became_active":"2024-09-21T02:21:50.910301+0000","last_became_peered":"2024-09-21T02:21:50.910301+0000","last_unstale":"2024-09-21T02:21:52.772529+0000","last_undegraded":"2024-09-21T02:21:52.772529+0000","last_fullsized":"2024-09-21T02:21:52.772529+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:37.678853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876147+0000","last_change":"2024-09-21T02:21:49.893741+0000","last_active":"2024-09-21T02:21:51.876147+0000","last_peered":"2024-09-21T02:21:51.876147+0000","last_clean":"2024-09-21T02:21:51.876147+0000","last_became_active":"2024-09-21T02:21:49.892808+0000","last_became_peered":"2024-09-21T02:21:49.892808+0000","last_unstale":"2024-09-21T02:21:51.876147+0000","last_undegraded":"2024-09-21T02:21:51.876147+0000","last_fullsized":"2024-09-21T02:21:51.876147+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:47.913939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":376,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877895+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:21:51.877895+0000","last_peered":"2024-09-21T02:21:51.877895+0000","last_clean":"2024-09-21T02:21:51.877895+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:21:51.877895+0000","last_undegraded":"2024-09-21T02:21:51.877895+0000","last_fullsized":"2024-09-21T02:21:51.877895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1941504,"data_stored":849683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":848049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2523136,"data_stored":1438931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1925120,"data_stored":847993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509548,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":1937408,"data_stored":849641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1925120,"data_stored":848029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41964,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329876,"statfs":{"total":96636764160,"available":96593793024,"internally_reserved":0,"allocated":2514944,"data_stored":1438327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1929216,"data_stored":848575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:21:55.063 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:21:55.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e40b60640 1 -- 172.21.15.170:0/3599600645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e28005160 msgr2=0x7f7e2800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e40b60640 1 --2- 172.21.15.170:0/3599600645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e28005160 0x7f7e2800b9d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0c7170 tx=0x7f7e3401d5d0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e3ad9d640 1 -- 172.21.15.170:0/3599600645 reap_dead start 2024-09-21T02:21:55.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e40b60640 1 -- 172.21.15.170:0/3599600645 shutdown_connections 2024-09-21T02:21:55.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e40b60640 1 -- 172.21.15.170:0/3599600645 >> 172.21.15.170:0/3599600645 conn(0x7f7e3c0b1c40 msgr2=0x7f7e3c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:55.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e40b60640 1 -- 172.21.15.170:0/3599600645 shutdown_connections 2024-09-21T02:21:55.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.260+0000 7f7e40b60640 1 -- 172.21.15.170:0/3599600645 wait complete. 2024-09-21T02:21:55.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 Processor -- start 2024-09-21T02:21:55.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- start start 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e28005160 0x7f7e3c15c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 0x7f7e3c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e3c0733c0 0x7f7e3c1f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c0cf760 con 0x7f7e3c09c930 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c0c3a10 con 0x7f7e3c0733c0 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3c0cf8e0 con 0x7f7e28005160 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 0x7f7e3c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 0x7f7e3c072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54690/0 (socket says 172.21.15.170:54690) 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 -- 172.21.15.170:0/1203020154 learned_addr learned my addr 172.21.15.170:0/1203020154 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:55.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e3a59c640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e28005160 0x7f7e3c15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e3ad9d640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e3c0733c0 0x7f7e3c1f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 -- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e28005160 msgr2=0x7f7e3c15c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e28005160 0x7f7e3c15c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 -- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e3c0733c0 msgr2=0x7f7e3c1f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e3c0733c0 0x7f7e3c1f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 -- 172.21.15.170:0/1203020154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c0afcb0 con 0x7f7e3c09c930 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e31d9b640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 0x7f7e3c072e80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c430 tx=0x7f7e2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240052d0 con 0x7f7e3c09c930 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c0aff70 con 0x7f7e3c09c930 2024-09-21T02:21:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e24007020 con 0x7f7e3c09c930 2024-09-21T02:21:55.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24007470 con 0x7f7e3c09c930 2024-09-21T02:21:55.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c0b0530 con 0x7f7e3c09c930 2024-09-21T02:21:55.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.264+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c0b3350 con 0x7f7e3c09c930 2024-09-21T02:21:55.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.268+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f7e240067e0 con 0x7f7e3c09c930 2024-09-21T02:21:55.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.268+0000 7f7e337fe640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e1c073a70 0x7f7e1c075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.268+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f7e24092000 con 0x7f7e3c09c930 2024-09-21T02:21:55.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.268+0000 7f7e3a59c640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e1c073a70 0x7f7e1c075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.268+0000 7f7e3a59c640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e1c073a70 0x7f7e1c075f80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c001890 tx=0x7f7e2c001d30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:55.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.272+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7e2405ea50 con 0x7f7e3c09c930 2024-09-21T02:21:55.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.440+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e3c06fb40 con 0x7f7e1c073a70 2024-09-21T02:21:55.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.452+0000 7f7e337fe640 1 -- 172.21.15.170:0/1203020154 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f7e3c06fb40 con 0x7f7e1c073a70 2024-09-21T02:21:55.454 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:55.456 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:21:55.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e1c073a70 msgr2=0x7f7e1c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e1c073a70 0x7f7e1c075f80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c001890 tx=0x7f7e2c001d30 comp rx=0 tx=0).stop 2024-09-21T02:21:55.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 msgr2=0x7f7e3c072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 0x7f7e3c072e80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c430 tx=0x7f7e2400c8f0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 shutdown_connections 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7e1c073a70 0x7f7e1c075f80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e3c0733c0 0x7f7e3c1f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e3c09c930 0x7f7e3c072e80 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 --2- 172.21.15.170:0/1203020154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e28005160 0x7f7e3c15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 >> 172.21.15.170:0/1203020154 conn(0x7f7e3c0b1c40 msgr2=0x7f7e3c0c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 shutdown_connections 2024-09-21T02:21:55.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.456+0000 7f7e40b60640 1 -- 172.21.15.170:0/1203020154 wait complete. 2024-09-21T02:21:55.475 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-21T02:21:54.933315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":294880,"kb_used_data":16232,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754679840,"statfs":{"total":773094113280,"available":772792156160,"internally_reserved":0,"allocated":16621568,"data_stored":7969228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001236"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773054+0000","last_change":"2024-09-21T02:21:50.931420+0000","last_active":"2024-09-21T02:21:52.773054+0000","last_peered":"2024-09-21T02:21:52.773054+0000","last_clean":"2024-09-21T02:21:52.773054+0000","last_became_active":"2024-09-21T02:21:50.931011+0000","last_became_peered":"2024-09-21T02:21:50.931011+0000","last_unstale":"2024-09-21T02:21:52.773054+0000","last_undegraded":"2024-09-21T02:21:52.773054+0000","last_fullsized":"2024-09-21T02:21:52.773054+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:29.751245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879756+0000","last_change":"2024-09-21T02:21:50.886516+0000","last_active":"2024-09-21T02:21:51.879756+0000","last_peered":"2024-09-21T02:21:51.879756+0000","last_clean":"2024-09-21T02:21:51.879756+0000","last_became_active":"2024-09-21T02:21:50.885997+0000","last_became_peered":"2024-09-21T02:21:50.885997+0000","last_unstale":"2024-09-21T02:21:51.879756+0000","last_undegraded":"2024-09-21T02:21:51.879756+0000","last_fullsized":"2024-09-21T02:21:51.879756+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:36.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878602+0000","last_change":"2024-09-21T02:21:50.888430+0000","last_active":"2024-09-21T02:21:51.878602+0000","last_peered":"2024-09-21T02:21:51.878602+0000","last_clean":"2024-09-21T02:21:51.878602+0000","last_became_active":"2024-09-21T02:21:50.887325+0000","last_became_peered":"2024-09-21T02:21:50.887325+0000","last_unstale":"2024-09-21T02:21:51.878602+0000","last_undegraded":"2024-09-21T02:21:51.878602+0000","last_fullsized":"2024-09-21T02:21:51.878602+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:20.391904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879291+0000","last_change":"2024-09-21T02:21:50.909802+0000","last_active":"2024-09-21T02:21:51.879291+0000","last_peered":"2024-09-21T02:21:51.879291+0000","last_clean":"2024-09-21T02:21:51.879291+0000","last_became_active":"2024-09-21T02:21:50.909496+0000","last_became_peered":"2024-09-21T02:21:50.909496+0000","last_unstale":"2024-09-21T02:21:51.879291+0000","last_undegraded":"2024-09-21T02:21:51.879291+0000","last_fullsized":"2024-09-21T02:21:51.879291+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:53.617495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774632+0000","last_change":"2024-09-21T02:21:50.945220+0000","last_active":"2024-09-21T02:21:52.774632+0000","last_peered":"2024-09-21T02:21:52.774632+0000","last_clean":"2024-09-21T02:21:52.774632+0000","last_became_active":"2024-09-21T02:21:50.943890+0000","last_became_peered":"2024-09-21T02:21:50.943890+0000","last_unstale":"2024-09-21T02:21:52.774632+0000","last_undegraded":"2024-09-21T02:21:52.774632+0000","last_fullsized":"2024-09-21T02:21:52.774632+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:26:18.011149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772367+0000","last_change":"2024-09-21T02:21:50.952232+0000","last_active":"2024-09-21T02:21:52.772367+0000","last_peered":"2024-09-21T02:21:52.772367+0000","last_clean":"2024-09-21T02:21:52.772367+0000","last_became_active":"2024-09-21T02:21:50.951638+0000","last_became_peered":"2024-09-21T02:21:50.951638+0000","last_unstale":"2024-09-21T02:21:52.772367+0000","last_undegraded":"2024-09-21T02:21:52.772367+0000","last_fullsized":"2024-09-21T02:21:52.772367+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:27.875060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771595+0000","last_change":"2024-09-21T02:21:50.908683+0000","last_active":"2024-09-21T02:21:52.771595+0000","last_peered":"2024-09-21T02:21:52.771595+0000","last_clean":"2024-09-21T02:21:52.771595+0000","last_became_active":"2024-09-21T02:21:50.908270+0000","last_became_peered":"2024-09-21T02:21:50.908270+0000","last_unstale":"2024-09-21T02:21:52.771595+0000","last_undegraded":"2024-09-21T02:21:52.771595+0000","last_fullsized":"2024-09-21T02:21:52.771595+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:25:05.158877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772134+0000","last_change":"2024-09-21T02:21:50.926342+0000","last_active":"2024-09-21T02:21:52.772134+0000","last_peered":"2024-09-21T02:21:52.772134+0000","last_clean":"2024-09-21T02:21:52.772134+0000","last_became_active":"2024-09-21T02:21:50.925616+0000","last_became_peered":"2024-09-21T02:21:50.925616+0000","last_unstale":"2024-09-21T02:21:52.772134+0000","last_undegraded":"2024-09-21T02:21:52.772134+0000","last_fullsized":"2024-09-21T02:21:52.772134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:04.024540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878115+0000","last_change":"2024-09-21T02:21:50.881337+0000","last_active":"2024-09-21T02:21:51.878115+0000","last_peered":"2024-09-21T02:21:51.878115+0000","last_clean":"2024-09-21T02:21:51.878115+0000","last_became_active":"2024-09-21T02:21:50.880993+0000","last_became_peered":"2024-09-21T02:21:50.880993+0000","last_unstale":"2024-09-21T02:21:51.878115+0000","last_undegraded":"2024-09-21T02:21:51.878115+0000","last_fullsized":"2024-09-21T02:21:51.878115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:03.407808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775280+0000","last_change":"2024-09-21T02:21:50.932172+0000","last_active":"2024-09-21T02:21:52.775280+0000","last_peered":"2024-09-21T02:21:52.775280+0000","last_clean":"2024-09-21T02:21:52.775280+0000","last_became_active":"2024-09-21T02:21:50.931495+0000","last_became_peered":"2024-09-21T02:21:50.931495+0000","last_unstale":"2024-09-21T02:21:52.775280+0000","last_undegraded":"2024-09-21T02:21:52.775280+0000","last_fullsized":"2024-09-21T02:21:52.775280+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:30.787407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771530+0000","last_change":"2024-09-21T02:21:50.890482+0000","last_active":"2024-09-21T02:21:52.771530+0000","last_peered":"2024-09-21T02:21:52.771530+0000","last_clean":"2024-09-21T02:21:52.771530+0000","last_became_active":"2024-09-21T02:21:50.890057+0000","last_became_peered":"2024-09-21T02:21:50.890057+0000","last_unstale":"2024-09-21T02:21:52.771530+0000","last_undegraded":"2024-09-21T02:21:52.771530+0000","last_fullsized":"2024-09-21T02:21:52.771530+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:45.772234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876884+0000","last_change":"2024-09-21T02:21:50.892927+0000","last_active":"2024-09-21T02:21:51.876884+0000","last_peered":"2024-09-21T02:21:51.876884+0000","last_clean":"2024-09-21T02:21:51.876884+0000","last_became_active":"2024-09-21T02:21:50.892579+0000","last_became_peered":"2024-09-21T02:21:50.892579+0000","last_unstale":"2024-09-21T02:21:51.876884+0000","last_undegraded":"2024-09-21T02:21:51.876884+0000","last_fullsized":"2024-09-21T02:21:51.876884+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:00:43.485832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773222+0000","last_change":"2024-09-21T02:21:50.936793+0000","last_active":"2024-09-21T02:21:52.773222+0000","last_peered":"2024-09-21T02:21:52.773222+0000","last_clean":"2024-09-21T02:21:52.773222+0000","last_became_active":"2024-09-21T02:21:50.936482+0000","last_became_peered":"2024-09-21T02:21:50.936482+0000","last_unstale":"2024-09-21T02:21:52.773222+0000","last_undegraded":"2024-09-21T02:21:52.773222+0000","last_fullsized":"2024-09-21T02:21:52.773222+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:56.547832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878114+0000","last_change":"2024-09-21T02:21:50.931201+0000","last_active":"2024-09-21T02:21:51.878114+0000","last_peered":"2024-09-21T02:21:51.878114+0000","last_clean":"2024-09-21T02:21:51.878114+0000","last_became_active":"2024-09-21T02:21:50.930823+0000","last_became_peered":"2024-09-21T02:21:50.930823+0000","last_unstale":"2024-09-21T02:21:51.878114+0000","last_undegraded":"2024-09-21T02:21:51.878114+0000","last_fullsized":"2024-09-21T02:21:51.878114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:02.137050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772127+0000","last_change":"2024-09-21T02:21:50.891151+0000","last_active":"2024-09-21T02:21:52.772127+0000","last_peered":"2024-09-21T02:21:52.772127+0000","last_clean":"2024-09-21T02:21:52.772127+0000","last_became_active":"2024-09-21T02:21:50.890219+0000","last_became_peered":"2024-09-21T02:21:50.890219+0000","last_unstale":"2024-09-21T02:21:52.772127+0000","last_undegraded":"2024-09-21T02:21:52.772127+0000","last_fullsized":"2024-09-21T02:21:52.772127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:37:37.038252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877966+0000","last_change":"2024-09-21T02:21:50.885425+0000","last_active":"2024-09-21T02:21:51.877966+0000","last_peered":"2024-09-21T02:21:51.877966+0000","last_clean":"2024-09-21T02:21:51.877966+0000","last_became_active":"2024-09-21T02:21:50.884764+0000","last_became_peered":"2024-09-21T02:21:50.884764+0000","last_unstale":"2024-09-21T02:21:51.877966+0000","last_undegraded":"2024-09-21T02:21:51.877966+0000","last_fullsized":"2024-09-21T02:21:51.877966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:15.636349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773690+0000","last_change":"2024-09-21T02:21:50.943513+0000","last_active":"2024-09-21T02:21:52.773690+0000","last_peered":"2024-09-21T02:21:52.773690+0000","last_clean":"2024-09-21T02:21:52.773690+0000","last_became_active":"2024-09-21T02:21:50.942861+0000","last_became_peered":"2024-09-21T02:21:50.942861+0000","last_unstale":"2024-09-21T02:21:52.773690+0000","last_undegraded":"2024-09-21T02:21:52.773690+0000","last_fullsized":"2024-09-21T02:21:52.773690+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:04.889121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773481+0000","last_change":"2024-09-21T02:21:50.891329+0000","last_active":"2024-09-21T02:21:52.773481+0000","last_peered":"2024-09-21T02:21:52.773481+0000","last_clean":"2024-09-21T02:21:52.773481+0000","last_became_active":"2024-09-21T02:21:50.890996+0000","last_became_peered":"2024-09-21T02:21:50.890996+0000","last_unstale":"2024-09-21T02:21:52.773481+0000","last_undegraded":"2024-09-21T02:21:52.773481+0000","last_fullsized":"2024-09-21T02:21:52.773481+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:47.035210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878722+0000","last_change":"2024-09-21T02:21:50.882529+0000","last_active":"2024-09-21T02:21:51.878722+0000","last_peered":"2024-09-21T02:21:51.878722+0000","last_clean":"2024-09-21T02:21:51.878722+0000","last_became_active":"2024-09-21T02:21:50.881737+0000","last_became_peered":"2024-09-21T02:21:50.881737+0000","last_unstale":"2024-09-21T02:21:51.878722+0000","last_undegraded":"2024-09-21T02:21:51.878722+0000","last_fullsized":"2024-09-21T02:21:51.878722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:06.987021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775798+0000","last_change":"2024-09-21T02:21:50.948599+0000","last_active":"2024-09-21T02:21:52.775798+0000","last_peered":"2024-09-21T02:21:52.775798+0000","last_clean":"2024-09-21T02:21:52.775798+0000","last_became_active":"2024-09-21T02:21:50.944949+0000","last_became_peered":"2024-09-21T02:21:50.944949+0000","last_unstale":"2024-09-21T02:21:52.775798+0000","last_undegraded":"2024-09-21T02:21:52.775798+0000","last_fullsized":"2024-09-21T02:21:52.775798+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:26.378832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878370+0000","last_change":"2024-09-21T02:21:50.882144+0000","last_active":"2024-09-21T02:21:51.878370+0000","last_peered":"2024-09-21T02:21:51.878370+0000","last_clean":"2024-09-21T02:21:51.878370+0000","last_became_active":"2024-09-21T02:21:50.881100+0000","last_became_peered":"2024-09-21T02:21:50.881100+0000","last_unstale":"2024-09-21T02:21:51.878370+0000","last_undegraded":"2024-09-21T02:21:51.878370+0000","last_fullsized":"2024-09-21T02:21:51.878370+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.650025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774501+0000","last_change":"2024-09-21T02:21:50.948792+0000","last_active":"2024-09-21T02:21:52.774501+0000","last_peered":"2024-09-21T02:21:52.774501+0000","last_clean":"2024-09-21T02:21:52.774501+0000","last_became_active":"2024-09-21T02:21:50.944850+0000","last_became_peered":"2024-09-21T02:21:50.944850+0000","last_unstale":"2024-09-21T02:21:52.774501+0000","last_undegraded":"2024-09-21T02:21:52.774501+0000","last_fullsized":"2024-09-21T02:21:52.774501+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:09.901651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771134+0000","last_change":"2024-09-21T02:21:50.950690+0000","last_active":"2024-09-21T02:21:52.771134+0000","last_peered":"2024-09-21T02:21:52.771134+0000","last_clean":"2024-09-21T02:21:52.771134+0000","last_became_active":"2024-09-21T02:21:50.950185+0000","last_became_peered":"2024-09-21T02:21:50.950185+0000","last_unstale":"2024-09-21T02:21:52.771134+0000","last_undegraded":"2024-09-21T02:21:52.771134+0000","last_fullsized":"2024-09-21T02:21:52.771134+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:39.110347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772391+0000","last_change":"2024-09-21T02:21:50.932062+0000","last_active":"2024-09-21T02:21:52.772391+0000","last_peered":"2024-09-21T02:21:52.772391+0000","last_clean":"2024-09-21T02:21:52.772391+0000","last_became_active":"2024-09-21T02:21:50.931337+0000","last_became_peered":"2024-09-21T02:21:50.931337+0000","last_unstale":"2024-09-21T02:21:52.772391+0000","last_undegraded":"2024-09-21T02:21:52.772391+0000","last_fullsized":"2024-09-21T02:21:52.772391+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:48.917825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.880475+0000","last_change":"2024-09-21T02:21:50.887104+0000","last_active":"2024-09-21T02:21:51.880475+0000","last_peered":"2024-09-21T02:21:51.880475+0000","last_clean":"2024-09-21T02:21:51.880475+0000","last_became_active":"2024-09-21T02:21:50.886773+0000","last_became_peered":"2024-09-21T02:21:50.886773+0000","last_unstale":"2024-09-21T02:21:51.880475+0000","last_undegraded":"2024-09-21T02:21:51.880475+0000","last_fullsized":"2024-09-21T02:21:51.880475+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:07.176400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876585+0000","last_change":"2024-09-21T02:21:50.891281+0000","last_active":"2024-09-21T02:21:51.876585+0000","last_peered":"2024-09-21T02:21:51.876585+0000","last_clean":"2024-09-21T02:21:51.876585+0000","last_became_active":"2024-09-21T02:21:50.890926+0000","last_became_peered":"2024-09-21T02:21:50.890926+0000","last_unstale":"2024-09-21T02:21:51.876585+0000","last_undegraded":"2024-09-21T02:21:51.876585+0000","last_fullsized":"2024-09-21T02:21:51.876585+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:49.814230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879260+0000","last_change":"2024-09-21T02:21:50.882309+0000","last_active":"2024-09-21T02:21:51.879260+0000","last_peered":"2024-09-21T02:21:51.879260+0000","last_clean":"2024-09-21T02:21:51.879260+0000","last_became_active":"2024-09-21T02:21:50.881367+0000","last_became_peered":"2024-09-21T02:21:50.881367+0000","last_unstale":"2024-09-21T02:21:51.879260+0000","last_undegraded":"2024-09-21T02:21:51.879260+0000","last_fullsized":"2024-09-21T02:21:51.879260+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:25.666140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774607+0000","last_change":"2024-09-21T02:21:50.936522+0000","last_active":"2024-09-21T02:21:52.774607+0000","last_peered":"2024-09-21T02:21:52.774607+0000","last_clean":"2024-09-21T02:21:52.774607+0000","last_became_active":"2024-09-21T02:21:50.934679+0000","last_became_peered":"2024-09-21T02:21:50.934679+0000","last_unstale":"2024-09-21T02:21:52.774607+0000","last_undegraded":"2024-09-21T02:21:52.774607+0000","last_fullsized":"2024-09-21T02:21:52.774607+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:47.336663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876096+0000","last_change":"2024-09-21T02:21:50.934226+0000","last_active":"2024-09-21T02:21:51.876096+0000","last_peered":"2024-09-21T02:21:51.876096+0000","last_clean":"2024-09-21T02:21:51.876096+0000","last_became_active":"2024-09-21T02:21:50.933960+0000","last_became_peered":"2024-09-21T02:21:50.933960+0000","last_unstale":"2024-09-21T02:21:51.876096+0000","last_undegraded":"2024-09-21T02:21:51.876096+0000","last_fullsized":"2024-09-21T02:21:51.876096+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:59.985384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876353+0000","last_change":"2024-09-21T02:21:50.887932+0000","last_active":"2024-09-21T02:21:51.876353+0000","last_peered":"2024-09-21T02:21:51.876353+0000","last_clean":"2024-09-21T02:21:51.876353+0000","last_became_active":"2024-09-21T02:21:50.887600+0000","last_became_peered":"2024-09-21T02:21:50.887600+0000","last_unstale":"2024-09-21T02:21:51.876353+0000","last_undegraded":"2024-09-21T02:21:51.876353+0000","last_fullsized":"2024-09-21T02:21:51.876353+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:13.449111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773186+0000","last_change":"2024-09-21T02:21:50.952058+0000","last_active":"2024-09-21T02:21:52.773186+0000","last_peered":"2024-09-21T02:21:52.773186+0000","last_clean":"2024-09-21T02:21:52.773186+0000","last_became_active":"2024-09-21T02:21:50.948466+0000","last_became_peered":"2024-09-21T02:21:50.948466+0000","last_unstale":"2024-09-21T02:21:52.773186+0000","last_undegraded":"2024-09-21T02:21:52.773186+0000","last_fullsized":"2024-09-21T02:21:52.773186+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:03.814658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772785+0000","last_change":"2024-09-21T02:21:50.939975+0000","last_active":"2024-09-21T02:21:52.772785+0000","last_peered":"2024-09-21T02:21:52.772785+0000","last_clean":"2024-09-21T02:21:52.772785+0000","last_became_active":"2024-09-21T02:21:50.939313+0000","last_became_peered":"2024-09-21T02:21:50.939313+0000","last_unstale":"2024-09-21T02:21:52.772785+0000","last_undegraded":"2024-09-21T02:21:52.772785+0000","last_fullsized":"2024-09-21T02:21:52.772785+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:47.035210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776982+0000","last_change":"2024-09-21T02:21:50.943255+0000","last_active":"2024-09-21T02:21:52.776982+0000","last_peered":"2024-09-21T02:21:52.776982+0000","last_clean":"2024-09-21T02:21:52.776982+0000","last_became_active":"2024-09-21T02:21:50.940007+0000","last_became_peered":"2024-09-21T02:21:50.940007+0000","last_unstale":"2024-09-21T02:21:52.776982+0000","last_undegraded":"2024-09-21T02:21:52.776982+0000","last_fullsized":"2024-09-21T02:21:52.776982+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.808475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"152'4","reported_seq":19,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879535+0000","last_change":"2024-09-21T02:21:49.890527+0000","last_active":"2024-09-21T02:21:51.879535+0000","last_peered":"2024-09-21T02:21:51.879535+0000","last_clean":"2024-09-21T02:21:51.879535+0000","last_became_active":"2024-09-21T02:21:49.889683+0000","last_became_peered":"2024-09-21T02:21:49.889683+0000","last_unstale":"2024-09-21T02:21:51.879535+0000","last_undegraded":"2024-09-21T02:21:51.879535+0000","last_fullsized":"2024-09-21T02:21:51.879535+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:08.897765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775823+0000","last_change":"2024-09-21T02:21:50.947004+0000","last_active":"2024-09-21T02:21:52.775823+0000","last_peered":"2024-09-21T02:21:52.775823+0000","last_clean":"2024-09-21T02:21:52.775823+0000","last_became_active":"2024-09-21T02:21:50.946289+0000","last_became_peered":"2024-09-21T02:21:50.946289+0000","last_unstale":"2024-09-21T02:21:52.775823+0000","last_undegraded":"2024-09-21T02:21:52.775823+0000","last_fullsized":"2024-09-21T02:21:52.775823+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:40.626169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876857+0000","last_change":"2024-09-21T02:21:49.899999+0000","last_active":"2024-09-21T02:21:51.876857+0000","last_peered":"2024-09-21T02:21:51.876857+0000","last_clean":"2024-09-21T02:21:51.876857+0000","last_became_active":"2024-09-21T02:21:49.899531+0000","last_became_peered":"2024-09-21T02:21:49.899531+0000","last_unstale":"2024-09-21T02:21:51.876857+0000","last_undegraded":"2024-09-21T02:21:51.876857+0000","last_fullsized":"2024-09-21T02:21:51.876857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:06.666382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879900+0000","last_change":"2024-09-21T02:21:50.888279+0000","last_active":"2024-09-21T02:21:51.879900+0000","last_peered":"2024-09-21T02:21:51.879900+0000","last_clean":"2024-09-21T02:21:51.879900+0000","last_became_active":"2024-09-21T02:21:50.887854+0000","last_became_peered":"2024-09-21T02:21:50.887854+0000","last_unstale":"2024-09-21T02:21:51.879900+0000","last_undegraded":"2024-09-21T02:21:51.879900+0000","last_fullsized":"2024-09-21T02:21:51.879900+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:01.859733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776588+0000","last_change":"2024-09-21T02:21:49.923894+0000","last_active":"2024-09-21T02:21:52.776588+0000","last_peered":"2024-09-21T02:21:52.776588+0000","last_clean":"2024-09-21T02:21:52.776588+0000","last_became_active":"2024-09-21T02:21:49.922968+0000","last_became_peered":"2024-09-21T02:21:49.922968+0000","last_unstale":"2024-09-21T02:21:52.776588+0000","last_undegraded":"2024-09-21T02:21:52.776588+0000","last_fullsized":"2024-09-21T02:21:52.776588+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:41:57.407489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879547+0000","last_change":"2024-09-21T02:21:50.923063+0000","last_active":"2024-09-21T02:21:51.879547+0000","last_peered":"2024-09-21T02:21:51.879547+0000","last_clean":"2024-09-21T02:21:51.879547+0000","last_became_active":"2024-09-21T02:21:50.922685+0000","last_became_peered":"2024-09-21T02:21:50.922685+0000","last_unstale":"2024-09-21T02:21:51.879547+0000","last_undegraded":"2024-09-21T02:21:51.879547+0000","last_fullsized":"2024-09-21T02:21:51.879547+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:18.593576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877367+0000","last_change":"2024-09-21T02:21:49.900228+0000","last_active":"2024-09-21T02:21:51.877367+0000","last_peered":"2024-09-21T02:21:51.877367+0000","last_clean":"2024-09-21T02:21:51.877367+0000","last_became_active":"2024-09-21T02:21:49.899791+0000","last_became_peered":"2024-09-21T02:21:49.899791+0000","last_unstale":"2024-09-21T02:21:51.877367+0000","last_undegraded":"2024-09-21T02:21:51.877367+0000","last_fullsized":"2024-09-21T02:21:51.877367+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:03.020654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772960+0000","last_change":"2024-09-21T02:21:50.949467+0000","last_active":"2024-09-21T02:21:52.772960+0000","last_peered":"2024-09-21T02:21:52.772960+0000","last_clean":"2024-09-21T02:21:52.772960+0000","last_became_active":"2024-09-21T02:21:50.948824+0000","last_became_peered":"2024-09-21T02:21:50.948824+0000","last_unstale":"2024-09-21T02:21:52.772960+0000","last_undegraded":"2024-09-21T02:21:52.772960+0000","last_fullsized":"2024-09-21T02:21:52.772960+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:55:30.375733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775216+0000","last_change":"2024-09-21T02:21:49.918189+0000","last_active":"2024-09-21T02:21:52.775216+0000","last_peered":"2024-09-21T02:21:52.775216+0000","last_clean":"2024-09-21T02:21:52.775216+0000","last_became_active":"2024-09-21T02:21:49.917375+0000","last_became_peered":"2024-09-21T02:21:49.917375+0000","last_unstale":"2024-09-21T02:21:52.775216+0000","last_undegraded":"2024-09-21T02:21:52.775216+0000","last_fullsized":"2024-09-21T02:21:52.775216+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:31.859793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775337+0000","last_change":"2024-09-21T02:21:50.947633+0000","last_active":"2024-09-21T02:21:52.775337+0000","last_peered":"2024-09-21T02:21:52.775337+0000","last_clean":"2024-09-21T02:21:52.775337+0000","last_became_active":"2024-09-21T02:21:50.944395+0000","last_became_peered":"2024-09-21T02:21:50.944395+0000","last_unstale":"2024-09-21T02:21:52.775337+0000","last_undegraded":"2024-09-21T02:21:52.775337+0000","last_fullsized":"2024-09-21T02:21:52.775337+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:01.053341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774970+0000","last_change":"2024-09-21T02:21:49.914067+0000","last_active":"2024-09-21T02:21:52.774970+0000","last_peered":"2024-09-21T02:21:52.774970+0000","last_clean":"2024-09-21T02:21:52.774970+0000","last_became_active":"2024-09-21T02:21:49.913102+0000","last_became_peered":"2024-09-21T02:21:49.913102+0000","last_unstale":"2024-09-21T02:21:52.774970+0000","last_undegraded":"2024-09-21T02:21:52.774970+0000","last_fullsized":"2024-09-21T02:21:52.774970+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:26:37.161206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772564+0000","last_change":"2024-09-21T02:21:50.900120+0000","last_active":"2024-09-21T02:21:52.772564+0000","last_peered":"2024-09-21T02:21:52.772564+0000","last_clean":"2024-09-21T02:21:52.772564+0000","last_became_active":"2024-09-21T02:21:50.899723+0000","last_became_peered":"2024-09-21T02:21:50.899723+0000","last_unstale":"2024-09-21T02:21:52.772564+0000","last_undegraded":"2024-09-21T02:21:52.772564+0000","last_fullsized":"2024-09-21T02:21:52.772564+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:35.407486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771325+0000","last_change":"2024-09-21T02:21:49.915755+0000","last_active":"2024-09-21T02:21:52.771325+0000","last_peered":"2024-09-21T02:21:52.771325+0000","last_clean":"2024-09-21T02:21:52.771325+0000","last_became_active":"2024-09-21T02:21:49.915293+0000","last_became_peered":"2024-09-21T02:21:49.915293+0000","last_unstale":"2024-09-21T02:21:52.771325+0000","last_undegraded":"2024-09-21T02:21:52.771325+0000","last_fullsized":"2024-09-21T02:21:52.771325+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:23.100912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879427+0000","last_change":"2024-09-21T02:21:50.922402+0000","last_active":"2024-09-21T02:21:51.879427+0000","last_peered":"2024-09-21T02:21:51.879427+0000","last_clean":"2024-09-21T02:21:51.879427+0000","last_became_active":"2024-09-21T02:21:50.921527+0000","last_became_peered":"2024-09-21T02:21:50.921527+0000","last_unstale":"2024-09-21T02:21:51.879427+0000","last_undegraded":"2024-09-21T02:21:51.879427+0000","last_fullsized":"2024-09-21T02:21:51.879427+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:16.556388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773059+0000","last_change":"2024-09-21T02:21:49.895696+0000","last_active":"2024-09-21T02:21:52.773059+0000","last_peered":"2024-09-21T02:21:52.773059+0000","last_clean":"2024-09-21T02:21:52.773059+0000","last_became_active":"2024-09-21T02:21:49.894887+0000","last_became_peered":"2024-09-21T02:21:49.894887+0000","last_unstale":"2024-09-21T02:21:52.773059+0000","last_undegraded":"2024-09-21T02:21:52.773059+0000","last_fullsized":"2024-09-21T02:21:52.773059+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:16.327882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776030+0000","last_change":"2024-09-21T02:21:50.932260+0000","last_active":"2024-09-21T02:21:52.776030+0000","last_peered":"2024-09-21T02:21:52.776030+0000","last_clean":"2024-09-21T02:21:52.776030+0000","last_became_active":"2024-09-21T02:21:50.929555+0000","last_became_peered":"2024-09-21T02:21:50.929555+0000","last_unstale":"2024-09-21T02:21:52.776030+0000","last_undegraded":"2024-09-21T02:21:52.776030+0000","last_fullsized":"2024-09-21T02:21:52.776030+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:33:44.390759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776104+0000","last_change":"2024-09-21T02:21:49.902786+0000","last_active":"2024-09-21T02:21:52.776104+0000","last_peered":"2024-09-21T02:21:52.776104+0000","last_clean":"2024-09-21T02:21:52.776104+0000","last_became_active":"2024-09-21T02:21:49.901771+0000","last_became_peered":"2024-09-21T02:21:49.901771+0000","last_unstale":"2024-09-21T02:21:52.776104+0000","last_undegraded":"2024-09-21T02:21:52.776104+0000","last_fullsized":"2024-09-21T02:21:52.776104+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:06.829878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877327+0000","last_change":"2024-09-21T02:21:50.892428+0000","last_active":"2024-09-21T02:21:51.877327+0000","last_peered":"2024-09-21T02:21:51.877327+0000","last_clean":"2024-09-21T02:21:51.877327+0000","last_became_active":"2024-09-21T02:21:50.892019+0000","last_became_peered":"2024-09-21T02:21:50.892019+0000","last_unstale":"2024-09-21T02:21:51.877327+0000","last_undegraded":"2024-09-21T02:21:51.877327+0000","last_fullsized":"2024-09-21T02:21:51.877327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:11.650025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"152'3","reported_seq":18,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776788+0000","last_change":"2024-09-21T02:21:49.922091+0000","last_active":"2024-09-21T02:21:52.776788+0000","last_peered":"2024-09-21T02:21:52.776788+0000","last_clean":"2024-09-21T02:21:52.776788+0000","last_became_active":"2024-09-21T02:21:49.921264+0000","last_became_peered":"2024-09-21T02:21:49.921264+0000","last_unstale":"2024-09-21T02:21:52.776788+0000","last_undegraded":"2024-09-21T02:21:52.776788+0000","last_fullsized":"2024-09-21T02:21:52.776788+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:40.077793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775378+0000","last_change":"2024-09-21T02:21:50.931450+0000","last_active":"2024-09-21T02:21:52.775378+0000","last_peered":"2024-09-21T02:21:52.775378+0000","last_clean":"2024-09-21T02:21:52.775378+0000","last_became_active":"2024-09-21T02:21:50.930828+0000","last_became_peered":"2024-09-21T02:21:50.930828+0000","last_unstale":"2024-09-21T02:21:52.775378+0000","last_undegraded":"2024-09-21T02:21:52.775378+0000","last_fullsized":"2024-09-21T02:21:52.775378+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:58.562930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"152'6","reported_seq":21,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776277+0000","last_change":"2024-09-21T02:21:49.918733+0000","last_active":"2024-09-21T02:21:52.776277+0000","last_peered":"2024-09-21T02:21:52.776277+0000","last_clean":"2024-09-21T02:21:52.776277+0000","last_became_active":"2024-09-21T02:21:49.918026+0000","last_became_peered":"2024-09-21T02:21:49.918026+0000","last_unstale":"2024-09-21T02:21:52.776277+0000","last_undegraded":"2024-09-21T02:21:52.776277+0000","last_fullsized":"2024-09-21T02:21:52.776277+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:26.005577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772141+0000","last_change":"2024-09-21T02:21:50.954218+0000","last_active":"2024-09-21T02:21:52.772141+0000","last_peered":"2024-09-21T02:21:52.772141+0000","last_clean":"2024-09-21T02:21:52.772141+0000","last_became_active":"2024-09-21T02:21:50.953549+0000","last_became_peered":"2024-09-21T02:21:50.953549+0000","last_unstale":"2024-09-21T02:21:52.772141+0000","last_undegraded":"2024-09-21T02:21:52.772141+0000","last_fullsized":"2024-09-21T02:21:52.772141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:38.135408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876836+0000","last_change":"2024-09-21T02:21:49.894254+0000","last_active":"2024-09-21T02:21:51.876836+0000","last_peered":"2024-09-21T02:21:51.876836+0000","last_clean":"2024-09-21T02:21:51.876836+0000","last_became_active":"2024-09-21T02:21:49.893836+0000","last_became_peered":"2024-09-21T02:21:49.893836+0000","last_unstale":"2024-09-21T02:21:51.876836+0000","last_undegraded":"2024-09-21T02:21:51.876836+0000","last_fullsized":"2024-09-21T02:21:51.876836+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:25:04.154991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772187+0000","last_change":"2024-09-21T02:21:50.922390+0000","last_active":"2024-09-21T02:21:52.772187+0000","last_peered":"2024-09-21T02:21:52.772187+0000","last_clean":"2024-09-21T02:21:52.772187+0000","last_became_active":"2024-09-21T02:21:50.922104+0000","last_became_peered":"2024-09-21T02:21:50.922104+0000","last_unstale":"2024-09-21T02:21:52.772187+0000","last_undegraded":"2024-09-21T02:21:52.772187+0000","last_fullsized":"2024-09-21T02:21:52.772187+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:04.024540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772004+0000","last_change":"2024-09-21T02:21:49.920371+0000","last_active":"2024-09-21T02:21:52.772004+0000","last_peered":"2024-09-21T02:21:52.772004+0000","last_clean":"2024-09-21T02:21:52.772004+0000","last_became_active":"2024-09-21T02:21:49.919578+0000","last_became_peered":"2024-09-21T02:21:49.919578+0000","last_unstale":"2024-09-21T02:21:52.772004+0000","last_undegraded":"2024-09-21T02:21:52.772004+0000","last_fullsized":"2024-09-21T02:21:52.772004+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:49.573706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876108+0000","last_change":"2024-09-21T02:21:50.910973+0000","last_active":"2024-09-21T02:21:51.876108+0000","last_peered":"2024-09-21T02:21:51.876108+0000","last_clean":"2024-09-21T02:21:51.876108+0000","last_became_active":"2024-09-21T02:21:50.910642+0000","last_became_peered":"2024-09-21T02:21:50.910642+0000","last_unstale":"2024-09-21T02:21:51.876108+0000","last_undegraded":"2024-09-21T02:21:51.876108+0000","last_fullsized":"2024-09-21T02:21:51.876108+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.808475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776659+0000","last_change":"2024-09-21T02:21:49.922108+0000","last_active":"2024-09-21T02:21:52.776659+0000","last_peered":"2024-09-21T02:21:52.776659+0000","last_clean":"2024-09-21T02:21:52.776659+0000","last_became_active":"2024-09-21T02:21:49.921355+0000","last_became_peered":"2024-09-21T02:21:49.921355+0000","last_unstale":"2024-09-21T02:21:52.776659+0000","last_undegraded":"2024-09-21T02:21:52.776659+0000","last_fullsized":"2024-09-21T02:21:52.776659+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:40.068577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773875+0000","last_change":"2024-09-21T02:21:50.949081+0000","last_active":"2024-09-21T02:21:52.773875+0000","last_peered":"2024-09-21T02:21:52.773875+0000","last_clean":"2024-09-21T02:21:52.773875+0000","last_became_active":"2024-09-21T02:21:50.943163+0000","last_became_peered":"2024-09-21T02:21:50.943163+0000","last_unstale":"2024-09-21T02:21:52.773875+0000","last_undegraded":"2024-09-21T02:21:52.773875+0000","last_fullsized":"2024-09-21T02:21:52.773875+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:26.960103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"152'8","reported_seq":23,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771647+0000","last_change":"2024-09-21T02:21:49.891653+0000","last_active":"2024-09-21T02:21:52.771647+0000","last_peered":"2024-09-21T02:21:52.771647+0000","last_clean":"2024-09-21T02:21:52.771647+0000","last_became_active":"2024-09-21T02:21:49.890965+0000","last_became_peered":"2024-09-21T02:21:49.890965+0000","last_unstale":"2024-09-21T02:21:52.771647+0000","last_undegraded":"2024-09-21T02:21:52.771647+0000","last_fullsized":"2024-09-21T02:21:52.771647+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:04.299884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775901+0000","last_change":"2024-09-21T02:21:50.931980+0000","last_active":"2024-09-21T02:21:52.775901+0000","last_peered":"2024-09-21T02:21:52.775901+0000","last_clean":"2024-09-21T02:21:52.775901+0000","last_became_active":"2024-09-21T02:21:50.931192+0000","last_became_peered":"2024-09-21T02:21:50.931192+0000","last_unstale":"2024-09-21T02:21:52.775901+0000","last_undegraded":"2024-09-21T02:21:52.775901+0000","last_fullsized":"2024-09-21T02:21:52.775901+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:51.867901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774717+0000","last_change":"2024-09-21T02:21:49.909749+0000","last_active":"2024-09-21T02:21:52.774717+0000","last_peered":"2024-09-21T02:21:52.774717+0000","last_clean":"2024-09-21T02:21:52.774717+0000","last_became_active":"2024-09-21T02:21:49.908867+0000","last_became_peered":"2024-09-21T02:21:49.908867+0000","last_unstale":"2024-09-21T02:21:52.774717+0000","last_undegraded":"2024-09-21T02:21:52.774717+0000","last_fullsized":"2024-09-21T02:21:52.774717+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:15.397198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878458+0000","last_change":"2024-09-21T02:21:50.896972+0000","last_active":"2024-09-21T02:21:51.878458+0000","last_peered":"2024-09-21T02:21:51.878458+0000","last_clean":"2024-09-21T02:21:51.878458+0000","last_became_active":"2024-09-21T02:21:50.896517+0000","last_became_peered":"2024-09-21T02:21:50.896517+0000","last_unstale":"2024-09-21T02:21:51.878458+0000","last_undegraded":"2024-09-21T02:21:51.878458+0000","last_fullsized":"2024-09-21T02:21:51.878458+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:49:20.391904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773827+0000","last_change":"2024-09-21T02:21:49.889518+0000","last_active":"2024-09-21T02:21:52.773827+0000","last_peered":"2024-09-21T02:21:52.773827+0000","last_clean":"2024-09-21T02:21:52.773827+0000","last_became_active":"2024-09-21T02:21:49.888574+0000","last_became_peered":"2024-09-21T02:21:49.888574+0000","last_unstale":"2024-09-21T02:21:52.773827+0000","last_undegraded":"2024-09-21T02:21:52.773827+0000","last_fullsized":"2024-09-21T02:21:52.773827+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:46.093955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.880023+0000","last_change":"2024-09-21T02:21:50.881983+0000","last_active":"2024-09-21T02:21:51.880023+0000","last_peered":"2024-09-21T02:21:51.880023+0000","last_clean":"2024-09-21T02:21:51.880023+0000","last_became_active":"2024-09-21T02:21:50.881541+0000","last_became_peered":"2024-09-21T02:21:50.881541+0000","last_unstale":"2024-09-21T02:21:51.880023+0000","last_undegraded":"2024-09-21T02:21:51.880023+0000","last_fullsized":"2024-09-21T02:21:51.880023+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:30.876175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"152'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772019+0000","last_change":"2024-09-21T02:21:49.892079+0000","last_active":"2024-09-21T02:21:52.772019+0000","last_peered":"2024-09-21T02:21:52.772019+0000","last_clean":"2024-09-21T02:21:52.772019+0000","last_became_active":"2024-09-21T02:21:49.891687+0000","last_became_peered":"2024-09-21T02:21:49.891687+0000","last_unstale":"2024-09-21T02:21:52.772019+0000","last_undegraded":"2024-09-21T02:21:52.772019+0000","last_fullsized":"2024-09-21T02:21:52.772019+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:30:53.782747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876884+0000","last_change":"2024-09-21T02:21:50.887537+0000","last_active":"2024-09-21T02:21:51.876884+0000","last_peered":"2024-09-21T02:21:51.876884+0000","last_clean":"2024-09-21T02:21:51.876884+0000","last_became_active":"2024-09-21T02:21:50.887213+0000","last_became_peered":"2024-09-21T02:21:50.887213+0000","last_unstale":"2024-09-21T02:21:51.876884+0000","last_undegraded":"2024-09-21T02:21:51.876884+0000","last_fullsized":"2024-09-21T02:21:51.876884+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:15.636349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771498+0000","last_change":"2024-09-21T02:21:49.898568+0000","last_active":"2024-09-21T02:21:52.771498+0000","last_peered":"2024-09-21T02:21:52.771498+0000","last_clean":"2024-09-21T02:21:52.771498+0000","last_became_active":"2024-09-21T02:21:49.897645+0000","last_became_peered":"2024-09-21T02:21:49.897645+0000","last_unstale":"2024-09-21T02:21:52.771498+0000","last_undegraded":"2024-09-21T02:21:52.771498+0000","last_fullsized":"2024-09-21T02:21:52.771498+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:29:49.171053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773033+0000","last_change":"2024-09-21T02:21:50.920935+0000","last_active":"2024-09-21T02:21:52.773033+0000","last_peered":"2024-09-21T02:21:52.773033+0000","last_clean":"2024-09-21T02:21:52.773033+0000","last_became_active":"2024-09-21T02:21:50.920016+0000","last_became_peered":"2024-09-21T02:21:50.920016+0000","last_unstale":"2024-09-21T02:21:52.773033+0000","last_undegraded":"2024-09-21T02:21:52.773033+0000","last_fullsized":"2024-09-21T02:21:52.773033+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:06:11.851393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.775570+0000","last_change":"2024-09-21T02:21:49.918276+0000","last_active":"2024-09-21T02:21:52.775570+0000","last_peered":"2024-09-21T02:21:52.775570+0000","last_clean":"2024-09-21T02:21:52.775570+0000","last_became_active":"2024-09-21T02:21:49.917548+0000","last_became_peered":"2024-09-21T02:21:49.917548+0000","last_unstale":"2024-09-21T02:21:52.775570+0000","last_undegraded":"2024-09-21T02:21:52.775570+0000","last_fullsized":"2024-09-21T02:21:52.775570+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:06.308291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772548+0000","last_change":"2024-09-21T02:21:50.936808+0000","last_active":"2024-09-21T02:21:52.772548+0000","last_peered":"2024-09-21T02:21:52.772548+0000","last_clean":"2024-09-21T02:21:52.772548+0000","last_became_active":"2024-09-21T02:21:50.936471+0000","last_became_peered":"2024-09-21T02:21:50.936471+0000","last_unstale":"2024-09-21T02:21:52.772548+0000","last_undegraded":"2024-09-21T02:21:52.772548+0000","last_fullsized":"2024-09-21T02:21:52.772548+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:48.917825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773465+0000","last_change":"2024-09-21T02:21:49.894700+0000","last_active":"2024-09-21T02:21:52.773465+0000","last_peered":"2024-09-21T02:21:52.773465+0000","last_clean":"2024-09-21T02:21:52.773465+0000","last_became_active":"2024-09-21T02:21:49.893021+0000","last_became_peered":"2024-09-21T02:21:49.893021+0000","last_unstale":"2024-09-21T02:21:52.773465+0000","last_undegraded":"2024-09-21T02:21:52.773465+0000","last_fullsized":"2024-09-21T02:21:52.773465+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:16.556540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771082+0000","last_change":"2024-09-21T02:21:50.893600+0000","last_active":"2024-09-21T02:21:52.771082+0000","last_peered":"2024-09-21T02:21:52.771082+0000","last_clean":"2024-09-21T02:21:52.771082+0000","last_became_active":"2024-09-21T02:21:50.893232+0000","last_became_peered":"2024-09-21T02:21:50.893232+0000","last_unstale":"2024-09-21T02:21:52.771082+0000","last_undegraded":"2024-09-21T02:21:52.771082+0000","last_fullsized":"2024-09-21T02:21:52.771082+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:37.767974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772378+0000","last_change":"2024-09-21T02:21:49.903683+0000","last_active":"2024-09-21T02:21:52.772378+0000","last_peered":"2024-09-21T02:21:52.772378+0000","last_clean":"2024-09-21T02:21:52.772378+0000","last_became_active":"2024-09-21T02:21:49.902756+0000","last_became_peered":"2024-09-21T02:21:49.902756+0000","last_unstale":"2024-09-21T02:21:52.772378+0000","last_undegraded":"2024-09-21T02:21:52.772378+0000","last_fullsized":"2024-09-21T02:21:52.772378+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:40.483264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771149+0000","last_change":"2024-09-21T02:21:50.920837+0000","last_active":"2024-09-21T02:21:52.771149+0000","last_peered":"2024-09-21T02:21:52.771149+0000","last_clean":"2024-09-21T02:21:52.771149+0000","last_became_active":"2024-09-21T02:21:50.920076+0000","last_became_peered":"2024-09-21T02:21:50.920076+0000","last_unstale":"2024-09-21T02:21:52.771149+0000","last_undegraded":"2024-09-21T02:21:52.771149+0000","last_fullsized":"2024-09-21T02:21:52.771149+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:56.466439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.771599+0000","last_change":"2024-09-21T02:21:49.893349+0000","last_active":"2024-09-21T02:21:52.771599+0000","last_peered":"2024-09-21T02:21:52.771599+0000","last_clean":"2024-09-21T02:21:52.771599+0000","last_became_active":"2024-09-21T02:21:49.892960+0000","last_became_peered":"2024-09-21T02:21:49.892960+0000","last_unstale":"2024-09-21T02:21:52.771599+0000","last_undegraded":"2024-09-21T02:21:52.771599+0000","last_fullsized":"2024-09-21T02:21:52.771599+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:57.296742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879710+0000","last_change":"2024-09-21T02:21:50.891057+0000","last_active":"2024-09-21T02:21:51.879710+0000","last_peered":"2024-09-21T02:21:51.879710+0000","last_clean":"2024-09-21T02:21:51.879710+0000","last_became_active":"2024-09-21T02:21:50.890394+0000","last_became_peered":"2024-09-21T02:21:50.890394+0000","last_unstale":"2024-09-21T02:21:51.879710+0000","last_undegraded":"2024-09-21T02:21:51.879710+0000","last_fullsized":"2024-09-21T02:21:51.879710+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:33.259929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"152'2","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.773362+0000","last_change":"2024-09-21T02:21:49.896095+0000","last_active":"2024-09-21T02:21:52.773362+0000","last_peered":"2024-09-21T02:21:52.773362+0000","last_clean":"2024-09-21T02:21:52.773362+0000","last_became_active":"2024-09-21T02:21:49.895284+0000","last_became_peered":"2024-09-21T02:21:49.895284+0000","last_unstale":"2024-09-21T02:21:52.773362+0000","last_undegraded":"2024-09-21T02:21:52.773362+0000","last_fullsized":"2024-09-21T02:21:52.773362+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:37.085768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878919+0000","last_change":"2024-09-21T02:21:50.893236+0000","last_active":"2024-09-21T02:21:51.878919+0000","last_peered":"2024-09-21T02:21:51.878919+0000","last_clean":"2024-09-21T02:21:51.878919+0000","last_became_active":"2024-09-21T02:21:50.889612+0000","last_became_peered":"2024-09-21T02:21:50.889612+0000","last_unstale":"2024-09-21T02:21:51.878919+0000","last_undegraded":"2024-09-21T02:21:51.878919+0000","last_fullsized":"2024-09-21T02:21:51.878919+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:19.557641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"153'1","reported_seq":16,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879995+0000","last_change":"2024-09-21T02:21:49.903670+0000","last_active":"2024-09-21T02:21:51.879995+0000","last_peered":"2024-09-21T02:21:51.879995+0000","last_clean":"2024-09-21T02:21:51.879995+0000","last_became_active":"2024-09-21T02:21:49.902913+0000","last_became_peered":"2024-09-21T02:21:49.902913+0000","last_unstale":"2024-09-21T02:21:51.879995+0000","last_undegraded":"2024-09-21T02:21:51.879995+0000","last_fullsized":"2024-09-21T02:21:51.879995+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:02.810772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878954+0000","last_change":"2024-09-21T02:21:50.896611+0000","last_active":"2024-09-21T02:21:51.878954+0000","last_peered":"2024-09-21T02:21:51.878954+0000","last_clean":"2024-09-21T02:21:51.878954+0000","last_became_active":"2024-09-21T02:21:50.895606+0000","last_became_peered":"2024-09-21T02:21:50.895606+0000","last_unstale":"2024-09-21T02:21:51.878954+0000","last_undegraded":"2024-09-21T02:21:51.878954+0000","last_fullsized":"2024-09-21T02:21:51.878954+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:10.510035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879289+0000","last_change":"2024-09-21T02:21:49.905594+0000","last_active":"2024-09-21T02:21:51.879289+0000","last_peered":"2024-09-21T02:21:51.879289+0000","last_clean":"2024-09-21T02:21:51.879289+0000","last_became_active":"2024-09-21T02:21:49.904826+0000","last_became_peered":"2024-09-21T02:21:49.904826+0000","last_unstale":"2024-09-21T02:21:51.879289+0000","last_undegraded":"2024-09-21T02:21:51.879289+0000","last_fullsized":"2024-09-21T02:21:51.879289+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:25.956217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776100+0000","last_change":"2024-09-21T02:21:50.944530+0000","last_active":"2024-09-21T02:21:52.776100+0000","last_peered":"2024-09-21T02:21:52.776100+0000","last_clean":"2024-09-21T02:21:52.776100+0000","last_became_active":"2024-09-21T02:21:50.943461+0000","last_became_peered":"2024-09-21T02:21:50.943461+0000","last_unstale":"2024-09-21T02:21:52.776100+0000","last_undegraded":"2024-09-21T02:21:52.776100+0000","last_fullsized":"2024-09-21T02:21:52.776100+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:59.985384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.776511+0000","last_change":"2024-09-21T02:21:49.918888+0000","last_active":"2024-09-21T02:21:52.776511+0000","last_peered":"2024-09-21T02:21:52.776511+0000","last_clean":"2024-09-21T02:21:52.776511+0000","last_became_active":"2024-09-21T02:21:49.918074+0000","last_became_peered":"2024-09-21T02:21:49.918074+0000","last_unstale":"2024-09-21T02:21:52.776511+0000","last_undegraded":"2024-09-21T02:21:52.776511+0000","last_fullsized":"2024-09-21T02:21:52.776511+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:47.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878729+0000","last_change":"2024-09-21T02:21:50.915579+0000","last_active":"2024-09-21T02:21:51.878729+0000","last_peered":"2024-09-21T02:21:51.878729+0000","last_clean":"2024-09-21T02:21:51.878729+0000","last_became_active":"2024-09-21T02:21:50.915236+0000","last_became_peered":"2024-09-21T02:21:50.915236+0000","last_unstale":"2024-09-21T02:21:51.878729+0000","last_undegraded":"2024-09-21T02:21:51.878729+0000","last_fullsized":"2024-09-21T02:21:51.878729+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:38.508821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.879602+0000","last_change":"2024-09-21T02:21:49.921193+0000","last_active":"2024-09-21T02:21:51.879602+0000","last_peered":"2024-09-21T02:21:51.879602+0000","last_clean":"2024-09-21T02:21:51.879602+0000","last_became_active":"2024-09-21T02:21:49.920790+0000","last_became_peered":"2024-09-21T02:21:49.920790+0000","last_unstale":"2024-09-21T02:21:51.879602+0000","last_undegraded":"2024-09-21T02:21:51.879602+0000","last_fullsized":"2024-09-21T02:21:51.879602+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:00.398615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772173+0000","last_change":"2024-09-21T02:21:50.888693+0000","last_active":"2024-09-21T02:21:52.772173+0000","last_peered":"2024-09-21T02:21:52.772173+0000","last_clean":"2024-09-21T02:21:52.772173+0000","last_became_active":"2024-09-21T02:21:50.888356+0000","last_became_peered":"2024-09-21T02:21:50.888356+0000","last_unstale":"2024-09-21T02:21:52.772173+0000","last_undegraded":"2024-09-21T02:21:52.772173+0000","last_fullsized":"2024-09-21T02:21:52.772173+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:30.260587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878880+0000","last_change":"2024-09-21T02:21:49.902562+0000","last_active":"2024-09-21T02:21:51.878880+0000","last_peered":"2024-09-21T02:21:51.878880+0000","last_clean":"2024-09-21T02:21:51.878880+0000","last_became_active":"2024-09-21T02:21:49.901933+0000","last_became_peered":"2024-09-21T02:21:49.901933+0000","last_unstale":"2024-09-21T02:21:51.878880+0000","last_undegraded":"2024-09-21T02:21:51.878880+0000","last_fullsized":"2024-09-21T02:21:51.878880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:15.742764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877695+0000","last_change":"2024-09-21T02:21:50.935783+0000","last_active":"2024-09-21T02:21:51.877695+0000","last_peered":"2024-09-21T02:21:51.877695+0000","last_clean":"2024-09-21T02:21:51.877695+0000","last_became_active":"2024-09-21T02:21:50.935462+0000","last_became_peered":"2024-09-21T02:21:50.935462+0000","last_unstale":"2024-09-21T02:21:51.877695+0000","last_undegraded":"2024-09-21T02:21:51.877695+0000","last_fullsized":"2024-09-21T02:21:51.877695+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:09.741008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.878553+0000","last_change":"2024-09-21T02:21:49.899956+0000","last_active":"2024-09-21T02:21:51.878553+0000","last_peered":"2024-09-21T02:21:51.878553+0000","last_clean":"2024-09-21T02:21:51.878553+0000","last_became_active":"2024-09-21T02:21:49.899423+0000","last_became_peered":"2024-09-21T02:21:49.899423+0000","last_unstale":"2024-09-21T02:21:51.878553+0000","last_undegraded":"2024-09-21T02:21:51.878553+0000","last_fullsized":"2024-09-21T02:21:51.878553+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:54.341679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.774580+0000","last_change":"2024-09-21T02:21:50.954014+0000","last_active":"2024-09-21T02:21:52.774580+0000","last_peered":"2024-09-21T02:21:52.774580+0000","last_clean":"2024-09-21T02:21:52.774580+0000","last_became_active":"2024-09-21T02:21:50.952065+0000","last_became_peered":"2024-09-21T02:21:50.952065+0000","last_unstale":"2024-09-21T02:21:52.774580+0000","last_undegraded":"2024-09-21T02:21:52.774580+0000","last_fullsized":"2024-09-21T02:21:52.774580+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:51.526321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772430+0000","last_change":"2024-09-21T02:21:49.904831+0000","last_active":"2024-09-21T02:21:52.772430+0000","last_peered":"2024-09-21T02:21:52.772430+0000","last_clean":"2024-09-21T02:21:52.772430+0000","last_became_active":"2024-09-21T02:21:49.904017+0000","last_became_peered":"2024-09-21T02:21:49.904017+0000","last_unstale":"2024-09-21T02:21:52.772430+0000","last_undegraded":"2024-09-21T02:21:52.772430+0000","last_fullsized":"2024-09-21T02:21:52.772430+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:08.357080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:52.772529+0000","last_change":"2024-09-21T02:21:50.911284+0000","last_active":"2024-09-21T02:21:52.772529+0000","last_peered":"2024-09-21T02:21:52.772529+0000","last_clean":"2024-09-21T02:21:52.772529+0000","last_became_active":"2024-09-21T02:21:50.910301+0000","last_became_peered":"2024-09-21T02:21:50.910301+0000","last_unstale":"2024-09-21T02:21:52.772529+0000","last_undegraded":"2024-09-21T02:21:52.772529+0000","last_fullsized":"2024-09-21T02:21:52.772529+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:49.851468+0000","last_clean_scrub_stamp":"2024-09-21T02:21:49.851468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:37.678853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.876147+0000","last_change":"2024-09-21T02:21:49.893741+0000","last_active":"2024-09-21T02:21:51.876147+0000","last_peered":"2024-09-21T02:21:51.876147+0000","last_clean":"2024-09-21T02:21:51.876147+0000","last_became_active":"2024-09-21T02:21:49.892808+0000","last_became_peered":"2024-09-21T02:21:49.892808+0000","last_unstale":"2024-09-21T02:21:51.876147+0000","last_undegraded":"2024-09-21T02:21:51.876147+0000","last_fullsized":"2024-09-21T02:21:51.876147+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:21:48.847582+0000","last_clean_scrub_stamp":"2024-09-21T02:21:48.847582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:47.913939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":376,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-21T02:21:51.877895+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:21:51.877895+0000","last_peered":"2024-09-21T02:21:51.877895+0000","last_clean":"2024-09-21T02:21:51.877895+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:21:51.877895+0000","last_undegraded":"2024-09-21T02:21:51.877895+0000","last_fullsized":"2024-09-21T02:21:51.877895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1941504,"data_stored":849683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":848049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2523136,"data_stored":1438931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1925120,"data_stored":847993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509548,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":1937408,"data_stored":849641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1925120,"data_stored":848029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41964,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329876,"statfs":{"total":96636764160,"available":96593793024,"internally_reserved":0,"allocated":2514944,"data_stored":1438327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1929216,"data_stored":848575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:21:55.479 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:21:55.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:21:55.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- 172.21.15.170:0/3558324331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 msgr2=0x7f4b30070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 --2- 172.21.15.170:0/3558324331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b30070490 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4b300720f0 tx=0x7f4b2801cd80 comp rx=0 tx=0).stop 2024-09-21T02:21:55.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- 172.21.15.170:0/3558324331 shutdown_connections 2024-09-21T02:21:55.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 --2- 172.21.15.170:0/3558324331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b30070490 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 --2- 172.21.15.170:0/3558324331 >> v2:172.21.15.191:3300/0 conn(0x7f4b300cec50 0x7f4b3006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- 172.21.15.170:0/3558324331 >> 172.21.15.170:0/3558324331 conn(0x7f4b300d0170 msgr2=0x7f4b300d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:55.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- 172.21.15.170:0/3558324331 shutdown_connections 2024-09-21T02:21:55.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- 172.21.15.170:0/3558324331 wait complete. 2024-09-21T02:21:55.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 Processor -- start 2024-09-21T02:21:55.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- start start 2024-09-21T02:21:55.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b300b2fc0 0x7f4b30140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b300cec50 0x7f4b301413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b300b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30169f50 con 0x7f4b300cf060 2024-09-21T02:21:55.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30169dd0 con 0x7f4b300b2fc0 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.672+0000 7f4b35f51640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3016a0d0 con 0x7f4b300cec50 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b300b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b300b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60474/0 (socket says 172.21.15.170:60474) 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 -- 172.21.15.170:0/570374300 learned_addr learned my addr 172.21.15.170:0/570374300 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2effd640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b300cec50 0x7f4b301413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2f7fe640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b300b2fc0 0x7f4b30140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 -- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b300cec50 msgr2=0x7f4b301413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b300cec50 0x7f4b301413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 -- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b300b2fc0 msgr2=0x7f4b30140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b300b2fc0 0x7f4b30140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 -- 172.21.15.170:0/570374300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b30155f90 con 0x7f4b300cf060 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2f7fe640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b300b2fc0 0x7f4b30140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:55.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2ffff640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b300b0940 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4b2801b430 tx=0x7f4b280a5b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:55.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280bb070 con 0x7f4b300cf060 2024-09-21T02:21:55.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b28004e00 con 0x7f4b300cf060 2024-09-21T02:21:55.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280b2040 con 0x7f4b300cf060 2024-09-21T02:21:55.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b30156270 con 0x7f4b300cf060 2024-09-21T02:21:55.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b30153660 con 0x7f4b300cf060 2024-09-21T02:21:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b30071880 con 0x7f4b300cf060 2024-09-21T02:21:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f4b28004320 con 0x7f4b300cf060 2024-09-21T02:21:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b17fff640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4b1c073a70 0x7f4b1c075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f4b2812ceb0 con 0x7f4b300cf060 2024-09-21T02:21:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2f7fe640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4b1c073a70 0x7f4b1c075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:55.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.676+0000 7f4b2f7fe640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4b1c073a70 0x7f4b1c075f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000a850 tx=0x7f4b20008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:55.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.680+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f4b280c4020 con 0x7f4b300cf060 2024-09-21T02:21:55.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.900+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4b300b4bd0 con 0x7f4b300cf060 2024-09-21T02:21:55.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.904+0000 7f4b17fff640 1 -- 172.21.15.170:0/570374300 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7f4b28016970 con 0x7f4b300cf060 2024-09-21T02:21:55.906 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:21:55.906 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:21:55.906 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:21:55.906 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:21:55.906 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:21:55.907 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4b1c073a70 msgr2=0x7f4b1c075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4b1c073a70 0x7f4b1c075f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000a850 tx=0x7f4b20008040 comp rx=0 tx=0).stop 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 msgr2=0x7f4b300b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b300b0940 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4b2801b430 tx=0x7f4b280a5b20 comp rx=0 tx=0).stop 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 shutdown_connections 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4b1c073a70 0x7f4b1c075f80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b300cec50 0x7f4b301413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b300cf060 0x7f4b300b0940 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 --2- 172.21.15.170:0/570374300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b300b2fc0 0x7f4b30140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 >> 172.21.15.170:0/570374300 conn(0x7f4b300d0170 msgr2=0x7f4b300b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:55.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 shutdown_connections 2024-09-21T02:21:55.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:55.908+0000 7f4b35f51640 1 -- 172.21.15.170:0/570374300 wait complete. 2024-09-21T02:21:55.927 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc3f36640 1 -- 172.21.15.170:0/1748576792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 msgr2=0x7f2fa8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/1748576792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fa8003e80 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc070c40 tx=0x7f2fac01cf80 comp rx=0 tx=0).stop 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc24d3640 1 -- 172.21.15.170:0/1748576792 reap_dead start 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc3f36640 1 -- 172.21.15.170:0/1748576792 shutdown_connections 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc3f36640 1 -- 172.21.15.170:0/1748576792 >> 172.21.15.170:0/1748576792 conn(0x7f2fbc0b1c40 msgr2=0x7f2fbc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc3f36640 1 -- 172.21.15.170:0/1748576792 shutdown_connections 2024-09-21T02:21:56.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.064+0000 7f2fc3f36640 1 -- 172.21.15.170:0/1748576792 wait complete. 2024-09-21T02:21:56.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 Processor -- start 2024-09-21T02:21:56.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 -- start start 2024-09-21T02:21:56.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fbc150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fbc1fa600 0x7f2fbc1faa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fbc0c6110 0x7f2fbc0c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc167e40 con 0x7f2fa8003a10 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc0cf970 con 0x7f2fbc0c6110 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2fbc167fc0 con 0x7f2fbc1fa600 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fbc150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fbc150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60482/0 (socket says 172.21.15.170:60482) 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 -- 172.21.15.170:0/3845513904 learned_addr learned my addr 172.21.15.170:0/3845513904 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc24d3640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fbc0c6110 0x7f2fbc0c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc14d1640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fbc1fa600 0x7f2fbc1faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 -- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fbc1fa600 msgr2=0x7f2fbc1faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fbc1fa600 0x7f2fbc1faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 -- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fbc0c6110 msgr2=0x7f2fbc0c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fbc0c6110 0x7f2fbc0c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 -- 172.21.15.170:0/3845513904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc0c9610 con 0x7f2fa8003a10 2024-09-21T02:21:56.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc1cd2640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fbc150680 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f2fac01ca40 tx=0x7f2fac00c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac006290 con 0x7f2fa8003a10 2024-09-21T02:21:56.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fac006450 con 0x7f2fa8003a10 2024-09-21T02:21:56.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003620 con 0x7f2fa8003a10 2024-09-21T02:21:56.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fbc0c6a60 con 0x7f2fa8003a10 2024-09-21T02:21:56.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.068+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fbc1f6fd0 con 0x7f2fa8003a10 2024-09-21T02:21:56.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.072+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fbc0b3350 con 0x7f2fa8003a10 2024-09-21T02:21:56.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.072+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f2fac0037c0 con 0x7f2fa8003a10 2024-09-21T02:21:56.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.072+0000 7f2fb27fc640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2fa0073a70 0x7f2fa0075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.072+0000 7f2fc14d1640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2fa0073a70 0x7f2fa0075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.072+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f2fac0b4020 con 0x7f2fa8003a10 2024-09-21T02:21:56.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.072+0000 7f2fc14d1640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2fa0073a70 0x7f2fa0075f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0c60a0 tx=0x7f2fb8009540 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.076+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2fbc0c6a60 con 0x7f2fa8003a10 2024-09-21T02:21:56.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fbc0b2180 con 0x7f2fa8003a10 2024-09-21T02:21:56.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fb27fc640 1 -- 172.21.15.170:0/3845513904 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5245 (secure 0 0 0) 0x7f2fbc0cd450 con 0x7f2fa8003a10 2024-09-21T02:21:56.285 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:21:56.285 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":110,"btime":"2024-09-21T02:21:53:926764+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:50.297124+0000","modified":"2024-09-21T02:21:53.314129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8934":{"gid":8934,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2398609544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2398609544},{"type":"v1","addr":"172.21.15.170:6835","nonce":2398609544}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8934,"qdb_cluster":[8934]},"id":12}]} 2024-09-21T02:21:56.285 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 110 2024-09-21T02:21:56.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2fa0073a70 msgr2=0x7f2fa0075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2fa0073a70 0x7f2fa0075f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0c60a0 tx=0x7f2fb8009540 comp rx=0 tx=0).stop 2024-09-21T02:21:56.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 msgr2=0x7f2fbc150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fbc150680 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f2fac01ca40 tx=0x7f2fac00c2f0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 shutdown_connections 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2fa0073a70 0x7f2fa0075f80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fbc1fa600 0x7f2fbc1faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fbc0c6110 0x7f2fbc0c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 --2- 172.21.15.170:0/3845513904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa8003a10 0x7f2fbc150680 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 >> 172.21.15.170:0/3845513904 conn(0x7f2fbc0b1c40 msgr2=0x7f2fbc0c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 shutdown_connections 2024-09-21T02:21:56.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.284+0000 7f2fc3f36640 1 -- 172.21.15.170:0/3845513904 wait complete. 2024-09-21T02:21:56.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:21:50.297124+0000', 'modified': '2024-09-21T02:21:53.314129+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8934}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8934': {'gid': 8934, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/2398609544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2398609544}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2398609544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8934, 'qdb_cluster': [8934]} 2024-09-21T02:21:56.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:21:56.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:21:56.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 -- 172.21.15.170:0/2805567011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 msgr2=0x7f72b80d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/2805567011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b80d14d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009900 tx=0x7f72a801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 -- 172.21.15.170:0/2805567011 shutdown_connections 2024-09-21T02:21:56.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/2805567011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72b80d3170 0x7f72b806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/2805567011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b80d14d0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 -- 172.21.15.170:0/2805567011 >> 172.21.15.170:0/2805567011 conn(0x7f72b808bf50 msgr2=0x7f72b80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 -- 172.21.15.170:0/2805567011 shutdown_connections 2024-09-21T02:21:56.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 -- 172.21.15.170:0/2805567011 wait complete. 2024-09-21T02:21:56.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 Processor -- start 2024-09-21T02:21:56.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 -- start start 2024-09-21T02:21:56.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.876+0000 7f72bdfc4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72b806df20 0x7f72b80c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b81521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72b80d3170 0x7f72b80c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72b816bf80 con 0x7f72b80d10c0 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72b816be00 con 0x7f72b806df20 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f72b816c100 con 0x7f72b80d3170 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b81521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b81521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60490/0 (socket says 172.21.15.170:60490) 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 -- 172.21.15.170:0/205763971 learned_addr learned my addr 172.21.15.170:0/205763971 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b77fe640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72b806df20 0x7f72b80c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b7fff640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72b80d3170 0x7f72b80c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 -- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72b80d3170 msgr2=0x7f72b80c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72b80d3170 0x7f72b80c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 -- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72b806df20 msgr2=0x7f72b80c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72b806df20 0x7f72b80c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 -- 172.21.15.170:0/205763971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b80c87b0 con 0x7f72b80d10c0 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b77fe640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72b806df20 0x7f72b80c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b81521c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b420 tx=0x7f72ac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f729ffff640 1 -- 172.21.15.170:0/205763971 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac011830 con 0x7f72b80d10c0 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f729ffff640 1 -- 172.21.15.170:0/205763971 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72ac016070 con 0x7f72b80d10c0 2024-09-21T02:21:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f729ffff640 1 -- 172.21.15.170:0/205763971 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac011d80 con 0x7f72b80d10c0 2024-09-21T02:21:56.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b80caf90 con 0x7f72b80d10c0 2024-09-21T02:21:56.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b80b5040 con 0x7f72b80d10c0 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f729ffff640 1 -- 172.21.15.170:0/205763971 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f72ac01a410 con 0x7f72b80d10c0 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f729ffff640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f72a4073a70 0x7f72a4075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f729ffff640 1 -- 172.21.15.170:0/205763971 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f72ac092290 con 0x7f72b80d10c0 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b77fe640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f72a4073a70 0x7f72a4075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.880+0000 7f72b77fe640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f72a4073a70 0x7f72a4075f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f72a80af000 tx=0x7f72a801b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 Processor -- start 2024-09-21T02:21:56.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- start start 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b48a460 0x55d92b9cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 0x55d92b9cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92b9cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d92b8a0410 con 0x55d92b87a950 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d92b7ddb40 con 0x55d92b48a460 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55d92b7ccf70 con 0x55d92b863ca0 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 0x55d92b9cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 0x55d92b9cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60506/0 (socket says 172.21.15.170:60506) 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3301423744 learned_addr learned my addr 172.21.15.170:0/3301423744 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:56.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3301423744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 msgr2=0x55d92b9cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92b9cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3301423744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b48a460 msgr2=0x55d92b9cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b48a460 0x55d92b9cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3301423744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92b7945b0 con 0x55d92b87a950 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 0x55d92b9cbbe0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f72ac05e680 tx=0x7f72ac05df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967ce50fe953cc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f729d7fa640 1 -- 172.21.15.170:0/3301423744 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac065600 con 0x55d92b87a950 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f729d7fa640 1 -- 172.21.15.170:0/3301423744 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72ac0631d0 con 0x55d92b87a950 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f729d7fa640 1 -- 172.21.15.170:0/3301423744 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac05db90 con 0x55d92b87a950 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- 172.21.15.170:0/3301423744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 msgr2=0x55d92b9cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 0x55d92b9cbbe0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f72ac05e680 tx=0x7f72ac05df90 comp rx=0 tx=0).stop 2024-09-21T02:21:56.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- 172.21.15.170:0/3301423744 shutdown_connections 2024-09-21T02:21:56.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92b9cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b87a950 0x55d92b9cbbe0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 --2- 172.21.15.170:0/3301423744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b48a460 0x55d92b9cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- 172.21.15.170:0/3301423744 >> 172.21.15.170:0/3301423744 conn(0x55d92b950d90 msgr2=0x55d92b980690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- 172.21.15.170:0/3301423744 shutdown_connections 2024-09-21T02:21:56.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.884+0000 7f72c528f000 1 -- 172.21.15.170:0/3301423744 wait complete. 2024-09-21T02:21:56.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- start start 2024-09-21T02:21:56.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 Processor -- start 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b48a460 0x55d92b9d2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 0x55d92b9d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92b9d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d92b7675f0 con 0x55d92b863ca0 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d92b7d0d80 con 0x55d92b87a950 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55d92b773520 con 0x55d92b48a460 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 10 client.0 _ll_register_callbacks cb 0x55d92b777830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 0x55d92b9d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 0x55d92b9d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60520/0 (socket says 172.21.15.170:60520) 2024-09-21T02:21:56.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3217393593 learned_addr learned my addr 172.21.15.170:0/3217393593 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b7fff640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92b9d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b77fe640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b48a460 0x55d92b9d2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b48a460 msgr2=0x55d92b9d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b48a460 0x55d92b9d2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 msgr2=0x55d92b9d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92b9d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 -- 172.21.15.170:0/3217393593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92b76b190 con 0x55d92b863ca0 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b7fff640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92b9d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 0x55d92b9d2e60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f72ac05b960 tx=0x7f72ac05ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0df36ab3982c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 1 -- 172.21.15.170:0/3217393593 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0631d0 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 1 -- 172.21.15.170:0/3217393593 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f72ac078bb0 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d92b765110 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 1 -- 172.21.15.170:0/3217393593 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f72ac078db0 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 10 client.9054 fetch_fsmap learned FSMap version 110 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55d92b851f10 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 1 -- 172.21.15.170:0/3217393593 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d92b765110 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 1 -- 172.21.15.170:0/3217393593 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 110) ==== 3502+0+0 (secure 0 0 0) 0x7f72ac065400 con 0x55d92b863ca0 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 10 client.9054 fetch_fsmap finished waiting for FSMap version 110 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 10 client.9054 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 20 client.9054 populate_metadata read hostname 'smithi170' 2024-09-21T02:21:56.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b46f7d0 0x55d92b9dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 4 client.9054 mds_command: new command op to 8934 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 --> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d92b7dfa90 con 0x55d92b46f7d0 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b77fe640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b46f7d0 0x55d92b9dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72b77fe640 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b46f7d0 0x55d92b9dcd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5acb99eb6e9f644e server_cookie=9f0eee289e32418d in_seq=0 out_seq=0 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.888+0000 7f72967fc640 10 client.9054 ms_handle_connect on v2:172.21.15.170:6834/2398609544 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72967fc640 1 -- 172.21.15.170:0/3217393593 <== mds.0 v2:172.21.15.170:6834/2398609544 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d92b7dfa90 con 0x55d92b46f7d0 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72967fc640 10 client.9054 handle_command_reply: tid=0 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 client.9054 shutdown 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 msgr2=0x55d92b9d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 0x55d92b9d2e60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f72ac05b960 tx=0x7f72ac05ae40 comp rx=0 tx=0).stop 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 shutdown_connections 2024-09-21T02:21:56.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b46f7d0 0x55d92b9dcd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92b9d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b863ca0 0x55d92b9d2e60 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 --2- 172.21.15.170:0/3217393593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b48a460 0x55d92b9d2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72967fc640 0 client.9054 ms_handle_reset on v2:172.21.15.170:6834/2398609544 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 >> 172.21.15.170:0/3217393593 conn(0x55d92b950d90 msgr2=0x55d92b987440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 shutdown_connections 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 1 -- 172.21.15.170:0/3217393593 wait complete. 2024-09-21T02:21:56.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.892+0000 7f72c528f000 20 client.9054 trim_cache size 0 max 16384 2024-09-21T02:21:56.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.908+0000 7f72c528f000 1 Processor -- start 2024-09-21T02:21:56.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.908+0000 7f72c528f000 1 -- start start 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92ba3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92ba3db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92ba3e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72b8205500 con 0x55d92b48a460 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72b80b6500 con 0x55d92b87a950 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f72b80b5840 con 0x55d92b863ca0 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92ba3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92ba3d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60522/0 (socket says 172.21.15.170:60522) 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 -- 172.21.15.170:0/381743889 learned_addr learned my addr 172.21.15.170:0/381743889 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92ba3db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b7fff640 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92ba3e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 -- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 msgr2=0x55d92ba3db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92ba3db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 -- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 msgr2=0x55d92ba3e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92ba3e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 -- 172.21.15.170:0/381743889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d92b828db0 con 0x55d92b48a460 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72b77fe640 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92ba3d630 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005f80 tx=0x7f72a801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b996e6ee5dcd9800 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f729cff9640 1 -- 172.21.15.170:0/381743889 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80d0070 con 0x55d92b48a460 2024-09-21T02:21:56.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f729cff9640 1 -- 172.21.15.170:0/381743889 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72a8005130 con 0x55d92b48a460 2024-09-21T02:21:56.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f729cff9640 1 -- 172.21.15.170:0/381743889 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80b2040 con 0x55d92b48a460 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- 172.21.15.170:0/381743889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 msgr2=0x55d92ba3d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92ba3d630 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005f80 tx=0x7f72a801c8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- 172.21.15.170:0/381743889 shutdown_connections 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b863ca0 0x55d92ba3db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b87a950 0x55d92ba3e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 --2- 172.21.15.170:0/381743889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92ba3d630 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- 172.21.15.170:0/381743889 >> 172.21.15.170:0/381743889 conn(0x55d92b950d90 msgr2=0x55d92b97fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- 172.21.15.170:0/381743889 shutdown_connections 2024-09-21T02:21:56.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- 172.21.15.170:0/381743889 wait complete. 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 -- start start 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.912+0000 7f72c528f000 1 Processor -- start 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92b9fc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b863ca0 0x55d92b9fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b87a950 0x55d92b9fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d92b7675f0 con 0x55d92b48a460 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72b80b5e90 con 0x55d92b863ca0 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55d92b7d0d80 con 0x55d92b87a950 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 10 client.0 _ll_register_callbacks cb 0x7f72ac063050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92b9fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92b9fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60524/0 (socket says 172.21.15.170:60524) 2024-09-21T02:21:56.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 -- 172.21.15.170:0/3624473043 learned_addr learned my addr 172.21.15.170:0/3624473043 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 -- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b87a950 msgr2=0x55d92b9fcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b87a950 0x55d92b9fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b863ca0 0x55d92b9fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 -- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b863ca0 msgr2=0x55d92b9fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b863ca0 0x55d92b9fc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 -- 172.21.15.170:0/3624473043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d92b793ab0 con 0x55d92b48a460 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b77fe640 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92b9fc260 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005f80 tx=0x7f72a8004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc1a4bb71465358 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 1 -- 172.21.15.170:0/3624473043 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80d0070 con 0x55d92b48a460 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 1 -- 172.21.15.170:0/3624473043 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f72a80054e0 con 0x55d92b48a460 2024-09-21T02:21:56.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a8005130 con 0x55d92b48a460 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 1 -- 172.21.15.170:0/3624473043 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f72a80d8050 con 0x55d92b48a460 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 10 client.9060 fetch_fsmap learned FSMap version 110 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55d92b772070 con 0x55d92b48a460 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 1 -- 172.21.15.170:0/3624473043 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72a8005130 con 0x55d92b48a460 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 1 -- 172.21.15.170:0/3624473043 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 110) ==== 3502+0+0 (secure 0 0 0) 0x7f72a80db020 con 0x55d92b48a460 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 10 client.9060 fetch_fsmap finished waiting for FSMap version 110 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 10 client.9060 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 20 client.9060 populate_metadata read hostname 'smithi170' 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b45d830 0x55d92b9fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 4 client.9060 mds_command: new command op to 8934 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 --> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] -- command(tid 0: {"prefix": "status"}) -- 0x55d92b7db540 con 0x55d92b45d830 2024-09-21T02:21:56.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b45d830 0x55d92b9fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 10 client.9060 ms_handle_connect on v2:172.21.15.170:6834/2398609544 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72b6ffd640 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b45d830 0x55d92b9fd610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757adb2c53ace11d server_cookie=45acba7a875be4fa in_seq=0 out_seq=0 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 1 -- 172.21.15.170:0/3624473043 <== mds.0 v2:172.21.15.170:6834/2398609544 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d92b7db540 con 0x55d92b45d830 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 10 client.9060 handle_command_reply: tid=0 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 client.9060 shutdown 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 msgr2=0x55d92b9fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92b9fc260 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005f80 tx=0x7f72a8004490 comp rx=0 tx=0).stop 2024-09-21T02:21:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 shutdown_connections 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:6834/2398609544,v1:172.21.15.170:6835/2398609544] conn(0x55d92b45d830 0x55d92b9fd610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55d92b87a950 0x55d92b9fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d92b863ca0 0x55d92b9fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72967fc640 0 client.9060 ms_handle_reset on v2:172.21.15.170:6834/2398609544 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 --2- 172.21.15.170:0/3624473043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d92b48a460 0x55d92b9fc260 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 >> 172.21.15.170:0/3624473043 conn(0x55d92b950d90 msgr2=0x55d92b9dc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 shutdown_connections 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 1 -- 172.21.15.170:0/3624473043 wait complete. 2024-09-21T02:21:56.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72c528f000 20 client.9060 trim_cache size 0 max 16384 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8934, 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 6.050817618, 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 110, 2024-09-21T02:21:56.926 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 153, 2024-09-21T02:21:56.927 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 153, 2024-09-21T02:21:56.927 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.918080847000001 2024-09-21T02:21:56.927 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f72a4073a70 msgr2=0x7f72a4075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f72a4073a70 0x7f72a4075f80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f72a80af000 tx=0x7f72a801b120 comp rx=0 tx=0).stop 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 msgr2=0x7f72b81521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b81521c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b420 tx=0x7f72ac00b8e0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 shutdown_connections 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f72a4073a70 0x7f72a4075f80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72b80d3170 0x7f72b80c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72b80d10c0 0x7f72b81521c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 --2- 172.21.15.170:0/205763971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72b806df20 0x7f72b80c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 >> 172.21.15.170:0/205763971 conn(0x7f72b808bf50 msgr2=0x7f72b80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 shutdown_connections 2024-09-21T02:21:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:56.916+0000 7f72bdfc4640 1 -- 172.21.15.170:0/205763971 wait complete. 2024-09-21T02:21:56.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 -- 172.21.15.170:0/3338342577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c1c004810 msgr2=0x7f7c1c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 --2- 172.21.15.170:0/3338342577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c1c004810 0x7f7c1c004c20 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7c34073750 tx=0x7f7c2801cbe0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c33fff640 1 -- 172.21.15.170:0/3338342577 reap_dead start 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 -- 172.21.15.170:0/3338342577 shutdown_connections 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 -- 172.21.15.170:0/3338342577 >> 172.21.15.170:0/3338342577 conn(0x7f7c3409a9c0 msgr2=0x7f7c340d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 -- 172.21.15.170:0/3338342577 shutdown_connections 2024-09-21T02:21:57.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 -- 172.21.15.170:0/3338342577 wait complete. 2024-09-21T02:21:57.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 Processor -- start 2024-09-21T02:21:57.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.076+0000 7f7c39aba640 1 -- start start 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004810 0x7f7c341f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 0x7f7c340cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c340cfbe0 0x7f7c341f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3406cf80 con 0x7f7c3415d810 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3406ce00 con 0x7f7c340cfbe0 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3406d100 con 0x7f7c1c004810 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 0x7f7c340cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:57.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 0x7f7c340cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60538/0 (socket says 172.21.15.170:60538) 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 -- 172.21.15.170:0/4189926719 learned_addr learned my addr 172.21.15.170:0/4189926719 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c337fe640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004810 0x7f7c341f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c33fff640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c340cfbe0 0x7f7c341f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 -- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004810 msgr2=0x7f7c341f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004810 0x7f7c341f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 -- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c340cfbe0 msgr2=0x7f7c341f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c340cfbe0 0x7f7c341f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 -- 172.21.15.170:0/4189926719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c340cfff0 con 0x7f7c3415d810 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c33fff640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c340cfbe0 0x7f7c341f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:57.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c337fe640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004810 0x7f7c341f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c32ffd640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 0x7f7c340cf6a0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7c28001ec0 tx=0x7f7c280056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280ab040 con 0x7f7c3415d810 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c280be070 con 0x7f7c3415d810 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c340c7700 con 0x7f7c3415d810 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28004a20 con 0x7f7c3415d810 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c340d0550 con 0x7f7c3415d810 2024-09-21T02:21:57.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c340c7470 con 0x7f7c3415d810 2024-09-21T02:21:57.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.080+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f7c280b6070 con 0x7f7c3415d810 2024-09-21T02:21:57.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.084+0000 7f7c1bfff640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c20073a70 0x7f7c20075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:57.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.084+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7f7c2812e3f0 con 0x7f7c3415d810 2024-09-21T02:21:57.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.084+0000 7f7c337fe640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c20073a70 0x7f7c20075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:57.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.084+0000 7f7c337fe640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c20073a70 0x7f7c20075f80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005870 tx=0x7f7c24005ef0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:57.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.088+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7c340cfff0 con 0x7f7c3415d810 2024-09-21T02:21:57.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.288+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7c340b1730 con 0x7f7c3415d810 2024-09-21T02:21:57.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.288+0000 7f7c1bfff640 1 -- 172.21.15.170:0/4189926719 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) ==== 53+0+73 (secure 0 0 0) 0x7f7c280168f0 con 0x7f7c3415d810 2024-09-21T02:21:57.289 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c20073a70 msgr2=0x7f7c20075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c20073a70 0x7f7c20075f80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005870 tx=0x7f7c24005ef0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 msgr2=0x7f7c340cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 0x7f7c340cf6a0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7c28001ec0 tx=0x7f7c280056e0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 shutdown_connections 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c20073a70 0x7f7c20075f80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c340cfbe0 0x7f7c341f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3415d810 0x7f7c340cf6a0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 --2- 172.21.15.170:0/4189926719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004810 0x7f7c341f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 >> 172.21.15.170:0/4189926719 conn(0x7f7c3409a9c0 msgr2=0x7f7c340b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 shutdown_connections 2024-09-21T02:21:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57.292+0000 7f7c39aba640 1 -- 172.21.15.170:0/4189926719 wait complete. 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:21:57.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:21:57.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:57.311 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:21:57.317 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:21:57.317 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:21:57.317 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:21:57.317 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.317 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:21:57.317 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.318 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:21:57.320 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.320 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:21:57.320 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:57.320 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:21:57.320 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:21:57.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:57.320 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:21:57.368 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:21:57.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:57.370 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:21:57.419 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:21:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:21:57.420 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:21:57.421 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:21:57.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:21:57.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:21:57.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:21:57.421 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:21:57.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:21:57.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:57.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:57.540 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:21:57.592 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:21:57.592 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:21:57.592 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:21:57.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:57.593 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:21:57.648 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:21:57.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:21:57.648 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:57.695 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:21:57.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:21:57.696 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:57.744 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:21:57.805 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:57.854 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:21:57.854 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:21:57.854 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:21:57.855 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:57.856 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:57.857 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:57.857 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:57.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:57.857 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:21:57.904 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:21:57.904 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:21:57.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:21:57.967 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:58.020 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:21:58.020 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:21:58.020 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:21:58.021 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:58.022 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:58.023 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:58.023 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:58.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:58.023 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:21:58.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:21:58.063+0000 7fd00089b580 -1 init, newargv = 0x5626d2efefd0 newargc=15 2024-09-21T02:21:58.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[52258]: starting ceph client 2024-09-21T02:21:58.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[52258]: starting fuse 2024-09-21T02:21:59.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:21:59.185 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:21:59.233 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:21:59.234 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:21:59.235 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:59.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:59.236 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:21:59.284 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:21:59.284 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:21:59.284 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:21:59.285 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:21:59.286 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:21:59.286 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:21:59.409 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.52258.asok 2024-09-21T02:21:59.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:21:59.421 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52258.asok 2024-09-21T02:21:59.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:59.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52258.asok status 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:21:59.596 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "52258", 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 9072, 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 9072 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:21:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3232207103 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 3232207103 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.9072 192.168.0.1:0/3232207103", 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/3232207103", 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 110, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 153, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:21:59.598 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:21:59.599 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:21:59.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:59.612 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:21:59.612 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:59.618 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:21:59.618 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:59.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:21:59.618 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:21:59.671 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:21:59.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 -- 172.21.15.170:0/2366769968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0ce1c0 msgr2=0x7ff15c0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:59.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 --2- 172.21.15.170:0/2366769968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0ce1c0 0x7ff15c0ce5d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7ff154009c80 tx=0x7ff15401cda0 comp rx=0 tx=0).stop 2024-09-21T02:21:59.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 -- 172.21.15.170:0/2366769968 shutdown_connections 2024-09-21T02:21:59.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 --2- 172.21.15.170:0/2366769968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0ce1c0 0x7ff15c0ce5d0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:59.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 --2- 172.21.15.170:0/2366769968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cf7e0 0x7ff15c070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:59.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 -- 172.21.15.170:0/2366769968 >> 172.21.15.170:0/2366769968 conn(0x7ff15c0b1c40 msgr2=0x7ff15c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:21:59.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 -- 172.21.15.170:0/2366769968 shutdown_connections 2024-09-21T02:21:59.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 -- 172.21.15.170:0/2366769968 wait complete. 2024-09-21T02:21:59.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 Processor -- start 2024-09-21T02:21:59.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.871+0000 7ff162aca640 1 -- start start 2024-09-21T02:21:59.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 0x7ff15c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:59.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c0ce1c0 0x7ff15c1520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:59.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cf7e0 0x7ff15c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:59.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c1690d0 con 0x7ff15c0cb460 2024-09-21T02:21:59.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c168f50 con 0x7ff15c0ce1c0 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff15c169250 con 0x7ff15c0cf7e0 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 0x7ff15c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 0x7ff15c154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60562/0 (socket says 172.21.15.170:60562) 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 -- 172.21.15.170:0/807268475 learned_addr learned my addr 172.21.15.170:0/807268475 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff161067640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cf7e0 0x7ff15c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff15bfff640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c0ce1c0 0x7ff15c1520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 -- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cf7e0 msgr2=0x7ff15c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cf7e0 0x7ff15c06eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 -- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c0ce1c0 msgr2=0x7ff15c1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c0ce1c0 0x7ff15c1520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 -- 172.21.15.170:0/807268475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c1f8d10 con 0x7ff15c0cb460 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff160866640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 0x7ff15c154500 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c9f0 tx=0x7ff15000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:59.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150007bb0 con 0x7ff15c0cb460 2024-09-21T02:21:59.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff150007e40 con 0x7ff15c0cb460 2024-09-21T02:21:59.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15000f690 con 0x7ff15c0cb460 2024-09-21T02:21:59.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c1f8ff0 con 0x7ff15c0cb460 2024-09-21T02:21:59.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c15ad10 con 0x7ff15c0cb460 2024-09-21T02:21:59.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15c15af70 con 0x7ff15c0cb460 2024-09-21T02:21:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.875+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7ff15000f830 con 0x7ff15c0cb460 2024-09-21T02:21:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.879+0000 7ff1597fa640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff140073a70 0x7ff140075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:21:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.879+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5737+0+0 (secure 0 0 0) 0x7ff150093530 con 0x7ff15c0cb460 2024-09-21T02:21:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.879+0000 7ff15bfff640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff140073a70 0x7ff140075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:21:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.879+0000 7ff15bfff640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff140073a70 0x7ff140075f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff15c1596d0 tx=0x7ff148009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:21:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:21:59.883+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff150014030 con 0x7ff15c0cb460 2024-09-21T02:22:00.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.091+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff15c0ce5d0 con 0x7ff15c0cb460 2024-09-21T02:22:00.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.091+0000 7ff1597fa640 1 -- 172.21.15.170:0/807268475 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) ==== 53+0+73 (secure 0 0 0) 0x7ff150060090 con 0x7ff15c0cb460 2024-09-21T02:22:00.094 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:22:00.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff140073a70 msgr2=0x7ff140075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:00.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff140073a70 0x7ff140075f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff15c1596d0 tx=0x7ff148009040 comp rx=0 tx=0).stop 2024-09-21T02:22:00.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 msgr2=0x7ff15c154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:00.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 0x7ff15c154500 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c9f0 tx=0x7ff15000ceb0 comp rx=0 tx=0).stop 2024-09-21T02:22:00.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 shutdown_connections 2024-09-21T02:22:00.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff140073a70 0x7ff140075f80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:00.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff15c0cf7e0 0x7ff15c06eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:00.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff15c0ce1c0 0x7ff15c1520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:00.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 --2- 172.21.15.170:0/807268475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff15c0cb460 0x7ff15c154500 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:00.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 >> 172.21.15.170:0/807268475 conn(0x7ff15c0b1c40 msgr2=0x7ff15c158750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:00.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 shutdown_connections 2024-09-21T02:22:00.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:00.095+0000 7ff162aca640 1 -- 172.21.15.170:0/807268475 wait complete. 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:22:00.115 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:22:00.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.116 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:22:00.122 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:22:00.122 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:22:00.122 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:22:00.122 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.122 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:22:00.122 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41989sec preferred_lft 41989sec 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.123 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:22:00.124 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:00.125 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:22:00.125 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:00.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.125 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:22:00.176 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:22:00.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.178 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:00.234 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:00.235 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:00.235 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:22:00.235 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:22:00.323 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:22:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:22:00.350 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:22:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:00.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.358 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:22:00.364 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:22:00.364 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:22:00.364 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:22:00.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.364 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:22:00.417 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:22:00.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:22:00.417 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:00.471 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:22:00.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:22:00.471 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:00.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:22:00.575 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:22:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:22:00.625 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:00.626 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:00.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.627 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:22:00.678 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:22:00.678 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:22:00.722 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:22:00.737 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:22:00.786 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:00.787 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:00.788 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:00.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:00.790 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:22:00.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:22:00.835+0000 7fc9a058d580 -1 init, newargv = 0x559f9c152fd0 newargc=15 2024-09-21T02:22:00.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35868]: starting ceph client 2024-09-21T02:22:00.842 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35868]: starting fuse 2024-09-21T02:22:01.938 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:22:01.955 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:22:02.003 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:22:02.004 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:02.005 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:02.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:02.006 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:22:02.054 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:22:02.054 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:22:02.054 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:22:02.054 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:22:02.054 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:22:02.055 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:22:02.056 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:22:02.057 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:02.057 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:22:02.057 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:22:02.176 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.35868.asok 2024-09-21T02:22:02.179 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:22:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:02.187 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35868.asok 2024-09-21T02:22:02.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:02.187 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35868.asok status 2024-09-21T02:22:02.367 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "35868", 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 7105, 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:22:02.368 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 7105 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2679868824 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2679868824 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:02.369 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.7105 192.168.0.1:0/2679868824", 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/2679868824", 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 110, 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 153, 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:22:02.370 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:22:02.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:02.383 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:22:02.383 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:02.389 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:22:02.389 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:02.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:02.390 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:02.445 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 -- 172.21.15.170:0/3015325032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 msgr2=0x7f5f600739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/3015325032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f600739e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5f600ce380 tx=0x7f5f5801cd60 comp rx=0 tx=0).stop 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 -- 172.21.15.170:0/3015325032 shutdown_connections 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/3015325032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f600739e0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/3015325032 >> v2:172.21.15.191:3300/0 conn(0x7f5f600ca7b0 0x7f5f600cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 -- 172.21.15.170:0/3015325032 >> 172.21.15.170:0/3015325032 conn(0x7f5f600b1c40 msgr2=0x7f5f600b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 -- 172.21.15.170:0/3015325032 shutdown_connections 2024-09-21T02:22:02.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 -- 172.21.15.170:0/3015325032 wait complete. 2024-09-21T02:22:02.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 Processor -- start 2024-09-21T02:22:02.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.615+0000 7f5f65ca2640 1 -- start start 2024-09-21T02:22:02.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f6006ffc0 0x7f5f6009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:02.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f601f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:02.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f600ca7b0 0x7f5f600ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6006f700 con 0x7f5f600735d0 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6006f580 con 0x7f5f6006ffc0 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6006f880 con 0x7f5f600ca7b0 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f601f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f601f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60574/0 (socket says 172.21.15.170:60574) 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 -- 172.21.15.170:0/558012919 learned_addr learned my addr 172.21.15.170:0/558012919 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5f7fe640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f6006ffc0 0x7f5f6009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5ffff640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f600ca7b0 0x7f5f600ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 -- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f600ca7b0 msgr2=0x7f5f600ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:02.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f600ca7b0 0x7f5f600ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 -- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f6006ffc0 msgr2=0x7f5f6009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f6006ffc0 0x7f5f6009a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 -- 172.21.15.170:0/558012919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6009af00 con 0x7f5f600735d0 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5ffff640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f600ca7b0 0x7f5f600ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f57fff640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f601f7a00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b3e0 tx=0x7f5f4800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48015020 con 0x7f5f600735d0 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f601f7f40 con 0x7f5f600735d0 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f48011940 con 0x7f5f600735d0 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48011d90 con 0x7f5f600735d0 2024-09-21T02:22:02.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f600aef20 con 0x7f5f600735d0 2024-09-21T02:22:02.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f600ae9b0 con 0x7f5f600735d0 2024-09-21T02:22:02.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.619+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f5f48018020 con 0x7f5f600735d0 2024-09-21T02:22:02.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.623+0000 7f5f5cff9640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5f40073aa0 0x7f5f40075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:02.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.623+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f5f48092ec0 con 0x7f5f600735d0 2024-09-21T02:22:02.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.623+0000 7f5f5f7fe640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5f40073aa0 0x7f5f40075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:02.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.623+0000 7f5f5f7fe640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5f40073aa0 0x7f5f40075fb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000b810 tx=0x7f5f50008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:02.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.627+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5f4801f020 con 0x7f5f600735d0 2024-09-21T02:22:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.839+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5f6006fdb0 con 0x7f5f600735d0 2024-09-21T02:22:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.979+0000 7f5f5cff9640 1 -- 172.21.15.170:0/558012919 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v111) ==== 136+0+0 (secure 0 0 0) 0x7f5f4805fa20 con 0x7f5f600735d0 2024-09-21T02:22:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5f40073aa0 msgr2=0x7f5f40075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5f40073aa0 0x7f5f40075fb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000b810 tx=0x7f5f50008040 comp rx=0 tx=0).stop 2024-09-21T02:22:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 msgr2=0x7f5f601f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f601f7a00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b3e0 tx=0x7f5f4800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 shutdown_connections 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5f40073aa0 0x7f5f40075fb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f600ca7b0 0x7f5f600ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f600735d0 0x7f5f601f7a00 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 --2- 172.21.15.170:0/558012919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f6006ffc0 0x7f5f6009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 >> 172.21.15.170:0/558012919 conn(0x7f5f600b1c40 msgr2=0x7f5f60075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 shutdown_connections 2024-09-21T02:22:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:02.983+0000 7f5f65ca2640 1 -- 172.21.15.170:0/558012919 wait complete. 2024-09-21T02:22:03.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 -- 172.21.15.170:0/1728332206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 msgr2=0x7f6b18004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 --2- 172.21.15.170:0/1728332206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b18004810 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009b60 tx=0x7f6b1c01d270 comp rx=0 tx=0).stop 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b3266e640 1 -- 172.21.15.170:0/1728332206 reap_dead start 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 -- 172.21.15.170:0/1728332206 shutdown_connections 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 -- 172.21.15.170:0/1728332206 >> 172.21.15.170:0/1728332206 conn(0x7f6b2c0b1c40 msgr2=0x7f6b2c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 -- 172.21.15.170:0/1728332206 shutdown_connections 2024-09-21T02:22:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 -- 172.21.15.170:0/1728332206 wait complete. 2024-09-21T02:22:03.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 Processor -- start 2024-09-21T02:22:03.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.171+0000 7f6b340d1640 1 -- start start 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b2c146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b2c0c72c0 0x7f6b2c0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b2c1f3e70 0x7f6b2c1f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c0c3980 con 0x7f6b18003fe0 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c0c3800 con 0x7f6b2c0c72c0 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2c0cf760 con 0x7f6b2c1f3e70 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b2c146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b2c146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60606/0 (socket says 172.21.15.170:60606) 2024-09-21T02:22:03.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 -- 172.21.15.170:0/2637790555 learned_addr learned my addr 172.21.15.170:0/2637790555 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b3166c640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b2c0c72c0 0x7f6b2c0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b3266e640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b2c1f3e70 0x7f6b2c1f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 -- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b2c1f3e70 msgr2=0x7f6b2c1f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b2c1f3e70 0x7f6b2c1f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 -- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b2c0c72c0 msgr2=0x7f6b2c0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b2c0c72c0 0x7f6b2c0c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 -- 172.21.15.170:0/2637790555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c0c7c10 con 0x7f6b18003fe0 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b31e6d640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b2c146120 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f6b280019d0 tx=0x7f6b2800ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800de60 con 0x7f6b18003fe0 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b28012770 con 0x7f6b18003fe0 2024-09-21T02:22:03.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1f47c0 con 0x7f6b18003fe0 2024-09-21T02:22:03.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28012b80 con 0x7f6b18003fe0 2024-09-21T02:22:03.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c09ae40 con 0x7f6b18003fe0 2024-09-21T02:22:03.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c0ccfa0 con 0x7f6b18003fe0 2024-09-21T02:22:03.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f6b2801a400 con 0x7f6b18003fe0 2024-09-21T02:22:03.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b227fc640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b10073ad0 0x7f6b10075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f6b28095b40 con 0x7f6b18003fe0 2024-09-21T02:22:03.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.175+0000 7f6b3166c640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b10073ad0 0x7f6b10075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.179+0000 7f6b3166c640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b10073ad0 0x7f6b10075fe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0ae000 tx=0x7f6b1c01a010 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:03.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.183+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6b2c0c7c10 con 0x7f6b18003fe0 2024-09-21T02:22:03.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.375+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6b2c0b4c20 con 0x7f6b10073ad0 2024-09-21T02:22:03.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b227fc640 1 -- 172.21.15.170:0/2637790555 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6b2c0b4c20 con 0x7f6b10073ad0 2024-09-21T02:22:03.380 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:22:03.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:03.380 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:22:03.380 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:03.380 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:22:03.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b10073ad0 msgr2=0x7f6b10075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b10073ad0 0x7f6b10075fe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0ae000 tx=0x7f6b1c01a010 comp rx=0 tx=0).stop 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 msgr2=0x7f6b2c146120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b2c146120 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f6b280019d0 tx=0x7f6b2800ff90 comp rx=0 tx=0).stop 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 shutdown_connections 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6b10073ad0 0x7f6b10075fe0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b2c1f3e70 0x7f6b2c1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b2c0c72c0 0x7f6b2c0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 --2- 172.21.15.170:0/2637790555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18003fe0 0x7f6b2c146120 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 >> 172.21.15.170:0/2637790555 conn(0x7f6b2c0b1c40 msgr2=0x7f6b2c15c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 shutdown_connections 2024-09-21T02:22:03.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.379+0000 7f6b340d1640 1 -- 172.21.15.170:0/2637790555 wait complete. 2024-09-21T02:22:03.400 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 -- 172.21.15.170:0/712609859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400ce070 msgr2=0x7f93400ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 --2- 172.21.15.170:0/712609859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400ce070 0x7f93400ce480 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f934006dc90 tx=0x7f933801cd80 comp rx=0 tx=0).stop 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 -- 172.21.15.170:0/712609859 shutdown_connections 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 --2- 172.21.15.170:0/712609859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400ce070 0x7f93400ce480 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 --2- 172.21.15.170:0/712609859 >> v2:172.21.15.191:3300/0 conn(0x7f93400740a0 0x7f93400744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 --2- 172.21.15.170:0/712609859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93400cf7d0 0x7f9340073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 -- 172.21.15.170:0/712609859 >> 172.21.15.170:0/712609859 conn(0x7f93400d0170 msgr2=0x7f93400d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 -- 172.21.15.170:0/712609859 shutdown_connections 2024-09-21T02:22:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 -- 172.21.15.170:0/712609859 wait complete. 2024-09-21T02:22:03.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 Processor -- start 2024-09-21T02:22:03.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.555+0000 7f93451b4640 1 -- start start 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93400740a0 0x7f93400ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93400ce070 0x7f93400c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 0x7f93400c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9340070370 con 0x7f93400cf7d0 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93400701f0 con 0x7f93400ce070 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f93400704f0 con 0x7f93400740a0 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 0x7f93400c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 0x7f93400c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60634/0 (socket says 172.21.15.170:60634) 2024-09-21T02:22:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 -- 172.21.15.170:0/1712389843 learned_addr learned my addr 172.21.15.170:0/1712389843 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933659c640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93400ce070 0x7f93400c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933ed9d640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93400740a0 0x7f93400ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 -- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93400740a0 msgr2=0x7f93400ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93400740a0 0x7f93400ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 -- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93400ce070 msgr2=0x7f93400c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93400ce070 0x7f93400c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 -- 172.21.15.170:0/1712389843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93400b2780 con 0x7f93400cf7d0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933f59e640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 0x7f93400c7420 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9338009bd0 tx=0x7f9338005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93380bb070 con 0x7f93400cf7d0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9338004d40 con 0x7f93400cf7d0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93400b2a60 con 0x7f93400cf7d0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93380b2040 con 0x7f93400cf7d0 2024-09-21T02:22:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93400adf70 con 0x7f93400cf7d0 2024-09-21T02:22:03.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93400ce480 con 0x7f93400cf7d0 2024-09-21T02:22:03.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f93380be070 con 0x7f93400cf7d0 2024-09-21T02:22:03.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f9337fff640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9320073a70 0x7f9320075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:03.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f933812ce40 con 0x7f93400cf7d0 2024-09-21T02:22:03.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.559+0000 7f933ed9d640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9320073a70 0x7f9320075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:03.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.563+0000 7f933ed9d640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9320073a70 0x7f9320075f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9330005fd0 tx=0x7f9330005e40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:03.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.563+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f93380c5020 con 0x7f93400cf7d0 2024-09-21T02:22:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.739+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f934015e050 con 0x7f93400cf7d0 2024-09-21T02:22:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.739+0000 7f9337fff640 1 -- 172.21.15.170:0/1712389843 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f9338016970 con 0x7f93400cf7d0 2024-09-21T02:22:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9320073a70 msgr2=0x7f9320075f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9320073a70 0x7f9320075f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9330005fd0 tx=0x7f9330005e40 comp rx=0 tx=0).stop 2024-09-21T02:22:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 msgr2=0x7f93400c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:03.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 0x7f93400c7420 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9338009bd0 tx=0x7f9338005b00 comp rx=0 tx=0).stop 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 shutdown_connections 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9320073a70 0x7f9320075f80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93400cf7d0 0x7f93400c7420 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93400ce070 0x7f93400c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 --2- 172.21.15.170:0/1712389843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93400740a0 0x7f93400ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 >> 172.21.15.170:0/1712389843 conn(0x7f93400d0170 msgr2=0x7f93400b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 shutdown_connections 2024-09-21T02:22:03.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:03.743+0000 7f93451b4640 1 -- 172.21.15.170:0/1712389843 wait complete. 2024-09-21T02:22:03.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:03.764 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol93210/0349e1db-d8c7-41dd-93f8-161fd2acf379) 2024-09-21T02:22:03.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:03.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:22:03.843 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:22:03.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:03.844 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol93210/0349e1db-d8c7-41dd-93f8-161fd2acf379) 2024-09-21T02:22:03.908 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:03.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol93210/.meta 2024-09-21T02:22:03.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:03.972 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33430/subvol68281/4b956d71-1d72-4df7-9e61-2d1af7be2c27) 2024-09-21T02:22:04.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:04.037 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:22:04.087 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:22:04.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:04.088 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp33430/subvol68281/4b956d71-1d72-4df7-9e61-2d1af7be2c27) 2024-09-21T02:22:04.158 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:04.158 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp33430/subvol68281/.meta 2024-09-21T02:22:04.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol93210 2024-09-21T02:22:04.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 -- 172.21.15.170:0/1275875468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954004d50 msgr2=0x7f5954005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 --2- 172.21.15.170:0/1275875468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954004d50 0x7f5954005160 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f59680b61d0 tx=0x7f596001cf40 comp rx=0 tx=0).stop 2024-09-21T02:22:04.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f596e9d2640 1 -- 172.21.15.170:0/1275875468 reap_dead start 2024-09-21T02:22:04.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 -- 172.21.15.170:0/1275875468 shutdown_connections 2024-09-21T02:22:04.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 -- 172.21.15.170:0/1275875468 >> 172.21.15.170:0/1275875468 conn(0x7f59680d0170 msgr2=0x7f59680d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:04.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 -- 172.21.15.170:0/1275875468 shutdown_connections 2024-09-21T02:22:04.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 -- 172.21.15.170:0/1275875468 wait complete. 2024-09-21T02:22:04.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 Processor -- start 2024-09-21T02:22:04.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.427+0000 7f5970435640 1 -- start start 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5954004d50 0x7f5968150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59681f69f0 0x7f596814db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 0x7f596806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f59680ceb90 con 0x7f59680c8300 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59680b5bb0 con 0x7f59681f69f0 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f59680ced10 con 0x7f5954004d50 2024-09-21T02:22:04.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 0x7f596806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 0x7f596806f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60646/0 (socket says 172.21.15.170:60646) 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 -- 172.21.15.170:0/1074610048 learned_addr learned my addr 172.21.15.170:0/1074610048 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596d9d0640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59681f69f0 0x7f596814db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e1d1640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5954004d50 0x7f5968150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 -- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5954004d50 msgr2=0x7f5968150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5954004d50 0x7f5968150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 -- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59681f69f0 msgr2=0x7f596814db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59681f69f0 0x7f596814db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 -- 172.21.15.170:0/1074610048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59681513a0 con 0x7f59680c8300 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f596e9d2640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 0x7f596806f7d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f596001ca00 tx=0x7f5960001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960006ca0 con 0x7f59680c8300 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5960006440 con 0x7f59680c8300 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59681f7420 con 0x7f59680c8300 2024-09-21T02:22:04.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960003860 con 0x7f59680c8300 2024-09-21T02:22:04.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5968150660 con 0x7f59680c8300 2024-09-21T02:22:04.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59680ce6e0 con 0x7f59680c8300 2024-09-21T02:22:04.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.431+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7f5960003a00 con 0x7f59680c8300 2024-09-21T02:22:04.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.435+0000 7f595effd640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f594c073aa0 0x7f594c075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.435+0000 7f596e1d1640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f594c073aa0 0x7f594c075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.435+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f596012d440 con 0x7f59680c8300 2024-09-21T02:22:04.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.435+0000 7f596e1d1640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f594c073aa0 0x7f594c075fb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5964005b50 tx=0x7f5964005be0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:04.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.435+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f59600bf060 con 0x7f59680c8300 2024-09-21T02:22:04.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.631+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol93210", "target": ["mon-mgr", ""]}) -- 0x7f59681504a0 con 0x7f594c073aa0 2024-09-21T02:22:04.642 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:04.639+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:04.642 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:04.639+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:04.642 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:04.639+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:04.642 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:04.639+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:04.642 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:04.639+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:04.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f595effd640 1 -- 172.21.15.170:0/1074610048 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f5964009be0 con 0x7f594c073aa0 2024-09-21T02:22:04.704 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvol93210/0349e1db-d8c7-41dd-93f8-161fd2acf379 2024-09-21T02:22:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f594c073aa0 msgr2=0x7f594c075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f594c073aa0 0x7f594c075fb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5964005b50 tx=0x7f5964005be0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 msgr2=0x7f596806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 0x7f596806f7d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f596001ca00 tx=0x7f5960001a60 comp rx=0 tx=0).stop 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 shutdown_connections 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f594c073aa0 0x7f594c075fb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59680c8300 0x7f596806f7d0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59681f69f0 0x7f596814db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 --2- 172.21.15.170:0/1074610048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5954004d50 0x7f5968150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 >> 172.21.15.170:0/1074610048 conn(0x7f59680d0170 msgr2=0x7f596815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 shutdown_connections 2024-09-21T02:22:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.703+0000 7f5970435640 1 -- 172.21.15.170:0/1074610048 wait complete. 2024-09-21T02:22:04.724 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol68281 subvol_grp33430 2024-09-21T02:22:04.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 -- 172.21.15.170:0/530060490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200d25e0 msgr2=0x7fc5200d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 --2- 172.21.15.170:0/530060490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200d25e0 0x7fc5200d29f0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc518009c80 tx=0x7fc51801cce0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 -- 172.21.15.170:0/530060490 shutdown_connections 2024-09-21T02:22:04.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 --2- 172.21.15.170:0/530060490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200d25e0 0x7fc5200d29f0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 --2- 172.21.15.170:0/530060490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5200cf860 0x7fc5200cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 -- 172.21.15.170:0/530060490 >> 172.21.15.170:0/530060490 conn(0x7fc52009d830 msgr2=0x7fc52008bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 -- 172.21.15.170:0/530060490 shutdown_connections 2024-09-21T02:22:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 -- 172.21.15.170:0/530060490 wait complete. 2024-09-21T02:22:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 Processor -- start 2024-09-21T02:22:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.879+0000 7fc528541640 1 -- start start 2024-09-21T02:22:04.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 0x7fc5201585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5200cf860 0x7fc5200b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5200d25e0 0x7fc52015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc52016bd70 con 0x7fc5200b95f0 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5200b5280 con 0x7fc5200d25e0 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc52016bef0 con 0x7fc5200cf860 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 0x7fc5201585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 0x7fc5201585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60660/0 (socket says 172.21.15.170:60660) 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 -- 172.21.15.170:0/3331349675 learned_addr learned my addr 172.21.15.170:0/3331349675 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc526ade640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5200d25e0 0x7fc52015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc525adc640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5200cf860 0x7fc5200b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 -- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5200cf860 msgr2=0x7fc5200b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5200cf860 0x7fc5200b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 -- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5200d25e0 msgr2=0x7fc52015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:04.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5200d25e0 0x7fc52015b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 -- 172.21.15.170:0/3331349675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5200cd7e0 con 0x7fc5200b95f0 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc526ade640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5200d25e0 0x7fc52015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc5262dd640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 0x7fc5201585e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc51000b440 tx=0x7fc51000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510015020 con 0x7fc5200b95f0 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc510011980 con 0x7fc5200b95f0 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5200cdac0 con 0x7fc5200b95f0 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510011dd0 con 0x7fc5200b95f0 2024-09-21T02:22:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5200ce080 con 0x7fc5200b95f0 2024-09-21T02:22:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5200ce460 con 0x7fc5200b95f0 2024-09-21T02:22:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 97999+0+0 (secure 0 0 0) 0x7fc510018020 con 0x7fc5200b95f0 2024-09-21T02:22:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc516ffd640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc504073aa0 0x7fc504075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc525adc640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc504073aa0 0x7fc504075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.883+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7fc510092dd0 con 0x7fc5200b95f0 2024-09-21T02:22:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.887+0000 7fc525adc640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc504073aa0 0x7fc504075fb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0058d0 tx=0x7fc51c005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:04.887+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc51001f030 con 0x7fc5200b95f0 2024-09-21T02:22:05.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.015+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 26) ==== 98050+0+0 (secure 0 0 0) 0x7fc51000f050 con 0x7fc5200b95f0 2024-09-21T02:22:05.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.095+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol68281", "group_name": "subvol_grp33430", "target": ["mon-mgr", ""]}) -- 0x7fc5200cfc70 con 0x7fc504073aa0 2024-09-21T02:22:05.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.175+0000 7fc516ffd640 1 -- 172.21.15.170:0/3331349675 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fc5200cfc70 con 0x7fc504073aa0 2024-09-21T02:22:05.178 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp33430/subvol68281/4b956d71-1d72-4df7-9e61-2d1af7be2c27 2024-09-21T02:22:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc504073aa0 msgr2=0x7fc504075fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc504073aa0 0x7fc504075fb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0058d0 tx=0x7fc51c005860 comp rx=0 tx=0).stop 2024-09-21T02:22:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 msgr2=0x7fc5201585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 0x7fc5201585e0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc51000b440 tx=0x7fc51000b900 comp rx=0 tx=0).stop 2024-09-21T02:22:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 shutdown_connections 2024-09-21T02:22:05.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc504073aa0 0x7fc504075fb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5200cf860 0x7fc5200b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5200d25e0 0x7fc52015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 --2- 172.21.15.170:0/3331349675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5200b95f0 0x7fc5201585e0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 >> 172.21.15.170:0/3331349675 conn(0x7fc52009d830 msgr2=0x7fc5200b7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 shutdown_connections 2024-09-21T02:22:05.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.179+0000 7fc528541640 1 -- 172.21.15.170:0/3331349675 wait complete. 2024-09-21T02:22:05.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:05.203 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol93210/.meta) 2024-09-21T02:22:05.220 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:22:05.220 INFO:teuthology.orchestra.run.smithi170.stdout:version = 2 2024-09-21T02:22:05.220 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:22:05.220 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/_nogroup/subvol93210/0349e1db-d8c7-41dd-93f8-161fd2acf379 2024-09-21T02:22:05.220 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:22:05.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:05.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:05.222 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp33430/subvol68281/.meta) 2024-09-21T02:22:05.282 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:22:05.282 INFO:teuthology.orchestra.run.smithi170.stdout:version = 2 2024-09-21T02:22:05.282 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:22:05.282 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/subvol_grp33430/subvol68281/4b956d71-1d72-4df7-9e61-2d1af7be2c27 2024-09-21T02:22:05.282 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:22:05.282 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:05.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol93210 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 -- 172.21.15.170:0/2628114293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0cea80 msgr2=0x7f035c0cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 --2- 172.21.15.170:0/2628114293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0cea80 0x7f035c0cee90 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f034c009930 tx=0x7f034c01c8e0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 -- 172.21.15.170:0/2628114293 shutdown_connections 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 --2- 172.21.15.170:0/2628114293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0cea80 0x7f035c0cee90 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 --2- 172.21.15.170:0/2628114293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b2fc0 0x7f035c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 -- 172.21.15.170:0/2628114293 >> 172.21.15.170:0/2628114293 conn(0x7f035c0d0170 msgr2=0x7f035c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 -- 172.21.15.170:0/2628114293 shutdown_connections 2024-09-21T02:22:05.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.491+0000 7f03629f7640 1 -- 172.21.15.170:0/2628114293 wait complete. 2024-09-21T02:22:05.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 Processor -- start 2024-09-21T02:22:05.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- start start 2024-09-21T02:22:05.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 0x7f035c1fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b2fc0 0x7f035c1fad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0cea80 0x7f035c141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f035c169f50 con 0x7f035c06ef40 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f035c169dd0 con 0x7f035c0b2fc0 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f035c16a0d0 con 0x7f035c0cea80 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 0x7f035c1fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 0x7f035c1fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39174/0 (socket says 172.21.15.170:39174) 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 -- 172.21.15.170:0/4051753076 learned_addr learned my addr 172.21.15.170:0/4051753076 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035b7fe640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b2fc0 0x7f035c1fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 -- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0cea80 msgr2=0x7f035c141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f0360f94640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0cea80 0x7f035c141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0cea80 0x7f035c141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 -- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b2fc0 msgr2=0x7f035c1fad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b2fc0 0x7f035c1fad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 -- 172.21.15.170:0/4051753076 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c09c990 con 0x7f035c06ef40 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035b7fe640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b2fc0 0x7f035c1fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f035bfff640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 0x7f035c1fa820 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f035000d950 tx=0x7f035000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350017070 con 0x7f035c06ef40 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0350009d20 con 0x7f035c06ef40 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c09cc10 con 0x7f035c06ef40 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03500106c0 con 0x7f035c06ef40 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c09d1d0 con 0x7f035c06ef40 2024-09-21T02:22:05.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.495+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c0af260 con 0x7f035c06ef40 2024-09-21T02:22:05.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.499+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 98050+0+0 (secure 0 0 0) 0x7f035001a070 con 0x7f035c06ef40 2024-09-21T02:22:05.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.499+0000 7f0358ff9640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0348073b30 0x7f0348076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.499+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f0350093110 con 0x7f035c06ef40 2024-09-21T02:22:05.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.499+0000 7f035b7fe640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0348073b30 0x7f0348076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.499+0000 7f035b7fe640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0348073b30 0x7f0348076040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f034c0af000 tx=0x7f034c01b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:05.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.503+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f035c09c990 con 0x7f035c06ef40 2024-09-21T02:22:05.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.703+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol93210", "target": ["mon-mgr", ""]}) -- 0x7f035c0aded0 con 0x7f0348073b30 2024-09-21T02:22:05.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f0358ff9640 1 -- 172.21.15.170:0/4051753076 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f035c0aded0 con 0x7f0348073b30 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0348073b30 msgr2=0x7f0348076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0348073b30 0x7f0348076040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f034c0af000 tx=0x7f034c01b120 comp rx=0 tx=0).stop 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 msgr2=0x7f035c1fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 0x7f035c1fa820 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f035000d950 tx=0x7f035000de10 comp rx=0 tx=0).stop 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 shutdown_connections 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0348073b30 0x7f0348076040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0cea80 0x7f035c141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b2fc0 0x7f035c1fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 --2- 172.21.15.170:0/4051753076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c06ef40 0x7f035c1fa820 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 >> 172.21.15.170:0/4051753076 conn(0x7f035c0d0170 msgr2=0x7f035c0acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:05.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 shutdown_connections 2024-09-21T02:22:05.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.735+0000 7f03629f7640 1 -- 172.21.15.170:0/4051753076 wait complete. 2024-09-21T02:22:05.742 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.739+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.742 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.739+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.742 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.739+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.742 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.739+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.743 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.739+0000 7efba12c8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.759 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.755+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.759 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.755+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.759 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.755+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.759 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.759+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.759 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:05.759+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:05.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol68281 subvol_grp33430 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- 172.21.15.170:0/1925008699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34200b4290 msgr2=0x7f34200b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 --2- 172.21.15.170:0/1925008699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34200b4290 0x7f34200b5120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3418009bf0 tx=0x7f341801cba0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- 172.21.15.170:0/1925008699 shutdown_connections 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 --2- 172.21.15.170:0/1925008699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34200b4290 0x7f34200b5120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 --2- 172.21.15.170:0/1925008699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f342015c770 0x7f34200b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- 172.21.15.170:0/1925008699 >> 172.21.15.170:0/1925008699 conn(0x7f34200ced50 msgr2=0x7f34200cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- 172.21.15.170:0/1925008699 shutdown_connections 2024-09-21T02:22:05.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- 172.21.15.170:0/1925008699 wait complete. 2024-09-21T02:22:05.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 Processor -- start 2024-09-21T02:22:05.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- start start 2024-09-21T02:22:05.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34200b4290 0x7f34201567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 0x7f3420156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342015c770 0x7f34201f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3420169f60 con 0x7f34200b7f20 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3420169de0 con 0x7f342015c770 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.919+0000 7f3425046640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f342016a0e0 con 0x7f34200b4290 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 0x7f3420156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 0x7f3420156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39214/0 (socket says 172.21.15.170:39214) 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 -- 172.21.15.170:0/1556766936 learned_addr learned my addr 172.21.15.170:0/1556766936 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341ed9d640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34200b4290 0x7f34201567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341f59e640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342015c770 0x7f34201f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 -- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34200b4290 msgr2=0x7f34201567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34200b4290 0x7f34201567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 -- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342015c770 msgr2=0x7f34201f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342015c770 0x7f34201f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 -- 172.21.15.170:0/1556766936 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342015b0b0 con 0x7f34200b7f20 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341659c640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 0x7f3420156d30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f34200d0510 tx=0x7f340800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:05.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408004220 con 0x7f34200b7f20 2024-09-21T02:22:05.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34080043c0 con 0x7f34200b7f20 2024-09-21T02:22:05.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342015b390 con 0x7f34200b7f20 2024-09-21T02:22:05.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340800cb10 con 0x7f34200b7f20 2024-09-21T02:22:05.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342015b950 con 0x7f34200b7f20 2024-09-21T02:22:05.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342015ae20 con 0x7f34200b7f20 2024-09-21T02:22:05.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 98050+0+0 (secure 0 0 0) 0x7f340800ccb0 con 0x7f34200b7f20 2024-09-21T02:22:05.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3417fff640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3400073b00 0x7f3400076010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:05.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341ed9d640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3400073b00 0x7f3400076010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:05.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f3408093250 con 0x7f34200b7f20 2024-09-21T02:22:05.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.923+0000 7f341ed9d640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3400073b00 0x7f3400076010 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3410007fd0 tx=0x7f3410007ea0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:05.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:05.931+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f340809c080 con 0x7f34200b7f20 2024-09-21T02:22:06.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.135+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol68281", "group_name": "subvol_grp33430", "target": ["mon-mgr", ""]}) -- 0x7f34200b5120 con 0x7f3400073b00 2024-09-21T02:22:06.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.163+0000 7f3417fff640 1 -- 172.21.15.170:0/1556766936 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34200b5120 con 0x7f3400073b00 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3400073b00 msgr2=0x7f3400076010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3400073b00 0x7f3400076010 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3410007fd0 tx=0x7f3410007ea0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 msgr2=0x7f3420156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 0x7f3420156d30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f34200d0510 tx=0x7f340800eb50 comp rx=0 tx=0).stop 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 shutdown_connections 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3400073b00 0x7f3400076010 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f342015c770 0x7f34201f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34200b7f20 0x7f3420156d30 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 --2- 172.21.15.170:0/1556766936 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34200b4290 0x7f34201567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 >> 172.21.15.170:0/1556766936 conn(0x7f34200ced50 msgr2=0x7f34200b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 shutdown_connections 2024-09-21T02:22:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.167+0000 7f3425046640 1 -- 172.21.15.170:0/1556766936 wait complete. 2024-09-21T02:22:06.194 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:22:06.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:22:06.247 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:22:06.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:06.255 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:22:06.255 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:22:06.373 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:22:06.447 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:22:06.447 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:06.447 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:06.447 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:06.453 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:06.453 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:06.501 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:22:06.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 -- 172.21.15.170:0/2568215870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cb460 msgr2=0x7f2c980cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 --2- 172.21.15.170:0/2568215870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cb460 0x7f2c980cb870 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2c980d1160 tx=0x7f2c8c01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 -- 172.21.15.170:0/2568215870 shutdown_connections 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 --2- 172.21.15.170:0/2568215870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cb460 0x7f2c980cb870 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 --2- 172.21.15.170:0/2568215870 >> v2:172.21.15.191:3300/0 conn(0x7f2c980cf7e0 0x7f2c98070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 -- 172.21.15.170:0/2568215870 >> 172.21.15.170:0/2568215870 conn(0x7f2c980b1c40 msgr2=0x7f2c980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 -- 172.21.15.170:0/2568215870 shutdown_connections 2024-09-21T02:22:06.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.707+0000 7f2ca0725640 1 -- 172.21.15.170:0/2568215870 wait complete. 2024-09-21T02:22:06.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 Processor -- start 2024-09-21T02:22:06.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 -- start start 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c980cb460 0x7f2c9809b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 0x7f2c9814a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c980cf7e0 0x7f2c980c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981690d0 con 0x7f2c980cc8c0 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98168f50 con 0x7f2c980cb460 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2c98169250 con 0x7f2c980cf7e0 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 0x7f2c9814a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 0x7f2c9814a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39228/0 (socket says 172.21.15.170:39228) 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 -- 172.21.15.170:0/728583359 learned_addr learned my addr 172.21.15.170:0/728583359 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:06.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9e4c1640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c980cb460 0x7f2c9809b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9ecc2640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c980cf7e0 0x7f2c980c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 -- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c980cf7e0 msgr2=0x7f2c980c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c980cf7e0 0x7f2c980c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 -- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c980cb460 msgr2=0x7f2c9809b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c980cb460 0x7f2c9809b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 -- 172.21.15.170:0/728583359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9814aa60 con 0x7f2c980cc8c0 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9e4c1640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c980cb460 0x7f2c9809b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9ecc2640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c980cf7e0 0x7f2c980c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c9dcc0640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 0x7f2c9814a520 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0b0040 tx=0x7f2c8c0a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0bb070 con 0x7f2c980cc8c0 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8c0044b0 con 0x7f2c980cc8c0 2024-09-21T02:22:06.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9809daa0 con 0x7f2c980cc8c0 2024-09-21T02:22:06.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0b2040 con 0x7f2c980cc8c0 2024-09-21T02:22:06.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.711+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9809e060 con 0x7f2c980cc8c0 2024-09-21T02:22:06.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.715+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c980cb870 con 0x7f2c980cc8c0 2024-09-21T02:22:06.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.715+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 98050+0+0 (secure 0 0 0) 0x7f2c8c0052d0 con 0x7f2c980cc8c0 2024-09-21T02:22:06.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.715+0000 7f2c96ffd640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2c84073b30 0x7f2c84076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:06.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.715+0000 7f2c9e4c1640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2c84073b30 0x7f2c84076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:06.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.715+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f2c8c12c270 con 0x7f2c980cc8c0 2024-09-21T02:22:06.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.715+0000 7f2c9e4c1640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2c84073b30 0x7f2c84076040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009e70 tx=0x7f2c90009850 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:06.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.719+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2c8c0164b0 con 0x7f2c980cc8c0 2024-09-21T02:22:06.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.911+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c980b09d0 con 0x7f2c980cc8c0 2024-09-21T02:22:06.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.911+0000 7f2c96ffd640 1 -- 172.21.15.170:0/728583359 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) ==== 80+0+0 (secure 0 0 0) 0x7f2c8c0f8d40 con 0x7f2c980cc8c0 2024-09-21T02:22:06.913 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:22:06.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2c84073b30 msgr2=0x7f2c84076040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2c84073b30 0x7f2c84076040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009e70 tx=0x7f2c90009850 comp rx=0 tx=0).stop 2024-09-21T02:22:06.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 msgr2=0x7f2c9814a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:06.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 0x7f2c9814a520 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0b0040 tx=0x7f2c8c0a57f0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 shutdown_connections 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2c84073b30 0x7f2c84076040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c980cf7e0 0x7f2c980c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c980cc8c0 0x7f2c9814a520 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 --2- 172.21.15.170:0/728583359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c980cb460 0x7f2c9809b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 >> 172.21.15.170:0/728583359 conn(0x7f2c980b1c40 msgr2=0x7f2c980b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 shutdown_connections 2024-09-21T02:22:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:06.915+0000 7f2ca0725640 1 -- 172.21.15.170:0/728583359 wait complete. 2024-09-21T02:22:06.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:22:06.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:06.936 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:06.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[52258]: fuse finished with error 0 and tester_r 0 2024-09-21T02:22:06.948 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:22:06.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:06.948 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:22:07.032 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:07.032 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:07.033 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:07.041 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:07.041 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:07.090 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:22:07.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 -- 172.21.15.170:0/362013920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0cfbf0 msgr2=0x7f613c0cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 --2- 172.21.15.170:0/362013920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0cfbf0 0x7f613c0cf3a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f613800a950 tx=0x7f613801db70 comp rx=0 tx=0).stop 2024-09-21T02:22:07.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 -- 172.21.15.170:0/362013920 shutdown_connections 2024-09-21T02:22:07.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 --2- 172.21.15.170:0/362013920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f613c0701a0 0x7f613c0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 --2- 172.21.15.170:0/362013920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0cfbf0 0x7f613c0cf3a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 -- 172.21.15.170:0/362013920 >> 172.21.15.170:0/362013920 conn(0x7f613c0b1c40 msgr2=0x7f613c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:07.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 -- 172.21.15.170:0/362013920 shutdown_connections 2024-09-21T02:22:07.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 -- 172.21.15.170:0/362013920 wait complete. 2024-09-21T02:22:07.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 Processor -- start 2024-09-21T02:22:07.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.247+0000 7f6142e86640 1 -- start start 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 0x7f613c0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0705b0 0x7f613c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f613c0cfbf0 0x7f613c1fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f613c168440 con 0x7f613c0701a0 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1682c0 con 0x7f613c0705b0 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f613c1685c0 con 0x7f613c0cfbf0 2024-09-21T02:22:07.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 0x7f613c0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 0x7f613c0b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39242/0 (socket says 172.21.15.170:39242) 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 -- 172.21.15.170:0/4008378339 learned_addr learned my addr 172.21.15.170:0/4008378339 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6141423640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f613c0cfbf0 0x7f613c1fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6133fff640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0705b0 0x7f613c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 -- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f613c0cfbf0 msgr2=0x7f613c1fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f613c0cfbf0 0x7f613c1fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 -- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0705b0 msgr2=0x7f613c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0705b0 0x7f613c0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 -- 172.21.15.170:0/4008378339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c0b3480 con 0x7f613c0701a0 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6133fff640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0705b0 0x7f613c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6140c22640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 0x7f613c0b2f40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f613800a8a0 tx=0x7f6138005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61380b3040 con 0x7f613c0701a0 2024-09-21T02:22:07.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61380053e0 con 0x7f613c0701a0 2024-09-21T02:22:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61380046d0 con 0x7f613c0701a0 2024-09-21T02:22:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c15d740 con 0x7f613c0701a0 2024-09-21T02:22:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c15dd00 con 0x7f613c0701a0 2024-09-21T02:22:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.251+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613c073ac0 con 0x7f613c0701a0 2024-09-21T02:22:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.255+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f61380bf060 con 0x7f613c0701a0 2024-09-21T02:22:07.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.255+0000 7f61317fa640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6128073c20 0x7f6128076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.255+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f613812f710 con 0x7f613c0701a0 2024-09-21T02:22:07.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.255+0000 7f6133fff640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6128073c20 0x7f6128076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.255+0000 7f6133fff640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6128073c20 0x7f6128076130 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f613c09aa70 tx=0x7f612c0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:07.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.259+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f6138017760 con 0x7f613c0701a0 2024-09-21T02:22:07.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.443+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f613c0737e0 con 0x7f613c0701a0 2024-09-21T02:22:07.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.443+0000 7f61317fa640 1 -- 172.21.15.170:0/4008378339 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) ==== 80+0+0 (secure 0 0 0) 0x7f61380f97f0 con 0x7f613c0701a0 2024-09-21T02:22:07.445 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:22:07.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.443+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6128073c20 msgr2=0x7f6128076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.443+0000 7f6142e86640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6128073c20 0x7f6128076130 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f613c09aa70 tx=0x7f612c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.443+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 msgr2=0x7f613c0b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.443+0000 7f6142e86640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 0x7f613c0b2f40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f613800a8a0 tx=0x7f6138005db0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 shutdown_connections 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6128073c20 0x7f6128076130 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f613c0cfbf0 0x7f613c1fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f613c0705b0 0x7f613c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 --2- 172.21.15.170:0/4008378339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c0701a0 0x7f613c0b2f40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 >> 172.21.15.170:0/4008378339 conn(0x7f613c0b1c40 msgr2=0x7f613c161480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 shutdown_connections 2024-09-21T02:22:07.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.447+0000 7f6142e86640 1 -- 172.21.15.170:0/4008378339 wait complete. 2024-09-21T02:22:07.468 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:22:07.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:07.469 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:07.481 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[35868]: fuse finished with error 0 and tester_r 0 2024-09-21T02:22:07.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:22:07.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:07.483 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:22:07.603 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:07.604 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:07.604 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:07.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:22:07.610 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:07.611 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:07.611 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:07.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:22:07.617 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec7933d640 1 -- 172.21.15.170:0/4013164332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 msgr2=0x7fec5c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec7933d640 1 --2- 172.21.15.170:0/4013164332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec5c003eb0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c2e0 tx=0x7fec6801cf60 comp rx=0 tx=0).stop 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec737fe640 1 -- 172.21.15.170:0/4013164332 reap_dead start 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec7933d640 1 -- 172.21.15.170:0/4013164332 shutdown_connections 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec7933d640 1 -- 172.21.15.170:0/4013164332 >> 172.21.15.170:0/4013164332 conn(0x7fec740b1c40 msgr2=0x7fec740b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec7933d640 1 -- 172.21.15.170:0/4013164332 shutdown_connections 2024-09-21T02:22:07.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.815+0000 7fec7933d640 1 -- 172.21.15.170:0/4013164332 wait complete. 2024-09-21T02:22:07.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 Processor -- start 2024-09-21T02:22:07.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 -- start start 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec7414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec7415fb10 0x7fec7415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec7415b6e0 0x7fec7415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec741691e0 con 0x7fec5c003a40 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fec74169060 con 0x7fec7415b6e0 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec7933d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fec74169360 con 0x7fec7415fb10 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec7414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec7414d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39266/0 (socket says 172.21.15.170:39266) 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 -- 172.21.15.170:0/964617318 learned_addr learned my addr 172.21.15.170:0/964617318 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec727fc640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec7415fb10 0x7fec7415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 -- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec7415fb10 msgr2=0x7fec7415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec7415fb10 0x7fec7415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 -- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec7415b6e0 msgr2=0x7fec7415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec7415b6e0 0x7fec7415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 -- 172.21.15.170:0/964617318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7415bd80 con 0x7fec5c003a40 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec727fc640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec7415fb10 0x7fec7415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec72ffd640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec7414d5c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fec6801b660 tx=0x7fec68001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.819+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680a5bf0 con 0x7fec5c003a40 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec68001500 con 0x7fec5c003a40 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7414db00 con 0x7fec5c003a40 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68006330 con 0x7fec5c003a40 2024-09-21T02:22:07.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec740c3f90 con 0x7fec5c003a40 2024-09-21T02:22:07.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7406fa30 con 0x7fec5c003a40 2024-09-21T02:22:07.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fec68003600 con 0x7fec5c003a40 2024-09-21T02:22:07.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec5b7fe640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec60073c20 0x7fec60076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:07.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7fec6812cd90 con 0x7fec5c003a40 2024-09-21T02:22:07.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec727fc640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec60073c20 0x7fec60076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:07.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.823+0000 7fec727fc640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec60073c20 0x7fec60076130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec7415cdf0 tx=0x7fec640091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:07.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:07.827+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fec7415bd80 con 0x7fec5c003a40 2024-09-21T02:22:08.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.039+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec740b4dd0 con 0x7fec5c003a40 2024-09-21T02:22:08.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.039+0000 7fec5b7fe640 1 -- 172.21.15.170:0/964617318 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5245 (secure 0 0 0) 0x7fec68016d10 con 0x7fec5c003a40 2024-09-21T02:22:08.044 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:08.044 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":111,"btime":"2024-09-21T02:22:02:983140+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:50.297124+0000","modified":"2024-09-21T02:21:53.314129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8934":{"gid":8934,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2398609544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2398609544},{"type":"v1","addr":"172.21.15.170:6835","nonce":2398609544}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8934,"qdb_cluster":[8934]},"id":12}]} 2024-09-21T02:22:08.044 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 111 2024-09-21T02:22:08.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec60073c20 msgr2=0x7fec60076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec60073c20 0x7fec60076130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fec7415cdf0 tx=0x7fec640091e0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 msgr2=0x7fec7414d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec7414d5c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fec6801b660 tx=0x7fec68001cd0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 shutdown_connections 2024-09-21T02:22:08.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fec60073c20 0x7fec60076130 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec7415fb10 0x7fec7415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec7415b6e0 0x7fec7415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 --2- 172.21.15.170:0/964617318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec5c003a40 0x7fec7414d5c0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 >> 172.21.15.170:0/964617318 conn(0x7fec740b1c40 msgr2=0x7fec74167b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:08.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 shutdown_connections 2024-09-21T02:22:08.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.043+0000 7fec7933d640 1 -- 172.21.15.170:0/964617318 wait complete. 2024-09-21T02:22:08.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 -- 172.21.15.170:0/2380435583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffafc005360 msgr2=0x7ffafc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2380435583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffafc005360 0x7ffafc00b9d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ffb00005fd0 tx=0x7ffb0001cdc0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 -- 172.21.15.170:0/2380435583 shutdown_connections 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2380435583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffafc005360 0x7ffafc00b9d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 -- 172.21.15.170:0/2380435583 >> 172.21.15.170:0/2380435583 conn(0x7ffb100b1c40 msgr2=0x7ffb100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 -- 172.21.15.170:0/2380435583 shutdown_connections 2024-09-21T02:22:08.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.219+0000 7ffb148ba640 1 -- 172.21.15.170:0/2380435583 wait complete. 2024-09-21T02:22:08.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 Processor -- start 2024-09-21T02:22:08.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 -- start start 2024-09-21T02:22:08.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffafc005360 0x7ffb101f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10074130 0x7ffb101f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 0x7ffb1009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffb100cf760 con 0x7ffb1009bef0 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb100c3a10 con 0x7ffb10074130 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ffb100cf8e0 con 0x7ffafc005360 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 0x7ffb1009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 0x7ffb1009c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39282/0 (socket says 172.21.15.170:39282) 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 -- 172.21.15.170:0/2758244517 learned_addr learned my addr 172.21.15.170:0/2758244517 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0dd9b640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10074130 0x7ffb101f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0e59c640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffafc005360 0x7ffb101f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 -- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffafc005360 msgr2=0x7ffb101f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffafc005360 0x7ffb101f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 -- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10074130 msgr2=0x7ffb101f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10074130 0x7ffb101f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 -- 172.21.15.170:0/2758244517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb101f5240 con 0x7ffb1009bef0 2024-09-21T02:22:08.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0e59c640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffafc005360 0x7ffb101f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb0ed9d640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 0x7ffb1009c300 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000ba60 tx=0x7ffb00004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb000ab040 con 0x7ffb1009bef0 2024-09-21T02:22:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb00003810 con 0x7ffb1009bef0 2024-09-21T02:22:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00003c40 con 0x7ffb1009bef0 2024-09-21T02:22:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1009ae50 con 0x7ffb1009bef0 2024-09-21T02:22:08.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.223+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1009b410 con 0x7ffb1009bef0 2024-09-21T02:22:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.227+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1009abc0 con 0x7ffb1009bef0 2024-09-21T02:22:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.227+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7ffb000af450 con 0x7ffb1009bef0 2024-09-21T02:22:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.227+0000 7ffb06ffd640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffaf0073c20 0x7ffaf0076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.227+0000 7ffb0e59c640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffaf0073c20 0x7ffaf0076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.227+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7ffb0012f720 con 0x7ffb1009bef0 2024-09-21T02:22:08.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.227+0000 7ffb0e59c640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffaf0073c20 0x7ffaf0076130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffb08004c30 tx=0x7ffb080020a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:08.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.231+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ffb00016ad0 con 0x7ffb1009bef0 2024-09-21T02:22:08.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.435+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb100b34c0 con 0x7ffb1009bef0 2024-09-21T02:22:08.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb06ffd640 1 -- 172.21.15.170:0/2758244517 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5245 (secure 0 0 0) 0x7ffb000f9990 con 0x7ffb1009bef0 2024-09-21T02:22:08.440 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:08.440 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":111,"btime":"2024-09-21T02:22:02:983140+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:50.297124+0000","modified":"2024-09-21T02:21:53.314129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8934":{"gid":8934,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2398609544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2398609544},{"type":"v1","addr":"172.21.15.170:6835","nonce":2398609544}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8934,"qdb_cluster":[8934]},"id":12}]} 2024-09-21T02:22:08.440 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 111 2024-09-21T02:22:08.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffaf0073c20 msgr2=0x7ffaf0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffaf0073c20 0x7ffaf0076130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffb08004c30 tx=0x7ffb080020a0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 msgr2=0x7ffb1009c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 0x7ffb1009c300 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000ba60 tx=0x7ffb00004620 comp rx=0 tx=0).stop 2024-09-21T02:22:08.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 shutdown_connections 2024-09-21T02:22:08.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ffaf0073c20 0x7ffaf0076130 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb1009bef0 0x7ffb1009c300 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10074130 0x7ffb101f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 --2- 172.21.15.170:0/2758244517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffafc005360 0x7ffb101f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 >> 172.21.15.170:0/2758244517 conn(0x7ffb100b1c40 msgr2=0x7ffb100c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:08.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 shutdown_connections 2024-09-21T02:22:08.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.439+0000 7ffb148ba640 1 -- 172.21.15.170:0/2758244517 wait complete. 2024-09-21T02:22:08.462 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:08.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1341412671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 msgr2=0x7fc4140cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1341412671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc4140cadf0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc4140b2d00 tx=0x7fc40c01cd50 comp rx=0 tx=0).stop 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1341412671 shutdown_connections 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1341412671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc4140cadf0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1341412671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc414070570 0x7fc4140ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1341412671 >> v2:172.21.15.191:3300/0 conn(0x7fc41415f990 0x7fc414070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1341412671 >> 172.21.15.170:0/1341412671 conn(0x7fc4140b1c40 msgr2=0x7fc4140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1341412671 shutdown_connections 2024-09-21T02:22:08.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1341412671 wait complete. 2024-09-21T02:22:08.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 Processor -- start 2024-09-21T02:22:08.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.615+0000 7fc41a4bf640 1 -- start start 2024-09-21T02:22:08.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc414070570 0x7fc414154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc41409c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41415f990 0x7fc41409c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc4140732d0 con 0x7fc4140cea00 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc414073150 con 0x7fc41415f990 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc414073450 con 0x7fc414070570 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc41409c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc41409c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39300/0 (socket says 172.21.15.170:39300) 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 -- 172.21.15.170:0/1480536422 learned_addr learned my addr 172.21.15.170:0/1480536422 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc413fff640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc414070570 0x7fc414154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc418a5c640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41415f990 0x7fc41409c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 -- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc414070570 msgr2=0x7fc414154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc414070570 0x7fc414154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 -- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41415f990 msgr2=0x7fc41409c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41415f990 0x7fc41409c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 -- 172.21.15.170:0/1480536422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41409bc20 con 0x7fc4140cea00 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc4137fe640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc41409c250 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d650 tx=0x7fc40800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408017070 con 0x7fc4140cea00 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc408004310 con 0x7fc4140cea00 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408005280 con 0x7fc4140cea00 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41415d060 con 0x7fc4140cea00 2024-09-21T02:22:08.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc414157160 con 0x7fc4140cea00 2024-09-21T02:22:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4140cadf0 con 0x7fc4140cea00 2024-09-21T02:22:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fc408005420 con 0x7fc4140cea00 2024-09-21T02:22:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc410ff9640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc400073c50 0x7fc400076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7fc408093080 con 0x7fc4140cea00 2024-09-21T02:22:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc413fff640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc400073c50 0x7fc400076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.619+0000 7fc413fff640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc400073c50 0x7fc400076160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc4040058d0 tx=0x7fc404005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:08.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.627+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc408020020 con 0x7fc4140cea00 2024-09-21T02:22:08.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.799+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4140ce2a0 con 0x7fc4140cea00 2024-09-21T02:22:08.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.799+0000 7fc410ff9640 1 -- 172.21.15.170:0/1480536422 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+17028 (secure 0 0 0) 0x7fc40805f940 con 0x7fc4140cea00 2024-09-21T02:22:08.803 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:08.803 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":155,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:01.972348+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:47.975445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:49.257998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.2","mappings":[{"from":7,"to":4}]},{"pgid":"25.1b","mappings":[{"from":7,"to":3}]},{"pgid":"25.1d","mappings":[{"from":1,"to":3}]},{"pgid":"25.29","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc400073c50 msgr2=0x7fc400076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc400073c50 0x7fc400076160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc4040058d0 tx=0x7fc404005860 comp rx=0 tx=0).stop 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 msgr2=0x7fc41409c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc41409c250 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d650 tx=0x7fc40800db10 comp rx=0 tx=0).stop 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 shutdown_connections 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc400073c50 0x7fc400076160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41415f990 0x7fc41409c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4140cea00 0x7fc41409c250 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 --2- 172.21.15.170:0/1480536422 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc414070570 0x7fc414154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 >> 172.21.15.170:0/1480536422 conn(0x7fc4140b1c40 msgr2=0x7fc4140b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:08.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 shutdown_connections 2024-09-21T02:22:08.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.803+0000 7fc41a4bf640 1 -- 172.21.15.170:0/1480536422 wait complete. 2024-09-21T02:22:08.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:22:08.827 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/473506084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b815faf0 msgr2=0x7fc5b815ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/473506084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b815faf0 0x7fc5b815ff00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80d2370 tx=0x7fc5ac01c800 comp rx=0 tx=0).stop 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/473506084 shutdown_connections 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/473506084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b815faf0 0x7fc5b815ff00 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/473506084 >> v2:172.21.15.191:3300/0 conn(0x7fc5b80b25f0 0x7fc5b80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/473506084 >> 172.21.15.170:0/473506084 conn(0x7fc5b809a9c0 msgr2=0x7fc5b80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/473506084 shutdown_connections 2024-09-21T02:22:08.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.975+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/473506084 wait complete. 2024-09-21T02:22:08.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 Processor -- start 2024-09-21T02:22:08.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 -- start start 2024-09-21T02:22:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 0x7fc5b815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b815faf0 0x7fc5b8165900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8160440 0x7fc5b815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b81693f0 con 0x7fc5b80b25f0 2024-09-21T02:22:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8169270 con 0x7fc5b815faf0 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b8169570 con 0x7fc5b8160440 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 0x7fc5b815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 0x7fc5b815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39322/0 (socket says 172.21.15.170:39322) 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 -- 172.21.15.170:0/728553763 learned_addr learned my addr 172.21.15.170:0/728553763 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b7fff640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8160440 0x7fc5b815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 -- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8160440 msgr2=0x7fc5b815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8160440 0x7fc5b815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b6ffd640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b815faf0 0x7fc5b8165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 -- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b815faf0 msgr2=0x7fc5b8165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b815faf0 0x7fc5b8165900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 -- 172.21.15.170:0/728553763 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b81f9290 con 0x7fc5b80b25f0 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b7fff640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8160440 0x7fc5b815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5b77fe640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 0x7fc5b815a9f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000da90 tx=0x7fc5b000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0017070 con 0x7fc5b80b25f0 2024-09-21T02:22:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5b0010460 con 0x7fc5b80b25f0 2024-09-21T02:22:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b81f9550 con 0x7fc5b80b25f0 2024-09-21T02:22:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000be60 con 0x7fc5b80b25f0 2024-09-21T02:22:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.979+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b81f9b10 con 0x7fc5b80b25f0 2024-09-21T02:22:08.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.983+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b80cf950 con 0x7fc5b80b25f0 2024-09-21T02:22:08.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.983+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fc5b001a070 con 0x7fc5b80b25f0 2024-09-21T02:22:08.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.983+0000 7fc59ffff640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc5a4073c20 0x7fc5a4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:08.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.983+0000 7fc5b6ffd640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc5a4073c20 0x7fc5a4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:08.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.983+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7fc5b00931c0 con 0x7fc5b80b25f0 2024-09-21T02:22:08.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.983+0000 7fc5b6ffd640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc5a4073c20 0x7fc5a4076130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0af000 tx=0x7fc5ac01b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:08.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:08.987+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc5b005fa70 con 0x7fc5b80b25f0 2024-09-21T02:22:09.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.199+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5b81fa160 con 0x7fc5b80b25f0 2024-09-21T02:22:09.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.199+0000 7fc59ffff640 1 -- 172.21.15.170:0/728553763 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5245 (secure 0 0 0) 0x7fc5b005f370 con 0x7fc5b80b25f0 2024-09-21T02:22:09.202 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:09.203 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":111,"btime":"2024-09-21T02:22:02:983140+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:50.297124+0000","modified":"2024-09-21T02:21:53.314129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8934":{"gid":8934,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2398609544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2398609544},{"type":"v1","addr":"172.21.15.170:6835","nonce":2398609544}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8934,"qdb_cluster":[8934]},"id":12}]} 2024-09-21T02:22:09.203 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 111 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc5a4073c20 msgr2=0x7fc5a4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc5a4073c20 0x7fc5a4076130 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0af000 tx=0x7fc5ac01b040 comp rx=0 tx=0).stop 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 msgr2=0x7fc5b815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 0x7fc5b815a9f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000da90 tx=0x7fc5b000df50 comp rx=0 tx=0).stop 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 shutdown_connections 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc5a4073c20 0x7fc5a4076130 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5b8160440 0x7fc5b815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b815faf0 0x7fc5b8165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 --2- 172.21.15.170:0/728553763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5b80b25f0 0x7fc5b815a9f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 >> 172.21.15.170:0/728553763 conn(0x7fc5b809a9c0 msgr2=0x7fc5b814be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:09.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 shutdown_connections 2024-09-21T02:22:09.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.203+0000 7fc5bdbcf640 1 -- 172.21.15.170:0/728553763 wait complete. 2024-09-21T02:22:09.224 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:09.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 -- 172.21.15.170:0/77713726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500ce1c0 msgr2=0x7f05500ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 --2- 172.21.15.170:0/77713726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500ce1c0 0x7f05500ce5d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0548009840 tx=0x7f054801c820 comp rx=0 tx=0).stop 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 -- 172.21.15.170:0/77713726 shutdown_connections 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 --2- 172.21.15.170:0/77713726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500ce1c0 0x7f05500ce5d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 --2- 172.21.15.170:0/77713726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05500cf7e0 0x7f0550070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 -- 172.21.15.170:0/77713726 >> 172.21.15.170:0/77713726 conn(0x7f05500b1c40 msgr2=0x7f05500b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 -- 172.21.15.170:0/77713726 shutdown_connections 2024-09-21T02:22:09.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.363+0000 7f0556ac0640 1 -- 172.21.15.170:0/77713726 wait complete. 2024-09-21T02:22:09.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 Processor -- start 2024-09-21T02:22:09.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- start start 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500cb460 0x7f055015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 0x7f05501555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05500cf7e0 0x7f0550142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05501690d0 con 0x7f05500ce1c0 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0550168f50 con 0x7f05500cb460 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f0550169250 con 0x7f05500cf7e0 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 0x7f05501555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 0x7f05501555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39326/0 (socket says 172.21.15.170:39326) 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 -- 172.21.15.170:0/3827575852 learned_addr learned my addr 172.21.15.170:0/3827575852 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f055505d640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05500cf7e0 0x7f0550142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f055485c640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500cb460 0x7f055015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 -- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05500cf7e0 msgr2=0x7f0550142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05500cf7e0 0x7f0550142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 -- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500cb460 msgr2=0x7f055015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500cb460 0x7f055015e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 -- 172.21.15.170:0/3827575852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055009b0d0 con 0x7f05500ce1c0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054ffff640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 0x7f05501555e0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f054000cbc0 tx=0x7f054000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540016020 con 0x7f05500ce1c0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f054000aae0 con 0x7f05500ce1c0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0540012420 con 0x7f05500ce1c0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055009b820 con 0x7f05500ce1c0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0550157db0 con 0x7f05500ce1c0 2024-09-21T02:22:09.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.367+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05500ce5d0 con 0x7f05500ce1c0 2024-09-21T02:22:09.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.371+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f0540019020 con 0x7f05500ce1c0 2024-09-21T02:22:09.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.371+0000 7f054d7fa640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0530073c20 0x7f0530076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.371+0000 7f055485c640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0530073c20 0x7f0530076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.371+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7f0540092ce0 con 0x7f05500ce1c0 2024-09-21T02:22:09.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.371+0000 7f055485c640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0530073c20 0x7f0530076130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0538009ad0 tx=0x7f05380091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:09.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.375+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f0540020030 con 0x7f05500ce1c0 2024-09-21T02:22:09.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.579+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f055009b2d0 con 0x7f05500ce1c0 2024-09-21T02:22:09.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.579+0000 7f054d7fa640 1 -- 172.21.15.170:0/3827575852 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5245 (secure 0 0 0) 0x7f054005f590 con 0x7f05500ce1c0 2024-09-21T02:22:09.583 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:09.583 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":111,"btime":"2024-09-21T02:22:02:983140+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:21:50.297124+0000","modified":"2024-09-21T02:21:53.314129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8934},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8934":{"gid":8934,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/2398609544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2398609544},{"type":"v1","addr":"172.21.15.170:6835","nonce":2398609544}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8934,"qdb_cluster":[8934]},"id":12}]} 2024-09-21T02:22:09.583 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 111 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0530073c20 msgr2=0x7f0530076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0530073c20 0x7f0530076130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0538009ad0 tx=0x7f05380091e0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 msgr2=0x7f05501555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 0x7f05501555e0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f054000cbc0 tx=0x7f054000ccb0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 shutdown_connections 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f0530073c20 0x7f0530076130 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05500cf7e0 0x7f0550142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05500ce1c0 0x7f05501555e0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 --2- 172.21.15.170:0/3827575852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05500cb460 0x7f055015e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 >> 172.21.15.170:0/3827575852 conn(0x7f05500b1c40 msgr2=0x7f055015aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:09.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 shutdown_connections 2024-09-21T02:22:09.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.583+0000 7f0556ac0640 1 -- 172.21.15.170:0/3827575852 wait complete. 2024-09-21T02:22:09.604 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:09.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe83b515640 1 -- 172.21.15.170:0/117491088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe820004d50 msgr2=0x7fe820005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe83b515640 1 --2- 172.21.15.170:0/117491088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe820004d50 0x7fe820005160 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009eb0 tx=0x7fe82c01d5f0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe839ab2640 1 -- 172.21.15.170:0/117491088 reap_dead start 2024-09-21T02:22:09.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe83b515640 1 -- 172.21.15.170:0/117491088 shutdown_connections 2024-09-21T02:22:09.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe83b515640 1 -- 172.21.15.170:0/117491088 >> 172.21.15.170:0/117491088 conn(0x7fe8340cd3c0 msgr2=0x7fe8340cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:09.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe83b515640 1 -- 172.21.15.170:0/117491088 shutdown_connections 2024-09-21T02:22:09.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.739+0000 7fe83b515640 1 -- 172.21.15.170:0/117491088 wait complete. 2024-09-21T02:22:09.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 Processor -- start 2024-09-21T02:22:09.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 -- start start 2024-09-21T02:22:09.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe820004d50 0x7fe834156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 0x7fe8341506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8341e7f40 0x7fe8341e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340cc6f0 con 0x7fe834153260 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340cc870 con 0x7fe820004d50 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe8340cc9f0 con 0x7fe8341e7f40 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 0x7fe8341506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 0x7fe8341506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39340/0 (socket says 172.21.15.170:39340) 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 -- 172.21.15.170:0/1246433482 learned_addr learned my addr 172.21.15.170:0/1246433482 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe839ab2640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8341e7f40 0x7fe8341e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe8392b1640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe820004d50 0x7fe834156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 -- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8341e7f40 msgr2=0x7fe8341e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8341e7f40 0x7fe8341e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 -- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe820004d50 msgr2=0x7fe834156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe820004d50 0x7fe834156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 -- 172.21.15.170:0/1246433482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8341e8890 con 0x7fe834153260 2024-09-21T02:22:09.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe838ab0640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 0x7fe8341506c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe82400e9a0 tx=0x7fe82400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82400ccc0 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe82400ce80 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8341f9320 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824005280 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.743+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8341f98e0 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.747+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8340ca460 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.747+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fe824005420 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.747+0000 7fe829ffb640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe818073c20 0x7fe818076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.747+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7fe8240937e0 con 0x7fe834153260 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.747+0000 7fe8392b1640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe818073c20 0x7fe818076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:09.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.747+0000 7fe8392b1640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe818073c20 0x7fe818076130 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8300046e0 tx=0x7fe830001d40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:09.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.751+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe8341e8890 con 0x7fe834153260 2024-09-21T02:22:09.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.915+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8340ccf80 con 0x7fe834153260 2024-09-21T02:22:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.915+0000 7fe829ffb640 1 -- 172.21.15.170:0/1246433482 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+17028 (secure 0 0 0) 0x7fe824060090 con 0x7fe834153260 2024-09-21T02:22:09.918 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:09.918 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":155,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:01.972348+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:21:47.975445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T02:21:49.257998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.2","mappings":[{"from":7,"to":4}]},{"pgid":"25.1b","mappings":[{"from":7,"to":3}]},{"pgid":"25.1d","mappings":[{"from":1,"to":3}]},{"pgid":"25.29","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:09.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe818073c20 msgr2=0x7fe818076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe818073c20 0x7fe818076130 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe8300046e0 tx=0x7fe830001d40 comp rx=0 tx=0).stop 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 msgr2=0x7fe8341506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 0x7fe8341506c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe82400e9a0 tx=0x7fe82400ee60 comp rx=0 tx=0).stop 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 shutdown_connections 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe818073c20 0x7fe818076130 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8341e7f40 0x7fe8341e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe834153260 0x7fe8341506c0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 --2- 172.21.15.170:0/1246433482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe820004d50 0x7fe834156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 >> 172.21.15.170:0/1246433482 conn(0x7fe8340cd3c0 msgr2=0x7fe8340cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 shutdown_connections 2024-09-21T02:22:09.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:09.919+0000 7fe83b515640 1 -- 172.21.15.170:0/1246433482 wait complete. 2024-09-21T02:22:09.940 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:22:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- 172.21.15.170:0/2272498481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c0b6930 msgr2=0x7fbd9c06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- 172.21.15.170:0/2272498481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c0b6930 0x7fbd9c06ce00 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ced50 tx=0x7fbd8401c8e0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- 172.21.15.170:0/2272498481 shutdown_connections 2024-09-21T02:22:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- 172.21.15.170:0/2272498481 >> v2:172.21.15.191:3300/0 conn(0x7fbd9c06d340 0x7fbd9c160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- 172.21.15.170:0/2272498481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c0b6930 0x7fbd9c06ce00 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- 172.21.15.170:0/2272498481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0b22b0 0x7fbd9c0b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- 172.21.15.170:0/2272498481 >> 172.21.15.170:0/2272498481 conn(0x7fbd9c09a9c0 msgr2=0x7fbd9c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:10.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- 172.21.15.170:0/2272498481 shutdown_connections 2024-09-21T02:22:10.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- 172.21.15.170:0/2272498481 wait complete. 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 Processor -- start 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- start start 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 0x7fbd9c14a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0b22b0 0x7fbd9c158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0b6930 0x7fbd9c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c15c420 con 0x7fbd9c06d340 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c15c2a0 con 0x7fbd9c0b22b0 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbda209a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c15c5a0 con 0x7fbd9c0b6930 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 0x7fbd9c14a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 0x7fbd9c14a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39360/0 (socket says 172.21.15.170:39360) 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9b7fe640 1 -- 172.21.15.170:0/1410709136 learned_addr learned my addr 172.21.15.170:0/1410709136 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9affd640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0b22b0 0x7fbd9c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9bfff640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0b6930 0x7fbd9c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9b7fe640 1 -- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0b6930 msgr2=0x7fbd9c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.095+0000 7fbd9b7fe640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0b6930 0x7fbd9c158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd9b7fe640 1 -- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0b22b0 msgr2=0x7fbd9c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd9b7fe640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0b22b0 0x7fbd9c158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd9b7fe640 1 -- 172.21.15.170:0/1410709136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0c6d80 con 0x7fbd9c06d340 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd9b7fe640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 0x7fbd9c14a790 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400cce0 tx=0x7fbd94007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94004b30 con 0x7fbd9c06d340 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd94004dc0 con 0x7fbd9c06d340 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94005290 con 0x7fbd9c06d340 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c0c7060 con 0x7fbd9c06d340 2024-09-21T02:22:10.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c0c7620 con 0x7fbd9c06d340 2024-09-21T02:22:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c15cb30 con 0x7fbd9c06d340 2024-09-21T02:22:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fbd94005430 con 0x7fbd9c06d340 2024-09-21T02:22:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd93fff640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbd7c073c20 0x7fbd7c076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5853+0+0 (secure 0 0 0) 0x7fbd94093830 con 0x7fbd9c06d340 2024-09-21T02:22:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd9affd640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbd7c073c20 0x7fbd7c076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.099+0000 7fbd9affd640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbd7c073c20 0x7fbd7c076130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd840af000 tx=0x7fbd8401b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.103+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fbd94014070 con 0x7fbd9c06d340 2024-09-21T02:22:10.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.311+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbd9c0b63a0 con 0x7fbd9c06d340 2024-09-21T02:22:10.316 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:22:10.315+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:22:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.315+0000 7fbd93fff640 1 -- 172.21.15.170:0/1410709136 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7fbd94060170 con 0x7fbd9c06d340 2024-09-21T02:22:10.319 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbd7c073c20 msgr2=0x7fbd7c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbd7c073c20 0x7fbd7c076130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd840af000 tx=0x7fbd8401b120 comp rx=0 tx=0).stop 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 msgr2=0x7fbd9c14a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 0x7fbd9c14a790 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400cce0 tx=0x7fbd94007610 comp rx=0 tx=0).stop 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 shutdown_connections 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fbd7c073c20 0x7fbd7c076130 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0b6930 0x7fbd9c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0b22b0 0x7fbd9c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 --2- 172.21.15.170:0/1410709136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd9c06d340 0x7fbd9c14a790 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.319+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 >> 172.21.15.170:0/1410709136 conn(0x7fbd9c09a9c0 msgr2=0x7fbd9c15b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:10.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.323+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 shutdown_connections 2024-09-21T02:22:10.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.323+0000 7fbda209a640 1 -- 172.21.15.170:0/1410709136 wait complete. 2024-09-21T02:22:10.344 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:22:10.401 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 -- 172.21.15.170:0/2625667070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040ce780 msgr2=0x7ff70406f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 --2- 172.21.15.170:0/2625667070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040ce780 0x7ff70406f9a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7ff7040c3c40 tx=0x7ff6f801c770 comp rx=0 tx=0).stop 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 -- 172.21.15.170:0/2625667070 shutdown_connections 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 --2- 172.21.15.170:0/2625667070 >> v2:172.21.15.191:3300/0 conn(0x7ff7040cb790 0x7ff70406fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 --2- 172.21.15.170:0/2625667070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040ce780 0x7ff70406f9a0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 -- 172.21.15.170:0/2625667070 >> 172.21.15.170:0/2625667070 conn(0x7ff7040d0170 msgr2=0x7ff7040d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:10.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 -- 172.21.15.170:0/2625667070 shutdown_connections 2024-09-21T02:22:10.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.495+0000 7ff709903640 1 -- 172.21.15.170:0/2625667070 wait complete. 2024-09-21T02:22:10.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 Processor -- start 2024-09-21T02:22:10.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 -- start start 2024-09-21T02:22:10.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 0x7ff704158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7040cb790 0x7ff70415da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7040ce780 0x7ff70415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff704073f20 con 0x7ff7040cb380 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff704073da0 con 0x7ff7040ce780 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff7040740a0 con 0x7ff7040cb790 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 0x7ff704158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 0x7ff704158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39388/0 (socket says 172.21.15.170:39388) 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 -- 172.21.15.170:0/2539335571 learned_addr learned my addr 172.21.15.170:0/2539335571 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff7027fc640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7040cb790 0x7ff70415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff7037fe640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7040ce780 0x7ff70415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 -- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7040cb790 msgr2=0x7ff70415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7040cb790 0x7ff70415da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 -- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7040ce780 msgr2=0x7ff70415dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7040ce780 0x7ff70415dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 -- 172.21.15.170:0/2539335571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff704159060 con 0x7ff7040cb380 2024-09-21T02:22:10.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff7037fe640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7040ce780 0x7ff70415dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:10.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff702ffd640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 0x7ff704158b20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00c750 tx=0x7ff6fc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:10.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc017070 con 0x7ff7040cb380 2024-09-21T02:22:10.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6fc005070 con 0x7ff7040cb380 2024-09-21T02:22:10.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7040c77e0 con 0x7ff7040cb380 2024-09-21T02:22:10.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00ce60 con 0x7ff7040cb380 2024-09-21T02:22:10.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.499+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7040c7da0 con 0x7ff7040cb380 2024-09-21T02:22:10.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.503+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff704074600 con 0x7ff7040cb380 2024-09-21T02:22:10.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.503+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7ff6fc01a070 con 0x7ff7040cb380 2024-09-21T02:22:10.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.503+0000 7ff6ebfff640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff6f0073c80 0x7ff6f0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:10.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.503+0000 7ff7027fc640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff6f0073c80 0x7ff6f0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:10.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.503+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5939+0+0 (secure 0 0 0) 0x7ff6fc092380 con 0x7ff7040cb380 2024-09-21T02:22:10.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.503+0000 7ff7027fc640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff6f0073c80 0x7ff6f0076190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80af000 tx=0x7ff6f801afb0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:10.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.507+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff6fc05ec40 con 0x7ff7040cb380 2024-09-21T02:22:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:10.715+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff70415a200 con 0x7ff7040cb380 2024-09-21T02:22:11.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.319+0000 7ff6ebfff640 1 -- 172.21.15.170:0/2539335571 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) ==== 104+0+0 (secure 0 0 0) 0x7ff6fc065070 con 0x7ff7040cb380 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff6f0073c80 msgr2=0x7ff6f0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff6f0073c80 0x7ff6f0076190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80af000 tx=0x7ff6f801afb0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 msgr2=0x7ff704158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 0x7ff704158b20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00c750 tx=0x7ff6fc00cc10 comp rx=0 tx=0).stop 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 shutdown_connections 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff6f0073c80 0x7ff6f0076190 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7040cb790 0x7ff70415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7040ce780 0x7ff70415dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 --2- 172.21.15.170:0/2539335571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7040cb380 0x7ff704158b20 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 >> 172.21.15.170:0/2539335571 conn(0x7ff7040d0170 msgr2=0x7ff7040ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 shutdown_connections 2024-09-21T02:22:11.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.323+0000 7ff709903640 1 -- 172.21.15.170:0/2539335571 wait complete. 2024-09-21T02:22:11.353 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:22:11.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 -- 172.21.15.170:0/3078030931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40706d0 msgr2=0x7fc0f40cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:11.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 --2- 172.21.15.170:0/3078030931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40706d0 0x7fc0f40cabc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40ce380 tx=0x7fc0dc01c800 comp rx=0 tx=0).stop 2024-09-21T02:22:11.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 -- 172.21.15.170:0/3078030931 shutdown_connections 2024-09-21T02:22:11.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 --2- 172.21.15.170:0/3078030931 >> v2:172.21.15.191:3300/0 conn(0x7fc0f40735d0 0x7fc0f40739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 --2- 172.21.15.170:0/3078030931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40706d0 0x7fc0f40cabc0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 -- 172.21.15.170:0/3078030931 >> 172.21.15.170:0/3078030931 conn(0x7fc0f40b1c40 msgr2=0x7fc0f40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 -- 172.21.15.170:0/3078030931 shutdown_connections 2024-09-21T02:22:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.515+0000 7fc0f9216640 1 -- 172.21.15.170:0/3078030931 wait complete. 2024-09-21T02:22:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 Processor -- start 2024-09-21T02:22:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 -- start start 2024-09-21T02:22:11.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 0x7fc0f414ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:11.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0f40ca7b0 0x7fc0f40b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:11.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0f414e890 0x7fc0f40b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:11.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4168c50 con 0x7fc0f40735d0 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4168ad0 con 0x7fc0f40ca7b0 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f4168dd0 con 0x7fc0f414e890 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 0x7fc0f414ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 0x7fc0f414ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39404/0 (socket says 172.21.15.170:39404) 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 -- 172.21.15.170:0/2266230546 learned_addr learned my addr 172.21.15.170:0/2266230546 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f359e640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0f414e890 0x7fc0f40b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0ebfff640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0f40ca7b0 0x7fc0f40b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 -- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0f414e890 msgr2=0x7fc0f40b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0f414e890 0x7fc0f40b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 -- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0f40ca7b0 msgr2=0x7fc0f40b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0f40ca7b0 0x7fc0f40b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 -- 172.21.15.170:0/2266230546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f41f9ac0 con 0x7fc0f40735d0 2024-09-21T02:22:11.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0ebfff640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0f40ca7b0 0x7fc0f40b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:11.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f2d9d640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 0x7fc0f414ac30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c390 tx=0x7fc0e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:11.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4017070 con 0x7fc0f40735d0 2024-09-21T02:22:11.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0e4003620 con 0x7fc0f40735d0 2024-09-21T02:22:11.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f41f9da0 con 0x7fc0f40735d0 2024-09-21T02:22:11.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4003a70 con 0x7fc0f40735d0 2024-09-21T02:22:11.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.519+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41fa360 con 0x7fc0f40735d0 2024-09-21T02:22:11.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.523+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f40cabc0 con 0x7fc0f40735d0 2024-09-21T02:22:11.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.523+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fc0e401a070 con 0x7fc0f40735d0 2024-09-21T02:22:11.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.523+0000 7fc0eb7fe640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0d4073c20 0x7fc0d4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:11.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.523+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5939+0+0 (secure 0 0 0) 0x7fc0e40920a0 con 0x7fc0f40735d0 2024-09-21T02:22:11.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.523+0000 7fc0ebfff640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0d4073c20 0x7fc0d4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:11.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.523+0000 7fc0ebfff640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0d4073c20 0x7fc0d4076130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0af000 tx=0x7fc0dc01b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:11.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.527+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc0e405e8d0 con 0x7fc0f40735d0 2024-09-21T02:22:11.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:11.687+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0f40cd260 con 0x7fc0f40735d0 2024-09-21T02:22:12.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.331+0000 7fc0eb7fe640 1 -- 172.21.15.170:0/2266230546 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v157) ==== 181+0+0 (secure 0 0 0) 0x7fc0e405e1d0 con 0x7fc0f40735d0 2024-09-21T02:22:12.333 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:22:12.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0d4073c20 msgr2=0x7fc0d4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:12.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0d4073c20 0x7fc0d4076130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0af000 tx=0x7fc0dc01b040 comp rx=0 tx=0).stop 2024-09-21T02:22:12.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 msgr2=0x7fc0f414ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:12.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 0x7fc0f414ac30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c390 tx=0x7fc0e400c850 comp rx=0 tx=0).stop 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 shutdown_connections 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc0d4073c20 0x7fc0d4076130 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0f414e890 0x7fc0f40b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0f40ca7b0 0x7fc0f40b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 --2- 172.21.15.170:0/2266230546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0f40735d0 0x7fc0f414ac30 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 >> 172.21.15.170:0/2266230546 conn(0x7fc0f40b1c40 msgr2=0x7fc0f4154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.335+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 shutdown_connections 2024-09-21T02:22:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.339+0000 7fc0f9216640 1 -- 172.21.15.170:0/2266230546 wait complete. 2024-09-21T02:22:12.365 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:22:12.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 -- 172.21.15.170:0/2539382517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15c330 msgr2=0x7f544c15c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:12.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 --2- 172.21.15.170:0/2539382517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15c330 0x7f544c15c740 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f5440009900 tx=0x7f544001c850 comp rx=0 tx=0).stop 2024-09-21T02:22:12.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 -- 172.21.15.170:0/2539382517 shutdown_connections 2024-09-21T02:22:12.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 --2- 172.21.15.170:0/2539382517 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f544c0cf100 0x7f544c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 --2- 172.21.15.170:0/2539382517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15c330 0x7f544c15c740 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 -- 172.21.15.170:0/2539382517 >> 172.21.15.170:0/2539382517 conn(0x7f544c0cdc80 msgr2=0x7f544c0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 -- 172.21.15.170:0/2539382517 shutdown_connections 2024-09-21T02:22:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 -- 172.21.15.170:0/2539382517 wait complete. 2024-09-21T02:22:12.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 Processor -- start 2024-09-21T02:22:12.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.527+0000 7f54517bc640 1 -- start start 2024-09-21T02:22:12.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f544c0cf100 0x7f544c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:12.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f544c15c330 0x7f544c165cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:12.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 0x7f544c158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f544c168de0 con 0x7f544c15f230 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f544c1596b0 con 0x7f544c0cf100 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f544c168f60 con 0x7f544c15c330 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 0x7f544c158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 0x7f544c158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39428/0 (socket says 172.21.15.170:39428) 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 -- 172.21.15.170:0/1433238062 learned_addr learned my addr 172.21.15.170:0/1433238062 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544a7fc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f544c15c330 0x7f544c165cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544affd640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f544c0cf100 0x7f544c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 -- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f544c15c330 msgr2=0x7f544c165cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f544c15c330 0x7f544c165cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 -- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f544c0cf100 msgr2=0x7f544c0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f544c0cf100 0x7f544c0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 -- 172.21.15.170:0/1433238062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c1fa960 con 0x7f544c15f230 2024-09-21T02:22:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544b7fe640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 0x7f544c158340 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f544400e940 tx=0x7f544400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444004d30 con 0x7f544c15f230 2024-09-21T02:22:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f544400cb20 con 0x7f544c15f230 2024-09-21T02:22:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c1f7380 con 0x7f544c15f230 2024-09-21T02:22:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444005020 con 0x7f544c15f230 2024-09-21T02:22:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c1f9e80 con 0x7f544c15f230 2024-09-21T02:22:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544c0b27a0 con 0x7f544c15f230 2024-09-21T02:22:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f54440051c0 con 0x7f544c15f230 2024-09-21T02:22:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54337fe640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5438073c20 0x7f5438076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f544affd640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5438073c20 0x7f5438076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.531+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5501+0+0 (secure 0 0 0) 0x7f5444093ae0 con 0x7f544c15f230 2024-09-21T02:22:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.535+0000 7f544affd640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5438073c20 0x7f5438076130 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f543c001650 tx=0x7f543c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:12.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.535+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5444016030 con 0x7f544c15f230 2024-09-21T02:22:12.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:12.719+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f544c0cf510 con 0x7f544c15f230 2024-09-21T02:22:13.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.343+0000 7f54337fe640 1 -- 172.21.15.170:0/1433238062 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v158) ==== 169+0+0 (secure 0 0 0) 0x7f54440602f0 con 0x7f544c15f230 2024-09-21T02:22:13.345 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:22:13.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.347+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5438073c20 msgr2=0x7f5438076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.347+0000 7f54517bc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5438073c20 0x7f5438076130 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f543c001650 tx=0x7f543c009040 comp rx=0 tx=0).stop 2024-09-21T02:22:13.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.347+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 msgr2=0x7f544c158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.347+0000 7f54517bc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 0x7f544c158340 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f544400e940 tx=0x7f544400ee00 comp rx=0 tx=0).stop 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 shutdown_connections 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5438073c20 0x7f5438076130 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f544c15c330 0x7f544c165cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f544c15f230 0x7f544c158340 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 --2- 172.21.15.170:0/1433238062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f544c0cf100 0x7f544c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 >> 172.21.15.170:0/1433238062 conn(0x7f544c0cdc80 msgr2=0x7f544c0adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 shutdown_connections 2024-09-21T02:22:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.351+0000 7f54517bc640 1 -- 172.21.15.170:0/1433238062 wait complete. 2024-09-21T02:22:13.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:22:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.535+0000 7fa214d5f640 1 -- 172.21.15.170:0/613686038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1fc004d50 msgr2=0x7fa1fc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.535+0000 7fa214d5f640 1 --2- 172.21.15.170:0/613686038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1fc004d50 0x7fa1fc005200 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa2100cdc20 tx=0x7fa20801d280 comp rx=0 tx=0).stop 2024-09-21T02:22:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.535+0000 7fa20ed9d640 1 -- 172.21.15.170:0/613686038 reap_dead start 2024-09-21T02:22:13.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- 172.21.15.170:0/613686038 shutdown_connections 2024-09-21T02:22:13.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- 172.21.15.170:0/613686038 >> 172.21.15.170:0/613686038 conn(0x7fa2100b1c40 msgr2=0x7fa2100b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:13.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- 172.21.15.170:0/613686038 shutdown_connections 2024-09-21T02:22:13.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- 172.21.15.170:0/613686038 wait complete. 2024-09-21T02:22:13.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 Processor -- start 2024-09-21T02:22:13.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- start start 2024-09-21T02:22:13.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc004d50 0x7fa2100ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 0x7fa21015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa21014c640 0x7fa2101524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100ced50 con 0x7fa21015e810 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100cebd0 con 0x7fa1fc004d50 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa214d5f640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa210070030 con 0x7fa21014c640 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 0x7fa21015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 0x7fa21015cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39456/0 (socket says 172.21.15.170:39456) 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 -- 172.21.15.170:0/387290412 learned_addr learned my addr 172.21.15.170:0/387290412 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20ed9d640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa21014c640 0x7fa2101524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20e59c640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc004d50 0x7fa2100ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 -- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa21014c640 msgr2=0x7fa2101524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa21014c640 0x7fa2101524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 -- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc004d50 msgr2=0x7fa2100ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc004d50 0x7fa2100ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.539+0000 7fa20dd9b640 1 -- 172.21.15.170:0/387290412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21015d770 con 0x7fa21015e810 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa20dd9b640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 0x7fa21015cf20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa200004cc0 tx=0x7fa200001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200015020 con 0x7fa21015e810 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa200011750 con 0x7fa21015e810 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21015da50 con 0x7fa21015e810 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200011b80 con 0x7fa21015e810 2024-09-21T02:22:13.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21015e010 con 0x7fa21015e810 2024-09-21T02:22:13.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21015d4e0 con 0x7fa21015e810 2024-09-21T02:22:13.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fa200018020 con 0x7fa21015e810 2024-09-21T02:22:13.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa206ffd640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1f0073c50 0x7fa1f0076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa20e59c640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1f0073c50 0x7fa1f0076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7fa200093660 con 0x7fa21015e810 2024-09-21T02:22:13.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.543+0000 7fa20e59c640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1f0073c50 0x7fa1f0076160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800ab80 tx=0x7fa1f800a2f0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.547+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa20001f020 con 0x7fa21015e810 2024-09-21T02:22:13.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.731+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fa21006de10 con 0x7fa21015e810 2024-09-21T02:22:13.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa206ffd640 1 -- 172.21.15.170:0/387290412 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v51) ==== 71+0+0 (secure 0 0 0) 0x7fa20005ffa0 con 0x7fa21015e810 2024-09-21T02:22:13.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1f0073c50 msgr2=0x7fa1f0076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1f0073c50 0x7fa1f0076160 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800ab80 tx=0x7fa1f800a2f0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 msgr2=0x7fa21015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 0x7fa21015cf20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa200004cc0 tx=0x7fa200001680 comp rx=0 tx=0).stop 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 shutdown_connections 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa1f0073c50 0x7fa1f0076160 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa21014c640 0x7fa2101524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21015e810 0x7fa21015cf20 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 --2- 172.21.15.170:0/387290412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc004d50 0x7fa2100ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 >> 172.21.15.170:0/387290412 conn(0x7fa2100b1c40 msgr2=0x7fa21006f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 shutdown_connections 2024-09-21T02:22:13.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.739+0000 7fa214d5f640 1 -- 172.21.15.170:0/387290412 wait complete. 2024-09-21T02:22:13.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-21T02:22:13.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 -- 172.21.15.170:0/452061035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 msgr2=0x7fd35406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/452061035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd35406fa30 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fd3540c7150 tx=0x7fd35001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 -- 172.21.15.170:0/452061035 shutdown_connections 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/452061035 >> v2:172.21.15.191:3300/0 conn(0x7fd3540cd210 0x7fd3540b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/452061035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd35406fa30 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 -- 172.21.15.170:0/452061035 >> 172.21.15.170:0/452061035 conn(0x7fd3540b1c40 msgr2=0x7fd3540b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 -- 172.21.15.170:0/452061035 shutdown_connections 2024-09-21T02:22:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.911+0000 7fd35c1f9640 1 -- 172.21.15.170:0/452061035 wait complete. 2024-09-21T02:22:13.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 Processor -- start 2024-09-21T02:22:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 -- start start 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3540b2f40 0x7fd3541f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd3541463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35415c5a0 0x7fd3541f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd354169fd0 con 0x7fd3540b5410 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd354169e50 con 0x7fd3540b2f40 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd35416a150 con 0x7fd35415c5a0 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd3541463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd3541463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39472/0 (socket says 172.21.15.170:39472) 2024-09-21T02:22:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 -- 172.21.15.170:0/1056099285 learned_addr learned my addr 172.21.15.170:0/1056099285 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359f95640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3540b2f40 0x7fd3541f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35a796640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35415c5a0 0x7fd3541f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 -- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35415c5a0 msgr2=0x7fd3541f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35415c5a0 0x7fd3541f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 -- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3540b2f40 msgr2=0x7fd3541f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3540b2f40 0x7fd3541f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 -- 172.21.15.170:0/1056099285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3541f7640 con 0x7fd3540b5410 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd359794640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd3541463e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b410 tx=0x7fd34400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344015030 con 0x7fd3540b5410 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd344004020 con 0x7fd3540b5410 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35415bac0 con 0x7fd3540b5410 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440044b0 con 0x7fd3540b5410 2024-09-21T02:22:13.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.915+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35415c080 con 0x7fd3540b5410 2024-09-21T02:22:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.919+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35406fa30 con 0x7fd3540b5410 2024-09-21T02:22:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.919+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fd344018070 con 0x7fd3540b5410 2024-09-21T02:22:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.919+0000 7fd34a7fc640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd338073c20 0x7fd338076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.919+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7fd344092ce0 con 0x7fd3540b5410 2024-09-21T02:22:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.919+0000 7fd359f95640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd338073c20 0x7fd338076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:13.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.919+0000 7fd359f95640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd338073c20 0x7fd338076130 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd3500af000 tx=0x7fd35001b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:13.923+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd344018e40 con 0x7fd3540b5410 2024-09-21T02:22:14.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.155+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fd3540ce370 con 0x7fd3540b5410 2024-09-21T02:22:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.343+0000 7fd34a7fc640 1 -- 172.21.15.170:0/1056099285 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v468) ==== 145+0+0 (secure 0 0 0) 0x7fd34405f620 con 0x7fd3540b5410 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd338073c20 msgr2=0x7fd338076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd338073c20 0x7fd338076130 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd3500af000 tx=0x7fd35001b100 comp rx=0 tx=0).stop 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 msgr2=0x7fd3541463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd3541463e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b410 tx=0x7fd34400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 shutdown_connections 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd338073c20 0x7fd338076130 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35415c5a0 0x7fd3541f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540b5410 0x7fd3541463e0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 --2- 172.21.15.170:0/1056099285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3540b2f40 0x7fd3541f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 >> 172.21.15.170:0/1056099285 conn(0x7fd3540b1c40 msgr2=0x7fd3540ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 shutdown_connections 2024-09-21T02:22:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.347+0000 7fd35c1f9640 1 -- 172.21.15.170:0/1056099285 wait complete. 2024-09-21T02:22:14.373 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-21T02:22:14.373 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-21T02:22:14.373 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T02:22:14.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 -- 172.21.15.170:0/593258688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c15f990 msgr2=0x7ff57c070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 --2- 172.21.15.170:0/593258688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c15f990 0x7ff57c070030 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0cdc00 tx=0x7ff56c01c6b0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 -- 172.21.15.170:0/593258688 shutdown_connections 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 --2- 172.21.15.170:0/593258688 >> v2:172.21.15.191:3300/0 conn(0x7ff57c0cadf0 0x7ff57c0cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 --2- 172.21.15.170:0/593258688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c15f990 0x7ff57c070030 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 -- 172.21.15.170:0/593258688 >> 172.21.15.170:0/593258688 conn(0x7ff57c0b1c40 msgr2=0x7ff57c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 -- 172.21.15.170:0/593258688 shutdown_connections 2024-09-21T02:22:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 -- 172.21.15.170:0/593258688 wait complete. 2024-09-21T02:22:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 Processor -- start 2024-09-21T02:22:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.527+0000 7ff5821fc640 1 -- start start 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 0x7ff57c1fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff57c0cadf0 0x7ff57c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff57c15f990 0x7ff57c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c169100 con 0x7ff57c070570 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c0cede0 con 0x7ff57c0cadf0 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c169280 con 0x7ff57c15f990 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 0x7ff57c1fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 0x7ff57c1fcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39484/0 (socket says 172.21.15.170:39484) 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 -- 172.21.15.170:0/2820110062 learned_addr learned my addr 172.21.15.170:0/2820110062 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57bfff640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff57c15f990 0x7ff57c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57affd640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff57c0cadf0 0x7ff57c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 -- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff57c15f990 msgr2=0x7ff57c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff57c15f990 0x7ff57c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 -- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff57c0cadf0 msgr2=0x7ff57c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff57c0cadf0 0x7ff57c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 -- 172.21.15.170:0/2820110062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c1f9130 con 0x7ff57c070570 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57affd640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff57c0cadf0 0x7ff57c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57b7fe640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 0x7ff57c1fcfd0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ff56c01ad60 tx=0x7ff56c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c004230 con 0x7ff57c070570 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff56c0044e0 con 0x7ff57c070570 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c1f9410 con 0x7ff57c070570 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003840 con 0x7ff57c070570 2024-09-21T02:22:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c1f99d0 con 0x7ff57c070570 2024-09-21T02:22:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff57c0cb200 con 0x7ff57c070570 2024-09-21T02:22:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7ff56c0bf020 con 0x7ff57c070570 2024-09-21T02:22:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff573fff640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff55c073c20 0x7ff55c076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7ff56c12ddf0 con 0x7ff57c070570 2024-09-21T02:22:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57affd640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff55c073c20 0x7ff55c076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.531+0000 7ff57affd640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff55c073c20 0x7ff55c076130 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0b4770 tx=0x7ff5640073c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:14.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.535+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff56c0c6020 con 0x7ff57c070570 2024-09-21T02:22:14.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.695+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff57c0ce2a0 con 0x7ff57c070570 2024-09-21T02:22:14.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.699+0000 7ff573fff640 1 -- 172.21.15.170:0/2820110062 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+13136 (secure 0 0 0) 0x7ff57c0cb740 con 0x7ff57c070570 2024-09-21T02:22:14.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:14.700 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":158,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:13.336697+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6835/2398609544":"2024-09-22T02:22:10.319303+0000","172.21.15.170:6834/2398609544":"2024-09-22T02:22:10.319303+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff55c073c20 msgr2=0x7ff55c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff55c073c20 0x7ff55c076130 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0b4770 tx=0x7ff5640073c0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 msgr2=0x7ff57c1fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 0x7ff57c1fcfd0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ff56c01ad60 tx=0x7ff56c004040 comp rx=0 tx=0).stop 2024-09-21T02:22:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 shutdown_connections 2024-09-21T02:22:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff55c073c20 0x7ff55c076130 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff57c15f990 0x7ff57c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff57c0cadf0 0x7ff57c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 --2- 172.21.15.170:0/2820110062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff57c070570 0x7ff57c1fcfd0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 >> 172.21.15.170:0/2820110062 conn(0x7ff57c0b1c40 msgr2=0x7ff57c0cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 shutdown_connections 2024-09-21T02:22:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.703+0000 7ff5821fc640 1 -- 172.21.15.170:0/2820110062 wait complete. 2024-09-21T02:22:14.723 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:13:14.921556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:22:14.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b89297640 1 -- 172.21.15.170:0/953764703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b6c004d50 msgr2=0x7f2b6c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b89297640 1 --2- 172.21.15.170:0/953764703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b6c004d50 0x7f2b6c005200 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2b840c8320 tx=0x7f2b7c01caf0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b837fe640 1 -- 172.21.15.170:0/953764703 reap_dead start 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b89297640 1 -- 172.21.15.170:0/953764703 shutdown_connections 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b89297640 1 -- 172.21.15.170:0/953764703 >> 172.21.15.170:0/953764703 conn(0x7f2b8409a9c0 msgr2=0x7f2b840d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b89297640 1 -- 172.21.15.170:0/953764703 shutdown_connections 2024-09-21T02:22:14.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.879+0000 7f2b89297640 1 -- 172.21.15.170:0/953764703 wait complete. 2024-09-21T02:22:14.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 Processor -- start 2024-09-21T02:22:14.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 -- start start 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b8415d7e0 0x7f2b840ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b840cae40 0x7f2b840af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 0x7f2b84154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b840d1d50 con 0x7f2b84154180 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b840d1bd0 con 0x7f2b840cae40 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2b840b0b60 con 0x7f2b8415d7e0 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 0x7f2b84154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 0x7f2b84154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39508/0 (socket says 172.21.15.170:39508) 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 -- 172.21.15.170:0/4193949348 learned_addr learned my addr 172.21.15.170:0/4193949348 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b82ffd640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b8415d7e0 0x7f2b840ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b827fc640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b840cae40 0x7f2b840af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 -- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b8415d7e0 msgr2=0x7f2b840ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b8415d7e0 0x7f2b840ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 -- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b840cae40 msgr2=0x7f2b840af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b840cae40 0x7f2b840af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 -- 172.21.15.170:0/4193949348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b840ae730 con 0x7f2b84154180 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b837fe640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 0x7f2b84154590 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c01d390 tx=0x7f2b7c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0a5b60 con 0x7f2b84154180 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b7c0a5df0 con 0x7f2b84154180 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b840aea10 con 0x7f2b84154180 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0062f0 con 0x7f2b84154180 2024-09-21T02:22:14.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.883+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b840aefd0 con 0x7f2b84154180 2024-09-21T02:22:14.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.887+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b840cc5b0 con 0x7f2b84154180 2024-09-21T02:22:14.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.887+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f2b7c006490 con 0x7f2b84154180 2024-09-21T02:22:14.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.887+0000 7f2b6b7fe640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b70073c20 0x7f2b70076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:14.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.887+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7f2b7c12ca70 con 0x7f2b84154180 2024-09-21T02:22:14.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.887+0000 7f2b82ffd640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b70073c20 0x7f2b70076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:14.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.887+0000 7f2b82ffd640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b70073c20 0x7f2b70076130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b840af580 tx=0x7f2b74005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:14.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:14.891+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2b7c016850 con 0x7f2b84154180 2024-09-21T02:22:15.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.059+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b840b1070 con 0x7f2b84154180 2024-09-21T02:22:15.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.059+0000 7f2b6b7fe640 1 -- 172.21.15.170:0/4193949348 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7f2b7c0f95b0 con 0x7f2b84154180 2024-09-21T02:22:15.062 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b70073c20 msgr2=0x7f2b70076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b70073c20 0x7f2b70076130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b840af580 tx=0x7f2b74005f40 comp rx=0 tx=0).stop 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 msgr2=0x7f2b84154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 0x7f2b84154590 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c01d390 tx=0x7f2b7c003ed0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 shutdown_connections 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b70073c20 0x7f2b70076130 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b84154180 0x7f2b84154590 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b840cae40 0x7f2b840af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 --2- 172.21.15.170:0/4193949348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b8415d7e0 0x7f2b840ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 >> 172.21.15.170:0/4193949348 conn(0x7f2b8409a9c0 msgr2=0x7f2b840b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 shutdown_connections 2024-09-21T02:22:15.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.063+0000 7f2b89297640 1 -- 172.21.15.170:0/4193949348 wait complete. 2024-09-21T02:22:15.083 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-21T02:22:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 -- 172.21.15.170:0/4036453241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 msgr2=0x7f5ed40cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4036453241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed40cb870 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40d1160 tx=0x7f5ec801c8c0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 -- 172.21.15.170:0/4036453241 shutdown_connections 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4036453241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed40cb870 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4036453241 >> v2:172.21.15.191:3300/0 conn(0x7f5ed40cf7e0 0x7f5ed4070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 -- 172.21.15.170:0/4036453241 >> 172.21.15.170:0/4036453241 conn(0x7f5ed40b1c40 msgr2=0x7f5ed40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 -- 172.21.15.170:0/4036453241 shutdown_connections 2024-09-21T02:22:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.227+0000 7f5eda02d640 1 -- 172.21.15.170:0/4036453241 wait complete. 2024-09-21T02:22:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 Processor -- start 2024-09-21T02:22:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 -- start start 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed41f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ed40cc8c0 0x7f5ed41fa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ed40cf7e0 0x7f5ed4148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41690d0 con 0x7f5ed40cb460 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4168f50 con 0x7f5ed40cc8c0 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed4169250 con 0x7f5ed40cf7e0 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed41f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed41f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39520/0 (socket says 172.21.15.170:39520) 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 -- 172.21.15.170:0/4056229413 learned_addr learned my addr 172.21.15.170:0/4056229413 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed2ffd640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ed40cc8c0 0x7f5ed41fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed3fff640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ed40cf7e0 0x7f5ed4148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 -- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ed40cf7e0 msgr2=0x7f5ed4148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ed40cf7e0 0x7f5ed4148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 -- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ed40cc8c0 msgr2=0x7f5ed41fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ed40cc8c0 0x7f5ed41fa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 -- 172.21.15.170:0/4056229413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41fa6b0 con 0x7f5ed40cb460 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed2ffd640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ed40cc8c0 0x7f5ed41fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ed37fe640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed41f7640 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400b410 tx=0x7f5ec400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4015020 con 0x7f5ed40cb460 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec4011920 con 0x7f5ed40cb460 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41f6b80 con 0x7f5ed40cb460 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4011d70 con 0x7f5ed40cb460 2024-09-21T02:22:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.231+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41faea0 con 0x7f5ed40cb460 2024-09-21T02:22:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.235+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed40cb870 con 0x7f5ed40cb460 2024-09-21T02:22:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.235+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f5ec4018020 con 0x7f5ed40cb460 2024-09-21T02:22:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.235+0000 7f5ebbfff640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5ec0073c20 0x7f5ec0076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.235+0000 7f5ed2ffd640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5ec0073c20 0x7f5ec0076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.235+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7f5ec4092c80 con 0x7f5ed40cb460 2024-09-21T02:22:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.235+0000 7f5ed2ffd640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5ec0073c20 0x7f5ec0076130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80af000 tx=0x7f5ec801b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:15.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.239+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5ec401f020 con 0x7f5ed40cb460 2024-09-21T02:22:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:15.463+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f5ed40b17f0 con 0x7f5ed40cb460 2024-09-21T02:22:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.359+0000 7f5ebbfff640 1 -- 172.21.15.170:0/4056229413 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v470) ==== 166+0+0 (secure 0 0 0) 0x7f5ec405f5c0 con 0x7f5ed40cb460 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5ec0073c20 msgr2=0x7f5ec0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5ec0073c20 0x7f5ec0076130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80af000 tx=0x7f5ec801b100 comp rx=0 tx=0).stop 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 msgr2=0x7f5ed41f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed41f7640 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400b410 tx=0x7f5ec400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 shutdown_connections 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5ec0073c20 0x7f5ec0076130 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ed40cf7e0 0x7f5ed4148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ed40cc8c0 0x7f5ed41fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 --2- 172.21.15.170:0/4056229413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40cb460 0x7f5ed41f7640 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 >> 172.21.15.170:0/4056229413 conn(0x7f5ed40b1c40 msgr2=0x7f5ed40b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:16.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 shutdown_connections 2024-09-21T02:22:16.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.363+0000 7f5eda02d640 1 -- 172.21.15.170:0/4056229413 wait complete. 2024-09-21T02:22:16.389 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T02:22:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 -- 172.21.15.170:0/711297942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cb460 msgr2=0x7f2ce00cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/711297942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cb460 0x7f2ce00cb870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009900 tx=0x7f2cd001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 -- 172.21.15.170:0/711297942 shutdown_connections 2024-09-21T02:22:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/711297942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cb460 0x7f2ce00cb870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/711297942 >> v2:172.21.15.191:3300/0 conn(0x7f2ce00cf7e0 0x7f2ce0070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 -- 172.21.15.170:0/711297942 >> 172.21.15.170:0/711297942 conn(0x7f2ce00b1c40 msgr2=0x7f2ce00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:16.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 -- 172.21.15.170:0/711297942 shutdown_connections 2024-09-21T02:22:16.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.547+0000 7f2ce86ed640 1 -- 172.21.15.170:0/711297942 wait complete. 2024-09-21T02:22:16.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 Processor -- start 2024-09-21T02:22:16.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 -- start start 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce00cb460 0x7f2ce0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 0x7f2ce009cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cf7e0 0x7f2ce009d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01690d0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0168f50 con 0x7f2ce00cb460 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce0169250 con 0x7f2ce00cf7e0 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 0x7f2ce009cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 0x7f2ce009cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45744/0 (socket says 172.21.15.170:45744) 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 -- 172.21.15.170:0/4021088369 learned_addr learned my addr 172.21.15.170:0/4021088369 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce6c8a640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cf7e0 0x7f2ce009d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce6489640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce00cb460 0x7f2ce0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 -- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cf7e0 msgr2=0x7f2ce009d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cf7e0 0x7f2ce009d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 -- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce00cb460 msgr2=0x7f2ce0158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce00cb460 0x7f2ce0158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 -- 172.21.15.170:0/4021088369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce01fae60 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce6c8a640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cf7e0 0x7f2ce009d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce5c88640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 0x7f2ce009cbd0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009850 tx=0x7f2cd00a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd00bc070 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce00b19e0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cd00043b0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd00b2040 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.551+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01569d0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.555+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce00cb870 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.555+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f2cd0005280 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.555+0000 7f2cd6ffd640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2cc4073c50 0x7f2cc4076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.555+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7f2cd012cab0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.555+0000 7f2ce6489640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2cc4073c50 0x7f2cc4076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.555+0000 7f2ce6489640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2cc4073c50 0x7f2cc4076160 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009a10 tx=0x7f2cdc0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.559+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2cd00164d0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.735+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2ce00b33a0 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.735+0000 7f2cd6ffd640 1 -- 172.21.15.170:0/4021088369 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v51) ==== 59+0+2294 (secure 0 0 0) 0x7f2cd0130910 con 0x7f2ce00cc8c0 2024-09-21T02:22:16.737 INFO:teuthology.orchestra.run.smithi170.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:22:16.737 INFO:teuthology.orchestra.run.smithi170.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:22:16.737 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:22:16.737 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_client 20/20 2024-09-21T02:22:16.737 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4154.426215 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6178.054356 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4752.993307 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3756.357941 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6230.201743 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4977.223537 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5474.467070 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5411.446910 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds 20/20 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:22:16.738 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced debug_ms 2/2 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_frag true 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds dev mds_verify_scatter true 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced client_mount_timeout 600 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_client 20/20 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced debug_ms 1/1 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:22:16.739 INFO:teuthology.orchestra.run.smithi170.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:22:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2cc4073c50 msgr2=0x7f2cc4076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2cc4073c50 0x7f2cc4076160 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009a10 tx=0x7f2cdc0091e0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 msgr2=0x7f2ce009cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 0x7f2ce009cbd0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009850 tx=0x7f2cd00a5730 comp rx=0 tx=0).stop 2024-09-21T02:22:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 shutdown_connections 2024-09-21T02:22:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2cc4073c50 0x7f2cc4076160 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ce00cf7e0 0x7f2ce009d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ce00cc8c0 0x7f2ce009cbd0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 --2- 172.21.15.170:0/4021088369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ce00cb460 0x7f2ce0158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 >> 172.21.15.170:0/4021088369 conn(0x7f2ce00b1c40 msgr2=0x7f2ce0155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 shutdown_connections 2024-09-21T02:22:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.739+0000 7f2ce86ed640 1 -- 172.21.15.170:0/4021088369 wait complete. 2024-09-21T02:22:16.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:22:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1f7db640 1 -- 172.21.15.170:0/3892754865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 msgr2=0x7f5a04003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/3892754865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a04003eb0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f5a14006fd0 tx=0x7f5a1401cfa0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1dd78640 1 -- 172.21.15.170:0/3892754865 reap_dead start 2024-09-21T02:22:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1f7db640 1 -- 172.21.15.170:0/3892754865 shutdown_connections 2024-09-21T02:22:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1f7db640 1 -- 172.21.15.170:0/3892754865 >> 172.21.15.170:0/3892754865 conn(0x7f5a180d2a30 msgr2=0x7f5a180cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1f7db640 1 -- 172.21.15.170:0/3892754865 shutdown_connections 2024-09-21T02:22:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.911+0000 7f5a1f7db640 1 -- 172.21.15.170:0/3892754865 wait complete. 2024-09-21T02:22:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 Processor -- start 2024-09-21T02:22:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 -- start start 2024-09-21T02:22:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a180c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a181573e0 0x7f5a180cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1815a180 0x7f5a180cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18167930 con 0x7f5a04003a40 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181677b0 con 0x7f5a1815a180 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1f7db640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5a18167ab0 con 0x7f5a181573e0 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a180c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a180c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45760/0 (socket says 172.21.15.170:45760) 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 -- 172.21.15.170:0/2039605887 learned_addr learned my addr 172.21.15.170:0/2039605887 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1dd78640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1815a180 0x7f5a180cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1cd76640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a181573e0 0x7f5a180cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 -- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a181573e0 msgr2=0x7f5a180cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a181573e0 0x7f5a180cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 -- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1815a180 msgr2=0x7f5a180cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1815a180 0x7f5a180cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 -- 172.21.15.170:0/2039605887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a180ca010 con 0x7f5a04003a40 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.915+0000 7f5a1d577640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a180c53d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5a1401ca60 tx=0x7f5a14001140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a140014e0 con 0x7f5a04003a40 2024-09-21T02:22:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a14001680 con 0x7f5a04003a40 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a180b2d60 con 0x7f5a04003a40 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a140062f0 con 0x7f5a04003a40 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a180c9a00 con 0x7f5a04003a40 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a180ce1e0 con 0x7f5a04003a40 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f5a14006490 con 0x7f5a04003a40 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a0dffb640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59fc073c20 0x7f59fc076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a1cd76640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59fc073c20 0x7f59fc076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7f5a1412c940 con 0x7f5a04003a40 2024-09-21T02:22:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.919+0000 7f5a1cd76640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59fc073c20 0x7f59fc076130 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a180700f0 tx=0x7f5a0800c6a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:16.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:16.923+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f5a14016d50 con 0x7f5a04003a40 2024-09-21T02:22:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.095+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5a1815ad30 con 0x7f5a04003a40 2024-09-21T02:22:17.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.099+0000 7f5a0dffb640 1 -- 172.21.15.170:0/2039605887 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f5a140f9280 con 0x7f5a04003a40 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59fc073c20 msgr2=0x7f59fc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59fc073c20 0x7f59fc076130 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a180700f0 tx=0x7f5a0800c6a0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 msgr2=0x7f5a180c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a180c53d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5a1401ca60 tx=0x7f5a14001140 comp rx=0 tx=0).stop 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 shutdown_connections 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f59fc073c20 0x7f59fc076130 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5a181573e0 0x7f5a180cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a1815a180 0x7f5a180cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 --2- 172.21.15.170:0/2039605887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a04003a40 0x7f5a180c53d0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 >> 172.21.15.170:0/2039605887 conn(0x7f5a180d2a30 msgr2=0x7f5a18156230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 shutdown_connections 2024-09-21T02:22:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.103+0000 7f5a1f7db640 1 -- 172.21.15.170:0/2039605887 wait complete. 2024-09-21T02:22:17.124 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:22:17.124 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:22:17.200 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:22:17.274 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:22:17.275 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:17.275 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:17.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:22:17.281 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:17.281 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:17.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:22:17.287 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:17.287 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:17.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:22:17.328 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:17.328 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:17.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:22:17.334 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T02:22:17.334 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 -- 172.21.15.170:0/2876148057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff80cd210 msgr2=0x7feff80b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 --2- 172.21.15.170:0/2876148057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff80cd210 0x7feff80b3350 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7feff80b3a10 tx=0x7feff001cc30 comp rx=0 tx=0).stop 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 -- 172.21.15.170:0/2876148057 shutdown_connections 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 --2- 172.21.15.170:0/2876148057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff80cd210 0x7feff80b3350 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 --2- 172.21.15.170:0/2876148057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff80b5410 0x7feff806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 -- 172.21.15.170:0/2876148057 >> 172.21.15.170:0/2876148057 conn(0x7feff80b1c40 msgr2=0x7feff80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 -- 172.21.15.170:0/2876148057 shutdown_connections 2024-09-21T02:22:17.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 -- 172.21.15.170:0/2876148057 wait complete. 2024-09-21T02:22:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.523+0000 7feffd156640 1 Processor -- start 2024-09-21T02:22:17.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- start start 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80b2f40 0x7feff80cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff80b5410 0x7feff80ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 0x7feff81f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feff80c3a20 con 0x7feff8156930 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feff80cf970 con 0x7feff80b2f40 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7feff806ff70 con 0x7feff80b5410 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 0x7feff81f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 0x7feff81f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45778/0 (socket says 172.21.15.170:45778) 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 -- 172.21.15.170:0/3544923713 learned_addr learned my addr 172.21.15.170:0/3544923713 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:17.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff6d9d640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80b2f40 0x7feff80cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff659c640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff80b5410 0x7feff80ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 -- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff80b5410 msgr2=0x7feff80ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff80b5410 0x7feff80ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 -- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80b2f40 msgr2=0x7feff80cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80b2f40 0x7feff80cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 -- 172.21.15.170:0/3544923713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff8074480 con 0x7feff8156930 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff6d9d640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80b2f40 0x7feff80cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feff759e640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 0x7feff81f5780 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feff001b2e0 tx=0x7feff0004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00bb070 con 0x7feff8156930 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feff0004850 con 0x7feff8156930 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00b2040 con 0x7feff8156930 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff8156580 con 0x7feff8156930 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff8073340 con 0x7feff8156930 2024-09-21T02:22:17.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.527+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff8072e50 con 0x7feff8156930 2024-09-21T02:22:17.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.531+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7feff0004cb0 con 0x7feff8156930 2024-09-21T02:22:17.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.531+0000 7fefdf7fe640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fefe4073c20 0x7fefe4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.531+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7feff012d500 con 0x7feff8156930 2024-09-21T02:22:17.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.531+0000 7feff6d9d640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fefe4073c20 0x7fefe4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.531+0000 7feff6d9d640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fefe4073c20 0x7fefe4076130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fefec005fd0 tx=0x7fefec005dc0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:17.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.535+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7feff00c4020 con 0x7feff8156930 2024-09-21T02:22:17.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.731+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feff806fa30 con 0x7feff8156930 2024-09-21T02:22:17.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7fefdf7fe640 1 -- 172.21.15.170:0/3544923713 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+3925 (secure 0 0 0) 0x7feff00168b0 con 0x7feff8156930 2024-09-21T02:22:17.736 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:17.736 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":113,"btime":"2024-09-21T02:22:11:322529+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/1097589548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1097589548},{"type":"v1","addr":"172.21.15.191:6801","nonce":1097589548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7027,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2324635132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2324635132},{"type":"v1","addr":"172.21.15.191:6835","nonce":2324635132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1737078629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1737078629},{"type":"v1","addr":"172.21.15.170:6833","nonce":1737078629}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/996317712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":996317712},{"type":"v1","addr":"172.21.15.170:6835","nonce":996317712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113}],"filesystems":[]} 2024-09-21T02:22:17.737 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 113 2024-09-21T02:22:17.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fefe4073c20 msgr2=0x7fefe4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fefe4073c20 0x7fefe4076130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fefec005fd0 tx=0x7fefec005dc0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 msgr2=0x7feff81f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 0x7feff81f5780 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feff001b2e0 tx=0x7feff0004660 comp rx=0 tx=0).stop 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 shutdown_connections 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fefe4073c20 0x7fefe4076130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feff80b5410 0x7feff80ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8156930 0x7feff81f5780 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 --2- 172.21.15.170:0/3544923713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80b2f40 0x7feff80cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 >> 172.21.15.170:0/3544923713 conn(0x7feff80b1c40 msgr2=0x7feff80d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 shutdown_connections 2024-09-21T02:22:17.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.735+0000 7feffd156640 1 -- 172.21.15.170:0/3544923713 wait complete. 2024-09-21T02:22:17.757 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T02:22:17.757 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T02:22:17.757 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:22:17.757 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:17.758 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T02:22:17.758 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T02:22:17.758 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:22:17.758 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:17.758 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T02:22:17.758 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T02:22:17.758 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:22:17.758 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:17.759 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T02:22:17.759 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T02:22:17.759 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:22:17.759 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:17.770 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:22:17.770 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T02:22:17.772 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:22:17.772 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T02:22:17.786 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:22:17.786 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T02:22:17.787 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:22:17.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T02:22:17.820 INFO:tasks.ceph.mds.a:Started 2024-09-21T02:22:17.820 DEBUG:teuthology.parallel:result is None 2024-09-21T02:22:17.821 INFO:tasks.ceph.mds.c:Started 2024-09-21T02:22:17.821 DEBUG:teuthology.parallel:result is None 2024-09-21T02:22:17.830 INFO:tasks.ceph.mds.d:Started 2024-09-21T02:22:17.830 DEBUG:teuthology.parallel:result is None 2024-09-21T02:22:17.831 INFO:tasks.ceph.mds.b:Started 2024-09-21T02:22:17.831 DEBUG:teuthology.parallel:result is None 2024-09-21T02:22:17.832 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T02:22:17.911 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2024-09-21T02:22:17.912 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:22:17.931 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-09-21T02:22:17.934 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-21T02:22:17.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- 172.21.15.170:0/40421511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 msgr2=0x7f301815ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 --2- 172.21.15.170:0/40421511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 0x7f301815ff00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3000009840 tx=0x7f300001c740 comp rx=0 tx=0).stop 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- 172.21.15.170:0/40421511 shutdown_connections 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 --2- 172.21.15.170:0/40421511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 0x7f301815ff00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 --2- 172.21.15.170:0/40421511 >> v2:172.21.15.191:3300/0 conn(0x7f30180b25f0 0x7f30180cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- 172.21.15.170:0/40421511 >> 172.21.15.170:0/40421511 conn(0x7f301809a9c0 msgr2=0x7f30180d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- 172.21.15.170:0/40421511 shutdown_connections 2024-09-21T02:22:17.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- 172.21.15.170:0/40421511 wait complete. 2024-09-21T02:22:17.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 Processor -- start 2024-09-21T02:22:17.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- start start 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30180b25f0 0x7f30180c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 0x7f3018167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 0x7f3018168300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3018167390 con 0x7f3018160440 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3018167210 con 0x7f30180b25f0 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301ce0e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3018167510 con 0x7f301815faf0 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 0x7f3018168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 0x7f3018168300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45820/0 (socket says 172.21.15.170:45820) 2024-09-21T02:22:17.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 -- 172.21.15.170:0/2190209750 learned_addr learned my addr 172.21.15.170:0/2190209750 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f301659c640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30180b25f0 0x7f30180c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3015d9b640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 0x7f3018167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 -- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 msgr2=0x7f3018167dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 0x7f3018167dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 -- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30180b25f0 msgr2=0x7f30180c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30180b25f0 0x7f30180c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.971+0000 7f3016d9d640 1 -- 172.21.15.170:0/2190209750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3018154fc0 con 0x7f3018160440 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f3016d9d640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 0x7f3018168300 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f300800cbb0 tx=0x7f300800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:17.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008016020 con 0x7f3018160440 2024-09-21T02:22:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f300800ab00 con 0x7f3018160440 2024-09-21T02:22:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008012420 con 0x7f3018160440 2024-09-21T02:22:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30181fda30 con 0x7f3018160440 2024-09-21T02:22:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181fdff0 con 0x7f3018160440 2024-09-21T02:22:17.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30181fd840 con 0x7f3018160440 2024-09-21T02:22:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f3008019020 con 0x7f3018160440 2024-09-21T02:22:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f300effd640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ff8073c20 0x7f2ff8076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4971+0+0 (secure 0 0 0) 0x7f3008093060 con 0x7f3018160440 2024-09-21T02:22:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f301659c640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ff8073c20 0x7f2ff8076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:17.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.975+0000 7f301659c640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ff8073c20 0x7f2ff8076130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f3010005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:17.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:17.979+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3008020020 con 0x7f3018160440 2024-09-21T02:22:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.151+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3018167a30 con 0x7f3018160440 2024-09-21T02:22:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.387+0000 7f300effd640 1 -- 172.21.15.170:0/2190209750 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v160) ==== 97+0+0 (secure 0 0 0) 0x7f300805f2c0 con 0x7f3018160440 2024-09-21T02:22:18.389 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2024-09-21T02:22:18.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ff8073c20 msgr2=0x7f2ff8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ff8073c20 0x7f2ff8076130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f3010005f40 comp rx=0 tx=0).stop 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 msgr2=0x7f3018168300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 0x7f3018168300 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f300800cbb0 tx=0x7f300800cca0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 shutdown_connections 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2ff8073c20 0x7f2ff8076130 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f301815faf0 0x7f3018167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3018160440 0x7f3018168300 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 --2- 172.21.15.170:0/2190209750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30180b25f0 0x7f30180c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 >> 172.21.15.170:0/2190209750 conn(0x7f301809a9c0 msgr2=0x7f30180b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 shutdown_connections 2024-09-21T02:22:18.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.391+0000 7f301ce0e640 1 -- 172.21.15.170:0/2190209750 wait complete. 2024-09-21T02:22:18.417 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T02:22:18.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/754052988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d00b3e30 msgr2=0x7fb8d00b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/754052988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d00b3e30 0x7fb8d00b4240 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009c80 tx=0x7fb8c801cda0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/754052988 shutdown_connections 2024-09-21T02:22:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/754052988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8d00d0100 0x7fb8d00d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/754052988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d00b3e30 0x7fb8d00b4240 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/754052988 >> 172.21.15.170:0/754052988 conn(0x7fb8d00cee30 msgr2=0x7fb8d00cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/754052988 shutdown_connections 2024-09-21T02:22:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.555+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/754052988 wait complete. 2024-09-21T02:22:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 Processor -- start 2024-09-21T02:22:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 -- start start 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 0x7fb8d0150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d00b3e30 0x7fb8d00cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8d00d0100 0x7fb8d00cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d016c1d0 con 0x7fb8d0072900 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d016c050 con 0x7fb8d00b3e30 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d016c350 con 0x7fb8d00d0100 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 0x7fb8d0150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 0x7fb8d0150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45848/0 (socket says 172.21.15.170:45848) 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 -- 172.21.15.170:0/942008484 learned_addr learned my addr 172.21.15.170:0/942008484 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d53a9640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8d00d0100 0x7fb8d00cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 -- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8d00d0100 msgr2=0x7fb8d00cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8cffff640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d00b3e30 0x7fb8d00cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8d00d0100 0x7fb8d00cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 -- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d00b3e30 msgr2=0x7fb8d00cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d00b3e30 0x7fb8d00cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 -- 172.21.15.170:0/942008484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d0150c90 con 0x7fb8d0072900 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d4ba8640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 0x7fb8d0150750 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009bd0 tx=0x7fb8c80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80bb070 con 0x7fb8d0072900 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8c8004e00 con 0x7fb8d0072900 2024-09-21T02:22:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80b2040 con 0x7fb8d0072900 2024-09-21T02:22:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d015bbd0 con 0x7fb8d0072900 2024-09-21T02:22:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.559+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d015be90 con 0x7fb8d0072900 2024-09-21T02:22:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.563+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d00b4240 con 0x7fb8d0072900 2024-09-21T02:22:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.563+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fb8c8004260 con 0x7fb8d0072900 2024-09-21T02:22:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.563+0000 7fb8cd7fa640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8b0073c50 0x7fb8b0076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.563+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4885+0+0 (secure 0 0 0) 0x7fb8c812cd20 con 0x7fb8d0072900 2024-09-21T02:22:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.563+0000 7fb8cffff640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8b0073c50 0x7fb8b0076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.563+0000 7fb8cffff640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8b0073c50 0x7fb8b0076160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80058d0 tx=0x7fb8b8005bf0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.567+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb8c8016990 con 0x7fb8d0072900 2024-09-21T02:22:18.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.787+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb8d0162620 con 0x7fb8d0072900 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.787+0000 7fb8cd7fa640 1 -- 172.21.15.170:0/942008484 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5304 (secure 0 0 0) 0x7fb8c80f95d0 con 0x7fb8d0072900 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mwX0pMxAAl5hmIfuB07LCQXp5il+xJg==", 2024-09-21T02:22:18.790 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m3PtdBBAApZtT8BPw+X6JWH+PYREZoA==", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.791 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mxT25OhAAJ5OAzpdGF7aJXuLuMDlD0A==", 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2024-09-21T02:22:18.792 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mTWE0DBAA+cPdg77j/Sm7O1lCrGCm7w==", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCLK+5miU88JRAACgK0+epfNC83x2NtPeE+/g==", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCOK+5mIqdZNRAAyz74A4uI7a5/ozSypYTh2A==", 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCSK+5mQpXPGBAALWQMZar2RIv7hartQNEJfA==", 2024-09-21T02:22:18.794 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCVK+5mlxtiKRAAkVKy9XpFUuzBC364FjmEZg==", 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCbK+5meAbhMBAAW/Q0PYPbtC1KbfBibv9u9A==", 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2024-09-21T02:22:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCfK+5mnQ54BhAAO32cTTCDchWpv8Frq4mSsQ==", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCiK+5m2prNFhAASKT1tvw1FGxmH3WPUE7kKg==", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQClK+5mlGbVKBAAMiyTrdXhLGqJrnuTY5ArOw==", 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA==", 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mhmRPGhAAOo2mhnjTRvhpmRiL1kxtXA==", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5m2qXMFRAAAI72W4Aig/1BdZ+VPnTwHQ==", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCKK+5mjSRzHhAAyDoU5C4VtrPJsh2XBbanbA==", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2024-09-21T02:22:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mc1F5FRAA2U9rxAsHGP/aQGPRmwEUJg==", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mSpYFLRAAEfWR+FZmqSHUw0fn6k3qlA==", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQCJK+5mY/ZXKBAANcSdFDrj3bFFppyd07U5bg==", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2024-09-21T02:22:18.802 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2024-09-21T02:22:18.803 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2024-09-21T02:22:18.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:18.803 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T02:22:18.803 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8b0073c50 msgr2=0x7fb8b0076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8b0073c50 0x7fb8b0076160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80058d0 tx=0x7fb8b8005bf0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 msgr2=0x7fb8d0150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 0x7fb8d0150750 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009bd0 tx=0x7fb8c80093f0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 shutdown_connections 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb8b0073c50 0x7fb8b0076160 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb8d00d0100 0x7fb8d00cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d00b3e30 0x7fb8d00cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 --2- 172.21.15.170:0/942008484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8d0072900 0x7fb8d0150750 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 >> 172.21.15.170:0/942008484 conn(0x7fb8d00cee30 msgr2=0x7fb8d00b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 shutdown_connections 2024-09-21T02:22:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.791+0000 7fb8d6e0c640 1 -- 172.21.15.170:0/942008484 wait complete. 2024-09-21T02:22:18.811 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 -- 172.21.15.170:0/2445461514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b2f40 msgr2=0x7f26480b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 --2- 172.21.15.170:0/2445461514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b2f40 0x7f26480b3350 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f26480b3a10 tx=0x7f263801cda0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 -- 172.21.15.170:0/2445461514 shutdown_connections 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 --2- 172.21.15.170:0/2445461514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b2f40 0x7f26480b3350 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 --2- 172.21.15.170:0/2445461514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26480b5410 0x7f264806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 -- 172.21.15.170:0/2445461514 >> 172.21.15.170:0/2445461514 conn(0x7f26480b1c40 msgr2=0x7f26480b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 -- 172.21.15.170:0/2445461514 shutdown_connections 2024-09-21T02:22:18.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 -- 172.21.15.170:0/2445461514 wait complete. 2024-09-21T02:22:18.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 Processor -- start 2024-09-21T02:22:18.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.963+0000 7f264f186640 1 -- start start 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26480b2f40 0x7f2648148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 0x7f264809b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26480ccb00 0x7f264809b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2648169fd0 con 0x7f26480b5410 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2648169e50 con 0x7f26480b2f40 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f264816a150 con 0x7f26480ccb00 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 0x7f264809b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 0x7f264809b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45862/0 (socket says 172.21.15.170:45862) 2024-09-21T02:22:18.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 -- 172.21.15.170:0/2753482324 learned_addr learned my addr 172.21.15.170:0/2753482324 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264cf22640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26480b2f40 0x7f2648148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264d723640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26480ccb00 0x7f264809b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 -- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26480ccb00 msgr2=0x7f264809b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26480ccb00 0x7f264809b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 -- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26480b2f40 msgr2=0x7f2648148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26480b2f40 0x7f2648148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 -- 172.21.15.170:0/2753482324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648155280 con 0x7f26480b5410 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264cf22640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26480b2f40 0x7f2648148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f2647fff640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 0x7f264809b240 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f263801b450 tx=0x7f2638005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380b2040 con 0x7f26480b5410 2024-09-21T02:22:18.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26380bb070 con 0x7f26480b5410 2024-09-21T02:22:18.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264809daf0 con 0x7f26480b5410 2024-09-21T02:22:18.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638004530 con 0x7f26480b5410 2024-09-21T02:22:18.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264809d810 con 0x7f26480b5410 2024-09-21T02:22:18.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26480b3350 con 0x7f26480b5410 2024-09-21T02:22:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f26380046d0 con 0x7f26480b5410 2024-09-21T02:22:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f26457fa640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2628073c70 0x7f2628076180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4885+0+0 (secure 0 0 0) 0x7f263812da50 con 0x7f26480b5410 2024-09-21T02:22:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264cf22640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2628073c70 0x7f2628076180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.967+0000 7f264cf22640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2628073c70 0x7f2628076180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2630005fd0 tx=0x7f2630005d00 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:18.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:18.975+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2638016990 con 0x7f26480b5410 2024-09-21T02:22:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.147+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f26480b1460 con 0x7f26480b5410 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.147+0000 7f26457fa640 1 -- 172.21.15.170:0/2753482324 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v162) ==== 84+0+65 (secure 0 0 0) 0x7f26380fa370 con 0x7f26480b5410 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2024-09-21T02:22:19.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:19.151 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2628073c70 msgr2=0x7f2628076180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2628073c70 0x7f2628076180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2630005fd0 tx=0x7f2630005d00 comp rx=0 tx=0).stop 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 msgr2=0x7f264809b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 0x7f264809b240 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f263801b450 tx=0x7f2638005b10 comp rx=0 tx=0).stop 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 shutdown_connections 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2628073c70 0x7f2628076180 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26480ccb00 0x7f264809b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26480b5410 0x7f264809b240 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 --2- 172.21.15.170:0/2753482324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26480b2f40 0x7f2648148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 >> 172.21.15.170:0/2753482324 conn(0x7f26480b1c40 msgr2=0x7f2648074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 shutdown_connections 2024-09-21T02:22:19.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.151+0000 7f264f186640 1 -- 172.21.15.170:0/2753482324 wait complete. 2024-09-21T02:22:19.173 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:22:19.174 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 -- 172.21.15.170:0/2763826982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 msgr2=0x7fc87c0ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 --2- 172.21.15.170:0/2763826982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c0ca720 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc87c071d80 tx=0x7fc87001c830 comp rx=0 tx=0).stop 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 -- 172.21.15.170:0/2763826982 shutdown_connections 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 --2- 172.21.15.170:0/2763826982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c0ca720 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 --2- 172.21.15.170:0/2763826982 >> v2:172.21.15.191:3300/0 conn(0x7fc87c09dc00 0x7fc87c15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 -- 172.21.15.170:0/2763826982 >> 172.21.15.170:0/2763826982 conn(0x7fc87c0b1c40 msgr2=0x7fc87c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 -- 172.21.15.170:0/2763826982 shutdown_connections 2024-09-21T02:22:19.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 -- 172.21.15.170:0/2763826982 wait complete. 2024-09-21T02:22:19.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 Processor -- start 2024-09-21T02:22:19.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.331+0000 7fc8820af640 1 -- start start 2024-09-21T02:22:19.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc87c09dc00 0x7fc87c09c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:19.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c1f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:19.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc87c0ccf60 0x7fc87c06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:19.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c169380 con 0x7fc87c0ccb50 2024-09-21T02:22:19.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c169200 con 0x7fc87c0ccf60 2024-09-21T02:22:19.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc87c169500 con 0x7fc87c09dc00 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c1f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c1f9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45888/0 (socket says 172.21.15.170:45888) 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 -- 172.21.15.170:0/3584972867 learned_addr learned my addr 172.21.15.170:0/3584972867 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87b7fe640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc87c09dc00 0x7fc87c09c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87bfff640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc87c0ccf60 0x7fc87c06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 -- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc87c09dc00 msgr2=0x7fc87c09c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc87c09dc00 0x7fc87c09c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 -- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc87c0ccf60 msgr2=0x7fc87c06f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc87c0ccf60 0x7fc87c06f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 -- 172.21.15.170:0/3584972867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c09aaf0 con 0x7fc87c0ccb50 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87bfff640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc87c0ccf60 0x7fc87c06f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87affd640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c1f9cd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc870009230 tx=0x7fc870004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:19.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870004430 con 0x7fc87c0ccb50 2024-09-21T02:22:19.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8700bd070 con 0x7fc87c0ccb50 2024-09-21T02:22:19.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8700053a0 con 0x7fc87c0ccb50 2024-09-21T02:22:19.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c09add0 con 0x7fc87c0ccb50 2024-09-21T02:22:19.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c09b390 con 0x7fc87c0ccb50 2024-09-21T02:22:19.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c15ad30 con 0x7fc87c0ccb50 2024-09-21T02:22:19.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fc870005540 con 0x7fc87c0ccb50 2024-09-21T02:22:19.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc863fff640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc868073c20 0x7fc868076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:19.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc87b7fe640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc868073c20 0x7fc868076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:19.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.335+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4885+0+0 (secure 0 0 0) 0x7fc87012e210 con 0x7fc87c0ccb50 2024-09-21T02:22:19.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.339+0000 7fc87b7fe640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc868073c20 0x7fc868076130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005870 tx=0x7fc86c00c440 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:19.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.339+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc8700c6020 con 0x7fc87c0ccb50 2024-09-21T02:22:19.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:19.511+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc87c09cd60 con 0x7fc87c0ccb50 2024-09-21T02:22:20.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.407+0000 7fc863fff640 1 -- 172.21.15.170:0/3584972867 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v163) ==== 138+0+0 (secure 0 0 0) 0x7fc8700164b0 con 0x7fc87c0ccb50 2024-09-21T02:22:20.411 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2024-09-21T02:22:20.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc868073c20 msgr2=0x7fc868076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:20.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc868073c20 0x7fc868076130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005870 tx=0x7fc86c00c440 comp rx=0 tx=0).stop 2024-09-21T02:22:20.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 msgr2=0x7fc87c1f9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:20.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c1f9cd0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc870009230 tx=0x7fc870004040 comp rx=0 tx=0).stop 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 shutdown_connections 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc868073c20 0x7fc868076130 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc87c0ccf60 0x7fc87c06f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc87c0ccb50 0x7fc87c1f9cd0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 --2- 172.21.15.170:0/3584972867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc87c09dc00 0x7fc87c09c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 >> 172.21.15.170:0/3584972867 conn(0x7fc87c0b1c40 msgr2=0x7fc87c0ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 shutdown_connections 2024-09-21T02:22:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.415+0000 7fc8820af640 1 -- 172.21.15.170:0/3584972867 wait complete. 2024-09-21T02:22:20.449 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 -- 172.21.15.170:0/875169970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc0d0700 msgr2=0x7f7dcc0d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/875169970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc0d0700 0x7f7dcc0d0b10 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009840 tx=0x7f7dc001c7f0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 -- 172.21.15.170:0/875169970 shutdown_connections 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/875169970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc0d0700 0x7f7dcc0d0b10 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/875169970 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dcc1606e0 0x7f7dcc160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 -- 172.21.15.170:0/875169970 >> 172.21.15.170:0/875169970 conn(0x7f7dcc0d2a30 msgr2=0x7f7dcc0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 -- 172.21.15.170:0/875169970 shutdown_connections 2024-09-21T02:22:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.615+0000 7f7dd2d58640 1 -- 172.21.15.170:0/875169970 wait complete. 2024-09-21T02:22:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 Processor -- start 2024-09-21T02:22:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 -- start start 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dcc0d0700 0x7f7dcc14a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dcc15d3e0 0x7f7dcc153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 0x7f7dcc1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc15dc90 con 0x7f7dcc1606e0 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc15db10 con 0x7f7dcc15d3e0 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd2d58640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc169cb0 con 0x7f7dcc0d0700 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd12f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 0x7f7dcc1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd12f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 0x7f7dcc1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45914/0 (socket says 172.21.15.170:45914) 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd12f5640 1 -- 172.21.15.170:0/3486730981 learned_addr learned my addr 172.21.15.170:0/3486730981 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dd0af4640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dcc0d0700 0x7f7dcc14a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.619+0000 7f7dcbfff640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dcc15d3e0 0x7f7dcc153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd12f5640 1 -- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dcc0d0700 msgr2=0x7f7dcc14a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd12f5640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dcc0d0700 0x7f7dcc14a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd12f5640 1 -- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dcc15d3e0 msgr2=0x7f7dcc153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd12f5640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dcc15d3e0 0x7f7dcc153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd12f5640 1 -- 172.21.15.170:0/3486730981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc14b300 con 0x7f7dcc1606e0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd12f5640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 0x7f7dcc1542d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400cb10 tx=0x7f7dc400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4016020 con 0x7f7dcc1606e0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dc400ce50 con 0x7f7dcc1606e0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400ad60 con 0x7f7dcc1606e0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc14f9d0 con 0x7f7dcc1606e0 2024-09-21T02:22:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc14ff90 con 0x7f7dcc1606e0 2024-09-21T02:22:20.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc14f7c0 con 0x7f7dcc1606e0 2024-09-21T02:22:20.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f7dc4019020 con 0x7f7dcc1606e0 2024-09-21T02:22:20.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dc97fa640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7db8073c20 0x7f7db8076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:20.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dd0af4640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7db8073c20 0x7f7db8076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:20.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.623+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 5255+0+0 (secure 0 0 0) 0x7f7dc40933d0 con 0x7f7dcc1606e0 2024-09-21T02:22:20.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.627+0000 7f7dd0af4640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7db8073c20 0x7f7db8076130 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0058d0 tx=0x7f7dbc005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:20.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.631+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7dc4020020 con 0x7f7dcc1606e0 2024-09-21T02:22:20.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:20.803+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7dcc0d0b10 con 0x7f7dcc1606e0 2024-09-21T02:22:21.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.419+0000 7f7dc97fa640 1 -- 172.21.15.170:0/3486730981 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v164) ==== 170+0+0 (secure 0 0 0) 0x7f7dc405fc60 con 0x7f7dcc1606e0 2024-09-21T02:22:21.423 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.423+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7db8073c20 msgr2=0x7f7db8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.423+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7db8073c20 0x7f7db8076130 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0058d0 tx=0x7f7dbc005860 comp rx=0 tx=0).stop 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.423+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 msgr2=0x7f7dcc1542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.423+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 0x7f7dcc1542d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400cb10 tx=0x7f7dc400cc00 comp rx=0 tx=0).stop 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 shutdown_connections 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7db8073c20 0x7f7db8076130 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc1606e0 0x7f7dcc1542d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dcc15d3e0 0x7f7dcc153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 --2- 172.21.15.170:0/3486730981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dcc0d0700 0x7f7dcc14a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 >> 172.21.15.170:0/3486730981 conn(0x7f7dcc0d2a30 msgr2=0x7f7dcc15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:21.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 shutdown_connections 2024-09-21T02:22:21.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.427+0000 7f7dd2d58640 1 -- 172.21.15.170:0/3486730981 wait complete. 2024-09-21T02:22:21.469 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 -- 172.21.15.170:0/2849901675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda980b2fc0 msgr2=0x7fda980ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2849901675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda980b2fc0 0x7fda980ce540 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fda9809e220 tx=0x7fda9401c8a0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 -- 172.21.15.170:0/2849901675 shutdown_connections 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2849901675 >> v2:172.21.15.191:3300/0 conn(0x7fda9806ef40 0x7fda9806f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2849901675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda980cea80 0x7fda980cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2849901675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda980b2fc0 0x7fda980ce540 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 -- 172.21.15.170:0/2849901675 >> 172.21.15.170:0/2849901675 conn(0x7fda980d0170 msgr2=0x7fda980d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:21.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 -- 172.21.15.170:0/2849901675 shutdown_connections 2024-09-21T02:22:21.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.627+0000 7fda9fc97640 1 -- 172.21.15.170:0/2849901675 wait complete. 2024-09-21T02:22:21.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 Processor -- start 2024-09-21T02:22:21.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 -- start start 2024-09-21T02:22:21.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 0x7fda9809b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:21.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda980b2fc0 0x7fda98159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:21.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda980cea80 0x7fda9815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:21.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda98070430 con 0x7fda9806ef40 2024-09-21T02:22:21.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fda980702b0 con 0x7fda980b2fc0 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fda980705b0 con 0x7fda980cea80 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 0x7fda9809b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 0x7fda9809b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45934/0 (socket says 172.21.15.170:45934) 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 -- 172.21.15.170:0/2208601127 learned_addr learned my addr 172.21.15.170:0/2208601127 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9e234640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda980cea80 0x7fda9815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9d232640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda980b2fc0 0x7fda98159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 -- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda980cea80 msgr2=0x7fda9815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda980cea80 0x7fda9815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 -- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda980b2fc0 msgr2=0x7fda98159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda980b2fc0 0x7fda98159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 -- 172.21.15.170:0/2208601127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda98157080 con 0x7fda9806ef40 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9da33640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 0x7fda9809b3d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fda940b0040 tx=0x7fda940a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda940ba020 con 0x7fda9806ef40 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda940043b0 con 0x7fda9806ef40 2024-09-21T02:22:21.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda940b2040 con 0x7fda9806ef40 2024-09-21T02:22:21.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9809cd40 con 0x7fda9806ef40 2024-09-21T02:22:21.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.631+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9809d300 con 0x7fda9806ef40 2024-09-21T02:22:21.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.635+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda98157a40 con 0x7fda9806ef40 2024-09-21T02:22:21.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.635+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fda940bd070 con 0x7fda9806ef40 2024-09-21T02:22:21.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.635+0000 7fda8e7fc640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda7c073c20 0x7fda7c076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:21.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.635+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5637+0+0 (secure 0 0 0) 0x7fda9412c2a0 con 0x7fda9806ef40 2024-09-21T02:22:21.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.635+0000 7fda9d232640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda7c073c20 0x7fda7c076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:21.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.635+0000 7fda9d232640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda7c073c20 0x7fda7c076130 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fda88005970 tx=0x7fda88005900 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:21.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.639+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fda94016490 con 0x7fda9806ef40 2024-09-21T02:22:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:21.859+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fda9815ca10 con 0x7fda9806ef40 2024-09-21T02:22:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.459+0000 7fda8e7fc640 1 -- 172.21.15.170:0/2208601127 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v116) ==== 174+0+0 (secure 0 0 0) 0x7fda940f8be0 con 0x7fda9806ef40 2024-09-21T02:22:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 26 and data pool 27 2024-09-21T02:22:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda7c073c20 msgr2=0x7fda7c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda7c073c20 0x7fda7c076130 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fda88005970 tx=0x7fda88005900 comp rx=0 tx=0).stop 2024-09-21T02:22:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 msgr2=0x7fda9809b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 0x7fda9809b3d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fda940b0040 tx=0x7fda940a55a0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 shutdown_connections 2024-09-21T02:22:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fda7c073c20 0x7fda7c076130 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda980cea80 0x7fda9815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda980b2fc0 0x7fda98159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 --2- 172.21.15.170:0/2208601127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda9806ef40 0x7fda9809b3d0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 >> 172.21.15.170:0/2208601127 conn(0x7fda980d0170 msgr2=0x7fda98158240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:22.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 shutdown_connections 2024-09-21T02:22:22.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.467+0000 7fda9fc97640 1 -- 172.21.15.170:0/2208601127 wait complete. 2024-09-21T02:22:22.507 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 -- 172.21.15.170:0/764537045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 msgr2=0x7fa5a4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/764537045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a4070340 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009900 tx=0x7fa5a001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 -- 172.21.15.170:0/764537045 shutdown_connections 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/764537045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a40ce1c0 0x7fa5a40ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/764537045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a4070340 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 -- 172.21.15.170:0/764537045 >> 172.21.15.170:0/764537045 conn(0x7fa5a40b1c40 msgr2=0x7fa5a40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 -- 172.21.15.170:0/764537045 shutdown_connections 2024-09-21T02:22:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.655+0000 7fa5ab36a640 1 -- 172.21.15.170:0/764537045 wait complete. 2024-09-21T02:22:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 Processor -- start 2024-09-21T02:22:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 -- start start 2024-09-21T02:22:22.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a40cb460 0x7fa5a40c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:22.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a40ce1c0 0x7fa5a40c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:22.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:22.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41690d0 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4168f50 con 0x7fa5a40cb460 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5ab36a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a4169250 con 0x7fa5a40ce1c0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a409a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45944/0 (socket says 172.21.15.170:45944) 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 -- 172.21.15.170:0/2225310451 learned_addr learned my addr 172.21.15.170:0/2225310451 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9106640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a40cb460 0x7fa5a40c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a8905640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a40ce1c0 0x7fa5a40c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 -- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a40ce1c0 msgr2=0x7fa5a40c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a40ce1c0 0x7fa5a40c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 -- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a40cb460 msgr2=0x7fa5a40c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a40cb460 0x7fa5a40c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 -- 172.21.15.170:0/2225310451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40c9c40 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.659+0000 7fa5a9907640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a409a9c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e910 tx=0x7fa59c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c004d30 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa59c00cb20 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a40c9f20 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c005020 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a40c50b0 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a4156d40 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fa59c0051c0 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa599ffb640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa588073c20 0x7fa588076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5737+0+0 (secure 0 0 0) 0x7fa59c0938e0 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa5a9106640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa588073c20 0x7fa588076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.663+0000 7fa5a9106640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa588073c20 0x7fa588076130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00af000 tx=0x7fa5a001b120 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.667+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa59c016030 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.843+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5a409ce60 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa599ffb640 1 -- 172.21.15.170:0/2225310451 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+16780 (secure 0 0 0) 0x7fa59c05fb40 con 0x7fa5a40cf7e0 2024-09-21T02:22:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:22.848 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":165,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:21.863813+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:22:19.529466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T02:22:20.819055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:22.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa588073c20 msgr2=0x7fa588076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa588073c20 0x7fa588076130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00af000 tx=0x7fa5a001b120 comp rx=0 tx=0).stop 2024-09-21T02:22:22.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 msgr2=0x7fa5a409a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a409a9c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e910 tx=0x7fa59c00edd0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 shutdown_connections 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa588073c20 0x7fa588076130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5a40ce1c0 0x7fa5a40c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40cf7e0 0x7fa5a409a9c0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 --2- 172.21.15.170:0/2225310451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5a40cb460 0x7fa5a40c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 >> 172.21.15.170:0/2225310451 conn(0x7fa5a40b1c40 msgr2=0x7fa5a40cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 shutdown_connections 2024-09-21T02:22:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:22.847+0000 7fa5ab36a640 1 -- 172.21.15.170:0/2225310451 wait complete. 2024-09-21T02:22:22.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:23.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 -- 172.21.15.170:0/1187477302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c06df10 msgr2=0x7f7c3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/1187477302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c06df10 0x7f7c3c06e320 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0d2380 tx=0x7f7c3401ccf0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 -- 172.21.15.170:0/1187477302 shutdown_connections 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/1187477302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c06df10 0x7f7c3c06e320 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/1187477302 >> v2:172.21.15.191:3300/0 conn(0x7f7c3c0d0100 0x7f7c3c0d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 -- 172.21.15.170:0/1187477302 >> 172.21.15.170:0/1187477302 conn(0x7f7c3c0cee30 msgr2=0x7f7c3c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 -- 172.21.15.170:0/1187477302 shutdown_connections 2024-09-21T02:22:23.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.027+0000 7f7c40a5b640 1 -- 172.21.15.170:0/1187477302 wait complete. 2024-09-21T02:22:23.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 Processor -- start 2024-09-21T02:22:23.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 -- start start 2024-09-21T02:22:23.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c3c06df10 0x7f7c3c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c3c0b3e30 0x7f7c3c0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 0x7f7c3c0b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c16be00 con 0x7f7c3c0d0100 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c16bc80 con 0x7f7c3c0b3e30 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c16bf80 con 0x7f7c3c06df10 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 0x7f7c3c0b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 0x7f7c3c0b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45962/0 (socket says 172.21.15.170:45962) 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 -- 172.21.15.170:0/188466433 learned_addr learned my addr 172.21.15.170:0/188466433 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c39d9b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c3c0b3e30 0x7f7c3c0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 -- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c3c06df10 msgr2=0x7f7c3c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3a59c640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c3c06df10 0x7f7c3c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c3c06df10 0x7f7c3c14e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 -- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c3c0b3e30 msgr2=0x7f7c3c0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c3c0b3e30 0x7f7c3c0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 -- 172.21.15.170:0/188466433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c158e50 con 0x7f7c3c0d0100 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c39d9b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c3c0b3e30 0x7f7c3c0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c3ad9d640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 0x7f7c3c0b33a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7c34019900 tx=0x7f7c34005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340bc070 con 0x7f7c3c0d0100 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c34004130 con 0x7f7c3c0d0100 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c14eb80 con 0x7f7c3c0d0100 2024-09-21T02:22:23.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c340b2040 con 0x7f7c3c0d0100 2024-09-21T02:22:23.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.031+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c1512f0 con 0x7f7c3c0d0100 2024-09-21T02:22:23.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.035+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c06e320 con 0x7f7c3c0d0100 2024-09-21T02:22:23.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.035+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f7c340bf070 con 0x7f7c3c0d0100 2024-09-21T02:22:23.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.035+0000 7f7c2affd640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c1c073c50 0x7f7c1c076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.035+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5737+0+0 (secure 0 0 0) 0x7f7c3412cb70 con 0x7f7c3c0d0100 2024-09-21T02:22:23.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.035+0000 7f7c3a59c640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c1c073c50 0x7f7c1c076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.035+0000 7f7c3a59c640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c1c073c50 0x7f7c1c076160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c001570 tx=0x7f7c2c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:23.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.039+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f7c3c14eb80 con 0x7f7c3c0d0100 2024-09-21T02:22:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.211+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c3c06fa40 con 0x7f7c3c0d0100 2024-09-21T02:22:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c2affd640 1 -- 172.21.15.170:0/188466433 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+16780 (secure 0 0 0) 0x7f7c34016970 con 0x7f7c3c0d0100 2024-09-21T02:22:23.216 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:23.216 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":165,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:21.863813+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:22:19.529466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T02:22:20.819055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c1c073c50 msgr2=0x7f7c1c076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c1c073c50 0x7f7c1c076160 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c001570 tx=0x7f7c2c009040 comp rx=0 tx=0).stop 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 msgr2=0x7f7c3c0b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 0x7f7c3c0b33a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7c34019900 tx=0x7f7c34005ae0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 shutdown_connections 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7c1c073c50 0x7f7c1c076160 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c3c0d0100 0x7f7c3c0b33a0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c3c0b3e30 0x7f7c3c0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 --2- 172.21.15.170:0/188466433 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c3c06df10 0x7f7c3c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 >> 172.21.15.170:0/188466433 conn(0x7f7c3c0cee30 msgr2=0x7f7c3c156400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 shutdown_connections 2024-09-21T02:22:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.215+0000 7f7c40a5b640 1 -- 172.21.15.170:0/188466433 wait complete. 2024-09-21T02:22:23.240 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:22:23.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f705e640 1 -- 172.21.15.170:0/2788206154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36d8004810 msgr2=0x7f36d8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f705e640 1 --2- 172.21.15.170:0/2788206154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36d8004810 0x7f36d8004c20 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f36e00016f0 tx=0x7f36e001c7b0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f55fb640 1 -- 172.21.15.170:0/2788206154 reap_dead start 2024-09-21T02:22:23.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f705e640 1 -- 172.21.15.170:0/2788206154 shutdown_connections 2024-09-21T02:22:23.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f705e640 1 -- 172.21.15.170:0/2788206154 >> 172.21.15.170:0/2788206154 conn(0x7f36f00cd5d0 msgr2=0x7f36f00cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:23.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f705e640 1 -- 172.21.15.170:0/2788206154 shutdown_connections 2024-09-21T02:22:23.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.395+0000 7f36f705e640 1 -- 172.21.15.170:0/2788206154 wait complete. 2024-09-21T02:22:23.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 Processor -- start 2024-09-21T02:22:23.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- start start 2024-09-21T02:22:23.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36f0157fc0 0x7f36f00c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f01f9700 0x7f36f0152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 0x7f36f00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f36f00ceff0 con 0x7f36f00c84e0 2024-09-21T02:22:23.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36f00cee70 con 0x7f36f01f9700 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f36f00cf170 con 0x7f36f0157fc0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 0x7f36f00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 0x7f36f00c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45974/0 (socket says 172.21.15.170:45974) 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 -- 172.21.15.170:0/107604345 learned_addr learned my addr 172.21.15.170:0/107604345 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f4dfa640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36f0157fc0 0x7f36f00c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36e7fff640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f01f9700 0x7f36f0152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 -- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36f0157fc0 msgr2=0x7f36f00c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36f0157fc0 0x7f36f00c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 -- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f01f9700 msgr2=0x7f36f0152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f01f9700 0x7f36f0152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 -- 172.21.15.170:0/107604345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f00c8e30 con 0x7f36f00c84e0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f55fb640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 0x7f36f00c88f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f36f0074a50 tx=0x7f36e8009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8011910 con 0x7f36f00c84e0 2024-09-21T02:22:23.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36e8019070 con 0x7f36f00c84e0 2024-09-21T02:22:23.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8011e60 con 0x7f36f00c84e0 2024-09-21T02:22:23.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f00ca9b0 con 0x7f36f00c84e0 2024-09-21T02:22:23.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f00ca6d0 con 0x7f36f00c84e0 2024-09-21T02:22:23.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.399+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f00cf6d0 con 0x7f36f00c84e0 2024-09-21T02:22:23.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.403+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f36e801e460 con 0x7f36f00c84e0 2024-09-21T02:22:23.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.403+0000 7f36e57fa640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f36dc073c20 0x7f36dc076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:23.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.403+0000 7f36f4dfa640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f36dc073c20 0x7f36dc076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:23.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.403+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5737+0+0 (secure 0 0 0) 0x7f36e80936b0 con 0x7f36f00c84e0 2024-09-21T02:22:23.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.403+0000 7f36f4dfa640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f36dc073c20 0x7f36dc076130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36ec001d80 tx=0x7f36ec001d10 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:23.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.407+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f36f00c8e30 con 0x7f36f00c84e0 2024-09-21T02:22:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:23.623+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f36f00cfa80 con 0x7f36f00c84e0 2024-09-21T02:22:24.406 INFO:tasks.ceph.mds.c.smithi170.stderr:2024-09-21T02:22:24.403+0000 7f6a1a8ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:22:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.479+0000 7f36e57fa640 1 -- 172.21.15.170:0/107604345 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v119) ==== 118+0+0 (secure 0 0 0) 0x7f36e805fff0 con 0x7f36f00c84e0 2024-09-21T02:22:24.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f36dc073c20 msgr2=0x7f36dc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:24.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f36dc073c20 0x7f36dc076130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36ec001d80 tx=0x7f36ec001d10 comp rx=0 tx=0).stop 2024-09-21T02:22:24.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 msgr2=0x7f36f00c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:24.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 0x7f36f00c88f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f36f0074a50 tx=0x7f36e8009a40 comp rx=0 tx=0).stop 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 shutdown_connections 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f36dc073c20 0x7f36dc076130 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36f00c84e0 0x7f36f00c88f0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f01f9700 0x7f36f0152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 --2- 172.21.15.170:0/107604345 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36f0157fc0 0x7f36f00c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 >> 172.21.15.170:0/107604345 conn(0x7f36f00cd5d0 msgr2=0x7f36f00cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 shutdown_connections 2024-09-21T02:22:24.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.483+0000 7f36f705e640 1 -- 172.21.15.170:0/107604345 wait complete. 2024-09-21T02:22:24.513 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T02:22:24.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 -- 172.21.15.170:0/846056062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb790 msgr2=0x7fb5ec06fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 --2- 172.21.15.170:0/846056062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb790 0x7fb5ec06fee0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009c80 tx=0x7fb5e401cd10 comp rx=0 tx=0).stop 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 -- 172.21.15.170:0/846056062 shutdown_connections 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 --2- 172.21.15.170:0/846056062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb790 0x7fb5ec06fee0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 --2- 172.21.15.170:0/846056062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0ce780 0x7fb5ec06f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 -- 172.21.15.170:0/846056062 >> 172.21.15.170:0/846056062 conn(0x7fb5ec0d0170 msgr2=0x7fb5ec0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:24.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 -- 172.21.15.170:0/846056062 shutdown_connections 2024-09-21T02:22:24.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.675+0000 7fb5f0929640 1 -- 172.21.15.170:0/846056062 wait complete. 2024-09-21T02:22:24.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 Processor -- start 2024-09-21T02:22:24.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- start start 2024-09-21T02:22:24.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 0x7fb5ec14f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:24.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0cb790 0x7fb5ec1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:24.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ec0ce780 0x7fb5ec156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec073710 con 0x7fb5ec0cb380 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec073590 con 0x7fb5ec0ce780 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec073890 con 0x7fb5ec0cb790 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 0x7fb5ec14f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 0x7fb5ec14f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46000/0 (socket says 172.21.15.170:46000) 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 -- 172.21.15.170:0/3114842973 learned_addr learned my addr 172.21.15.170:0/3114842973 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ead9d640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ec0ce780 0x7fb5ec156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5e9d9b640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0cb790 0x7fb5ec1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 -- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0cb790 msgr2=0x7fb5ec1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0cb790 0x7fb5ec1567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 -- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ec0ce780 msgr2=0x7fb5ec156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ec0ce780 0x7fb5ec156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 -- 172.21.15.170:0/3114842973 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1508b0 con 0x7fb5ec0cb380 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ead9d640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ec0ce780 0x7fb5ec156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5e9d9b640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0cb790 0x7fb5ec1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5ea59c640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 0x7fb5ec14f780 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c3c0 tx=0x7fb5dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0052d0 con 0x7fb5ec0cb380 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5dc007020 con 0x7fb5ec0cb380 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1519e0 con 0x7fb5ec0cb380 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc007470 con 0x7fb5ec0cb380 2024-09-21T02:22:24.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec0b19c0 con 0x7fb5ec0cb380 2024-09-21T02:22:24.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.679+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec0ce070 con 0x7fb5ec0cb380 2024-09-21T02:22:24.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.683+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fb5dc0067e0 con 0x7fb5ec0cb380 2024-09-21T02:22:24.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.683+0000 7fb5daffd640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb5cc073c20 0x7fb5cc076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:24.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.683+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fb5dc092710 con 0x7fb5ec0cb380 2024-09-21T02:22:24.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.683+0000 7fb5e9d9b640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb5cc073c20 0x7fb5cc076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:24.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.683+0000 7fb5e9d9b640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb5cc073c20 0x7fb5cc076130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00058d0 tx=0x7fb5e0005cc0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:24.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.687+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb5dc05f030 con 0x7fb5ec0cb380 2024-09-21T02:22:24.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:24.879+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb5ec0b1c00 con 0x7fb5ec0cb380 2024-09-21T02:22:25.052 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:22:25.054 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:22:25.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.483+0000 7fb5daffd640 1 -- 172.21.15.170:0/3114842973 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v120) ==== 122+0+0 (secure 0 0 0) 0x7fb5dc05e930 con 0x7fb5ec0cb380 2024-09-21T02:22:25.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb5cc073c20 msgr2=0x7fb5cc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb5cc073c20 0x7fb5cc076130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb5e00058d0 tx=0x7fb5e0005cc0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 msgr2=0x7fb5ec14f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 0x7fb5ec14f780 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c3c0 tx=0x7fb5dc00c880 comp rx=0 tx=0).stop 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 shutdown_connections 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb5cc073c20 0x7fb5cc076130 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb5ec0cb790 0x7fb5ec1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5ec0ce780 0x7fb5ec156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 --2- 172.21.15.170:0/3114842973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5ec0cb380 0x7fb5ec14f780 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 >> 172.21.15.170:0/3114842973 conn(0x7fb5ec0d0170 msgr2=0x7fb5ec0ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 shutdown_connections 2024-09-21T02:22:25.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.487+0000 7fb5f0929640 1 -- 172.21.15.170:0/3114842973 wait complete. 2024-09-21T02:22:25.514 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T02:22:25.515 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 -- 172.21.15.170:0/4099508931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b80cd210 msgr2=0x7fd6b80b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 --2- 172.21.15.170:0/4099508931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b80cd210 0x7fd6b80b3350 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80b3a10 tx=0x7fd6b001cda0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 -- 172.21.15.170:0/4099508931 shutdown_connections 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 --2- 172.21.15.170:0/4099508931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b80cd210 0x7fd6b80b3350 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 --2- 172.21.15.170:0/4099508931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6b80b5410 0x7fd6b806fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 -- 172.21.15.170:0/4099508931 >> 172.21.15.170:0/4099508931 conn(0x7fd6b80b1c40 msgr2=0x7fd6b80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 -- 172.21.15.170:0/4099508931 shutdown_connections 2024-09-21T02:22:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.663+0000 7fd6be21e640 1 -- 172.21.15.170:0/4099508931 wait complete. 2024-09-21T02:22:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 Processor -- start 2024-09-21T02:22:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- start start 2024-09-21T02:22:25.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6b80b2f40 0x7fd6b8145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80b5410 0x7fd6b81532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 0x7fd6b809cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8169fd0 con 0x7fd6b8153810 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8169e50 con 0x7fd6b80b5410 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b816a150 con 0x7fd6b80b2f40 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 0x7fd6b809cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 0x7fd6b809cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43996/0 (socket says 172.21.15.170:43996) 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 -- 172.21.15.170:0/2969659097 learned_addr learned my addr 172.21.15.170:0/2969659097 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b77fe640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6b80b2f40 0x7fd6b8145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b6ffd640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80b5410 0x7fd6b81532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 -- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6b80b2f40 msgr2=0x7fd6b8145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6b80b2f40 0x7fd6b8145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 -- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80b5410 msgr2=0x7fd6b81532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80b5410 0x7fd6b81532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 -- 172.21.15.170:0/2969659097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b80c7180 con 0x7fd6b8153810 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b6ffd640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80b5410 0x7fd6b81532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6b7fff640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 0x7fd6b809cda0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009bd0 tx=0x7fd6b00a5b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00bb070 con 0x7fd6b8153810 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6b0004e00 con 0x7fd6b8153810 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00b2040 con 0x7fd6b8153810 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b80ca030 con 0x7fd6b8153810 2024-09-21T02:22:25.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b809c5a0 con 0x7fd6b8153810 2024-09-21T02:22:25.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.667+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b80c9da0 con 0x7fd6b8153810 2024-09-21T02:22:25.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.671+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fd6b0004320 con 0x7fd6b8153810 2024-09-21T02:22:25.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.671+0000 7fd69ffff640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6a4073c20 0x7fd6a4076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:25.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.671+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fd6b012d3e0 con 0x7fd6b8153810 2024-09-21T02:22:25.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.671+0000 7fd6b77fe640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6a4073c20 0x7fd6a4076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:25.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.671+0000 7fd6b77fe640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6a4073c20 0x7fd6a4076130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800b9a0 tx=0x7fd6a8005c30 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:25.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.675+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd6b00c4020 con 0x7fd6b8153810 2024-09-21T02:22:25.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.879+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6b806fa30 con 0x7fd6b8153810 2024-09-21T02:22:25.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.879+0000 7fd69ffff640 1 -- 172.21.15.170:0/2969659097 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5245 (secure 0 0 0) 0x7fd6b0016990 con 0x7fd6b8153810 2024-09-21T02:22:25.881 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:25.882 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":120,"btime":"2024-09-21T02:22:25:485673+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:22:21.863736+0000","modified":"2024-09-21T02:22:24.886454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9303":{"gid":9303,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1712756729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1712756729},{"type":"v1","addr":"172.21.15.170:6835","nonce":1712756729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9303,"qdb_cluster":[9303]},"id":13}]} 2024-09-21T02:22:25.882 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 120 2024-09-21T02:22:25.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6a4073c20 msgr2=0x7fd6a4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6a4073c20 0x7fd6a4076130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800b9a0 tx=0x7fd6a8005c30 comp rx=0 tx=0).stop 2024-09-21T02:22:25.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 msgr2=0x7fd6b809cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 0x7fd6b809cda0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009bd0 tx=0x7fd6b00a5b00 comp rx=0 tx=0).stop 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 shutdown_connections 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd6a4073c20 0x7fd6a4076130 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b8153810 0x7fd6b809cda0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80b5410 0x7fd6b81532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 --2- 172.21.15.170:0/2969659097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6b80b2f40 0x7fd6b8145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 >> 172.21.15.170:0/2969659097 conn(0x7fd6b80b1c40 msgr2=0x7fd6b80c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 shutdown_connections 2024-09-21T02:22:25.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:25.883+0000 7fd6be21e640 1 -- 172.21.15.170:0/2969659097 wait complete. 2024-09-21T02:22:25.902 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:26.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 -- 172.21.15.170:0/2274673931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0074680 msgr2=0x7fcdf015ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2274673931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0074680 0x7fcdf015ae90 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009900 tx=0x7fcde001c820 comp rx=0 tx=0).stop 2024-09-21T02:22:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 -- 172.21.15.170:0/2274673931 shutdown_connections 2024-09-21T02:22:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2274673931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0074680 0x7fcdf015ae90 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 -- 172.21.15.170:0/2274673931 >> 172.21.15.170:0/2274673931 conn(0x7fcdf00d1240 msgr2=0x7fcdf00d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 -- 172.21.15.170:0/2274673931 shutdown_connections 2024-09-21T02:22:26.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.039+0000 7fcdf576d640 1 -- 172.21.15.170:0/2274673931 wait complete. 2024-09-21T02:22:26.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 Processor -- start 2024-09-21T02:22:26.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 -- start start 2024-09-21T02:22:26.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 0x7fcdf0155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdf00cd5c0 0x7fcdf009e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdf015c680 0x7fcdf015d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf0169c80 con 0x7fcdf0071670 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf006e200 con 0x7fcdf015c680 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf0169e00 con 0x7fcdf00cd5c0 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 0x7fcdf0155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 0x7fcdf0155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44006/0 (socket says 172.21.15.170:44006) 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 -- 172.21.15.170:0/2550819618 learned_addr learned my addr 172.21.15.170:0/2550819618 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdee7fc640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdf00cd5c0 0x7fcdf009e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 -- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdf00cd5c0 msgr2=0x7fcdf009e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdf00cd5c0 0x7fcdf009e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 -- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdf015c680 msgr2=0x7fcdf015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdef7fe640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdf015c680 0x7fcdf015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdf015c680 0x7fcdf015d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 -- 172.21.15.170:0/2550819618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf00cb070 con 0x7fcdf0071670 2024-09-21T02:22:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdeeffd640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 0x7fcdf0155910 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c3c0 tx=0x7fcdd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8017070 con 0x7fcdf0071670 2024-09-21T02:22:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd80053c0 con 0x7fcdf0071670 2024-09-21T02:22:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf00cb350 con 0x7fcdf0071670 2024-09-21T02:22:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8003860 con 0x7fcdf0071670 2024-09-21T02:22:26.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.043+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf009ed00 con 0x7fcdf0071670 2024-09-21T02:22:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.047+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf0073f70 con 0x7fcdf0071670 2024-09-21T02:22:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.047+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fcdd801a070 con 0x7fcdf0071670 2024-09-21T02:22:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.047+0000 7fcde77fe640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcdd0073c20 0x7fcdd0076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.047+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fcdd8092100 con 0x7fcdf0071670 2024-09-21T02:22:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.047+0000 7fcdee7fc640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcdd0073c20 0x7fcdd0076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.047+0000 7fcdee7fc640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcdd0073c20 0x7fcdd0076130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcde00af000 tx=0x7fcde001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:26.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.051+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fcdd805e9b0 con 0x7fcdf0071670 2024-09-21T02:22:26.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.223+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdf00cb520 con 0x7fcdf0071670 2024-09-21T02:22:26.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcde77fe640 1 -- 172.21.15.170:0/2550819618 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16767 (secure 0 0 0) 0x7fcdd805e2b0 con 0x7fcdf0071670 2024-09-21T02:22:26.227 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:26.228 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":166,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:23.429025+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:22:19.529466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T02:22:20.819055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcdd0073c20 msgr2=0x7fcdd0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcdd0073c20 0x7fcdd0076130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcde00af000 tx=0x7fcde001b060 comp rx=0 tx=0).stop 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 msgr2=0x7fcdf0155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 0x7fcdf0155910 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c3c0 tx=0x7fcdd800c880 comp rx=0 tx=0).stop 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 shutdown_connections 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fcdd0073c20 0x7fcdd0076130 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdf00cd5c0 0x7fcdf009e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdf015c680 0x7fcdf015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 --2- 172.21.15.170:0/2550819618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf0071670 0x7fcdf0155910 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 >> 172.21.15.170:0/2550819618 conn(0x7fcdf00d1240 msgr2=0x7fcdf00d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 shutdown_connections 2024-09-21T02:22:26.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.227+0000 7fcdf576d640 1 -- 172.21.15.170:0/2550819618 wait complete. 2024-09-21T02:22:26.250 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:22:26.251 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 -- 172.21.15.170:0/4224785969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 msgr2=0x7fafd80ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/4224785969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd80ce8d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fafd80cdc00 tx=0x7fafc001c6e0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 -- 172.21.15.170:0/4224785969 shutdown_connections 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/4224785969 >> v2:172.21.15.191:3300/0 conn(0x7fafd80cadf0 0x7fafd80cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/4224785969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd80ce8d0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 -- 172.21.15.170:0/4224785969 >> 172.21.15.170:0/4224785969 conn(0x7fafd80b1c40 msgr2=0x7fafd80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 -- 172.21.15.170:0/4224785969 shutdown_connections 2024-09-21T02:22:26.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.403+0000 7fafdd8e9640 1 -- 172.21.15.170:0/4224785969 wait complete. 2024-09-21T02:22:26.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 Processor -- start 2024-09-21T02:22:26.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 -- start start 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80cadf0 0x7fafd815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd814b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd815f990 0x7fafd8165600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8169100 con 0x7fafd80ce4c0 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8074f00 con 0x7fafd80cadf0 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8169280 con 0x7fafd815f990 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd814b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd814b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44016/0 (socket says 172.21.15.170:44016) 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 -- 172.21.15.170:0/214075117 learned_addr learned my addr 172.21.15.170:0/214075117 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafd6ffd640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80cadf0 0x7fafd815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafd77fe640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd815f990 0x7fafd8165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 -- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd815f990 msgr2=0x7fafd8165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd815f990 0x7fafd8165600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 -- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80cadf0 msgr2=0x7fafd815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80cadf0 0x7fafd815d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 -- 172.21.15.170:0/214075117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd809b9d0 con 0x7fafd80ce4c0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafcffff640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd814b440 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fafc001ad90 tx=0x7fafc00a8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00042c0 con 0x7fafd80ce4c0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafc0005030 con 0x7fafd80ce4c0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd809ac10 con 0x7fafd80ce4c0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0005480 con 0x7fafd80ce4c0 2024-09-21T02:22:26.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.407+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd809b1d0 con 0x7fafd80ce4c0 2024-09-21T02:22:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.411+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd814bf10 con 0x7fafd80ce4c0 2024-09-21T02:22:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.411+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fafc00046d0 con 0x7fafd80ce4c0 2024-09-21T02:22:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.411+0000 7fafd4ff9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafbc073c50 0x7fafbc076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.411+0000 7fafd6ffd640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafbc073c50 0x7fafbc076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.411+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fafc012d760 con 0x7fafd80ce4c0 2024-09-21T02:22:26.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.411+0000 7fafd6ffd640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafbc073c50 0x7fafbc076160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b810 tx=0x7fafc8008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.415+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fafc00bf060 con 0x7fafd80ce4c0 2024-09-21T02:22:26.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.587+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafd809a9c0 con 0x7fafbc073c50 2024-09-21T02:22:26.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.595+0000 7fafd4ff9640 1 -- 172.21.15.170:0/214075117 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7fafd809a9c0 con 0x7fafbc073c50 2024-09-21T02:22:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:26.599 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:22:26.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafbc073c50 msgr2=0x7fafbc076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafbc073c50 0x7fafbc076160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b810 tx=0x7fafc8008040 comp rx=0 tx=0).stop 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 msgr2=0x7fafd814b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd814b440 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fafc001ad90 tx=0x7fafc00a8620 comp rx=0 tx=0).stop 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 shutdown_connections 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fafbc073c50 0x7fafbc076160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd815f990 0x7fafd8165600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd80ce4c0 0x7fafd814b440 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 --2- 172.21.15.170:0/214075117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd80cadf0 0x7fafd815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 >> 172.21.15.170:0/214075117 conn(0x7fafd80b1c40 msgr2=0x7fafd806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 shutdown_connections 2024-09-21T02:22:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.599+0000 7fafdd8e9640 1 -- 172.21.15.170:0/214075117 wait complete. 2024-09-21T02:22:26.621 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-21T02:22:24.942337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304520,"kb_used_data":17544,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754670200,"statfs":{"total":773094113280,"available":772782284800,"internally_reserved":0,"allocated":17965056,"data_stored":8577660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":293654065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.527163"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440053+0000","last_change":"2024-09-21T02:22:22.457835+0000","last_active":"2024-09-21T02:22:23.440053+0000","last_peered":"2024-09-21T02:22:23.440053+0000","last_clean":"2024-09-21T02:22:23.440053+0000","last_became_active":"2024-09-21T02:22:22.456937+0000","last_became_peered":"2024-09-21T02:22:22.456937+0000","last_unstale":"2024-09-21T02:22:23.440053+0000","last_undegraded":"2024-09-21T02:22:23.440053+0000","last_fullsized":"2024-09-21T02:22:23.440053+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:36.199399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433567+0000","last_change":"2024-09-21T02:22:22.461326+0000","last_active":"2024-09-21T02:22:23.433567+0000","last_peered":"2024-09-21T02:22:23.433567+0000","last_clean":"2024-09-21T02:22:23.433567+0000","last_became_active":"2024-09-21T02:22:22.460591+0000","last_became_peered":"2024-09-21T02:22:22.460591+0000","last_unstale":"2024-09-21T02:22:23.433567+0000","last_undegraded":"2024-09-21T02:22:23.433567+0000","last_fullsized":"2024-09-21T02:22:23.433567+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:42.420027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490469+0000","last_change":"2024-09-21T02:22:22.504027+0000","last_active":"2024-09-21T02:22:23.490469+0000","last_peered":"2024-09-21T02:22:23.490469+0000","last_clean":"2024-09-21T02:22:23.490469+0000","last_became_active":"2024-09-21T02:22:22.503340+0000","last_became_peered":"2024-09-21T02:22:22.503340+0000","last_unstale":"2024-09-21T02:22:23.490469+0000","last_undegraded":"2024-09-21T02:22:23.490469+0000","last_fullsized":"2024-09-21T02:22:23.490469+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:52.239116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489067+0000","last_change":"2024-09-21T02:22:22.482951+0000","last_active":"2024-09-21T02:22:23.489067+0000","last_peered":"2024-09-21T02:22:23.489067+0000","last_clean":"2024-09-21T02:22:23.489067+0000","last_became_active":"2024-09-21T02:22:22.474858+0000","last_became_peered":"2024-09-21T02:22:22.474858+0000","last_unstale":"2024-09-21T02:22:23.489067+0000","last_undegraded":"2024-09-21T02:22:23.489067+0000","last_fullsized":"2024-09-21T02:22:23.489067+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.079204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491311+0000","last_change":"2024-09-21T02:22:22.503427+0000","last_active":"2024-09-21T02:22:23.491311+0000","last_peered":"2024-09-21T02:22:23.491311+0000","last_clean":"2024-09-21T02:22:23.491311+0000","last_became_active":"2024-09-21T02:22:22.501055+0000","last_became_peered":"2024-09-21T02:22:22.501055+0000","last_unstale":"2024-09-21T02:22:23.491311+0000","last_undegraded":"2024-09-21T02:22:23.491311+0000","last_fullsized":"2024-09-21T02:22:23.491311+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:47.372392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439234+0000","last_change":"2024-09-21T02:22:22.450895+0000","last_active":"2024-09-21T02:22:23.439234+0000","last_peered":"2024-09-21T02:22:23.439234+0000","last_clean":"2024-09-21T02:22:23.439234+0000","last_became_active":"2024-09-21T02:22:22.449130+0000","last_became_peered":"2024-09-21T02:22:22.449130+0000","last_unstale":"2024-09-21T02:22:23.439234+0000","last_undegraded":"2024-09-21T02:22:23.439234+0000","last_fullsized":"2024-09-21T02:22:23.439234+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:44.203710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436729+0000","last_change":"2024-09-21T02:22:22.466509+0000","last_active":"2024-09-21T02:22:23.436729+0000","last_peered":"2024-09-21T02:22:23.436729+0000","last_clean":"2024-09-21T02:22:23.436729+0000","last_became_active":"2024-09-21T02:22:22.464105+0000","last_became_peered":"2024-09-21T02:22:22.464105+0000","last_unstale":"2024-09-21T02:22:23.436729+0000","last_undegraded":"2024-09-21T02:22:23.436729+0000","last_fullsized":"2024-09-21T02:22:23.436729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:28.110932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436758+0000","last_change":"2024-09-21T02:22:22.476525+0000","last_active":"2024-09-21T02:22:23.436758+0000","last_peered":"2024-09-21T02:22:23.436758+0000","last_clean":"2024-09-21T02:22:23.436758+0000","last_became_active":"2024-09-21T02:22:22.475747+0000","last_became_peered":"2024-09-21T02:22:22.475747+0000","last_unstale":"2024-09-21T02:22:23.436758+0000","last_undegraded":"2024-09-21T02:22:23.436758+0000","last_fullsized":"2024-09-21T02:22:23.436758+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:56.637284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.482148+0000","last_change":"2024-09-21T02:22:22.469549+0000","last_active":"2024-09-21T02:22:23.482148+0000","last_peered":"2024-09-21T02:22:23.482148+0000","last_clean":"2024-09-21T02:22:23.482148+0000","last_became_active":"2024-09-21T02:22:22.463504+0000","last_became_peered":"2024-09-21T02:22:22.463504+0000","last_unstale":"2024-09-21T02:22:23.482148+0000","last_undegraded":"2024-09-21T02:22:23.482148+0000","last_fullsized":"2024-09-21T02:22:23.482148+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.073135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492640+0000","last_change":"2024-09-21T02:22:22.508739+0000","last_active":"2024-09-21T02:22:23.492640+0000","last_peered":"2024-09-21T02:22:23.492640+0000","last_clean":"2024-09-21T02:22:23.492640+0000","last_became_active":"2024-09-21T02:22:22.508106+0000","last_became_peered":"2024-09-21T02:22:22.508106+0000","last_unstale":"2024-09-21T02:22:23.492640+0000","last_undegraded":"2024-09-21T02:22:23.492640+0000","last_fullsized":"2024-09-21T02:22:23.492640+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:17.335335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440972+0000","last_change":"2024-09-21T02:22:22.469027+0000","last_active":"2024-09-21T02:22:23.440972+0000","last_peered":"2024-09-21T02:22:23.440972+0000","last_clean":"2024-09-21T02:22:23.440972+0000","last_became_active":"2024-09-21T02:22:22.465839+0000","last_became_peered":"2024-09-21T02:22:22.465839+0000","last_unstale":"2024-09-21T02:22:23.440972+0000","last_undegraded":"2024-09-21T02:22:23.440972+0000","last_fullsized":"2024-09-21T02:22:23.440972+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:58.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486976+0000","last_change":"2024-09-21T02:22:22.445658+0000","last_active":"2024-09-21T02:22:23.486976+0000","last_peered":"2024-09-21T02:22:23.486976+0000","last_clean":"2024-09-21T02:22:23.486976+0000","last_became_active":"2024-09-21T02:22:22.444980+0000","last_became_peered":"2024-09-21T02:22:22.444980+0000","last_unstale":"2024-09-21T02:22:23.486976+0000","last_undegraded":"2024-09-21T02:22:23.486976+0000","last_fullsized":"2024-09-21T02:22:23.486976+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:15.329863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488798+0000","last_change":"2024-09-21T02:22:22.477438+0000","last_active":"2024-09-21T02:22:23.488798+0000","last_peered":"2024-09-21T02:22:23.488798+0000","last_clean":"2024-09-21T02:22:23.488798+0000","last_became_active":"2024-09-21T02:22:22.477113+0000","last_became_peered":"2024-09-21T02:22:22.477113+0000","last_unstale":"2024-09-21T02:22:23.488798+0000","last_undegraded":"2024-09-21T02:22:23.488798+0000","last_fullsized":"2024-09-21T02:22:23.488798+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:40.067516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489720+0000","last_change":"2024-09-21T02:22:22.471371+0000","last_active":"2024-09-21T02:22:23.489720+0000","last_peered":"2024-09-21T02:22:23.489720+0000","last_clean":"2024-09-21T02:22:23.489720+0000","last_became_active":"2024-09-21T02:22:22.470542+0000","last_became_peered":"2024-09-21T02:22:22.470542+0000","last_unstale":"2024-09-21T02:22:23.489720+0000","last_undegraded":"2024-09-21T02:22:23.489720+0000","last_fullsized":"2024-09-21T02:22:23.489720+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:17.581268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437523+0000","last_change":"2024-09-21T02:22:22.448947+0000","last_active":"2024-09-21T02:22:23.437523+0000","last_peered":"2024-09-21T02:22:23.437523+0000","last_clean":"2024-09-21T02:22:23.437523+0000","last_became_active":"2024-09-21T02:22:22.448217+0000","last_became_peered":"2024-09-21T02:22:22.448217+0000","last_unstale":"2024-09-21T02:22:23.437523+0000","last_undegraded":"2024-09-21T02:22:23.437523+0000","last_fullsized":"2024-09-21T02:22:23.437523+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:48.407997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436762+0000","last_change":"2024-09-21T02:22:22.468067+0000","last_active":"2024-09-21T02:22:23.436762+0000","last_peered":"2024-09-21T02:22:23.436762+0000","last_clean":"2024-09-21T02:22:23.436762+0000","last_became_active":"2024-09-21T02:22:22.467230+0000","last_became_peered":"2024-09-21T02:22:22.467230+0000","last_unstale":"2024-09-21T02:22:23.436762+0000","last_undegraded":"2024-09-21T02:22:23.436762+0000","last_fullsized":"2024-09-21T02:22:23.436762+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:45.339630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440732+0000","last_change":"2024-09-21T02:22:22.465773+0000","last_active":"2024-09-21T02:22:23.440732+0000","last_peered":"2024-09-21T02:22:23.440732+0000","last_clean":"2024-09-21T02:22:23.440732+0000","last_became_active":"2024-09-21T02:22:22.463773+0000","last_became_peered":"2024-09-21T02:22:22.463773+0000","last_unstale":"2024-09-21T02:22:23.440732+0000","last_undegraded":"2024-09-21T02:22:23.440732+0000","last_fullsized":"2024-09-21T02:22:23.440732+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:31.096497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492632+0000","last_change":"2024-09-21T02:22:22.501647+0000","last_active":"2024-09-21T02:22:23.492632+0000","last_peered":"2024-09-21T02:22:23.492632+0000","last_clean":"2024-09-21T02:22:23.492632+0000","last_became_active":"2024-09-21T02:22:22.500886+0000","last_became_peered":"2024-09-21T02:22:22.500886+0000","last_unstale":"2024-09-21T02:22:23.492632+0000","last_undegraded":"2024-09-21T02:22:23.492632+0000","last_fullsized":"2024-09-21T02:22:23.492632+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:50.191975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492512+0000","last_change":"2024-09-21T02:22:22.507774+0000","last_active":"2024-09-21T02:22:23.492512+0000","last_peered":"2024-09-21T02:22:23.492512+0000","last_clean":"2024-09-21T02:22:23.492512+0000","last_became_active":"2024-09-21T02:22:22.504212+0000","last_became_peered":"2024-09-21T02:22:22.504212+0000","last_unstale":"2024-09-21T02:22:23.492512+0000","last_undegraded":"2024-09-21T02:22:23.492512+0000","last_fullsized":"2024-09-21T02:22:23.492512+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:57.326942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490419+0000","last_change":"2024-09-21T02:22:22.508566+0000","last_active":"2024-09-21T02:22:23.490419+0000","last_peered":"2024-09-21T02:22:23.490419+0000","last_clean":"2024-09-21T02:22:23.490419+0000","last_became_active":"2024-09-21T02:22:22.507845+0000","last_became_peered":"2024-09-21T02:22:22.507845+0000","last_unstale":"2024-09-21T02:22:23.490419+0000","last_undegraded":"2024-09-21T02:22:23.490419+0000","last_fullsized":"2024-09-21T02:22:23.490419+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:26.349734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486420+0000","last_change":"2024-09-21T02:22:22.467579+0000","last_active":"2024-09-21T02:22:23.486420+0000","last_peered":"2024-09-21T02:22:23.486420+0000","last_clean":"2024-09-21T02:22:23.486420+0000","last_became_active":"2024-09-21T02:22:22.466740+0000","last_became_peered":"2024-09-21T02:22:22.466740+0000","last_unstale":"2024-09-21T02:22:23.486420+0000","last_undegraded":"2024-09-21T02:22:23.486420+0000","last_fullsized":"2024-09-21T02:22:23.486420+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:31.981683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433382+0000","last_change":"2024-09-21T02:22:22.496861+0000","last_active":"2024-09-21T02:22:23.433382+0000","last_peered":"2024-09-21T02:22:23.433382+0000","last_clean":"2024-09-21T02:22:23.433382+0000","last_became_active":"2024-09-21T02:22:22.496456+0000","last_became_peered":"2024-09-21T02:22:22.496456+0000","last_unstale":"2024-09-21T02:22:23.433382+0000","last_undegraded":"2024-09-21T02:22:23.433382+0000","last_fullsized":"2024-09-21T02:22:23.433382+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:38.888252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488339+0000","last_change":"2024-09-21T02:22:22.474716+0000","last_active":"2024-09-21T02:22:23.488339+0000","last_peered":"2024-09-21T02:22:23.488339+0000","last_clean":"2024-09-21T02:22:23.488339+0000","last_became_active":"2024-09-21T02:22:22.460755+0000","last_became_peered":"2024-09-21T02:22:22.460755+0000","last_unstale":"2024-09-21T02:22:23.488339+0000","last_undegraded":"2024-09-21T02:22:23.488339+0000","last_fullsized":"2024-09-21T02:22:23.488339+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:36.882040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.482225+0000","last_change":"2024-09-21T02:22:22.468547+0000","last_active":"2024-09-21T02:22:23.482225+0000","last_peered":"2024-09-21T02:22:23.482225+0000","last_clean":"2024-09-21T02:22:23.482225+0000","last_became_active":"2024-09-21T02:22:22.463619+0000","last_became_peered":"2024-09-21T02:22:22.463619+0000","last_unstale":"2024-09-21T02:22:23.482225+0000","last_undegraded":"2024-09-21T02:22:23.482225+0000","last_fullsized":"2024-09-21T02:22:23.482225+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:57.229241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488908+0000","last_change":"2024-09-21T02:22:22.484153+0000","last_active":"2024-09-21T02:22:23.488908+0000","last_peered":"2024-09-21T02:22:23.488908+0000","last_clean":"2024-09-21T02:22:23.488908+0000","last_became_active":"2024-09-21T02:22:22.483794+0000","last_became_peered":"2024-09-21T02:22:22.483794+0000","last_unstale":"2024-09-21T02:22:23.488908+0000","last_undegraded":"2024-09-21T02:22:23.488908+0000","last_fullsized":"2024-09-21T02:22:23.488908+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:24.330388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438579+0000","last_change":"2024-09-21T02:22:22.454233+0000","last_active":"2024-09-21T02:22:23.438579+0000","last_peered":"2024-09-21T02:22:23.438579+0000","last_clean":"2024-09-21T02:22:23.438579+0000","last_became_active":"2024-09-21T02:22:22.452979+0000","last_became_peered":"2024-09-21T02:22:22.452979+0000","last_unstale":"2024-09-21T02:22:23.438579+0000","last_undegraded":"2024-09-21T02:22:23.438579+0000","last_fullsized":"2024-09-21T02:22:23.438579+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:35.243629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486719+0000","last_change":"2024-09-21T02:22:22.466506+0000","last_active":"2024-09-21T02:22:23.486719+0000","last_peered":"2024-09-21T02:22:23.486719+0000","last_clean":"2024-09-21T02:22:23.486719+0000","last_became_active":"2024-09-21T02:22:22.466159+0000","last_became_peered":"2024-09-21T02:22:22.466159+0000","last_unstale":"2024-09-21T02:22:23.486719+0000","last_undegraded":"2024-09-21T02:22:23.486719+0000","last_fullsized":"2024-09-21T02:22:23.486719+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:48.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438194+0000","last_change":"2024-09-21T02:22:22.462364+0000","last_active":"2024-09-21T02:22:23.438194+0000","last_peered":"2024-09-21T02:22:23.438194+0000","last_clean":"2024-09-21T02:22:23.438194+0000","last_became_active":"2024-09-21T02:22:22.461478+0000","last_became_peered":"2024-09-21T02:22:22.461478+0000","last_unstale":"2024-09-21T02:22:23.438194+0000","last_undegraded":"2024-09-21T02:22:23.438194+0000","last_fullsized":"2024-09-21T02:22:23.438194+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:44.914958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433490+0000","last_change":"2024-09-21T02:22:22.453604+0000","last_active":"2024-09-21T02:22:23.433490+0000","last_peered":"2024-09-21T02:22:23.433490+0000","last_clean":"2024-09-21T02:22:23.433490+0000","last_became_active":"2024-09-21T02:22:22.453307+0000","last_became_peered":"2024-09-21T02:22:22.453307+0000","last_unstale":"2024-09-21T02:22:23.433490+0000","last_undegraded":"2024-09-21T02:22:23.433490+0000","last_fullsized":"2024-09-21T02:22:23.433490+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:43.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488487+0000","last_change":"2024-09-21T02:22:22.484753+0000","last_active":"2024-09-21T02:22:23.488487+0000","last_peered":"2024-09-21T02:22:23.488487+0000","last_clean":"2024-09-21T02:22:23.488487+0000","last_became_active":"2024-09-21T02:22:22.484443+0000","last_became_peered":"2024-09-21T02:22:22.484443+0000","last_unstale":"2024-09-21T02:22:23.488487+0000","last_undegraded":"2024-09-21T02:22:23.488487+0000","last_fullsized":"2024-09-21T02:22:23.488487+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:30.654175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489181+0000","last_change":"2024-09-21T02:22:22.496770+0000","last_active":"2024-09-21T02:22:23.489181+0000","last_peered":"2024-09-21T02:22:23.489181+0000","last_clean":"2024-09-21T02:22:23.489181+0000","last_became_active":"2024-09-21T02:22:22.495967+0000","last_became_peered":"2024-09-21T02:22:22.495967+0000","last_unstale":"2024-09-21T02:22:23.489181+0000","last_undegraded":"2024-09-21T02:22:23.489181+0000","last_fullsized":"2024-09-21T02:22:23.489181+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:21.738039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488586+0000","last_change":"2024-09-21T02:22:22.479588+0000","last_active":"2024-09-21T02:22:23.488586+0000","last_peered":"2024-09-21T02:22:23.488586+0000","last_clean":"2024-09-21T02:22:23.488586+0000","last_became_active":"2024-09-21T02:22:22.478636+0000","last_became_peered":"2024-09-21T02:22:22.478636+0000","last_unstale":"2024-09-21T02:22:23.488586+0000","last_undegraded":"2024-09-21T02:22:23.488586+0000","last_fullsized":"2024-09-21T02:22:23.488586+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.247218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486768+0000","last_change":"2024-09-21T02:22:22.468975+0000","last_active":"2024-09-21T02:22:23.486768+0000","last_peered":"2024-09-21T02:22:23.486768+0000","last_clean":"2024-09-21T02:22:23.486768+0000","last_became_active":"2024-09-21T02:22:22.468642+0000","last_became_peered":"2024-09-21T02:22:22.468642+0000","last_unstale":"2024-09-21T02:22:23.486768+0000","last_undegraded":"2024-09-21T02:22:23.486768+0000","last_fullsized":"2024-09-21T02:22:23.486768+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:32.303310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"165'4","reported_seq":19,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486308+0000","last_change":"2024-09-21T02:22:21.481003+0000","last_active":"2024-09-21T02:22:23.486308+0000","last_peered":"2024-09-21T02:22:23.486308+0000","last_clean":"2024-09-21T02:22:23.486308+0000","last_became_active":"2024-09-21T02:22:21.480530+0000","last_became_peered":"2024-09-21T02:22:21.480530+0000","last_unstale":"2024-09-21T02:22:23.486308+0000","last_undegraded":"2024-09-21T02:22:23.486308+0000","last_fullsized":"2024-09-21T02:22:23.486308+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:21.830021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488690+0000","last_change":"2024-09-21T02:22:22.475603+0000","last_active":"2024-09-21T02:22:23.488690+0000","last_peered":"2024-09-21T02:22:23.488690+0000","last_clean":"2024-09-21T02:22:23.488690+0000","last_became_active":"2024-09-21T02:22:22.475158+0000","last_became_peered":"2024-09-21T02:22:22.475158+0000","last_unstale":"2024-09-21T02:22:23.488690+0000","last_undegraded":"2024-09-21T02:22:23.488690+0000","last_fullsized":"2024-09-21T02:22:23.488690+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:18.517560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433538+0000","last_change":"2024-09-21T02:22:21.450012+0000","last_active":"2024-09-21T02:22:23.433538+0000","last_peered":"2024-09-21T02:22:23.433538+0000","last_clean":"2024-09-21T02:22:23.433538+0000","last_became_active":"2024-09-21T02:22:21.449316+0000","last_became_peered":"2024-09-21T02:22:21.449316+0000","last_unstale":"2024-09-21T02:22:23.433538+0000","last_undegraded":"2024-09-21T02:22:23.433538+0000","last_fullsized":"2024-09-21T02:22:23.433538+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:19.975657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490364+0000","last_change":"2024-09-21T02:22:22.501226+0000","last_active":"2024-09-21T02:22:23.490364+0000","last_peered":"2024-09-21T02:22:23.490364+0000","last_clean":"2024-09-21T02:22:23.490364+0000","last_became_active":"2024-09-21T02:22:22.498228+0000","last_became_peered":"2024-09-21T02:22:22.498228+0000","last_unstale":"2024-09-21T02:22:23.490364+0000","last_undegraded":"2024-09-21T02:22:23.490364+0000","last_fullsized":"2024-09-21T02:22:23.490364+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:09.652755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440551+0000","last_change":"2024-09-21T02:22:21.470893+0000","last_active":"2024-09-21T02:22:23.440551+0000","last_peered":"2024-09-21T02:22:23.440551+0000","last_clean":"2024-09-21T02:22:23.440551+0000","last_became_active":"2024-09-21T02:22:21.470125+0000","last_became_peered":"2024-09-21T02:22:21.470125+0000","last_unstale":"2024-09-21T02:22:23.440551+0000","last_undegraded":"2024-09-21T02:22:23.440551+0000","last_fullsized":"2024-09-21T02:22:23.440551+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:09.343888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492556+0000","last_change":"2024-09-21T02:22:22.504645+0000","last_active":"2024-09-21T02:22:23.492556+0000","last_peered":"2024-09-21T02:22:23.492556+0000","last_clean":"2024-09-21T02:22:23.492556+0000","last_became_active":"2024-09-21T02:22:22.503399+0000","last_became_peered":"2024-09-21T02:22:22.503399+0000","last_unstale":"2024-09-21T02:22:23.492556+0000","last_undegraded":"2024-09-21T02:22:23.492556+0000","last_fullsized":"2024-09-21T02:22:23.492556+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:09.331075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489280+0000","last_change":"2024-09-21T02:22:21.445985+0000","last_active":"2024-09-21T02:22:23.489280+0000","last_peered":"2024-09-21T02:22:23.489280+0000","last_clean":"2024-09-21T02:22:23.489280+0000","last_became_active":"2024-09-21T02:22:21.445023+0000","last_became_peered":"2024-09-21T02:22:21.445023+0000","last_unstale":"2024-09-21T02:22:23.489280+0000","last_undegraded":"2024-09-21T02:22:23.489280+0000","last_fullsized":"2024-09-21T02:22:23.489280+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.025700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488811+0000","last_change":"2024-09-21T02:22:22.496514+0000","last_active":"2024-09-21T02:22:23.488811+0000","last_peered":"2024-09-21T02:22:23.488811+0000","last_clean":"2024-09-21T02:22:23.488811+0000","last_became_active":"2024-09-21T02:22:22.496060+0000","last_became_peered":"2024-09-21T02:22:22.496060+0000","last_unstale":"2024-09-21T02:22:23.488811+0000","last_undegraded":"2024-09-21T02:22:23.488811+0000","last_fullsized":"2024-09-21T02:22:23.488811+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:49.474503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439631+0000","last_change":"2024-09-21T02:22:21.479742+0000","last_active":"2024-09-21T02:22:23.439631+0000","last_peered":"2024-09-21T02:22:23.439631+0000","last_clean":"2024-09-21T02:22:23.439631+0000","last_became_active":"2024-09-21T02:22:21.478735+0000","last_became_peered":"2024-09-21T02:22:21.478735+0000","last_unstale":"2024-09-21T02:22:23.439631+0000","last_undegraded":"2024-09-21T02:22:23.439631+0000","last_fullsized":"2024-09-21T02:22:23.439631+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:08.648916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437032+0000","last_change":"2024-09-21T02:22:22.460708+0000","last_active":"2024-09-21T02:22:23.437032+0000","last_peered":"2024-09-21T02:22:23.437032+0000","last_clean":"2024-09-21T02:22:23.437032+0000","last_became_active":"2024-09-21T02:22:22.459772+0000","last_became_peered":"2024-09-21T02:22:22.459772+0000","last_unstale":"2024-09-21T02:22:23.437032+0000","last_undegraded":"2024-09-21T02:22:23.437032+0000","last_fullsized":"2024-09-21T02:22:23.437032+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:06.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438403+0000","last_change":"2024-09-21T02:22:21.476224+0000","last_active":"2024-09-21T02:22:23.438403+0000","last_peered":"2024-09-21T02:22:23.438403+0000","last_clean":"2024-09-21T02:22:23.438403+0000","last_became_active":"2024-09-21T02:22:21.474796+0000","last_became_peered":"2024-09-21T02:22:21.474796+0000","last_unstale":"2024-09-21T02:22:23.438403+0000","last_undegraded":"2024-09-21T02:22:23.438403+0000","last_fullsized":"2024-09-21T02:22:23.438403+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:18.592314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481616+0000","last_change":"2024-09-21T02:22:22.457333+0000","last_active":"2024-09-21T02:22:23.481616+0000","last_peered":"2024-09-21T02:22:23.481616+0000","last_clean":"2024-09-21T02:22:23.481616+0000","last_became_active":"2024-09-21T02:22:22.456486+0000","last_became_peered":"2024-09-21T02:22:22.456486+0000","last_unstale":"2024-09-21T02:22:23.481616+0000","last_undegraded":"2024-09-21T02:22:23.481616+0000","last_fullsized":"2024-09-21T02:22:23.481616+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:38.550121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488018+0000","last_change":"2024-09-21T02:22:21.465616+0000","last_active":"2024-09-21T02:22:23.488018+0000","last_peered":"2024-09-21T02:22:23.488018+0000","last_clean":"2024-09-21T02:22:23.488018+0000","last_became_active":"2024-09-21T02:22:21.465312+0000","last_became_peered":"2024-09-21T02:22:21.465312+0000","last_unstale":"2024-09-21T02:22:23.488018+0000","last_undegraded":"2024-09-21T02:22:23.488018+0000","last_fullsized":"2024-09-21T02:22:23.488018+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:43.898880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488975+0000","last_change":"2024-09-21T02:22:22.485325+0000","last_active":"2024-09-21T02:22:23.488975+0000","last_peered":"2024-09-21T02:22:23.488975+0000","last_clean":"2024-09-21T02:22:23.488975+0000","last_became_active":"2024-09-21T02:22:22.483953+0000","last_became_peered":"2024-09-21T02:22:22.483953+0000","last_unstale":"2024-09-21T02:22:23.488975+0000","last_undegraded":"2024-09-21T02:22:23.488975+0000","last_fullsized":"2024-09-21T02:22:23.488975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:59.711639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492576+0000","last_change":"2024-09-21T02:22:21.474042+0000","last_active":"2024-09-21T02:22:23.492576+0000","last_peered":"2024-09-21T02:22:23.492576+0000","last_clean":"2024-09-21T02:22:23.492576+0000","last_became_active":"2024-09-21T02:22:21.473281+0000","last_became_peered":"2024-09-21T02:22:21.473281+0000","last_unstale":"2024-09-21T02:22:23.492576+0000","last_undegraded":"2024-09-21T02:22:23.492576+0000","last_fullsized":"2024-09-21T02:22:23.492576+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:41.069296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489324+0000","last_change":"2024-09-21T02:22:22.474638+0000","last_active":"2024-09-21T02:22:23.489324+0000","last_peered":"2024-09-21T02:22:23.489324+0000","last_clean":"2024-09-21T02:22:23.489324+0000","last_became_active":"2024-09-21T02:22:22.473231+0000","last_became_peered":"2024-09-21T02:22:22.473231+0000","last_unstale":"2024-09-21T02:22:23.489324+0000","last_undegraded":"2024-09-21T02:22:23.489324+0000","last_fullsized":"2024-09-21T02:22:23.489324+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:12.827287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489184+0000","last_change":"2024-09-21T02:22:21.448461+0000","last_active":"2024-09-21T02:22:23.489184+0000","last_peered":"2024-09-21T02:22:23.489184+0000","last_clean":"2024-09-21T02:22:23.489184+0000","last_became_active":"2024-09-21T02:22:21.447720+0000","last_became_peered":"2024-09-21T02:22:21.447720+0000","last_unstale":"2024-09-21T02:22:23.489184+0000","last_undegraded":"2024-09-21T02:22:23.489184+0000","last_fullsized":"2024-09-21T02:22:23.489184+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:19.892947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481975+0000","last_change":"2024-09-21T02:22:22.468831+0000","last_active":"2024-09-21T02:22:23.481975+0000","last_peered":"2024-09-21T02:22:23.481975+0000","last_clean":"2024-09-21T02:22:23.481975+0000","last_became_active":"2024-09-21T02:22:22.463670+0000","last_became_peered":"2024-09-21T02:22:22.463670+0000","last_unstale":"2024-09-21T02:22:23.481975+0000","last_undegraded":"2024-09-21T02:22:23.481975+0000","last_fullsized":"2024-09-21T02:22:23.481975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:51.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"165'3","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440895+0000","last_change":"2024-09-21T02:22:21.481467+0000","last_active":"2024-09-21T02:22:23.440895+0000","last_peered":"2024-09-21T02:22:23.440895+0000","last_clean":"2024-09-21T02:22:23.440895+0000","last_became_active":"2024-09-21T02:22:21.480582+0000","last_became_peered":"2024-09-21T02:22:21.480582+0000","last_unstale":"2024-09-21T02:22:23.440895+0000","last_undegraded":"2024-09-21T02:22:23.440895+0000","last_fullsized":"2024-09-21T02:22:23.440895+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:30.217816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439942+0000","last_change":"2024-09-21T02:22:22.467611+0000","last_active":"2024-09-21T02:22:23.439942+0000","last_peered":"2024-09-21T02:22:23.439942+0000","last_clean":"2024-09-21T02:22:23.439942+0000","last_became_active":"2024-09-21T02:22:22.466706+0000","last_became_peered":"2024-09-21T02:22:22.466706+0000","last_unstale":"2024-09-21T02:22:23.439942+0000","last_undegraded":"2024-09-21T02:22:23.439942+0000","last_fullsized":"2024-09-21T02:22:23.439942+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:41.744685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"165'6","reported_seq":21,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481728+0000","last_change":"2024-09-21T02:22:21.448858+0000","last_active":"2024-09-21T02:22:23.481728+0000","last_peered":"2024-09-21T02:22:23.481728+0000","last_clean":"2024-09-21T02:22:23.481728+0000","last_became_active":"2024-09-21T02:22:21.448497+0000","last_became_peered":"2024-09-21T02:22:21.448497+0000","last_unstale":"2024-09-21T02:22:23.481728+0000","last_undegraded":"2024-09-21T02:22:23.481728+0000","last_fullsized":"2024-09-21T02:22:23.481728+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:20.373492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491444+0000","last_change":"2024-09-21T02:22:22.481160+0000","last_active":"2024-09-21T02:22:23.491444+0000","last_peered":"2024-09-21T02:22:23.491444+0000","last_clean":"2024-09-21T02:22:23.491444+0000","last_became_active":"2024-09-21T02:22:22.480341+0000","last_became_peered":"2024-09-21T02:22:22.480341+0000","last_unstale":"2024-09-21T02:22:23.491444+0000","last_undegraded":"2024-09-21T02:22:23.491444+0000","last_fullsized":"2024-09-21T02:22:23.491444+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:26.627544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481852+0000","last_change":"2024-09-21T02:22:21.451094+0000","last_active":"2024-09-21T02:22:23.481852+0000","last_peered":"2024-09-21T02:22:23.481852+0000","last_clean":"2024-09-21T02:22:23.481852+0000","last_became_active":"2024-09-21T02:22:21.450270+0000","last_became_peered":"2024-09-21T02:22:21.450270+0000","last_unstale":"2024-09-21T02:22:23.481852+0000","last_undegraded":"2024-09-21T02:22:23.481852+0000","last_fullsized":"2024-09-21T02:22:23.481852+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:14.367736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491626+0000","last_change":"2024-09-21T02:22:22.502467+0000","last_active":"2024-09-21T02:22:23.491626+0000","last_peered":"2024-09-21T02:22:23.491626+0000","last_clean":"2024-09-21T02:22:23.491626+0000","last_became_active":"2024-09-21T02:22:22.501820+0000","last_became_peered":"2024-09-21T02:22:22.501820+0000","last_unstale":"2024-09-21T02:22:23.491626+0000","last_undegraded":"2024-09-21T02:22:23.491626+0000","last_fullsized":"2024-09-21T02:22:23.491626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:47.906300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433515+0000","last_change":"2024-09-21T02:22:21.469222+0000","last_active":"2024-09-21T02:22:23.433515+0000","last_peered":"2024-09-21T02:22:23.433515+0000","last_clean":"2024-09-21T02:22:23.433515+0000","last_became_active":"2024-09-21T02:22:21.468703+0000","last_became_peered":"2024-09-21T02:22:21.468703+0000","last_unstale":"2024-09-21T02:22:23.433515+0000","last_undegraded":"2024-09-21T02:22:23.433515+0000","last_fullsized":"2024-09-21T02:22:23.433515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:31.233975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489458+0000","last_change":"2024-09-21T02:22:22.497046+0000","last_active":"2024-09-21T02:22:23.489458+0000","last_peered":"2024-09-21T02:22:23.489458+0000","last_clean":"2024-09-21T02:22:23.489458+0000","last_became_active":"2024-09-21T02:22:22.495953+0000","last_became_peered":"2024-09-21T02:22:22.495953+0000","last_unstale":"2024-09-21T02:22:23.489458+0000","last_undegraded":"2024-09-21T02:22:23.489458+0000","last_fullsized":"2024-09-21T02:22:23.489458+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:48.894869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438698+0000","last_change":"2024-09-21T02:22:21.461064+0000","last_active":"2024-09-21T02:22:23.438698+0000","last_peered":"2024-09-21T02:22:23.438698+0000","last_clean":"2024-09-21T02:22:23.438698+0000","last_became_active":"2024-09-21T02:22:21.460127+0000","last_became_peered":"2024-09-21T02:22:21.460127+0000","last_unstale":"2024-09-21T02:22:23.438698+0000","last_undegraded":"2024-09-21T02:22:23.438698+0000","last_fullsized":"2024-09-21T02:22:23.438698+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:58.845230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437145+0000","last_change":"2024-09-21T02:22:22.448417+0000","last_active":"2024-09-21T02:22:23.437145+0000","last_peered":"2024-09-21T02:22:23.437145+0000","last_clean":"2024-09-21T02:22:23.437145+0000","last_became_active":"2024-09-21T02:22:22.447355+0000","last_became_peered":"2024-09-21T02:22:22.447355+0000","last_unstale":"2024-09-21T02:22:23.437145+0000","last_undegraded":"2024-09-21T02:22:23.437145+0000","last_fullsized":"2024-09-21T02:22:23.437145+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:26.961413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"165'8","reported_seq":23,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437721+0000","last_change":"2024-09-21T02:22:21.468968+0000","last_active":"2024-09-21T02:22:23.437721+0000","last_peered":"2024-09-21T02:22:23.437721+0000","last_clean":"2024-09-21T02:22:23.437721+0000","last_became_active":"2024-09-21T02:22:21.468178+0000","last_became_peered":"2024-09-21T02:22:21.468178+0000","last_unstale":"2024-09-21T02:22:23.437721+0000","last_undegraded":"2024-09-21T02:22:23.437721+0000","last_fullsized":"2024-09-21T02:22:23.437721+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:54.424298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486627+0000","last_change":"2024-09-21T02:22:22.486901+0000","last_active":"2024-09-21T02:22:23.486627+0000","last_peered":"2024-09-21T02:22:23.486627+0000","last_clean":"2024-09-21T02:22:23.486627+0000","last_became_active":"2024-09-21T02:22:22.486512+0000","last_became_peered":"2024-09-21T02:22:22.486512+0000","last_unstale":"2024-09-21T02:22:23.486627+0000","last_undegraded":"2024-09-21T02:22:23.486627+0000","last_fullsized":"2024-09-21T02:22:23.486627+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:05.678401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.442557+0000","last_change":"2024-09-21T02:22:21.479020+0000","last_active":"2024-09-21T02:22:23.442557+0000","last_peered":"2024-09-21T02:22:23.442557+0000","last_clean":"2024-09-21T02:22:23.442557+0000","last_became_active":"2024-09-21T02:22:21.475721+0000","last_became_peered":"2024-09-21T02:22:21.475721+0000","last_unstale":"2024-09-21T02:22:23.442557+0000","last_undegraded":"2024-09-21T02:22:23.442557+0000","last_fullsized":"2024-09-21T02:22:23.442557+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:47.891029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488927+0000","last_change":"2024-09-21T02:22:22.476386+0000","last_active":"2024-09-21T02:22:23.488927+0000","last_peered":"2024-09-21T02:22:23.488927+0000","last_clean":"2024-09-21T02:22:23.488927+0000","last_became_active":"2024-09-21T02:22:22.475984+0000","last_became_peered":"2024-09-21T02:22:22.475984+0000","last_unstale":"2024-09-21T02:22:23.488927+0000","last_undegraded":"2024-09-21T02:22:23.488927+0000","last_fullsized":"2024-09-21T02:22:23.488927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:26.379696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481861+0000","last_change":"2024-09-21T02:22:21.470128+0000","last_active":"2024-09-21T02:22:23.481861+0000","last_peered":"2024-09-21T02:22:23.481861+0000","last_clean":"2024-09-21T02:22:23.481861+0000","last_became_active":"2024-09-21T02:22:21.469768+0000","last_became_peered":"2024-09-21T02:22:21.469768+0000","last_unstale":"2024-09-21T02:22:23.481861+0000","last_undegraded":"2024-09-21T02:22:23.481861+0000","last_fullsized":"2024-09-21T02:22:23.481861+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:44.008372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491046+0000","last_change":"2024-09-21T02:22:22.508645+0000","last_active":"2024-09-21T02:22:23.491046+0000","last_peered":"2024-09-21T02:22:23.491046+0000","last_clean":"2024-09-21T02:22:23.491046+0000","last_became_active":"2024-09-21T02:22:22.507972+0000","last_became_peered":"2024-09-21T02:22:22.507972+0000","last_unstale":"2024-09-21T02:22:23.491046+0000","last_undegraded":"2024-09-21T02:22:23.491046+0000","last_fullsized":"2024-09-21T02:22:23.491046+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:18.675323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490199+0000","last_change":"2024-09-21T02:22:21.453206+0000","last_active":"2024-09-21T02:22:23.490199+0000","last_peered":"2024-09-21T02:22:23.490199+0000","last_clean":"2024-09-21T02:22:23.490199+0000","last_became_active":"2024-09-21T02:22:21.452175+0000","last_became_peered":"2024-09-21T02:22:21.452175+0000","last_unstale":"2024-09-21T02:22:23.490199+0000","last_undegraded":"2024-09-21T02:22:23.490199+0000","last_fullsized":"2024-09-21T02:22:23.490199+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:03.484364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.441150+0000","last_change":"2024-09-21T02:22:22.473490+0000","last_active":"2024-09-21T02:22:23.441150+0000","last_peered":"2024-09-21T02:22:23.441150+0000","last_clean":"2024-09-21T02:22:23.441150+0000","last_became_active":"2024-09-21T02:22:22.465332+0000","last_became_peered":"2024-09-21T02:22:22.465332+0000","last_unstale":"2024-09-21T02:22:23.441150+0000","last_undegraded":"2024-09-21T02:22:23.441150+0000","last_fullsized":"2024-09-21T02:22:23.441150+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:52.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438629+0000","last_change":"2024-09-21T02:22:21.474322+0000","last_active":"2024-09-21T02:22:23.438629+0000","last_peered":"2024-09-21T02:22:23.438629+0000","last_clean":"2024-09-21T02:22:23.438629+0000","last_became_active":"2024-09-21T02:22:21.473688+0000","last_became_peered":"2024-09-21T02:22:21.473688+0000","last_unstale":"2024-09-21T02:22:23.438629+0000","last_undegraded":"2024-09-21T02:22:23.438629+0000","last_fullsized":"2024-09-21T02:22:23.438629+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:28.859889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490966+0000","last_change":"2024-09-21T02:22:22.508860+0000","last_active":"2024-09-21T02:22:23.490966+0000","last_peered":"2024-09-21T02:22:23.490966+0000","last_clean":"2024-09-21T02:22:23.490966+0000","last_became_active":"2024-09-21T02:22:22.508254+0000","last_became_peered":"2024-09-21T02:22:22.508254+0000","last_unstale":"2024-09-21T02:22:23.490966+0000","last_undegraded":"2024-09-21T02:22:23.490966+0000","last_fullsized":"2024-09-21T02:22:23.490966+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:58.267000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488961+0000","last_change":"2024-09-21T02:22:21.463499+0000","last_active":"2024-09-21T02:22:23.488961+0000","last_peered":"2024-09-21T02:22:23.488961+0000","last_clean":"2024-09-21T02:22:23.488961+0000","last_became_active":"2024-09-21T02:22:21.462105+0000","last_became_peered":"2024-09-21T02:22:21.462105+0000","last_unstale":"2024-09-21T02:22:23.488961+0000","last_undegraded":"2024-09-21T02:22:23.488961+0000","last_fullsized":"2024-09-21T02:22:23.488961+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:47.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439521+0000","last_change":"2024-09-21T02:22:22.472217+0000","last_active":"2024-09-21T02:22:23.439521+0000","last_peered":"2024-09-21T02:22:23.439521+0000","last_clean":"2024-09-21T02:22:23.439521+0000","last_became_active":"2024-09-21T02:22:22.471470+0000","last_became_peered":"2024-09-21T02:22:22.471470+0000","last_unstale":"2024-09-21T02:22:23.439521+0000","last_undegraded":"2024-09-21T02:22:23.439521+0000","last_fullsized":"2024-09-21T02:22:23.439521+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:49.362864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437790+0000","last_change":"2024-09-21T02:22:21.470671+0000","last_active":"2024-09-21T02:22:23.437790+0000","last_peered":"2024-09-21T02:22:23.437790+0000","last_clean":"2024-09-21T02:22:23.437790+0000","last_became_active":"2024-09-21T02:22:21.469843+0000","last_became_peered":"2024-09-21T02:22:21.469843+0000","last_unstale":"2024-09-21T02:22:23.437790+0000","last_undegraded":"2024-09-21T02:22:23.437790+0000","last_fullsized":"2024-09-21T02:22:23.437790+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:33.235495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488322+0000","last_change":"2024-09-21T02:22:22.476020+0000","last_active":"2024-09-21T02:22:23.488322+0000","last_peered":"2024-09-21T02:22:23.488322+0000","last_clean":"2024-09-21T02:22:23.488322+0000","last_became_active":"2024-09-21T02:22:22.475427+0000","last_became_peered":"2024-09-21T02:22:22.475427+0000","last_unstale":"2024-09-21T02:22:23.488322+0000","last_undegraded":"2024-09-21T02:22:23.488322+0000","last_fullsized":"2024-09-21T02:22:23.488322+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:29.399926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489757+0000","last_change":"2024-09-21T02:22:21.474582+0000","last_active":"2024-09-21T02:22:23.489757+0000","last_peered":"2024-09-21T02:22:23.489757+0000","last_clean":"2024-09-21T02:22:23.489757+0000","last_became_active":"2024-09-21T02:22:21.473601+0000","last_became_peered":"2024-09-21T02:22:21.473601+0000","last_unstale":"2024-09-21T02:22:23.489757+0000","last_undegraded":"2024-09-21T02:22:23.489757+0000","last_fullsized":"2024-09-21T02:22:23.489757+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:50.116902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489550+0000","last_change":"2024-09-21T02:22:22.474853+0000","last_active":"2024-09-21T02:22:23.489550+0000","last_peered":"2024-09-21T02:22:23.489550+0000","last_clean":"2024-09-21T02:22:23.489550+0000","last_became_active":"2024-09-21T02:22:22.462271+0000","last_became_peered":"2024-09-21T02:22:22.462271+0000","last_unstale":"2024-09-21T02:22:23.489550+0000","last_undegraded":"2024-09-21T02:22:23.489550+0000","last_fullsized":"2024-09-21T02:22:23.489550+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:09.589293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438172+0000","last_change":"2024-09-21T02:22:21.481357+0000","last_active":"2024-09-21T02:22:23.438172+0000","last_peered":"2024-09-21T02:22:23.438172+0000","last_clean":"2024-09-21T02:22:23.438172+0000","last_became_active":"2024-09-21T02:22:21.480659+0000","last_became_peered":"2024-09-21T02:22:21.480659+0000","last_unstale":"2024-09-21T02:22:23.438172+0000","last_undegraded":"2024-09-21T02:22:23.438172+0000","last_fullsized":"2024-09-21T02:22:23.438172+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:30.738468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486795+0000","last_change":"2024-09-21T02:22:22.447997+0000","last_active":"2024-09-21T02:22:23.486795+0000","last_peered":"2024-09-21T02:22:23.486795+0000","last_clean":"2024-09-21T02:22:23.486795+0000","last_became_active":"2024-09-21T02:22:22.447015+0000","last_became_peered":"2024-09-21T02:22:22.447015+0000","last_unstale":"2024-09-21T02:22:23.486795+0000","last_undegraded":"2024-09-21T02:22:23.486795+0000","last_fullsized":"2024-09-21T02:22:23.486795+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:38.617086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436812+0000","last_change":"2024-09-21T02:22:21.456293+0000","last_active":"2024-09-21T02:22:23.436812+0000","last_peered":"2024-09-21T02:22:23.436812+0000","last_clean":"2024-09-21T02:22:23.436812+0000","last_became_active":"2024-09-21T02:22:21.455368+0000","last_became_peered":"2024-09-21T02:22:21.455368+0000","last_unstale":"2024-09-21T02:22:23.436812+0000","last_undegraded":"2024-09-21T02:22:23.436812+0000","last_fullsized":"2024-09-21T02:22:23.436812+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:49.978614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.441623+0000","last_change":"2024-09-21T02:22:22.451994+0000","last_active":"2024-09-21T02:22:23.441623+0000","last_peered":"2024-09-21T02:22:23.441623+0000","last_clean":"2024-09-21T02:22:23.441623+0000","last_became_active":"2024-09-21T02:22:22.451174+0000","last_became_peered":"2024-09-21T02:22:22.451174+0000","last_unstale":"2024-09-21T02:22:23.441623+0000","last_undegraded":"2024-09-21T02:22:23.441623+0000","last_fullsized":"2024-09-21T02:22:23.441623+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:07.426362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"166'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.480541+0000","last_change":"2024-09-21T02:22:21.454116+0000","last_active":"2024-09-21T02:22:23.480541+0000","last_peered":"2024-09-21T02:22:23.480541+0000","last_clean":"2024-09-21T02:22:23.480541+0000","last_became_active":"2024-09-21T02:22:21.453802+0000","last_became_peered":"2024-09-21T02:22:21.453802+0000","last_unstale":"2024-09-21T02:22:23.480541+0000","last_undegraded":"2024-09-21T02:22:23.480541+0000","last_fullsized":"2024-09-21T02:22:23.480541+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:51.978522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433413+0000","last_change":"2024-09-21T02:22:22.454925+0000","last_active":"2024-09-21T02:22:23.433413+0000","last_peered":"2024-09-21T02:22:23.433413+0000","last_clean":"2024-09-21T02:22:23.433413+0000","last_became_active":"2024-09-21T02:22:22.454050+0000","last_became_peered":"2024-09-21T02:22:22.454050+0000","last_unstale":"2024-09-21T02:22:23.433413+0000","last_undegraded":"2024-09-21T02:22:23.433413+0000","last_fullsized":"2024-09-21T02:22:23.433413+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:05.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.441325+0000","last_change":"2024-09-21T02:22:21.471338+0000","last_active":"2024-09-21T02:22:23.441325+0000","last_peered":"2024-09-21T02:22:23.441325+0000","last_clean":"2024-09-21T02:22:23.441325+0000","last_became_active":"2024-09-21T02:22:21.470418+0000","last_became_peered":"2024-09-21T02:22:21.470418+0000","last_unstale":"2024-09-21T02:22:23.441325+0000","last_undegraded":"2024-09-21T02:22:23.441325+0000","last_fullsized":"2024-09-21T02:22:23.441325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:17.671483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486612+0000","last_change":"2024-09-21T02:22:22.468019+0000","last_active":"2024-09-21T02:22:23.486612+0000","last_peered":"2024-09-21T02:22:23.486612+0000","last_clean":"2024-09-21T02:22:23.486612+0000","last_became_active":"2024-09-21T02:22:22.467677+0000","last_became_peered":"2024-09-21T02:22:22.467677+0000","last_unstale":"2024-09-21T02:22:23.486612+0000","last_undegraded":"2024-09-21T02:22:23.486612+0000","last_fullsized":"2024-09-21T02:22:23.486612+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:03:26.561468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438739+0000","last_change":"2024-09-21T02:22:21.453344+0000","last_active":"2024-09-21T02:22:23.438739+0000","last_peered":"2024-09-21T02:22:23.438739+0000","last_clean":"2024-09-21T02:22:23.438739+0000","last_became_active":"2024-09-21T02:22:21.452601+0000","last_became_peered":"2024-09-21T02:22:21.452601+0000","last_unstale":"2024-09-21T02:22:23.438739+0000","last_undegraded":"2024-09-21T02:22:23.438739+0000","last_fullsized":"2024-09-21T02:22:23.438739+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:20.734200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439440+0000","last_change":"2024-09-21T02:22:22.461537+0000","last_active":"2024-09-21T02:22:23.439440+0000","last_peered":"2024-09-21T02:22:23.439440+0000","last_clean":"2024-09-21T02:22:23.439440+0000","last_became_active":"2024-09-21T02:22:22.459891+0000","last_became_peered":"2024-09-21T02:22:22.459891+0000","last_unstale":"2024-09-21T02:22:23.439440+0000","last_undegraded":"2024-09-21T02:22:23.439440+0000","last_fullsized":"2024-09-21T02:22:23.439440+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:08.668744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488930+0000","last_change":"2024-09-21T02:22:21.462431+0000","last_active":"2024-09-21T02:22:23.488930+0000","last_peered":"2024-09-21T02:22:23.488930+0000","last_clean":"2024-09-21T02:22:23.488930+0000","last_became_active":"2024-09-21T02:22:21.461925+0000","last_became_peered":"2024-09-21T02:22:21.461925+0000","last_unstale":"2024-09-21T02:22:23.488930+0000","last_undegraded":"2024-09-21T02:22:23.488930+0000","last_fullsized":"2024-09-21T02:22:23.488930+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:20.461260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433724+0000","last_change":"2024-09-21T02:22:22.471896+0000","last_active":"2024-09-21T02:22:23.433724+0000","last_peered":"2024-09-21T02:22:23.433724+0000","last_clean":"2024-09-21T02:22:23.433724+0000","last_became_active":"2024-09-21T02:22:22.471020+0000","last_became_peered":"2024-09-21T02:22:22.471020+0000","last_unstale":"2024-09-21T02:22:23.433724+0000","last_undegraded":"2024-09-21T02:22:23.433724+0000","last_fullsized":"2024-09-21T02:22:23.433724+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:28.861772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491265+0000","last_change":"2024-09-21T02:22:21.476491+0000","last_active":"2024-09-21T02:22:23.491265+0000","last_peered":"2024-09-21T02:22:23.491265+0000","last_clean":"2024-09-21T02:22:23.491265+0000","last_became_active":"2024-09-21T02:22:21.475140+0000","last_became_peered":"2024-09-21T02:22:21.475140+0000","last_unstale":"2024-09-21T02:22:23.491265+0000","last_undegraded":"2024-09-21T02:22:23.491265+0000","last_fullsized":"2024-09-21T02:22:23.491265+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:56.225401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437378+0000","last_change":"2024-09-21T02:22:22.452173+0000","last_active":"2024-09-21T02:22:23.437378+0000","last_peered":"2024-09-21T02:22:23.437378+0000","last_clean":"2024-09-21T02:22:23.437378+0000","last_became_active":"2024-09-21T02:22:22.450075+0000","last_became_peered":"2024-09-21T02:22:22.450075+0000","last_unstale":"2024-09-21T02:22:23.437378+0000","last_undegraded":"2024-09-21T02:22:23.437378+0000","last_fullsized":"2024-09-21T02:22:23.437378+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:25.144752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486415+0000","last_change":"2024-09-21T02:22:21.454568+0000","last_active":"2024-09-21T02:22:23.486415+0000","last_peered":"2024-09-21T02:22:23.486415+0000","last_clean":"2024-09-21T02:22:23.486415+0000","last_became_active":"2024-09-21T02:22:21.454228+0000","last_became_peered":"2024-09-21T02:22:21.454228+0000","last_unstale":"2024-09-21T02:22:23.486415+0000","last_undegraded":"2024-09-21T02:22:23.486415+0000","last_fullsized":"2024-09-21T02:22:23.486415+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:09.390858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489013+0000","last_change":"2024-09-21T02:22:22.475972+0000","last_active":"2024-09-21T02:22:23.489013+0000","last_peered":"2024-09-21T02:22:23.489013+0000","last_clean":"2024-09-21T02:22:23.489013+0000","last_became_active":"2024-09-21T02:22:22.475470+0000","last_became_peered":"2024-09-21T02:22:22.475470+0000","last_unstale":"2024-09-21T02:22:23.489013+0000","last_undegraded":"2024-09-21T02:22:23.489013+0000","last_fullsized":"2024-09-21T02:22:23.489013+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:33:29.730779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488129+0000","last_change":"2024-09-21T02:22:21.465287+0000","last_active":"2024-09-21T02:22:23.488129+0000","last_peered":"2024-09-21T02:22:23.488129+0000","last_clean":"2024-09-21T02:22:23.488129+0000","last_became_active":"2024-09-21T02:22:21.464854+0000","last_became_peered":"2024-09-21T02:22:21.464854+0000","last_unstale":"2024-09-21T02:22:23.488129+0000","last_undegraded":"2024-09-21T02:22:23.488129+0000","last_fullsized":"2024-09-21T02:22:23.488129+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:39.671206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490863+0000","last_change":"2024-09-21T02:22:22.501874+0000","last_active":"2024-09-21T02:22:23.490863+0000","last_peered":"2024-09-21T02:22:23.490863+0000","last_clean":"2024-09-21T02:22:23.490863+0000","last_became_active":"2024-09-21T02:22:22.500983+0000","last_became_peered":"2024-09-21T02:22:22.500983+0000","last_unstale":"2024-09-21T02:22:23.490863+0000","last_undegraded":"2024-09-21T02:22:23.490863+0000","last_fullsized":"2024-09-21T02:22:23.490863+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:17.715335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433656+0000","last_change":"2024-09-21T02:22:21.453719+0000","last_active":"2024-09-21T02:22:23.433656+0000","last_peered":"2024-09-21T02:22:23.433656+0000","last_clean":"2024-09-21T02:22:23.433656+0000","last_became_active":"2024-09-21T02:22:21.453306+0000","last_became_peered":"2024-09-21T02:22:21.453306+0000","last_unstale":"2024-09-21T02:22:23.433656+0000","last_undegraded":"2024-09-21T02:22:23.433656+0000","last_fullsized":"2024-09-21T02:22:23.433656+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:44.439955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":401,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488207+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:22:23.488207+0000","last_peered":"2024-09-21T02:22:23.488207+0000","last_clean":"2024-09-21T02:22:23.488207+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:22:23.488207+0000","last_undegraded":"2024-09-21T02:22:23.488207+0000","last_fullsized":"2024-09-21T02:22:23.488207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2093056,"data_stored":924035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509552,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2093056,"data_stored":924083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2699264,"data_stored":1515109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542258,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2093056,"data_stored":924103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2101248,"data_stored":925625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41572,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330268,"statfs":{"total":96636764160,"available":96594194432,"internally_reserved":0,"allocated":2109440,"data_stored":925729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509553,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42132,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329708,"statfs":{"total":96636764160,"available":96593620992,"internally_reserved":0,"allocated":2682880,"data_stored":1514381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41552,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330288,"statfs":{"total":96636764160,"available":96594214912,"internally_reserved":0,"allocated":2093056,"data_stored":924595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:22:26.625 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:22:26.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.815+0000 7ff97d6dc640 1 -- 172.21.15.170:0/472798448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff97806d510 msgr2=0x7ff9780b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.815+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/472798448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff97806d510 0x7ff9780b1730 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff968009c80 tx=0x7ff96801cda0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.815+0000 7ff97d6dc640 1 -- 172.21.15.170:0/472798448 shutdown_connections 2024-09-21T02:22:26.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.815+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/472798448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9780b6930 0x7ff978160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.815+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/472798448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff97806d510 0x7ff9780b1730 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.815+0000 7ff97d6dc640 1 -- 172.21.15.170:0/472798448 >> 172.21.15.170:0/472798448 conn(0x7ff97809a9c0 msgr2=0x7ff9780d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:26.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.819+0000 7ff97d6dc640 1 -- 172.21.15.170:0/472798448 shutdown_connections 2024-09-21T02:22:26.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.819+0000 7ff97d6dc640 1 -- 172.21.15.170:0/472798448 wait complete. 2024-09-21T02:22:26.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.819+0000 7ff97d6dc640 1 Processor -- start 2024-09-21T02:22:26.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.819+0000 7ff97d6dc640 1 -- start start 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.819+0000 7ff97d6dc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9780b22b0 0x7ff978154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6930 0x7ff9780c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 0x7ff9780af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff97815c620 con 0x7ff97815ba00 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff97815c4a0 con 0x7ff9780b6930 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff97815c7a0 con 0x7ff9780b22b0 2024-09-21T02:22:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 0x7ff9780af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 0x7ff9780af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44028/0 (socket says 172.21.15.170:44028) 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 -- 172.21.15.170:0/2363777626 learned_addr learned my addr 172.21.15.170:0/2363777626 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9767fc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6930 0x7ff9780c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff976ffd640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9780b22b0 0x7ff978154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 -- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9780b22b0 msgr2=0x7ff978154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9780b22b0 0x7ff978154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 -- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6930 msgr2=0x7ff9780c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6930 0x7ff9780c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 -- 172.21.15.170:0/2363777626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9780c6d90 con 0x7ff97815ba00 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff9777fe640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 0x7ff9780af2b0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff97000e970 tx=0x7ff97000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97000ccc0 con 0x7ff97815ba00 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9781f6cc0 con 0x7ff97815ba00 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff970004020 con 0x7ff97815ba00 2024-09-21T02:22:26.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97000ce60 con 0x7ff97815ba00 2024-09-21T02:22:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781f7280 con 0x7ff97815ba00 2024-09-21T02:22:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780c6f60 con 0x7ff97815ba00 2024-09-21T02:22:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7ff970003600 con 0x7ff97815ba00 2024-09-21T02:22:26.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff96f7fe640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff958073c50 0x7ff958076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:26.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff976ffd640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff958073c50 0x7ff958076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:26.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.823+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7ff970093620 con 0x7ff97815ba00 2024-09-21T02:22:26.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.827+0000 7ff976ffd640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff958073c50 0x7ff958076160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff960001640 tx=0x7ff960009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:26.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.831+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff97005fee0 con 0x7ff97815ba00 2024-09-21T02:22:26.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.987+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9780b1730 con 0x7ff958073c50 2024-09-21T02:22:26.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.995+0000 7ff96f7fe640 1 -- 172.21.15.170:0/2363777626 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7ff9780b1730 con 0x7ff958073c50 2024-09-21T02:22:26.999 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:27.001 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff958073c50 msgr2=0x7ff958076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff958073c50 0x7ff958076160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff960001640 tx=0x7ff960009040 comp rx=0 tx=0).stop 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 msgr2=0x7ff9780af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 0x7ff9780af2b0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff97000e970 tx=0x7ff97000ee30 comp rx=0 tx=0).stop 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 shutdown_connections 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff958073c50 0x7ff958076160 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff97815ba00 0x7ff9780af2b0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9780b6930 0x7ff9780c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 --2- 172.21.15.170:0/2363777626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9780b22b0 0x7ff978154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 >> 172.21.15.170:0/2363777626 conn(0x7ff97809a9c0 msgr2=0x7ff9780b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:27.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 shutdown_connections 2024-09-21T02:22:27.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:26.999+0000 7ff97d6dc640 1 -- 172.21.15.170:0/2363777626 wait complete. 2024-09-21T02:22:27.021 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-21T02:22:26.943429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304520,"kb_used_data":17544,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754670200,"statfs":{"total":773094113280,"available":772782284800,"internally_reserved":0,"allocated":17965056,"data_stored":8577660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":293654065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002034"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440053+0000","last_change":"2024-09-21T02:22:22.457835+0000","last_active":"2024-09-21T02:22:23.440053+0000","last_peered":"2024-09-21T02:22:23.440053+0000","last_clean":"2024-09-21T02:22:23.440053+0000","last_became_active":"2024-09-21T02:22:22.456937+0000","last_became_peered":"2024-09-21T02:22:22.456937+0000","last_unstale":"2024-09-21T02:22:23.440053+0000","last_undegraded":"2024-09-21T02:22:23.440053+0000","last_fullsized":"2024-09-21T02:22:23.440053+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:36.199399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433567+0000","last_change":"2024-09-21T02:22:22.461326+0000","last_active":"2024-09-21T02:22:23.433567+0000","last_peered":"2024-09-21T02:22:23.433567+0000","last_clean":"2024-09-21T02:22:23.433567+0000","last_became_active":"2024-09-21T02:22:22.460591+0000","last_became_peered":"2024-09-21T02:22:22.460591+0000","last_unstale":"2024-09-21T02:22:23.433567+0000","last_undegraded":"2024-09-21T02:22:23.433567+0000","last_fullsized":"2024-09-21T02:22:23.433567+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:42.420027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490469+0000","last_change":"2024-09-21T02:22:22.504027+0000","last_active":"2024-09-21T02:22:23.490469+0000","last_peered":"2024-09-21T02:22:23.490469+0000","last_clean":"2024-09-21T02:22:23.490469+0000","last_became_active":"2024-09-21T02:22:22.503340+0000","last_became_peered":"2024-09-21T02:22:22.503340+0000","last_unstale":"2024-09-21T02:22:23.490469+0000","last_undegraded":"2024-09-21T02:22:23.490469+0000","last_fullsized":"2024-09-21T02:22:23.490469+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:52.239116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489067+0000","last_change":"2024-09-21T02:22:22.482951+0000","last_active":"2024-09-21T02:22:23.489067+0000","last_peered":"2024-09-21T02:22:23.489067+0000","last_clean":"2024-09-21T02:22:23.489067+0000","last_became_active":"2024-09-21T02:22:22.474858+0000","last_became_peered":"2024-09-21T02:22:22.474858+0000","last_unstale":"2024-09-21T02:22:23.489067+0000","last_undegraded":"2024-09-21T02:22:23.489067+0000","last_fullsized":"2024-09-21T02:22:23.489067+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:16.079204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491311+0000","last_change":"2024-09-21T02:22:22.503427+0000","last_active":"2024-09-21T02:22:23.491311+0000","last_peered":"2024-09-21T02:22:23.491311+0000","last_clean":"2024-09-21T02:22:23.491311+0000","last_became_active":"2024-09-21T02:22:22.501055+0000","last_became_peered":"2024-09-21T02:22:22.501055+0000","last_unstale":"2024-09-21T02:22:23.491311+0000","last_undegraded":"2024-09-21T02:22:23.491311+0000","last_fullsized":"2024-09-21T02:22:23.491311+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:47.372392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439234+0000","last_change":"2024-09-21T02:22:22.450895+0000","last_active":"2024-09-21T02:22:23.439234+0000","last_peered":"2024-09-21T02:22:23.439234+0000","last_clean":"2024-09-21T02:22:23.439234+0000","last_became_active":"2024-09-21T02:22:22.449130+0000","last_became_peered":"2024-09-21T02:22:22.449130+0000","last_unstale":"2024-09-21T02:22:23.439234+0000","last_undegraded":"2024-09-21T02:22:23.439234+0000","last_fullsized":"2024-09-21T02:22:23.439234+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:44.203710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436729+0000","last_change":"2024-09-21T02:22:22.466509+0000","last_active":"2024-09-21T02:22:23.436729+0000","last_peered":"2024-09-21T02:22:23.436729+0000","last_clean":"2024-09-21T02:22:23.436729+0000","last_became_active":"2024-09-21T02:22:22.464105+0000","last_became_peered":"2024-09-21T02:22:22.464105+0000","last_unstale":"2024-09-21T02:22:23.436729+0000","last_undegraded":"2024-09-21T02:22:23.436729+0000","last_fullsized":"2024-09-21T02:22:23.436729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:28.110932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436758+0000","last_change":"2024-09-21T02:22:22.476525+0000","last_active":"2024-09-21T02:22:23.436758+0000","last_peered":"2024-09-21T02:22:23.436758+0000","last_clean":"2024-09-21T02:22:23.436758+0000","last_became_active":"2024-09-21T02:22:22.475747+0000","last_became_peered":"2024-09-21T02:22:22.475747+0000","last_unstale":"2024-09-21T02:22:23.436758+0000","last_undegraded":"2024-09-21T02:22:23.436758+0000","last_fullsized":"2024-09-21T02:22:23.436758+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:56.637284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.482148+0000","last_change":"2024-09-21T02:22:22.469549+0000","last_active":"2024-09-21T02:22:23.482148+0000","last_peered":"2024-09-21T02:22:23.482148+0000","last_clean":"2024-09-21T02:22:23.482148+0000","last_became_active":"2024-09-21T02:22:22.463504+0000","last_became_peered":"2024-09-21T02:22:22.463504+0000","last_unstale":"2024-09-21T02:22:23.482148+0000","last_undegraded":"2024-09-21T02:22:23.482148+0000","last_fullsized":"2024-09-21T02:22:23.482148+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.073135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492640+0000","last_change":"2024-09-21T02:22:22.508739+0000","last_active":"2024-09-21T02:22:23.492640+0000","last_peered":"2024-09-21T02:22:23.492640+0000","last_clean":"2024-09-21T02:22:23.492640+0000","last_became_active":"2024-09-21T02:22:22.508106+0000","last_became_peered":"2024-09-21T02:22:22.508106+0000","last_unstale":"2024-09-21T02:22:23.492640+0000","last_undegraded":"2024-09-21T02:22:23.492640+0000","last_fullsized":"2024-09-21T02:22:23.492640+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:17.335335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440972+0000","last_change":"2024-09-21T02:22:22.469027+0000","last_active":"2024-09-21T02:22:23.440972+0000","last_peered":"2024-09-21T02:22:23.440972+0000","last_clean":"2024-09-21T02:22:23.440972+0000","last_became_active":"2024-09-21T02:22:22.465839+0000","last_became_peered":"2024-09-21T02:22:22.465839+0000","last_unstale":"2024-09-21T02:22:23.440972+0000","last_undegraded":"2024-09-21T02:22:23.440972+0000","last_fullsized":"2024-09-21T02:22:23.440972+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:17:58.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486976+0000","last_change":"2024-09-21T02:22:22.445658+0000","last_active":"2024-09-21T02:22:23.486976+0000","last_peered":"2024-09-21T02:22:23.486976+0000","last_clean":"2024-09-21T02:22:23.486976+0000","last_became_active":"2024-09-21T02:22:22.444980+0000","last_became_peered":"2024-09-21T02:22:22.444980+0000","last_unstale":"2024-09-21T02:22:23.486976+0000","last_undegraded":"2024-09-21T02:22:23.486976+0000","last_fullsized":"2024-09-21T02:22:23.486976+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:15.329863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488798+0000","last_change":"2024-09-21T02:22:22.477438+0000","last_active":"2024-09-21T02:22:23.488798+0000","last_peered":"2024-09-21T02:22:23.488798+0000","last_clean":"2024-09-21T02:22:23.488798+0000","last_became_active":"2024-09-21T02:22:22.477113+0000","last_became_peered":"2024-09-21T02:22:22.477113+0000","last_unstale":"2024-09-21T02:22:23.488798+0000","last_undegraded":"2024-09-21T02:22:23.488798+0000","last_fullsized":"2024-09-21T02:22:23.488798+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:40.067516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489720+0000","last_change":"2024-09-21T02:22:22.471371+0000","last_active":"2024-09-21T02:22:23.489720+0000","last_peered":"2024-09-21T02:22:23.489720+0000","last_clean":"2024-09-21T02:22:23.489720+0000","last_became_active":"2024-09-21T02:22:22.470542+0000","last_became_peered":"2024-09-21T02:22:22.470542+0000","last_unstale":"2024-09-21T02:22:23.489720+0000","last_undegraded":"2024-09-21T02:22:23.489720+0000","last_fullsized":"2024-09-21T02:22:23.489720+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:03:17.581268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437523+0000","last_change":"2024-09-21T02:22:22.448947+0000","last_active":"2024-09-21T02:22:23.437523+0000","last_peered":"2024-09-21T02:22:23.437523+0000","last_clean":"2024-09-21T02:22:23.437523+0000","last_became_active":"2024-09-21T02:22:22.448217+0000","last_became_peered":"2024-09-21T02:22:22.448217+0000","last_unstale":"2024-09-21T02:22:23.437523+0000","last_undegraded":"2024-09-21T02:22:23.437523+0000","last_fullsized":"2024-09-21T02:22:23.437523+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:48.407997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436762+0000","last_change":"2024-09-21T02:22:22.468067+0000","last_active":"2024-09-21T02:22:23.436762+0000","last_peered":"2024-09-21T02:22:23.436762+0000","last_clean":"2024-09-21T02:22:23.436762+0000","last_became_active":"2024-09-21T02:22:22.467230+0000","last_became_peered":"2024-09-21T02:22:22.467230+0000","last_unstale":"2024-09-21T02:22:23.436762+0000","last_undegraded":"2024-09-21T02:22:23.436762+0000","last_fullsized":"2024-09-21T02:22:23.436762+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:22:45.339630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440732+0000","last_change":"2024-09-21T02:22:22.465773+0000","last_active":"2024-09-21T02:22:23.440732+0000","last_peered":"2024-09-21T02:22:23.440732+0000","last_clean":"2024-09-21T02:22:23.440732+0000","last_became_active":"2024-09-21T02:22:22.463773+0000","last_became_peered":"2024-09-21T02:22:22.463773+0000","last_unstale":"2024-09-21T02:22:23.440732+0000","last_undegraded":"2024-09-21T02:22:23.440732+0000","last_fullsized":"2024-09-21T02:22:23.440732+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:31.096497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492632+0000","last_change":"2024-09-21T02:22:22.501647+0000","last_active":"2024-09-21T02:22:23.492632+0000","last_peered":"2024-09-21T02:22:23.492632+0000","last_clean":"2024-09-21T02:22:23.492632+0000","last_became_active":"2024-09-21T02:22:22.500886+0000","last_became_peered":"2024-09-21T02:22:22.500886+0000","last_unstale":"2024-09-21T02:22:23.492632+0000","last_undegraded":"2024-09-21T02:22:23.492632+0000","last_fullsized":"2024-09-21T02:22:23.492632+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:50.191975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492512+0000","last_change":"2024-09-21T02:22:22.507774+0000","last_active":"2024-09-21T02:22:23.492512+0000","last_peered":"2024-09-21T02:22:23.492512+0000","last_clean":"2024-09-21T02:22:23.492512+0000","last_became_active":"2024-09-21T02:22:22.504212+0000","last_became_peered":"2024-09-21T02:22:22.504212+0000","last_unstale":"2024-09-21T02:22:23.492512+0000","last_undegraded":"2024-09-21T02:22:23.492512+0000","last_fullsized":"2024-09-21T02:22:23.492512+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:57.326942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490419+0000","last_change":"2024-09-21T02:22:22.508566+0000","last_active":"2024-09-21T02:22:23.490419+0000","last_peered":"2024-09-21T02:22:23.490419+0000","last_clean":"2024-09-21T02:22:23.490419+0000","last_became_active":"2024-09-21T02:22:22.507845+0000","last_became_peered":"2024-09-21T02:22:22.507845+0000","last_unstale":"2024-09-21T02:22:23.490419+0000","last_undegraded":"2024-09-21T02:22:23.490419+0000","last_fullsized":"2024-09-21T02:22:23.490419+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:31:26.349734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486420+0000","last_change":"2024-09-21T02:22:22.467579+0000","last_active":"2024-09-21T02:22:23.486420+0000","last_peered":"2024-09-21T02:22:23.486420+0000","last_clean":"2024-09-21T02:22:23.486420+0000","last_became_active":"2024-09-21T02:22:22.466740+0000","last_became_peered":"2024-09-21T02:22:22.466740+0000","last_unstale":"2024-09-21T02:22:23.486420+0000","last_undegraded":"2024-09-21T02:22:23.486420+0000","last_fullsized":"2024-09-21T02:22:23.486420+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:31.981683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433382+0000","last_change":"2024-09-21T02:22:22.496861+0000","last_active":"2024-09-21T02:22:23.433382+0000","last_peered":"2024-09-21T02:22:23.433382+0000","last_clean":"2024-09-21T02:22:23.433382+0000","last_became_active":"2024-09-21T02:22:22.496456+0000","last_became_peered":"2024-09-21T02:22:22.496456+0000","last_unstale":"2024-09-21T02:22:23.433382+0000","last_undegraded":"2024-09-21T02:22:23.433382+0000","last_fullsized":"2024-09-21T02:22:23.433382+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:38.888252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488339+0000","last_change":"2024-09-21T02:22:22.474716+0000","last_active":"2024-09-21T02:22:23.488339+0000","last_peered":"2024-09-21T02:22:23.488339+0000","last_clean":"2024-09-21T02:22:23.488339+0000","last_became_active":"2024-09-21T02:22:22.460755+0000","last_became_peered":"2024-09-21T02:22:22.460755+0000","last_unstale":"2024-09-21T02:22:23.488339+0000","last_undegraded":"2024-09-21T02:22:23.488339+0000","last_fullsized":"2024-09-21T02:22:23.488339+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:36.882040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.482225+0000","last_change":"2024-09-21T02:22:22.468547+0000","last_active":"2024-09-21T02:22:23.482225+0000","last_peered":"2024-09-21T02:22:23.482225+0000","last_clean":"2024-09-21T02:22:23.482225+0000","last_became_active":"2024-09-21T02:22:22.463619+0000","last_became_peered":"2024-09-21T02:22:22.463619+0000","last_unstale":"2024-09-21T02:22:23.482225+0000","last_undegraded":"2024-09-21T02:22:23.482225+0000","last_fullsized":"2024-09-21T02:22:23.482225+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:57.229241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488908+0000","last_change":"2024-09-21T02:22:22.484153+0000","last_active":"2024-09-21T02:22:23.488908+0000","last_peered":"2024-09-21T02:22:23.488908+0000","last_clean":"2024-09-21T02:22:23.488908+0000","last_became_active":"2024-09-21T02:22:22.483794+0000","last_became_peered":"2024-09-21T02:22:22.483794+0000","last_unstale":"2024-09-21T02:22:23.488908+0000","last_undegraded":"2024-09-21T02:22:23.488908+0000","last_fullsized":"2024-09-21T02:22:23.488908+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:02:24.330388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438579+0000","last_change":"2024-09-21T02:22:22.454233+0000","last_active":"2024-09-21T02:22:23.438579+0000","last_peered":"2024-09-21T02:22:23.438579+0000","last_clean":"2024-09-21T02:22:23.438579+0000","last_became_active":"2024-09-21T02:22:22.452979+0000","last_became_peered":"2024-09-21T02:22:22.452979+0000","last_unstale":"2024-09-21T02:22:23.438579+0000","last_undegraded":"2024-09-21T02:22:23.438579+0000","last_fullsized":"2024-09-21T02:22:23.438579+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:35.243629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486719+0000","last_change":"2024-09-21T02:22:22.466506+0000","last_active":"2024-09-21T02:22:23.486719+0000","last_peered":"2024-09-21T02:22:23.486719+0000","last_clean":"2024-09-21T02:22:23.486719+0000","last_became_active":"2024-09-21T02:22:22.466159+0000","last_became_peered":"2024-09-21T02:22:22.466159+0000","last_unstale":"2024-09-21T02:22:23.486719+0000","last_undegraded":"2024-09-21T02:22:23.486719+0000","last_fullsized":"2024-09-21T02:22:23.486719+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:48.442960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438194+0000","last_change":"2024-09-21T02:22:22.462364+0000","last_active":"2024-09-21T02:22:23.438194+0000","last_peered":"2024-09-21T02:22:23.438194+0000","last_clean":"2024-09-21T02:22:23.438194+0000","last_became_active":"2024-09-21T02:22:22.461478+0000","last_became_peered":"2024-09-21T02:22:22.461478+0000","last_unstale":"2024-09-21T02:22:23.438194+0000","last_undegraded":"2024-09-21T02:22:23.438194+0000","last_fullsized":"2024-09-21T02:22:23.438194+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:44.914958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433490+0000","last_change":"2024-09-21T02:22:22.453604+0000","last_active":"2024-09-21T02:22:23.433490+0000","last_peered":"2024-09-21T02:22:23.433490+0000","last_clean":"2024-09-21T02:22:23.433490+0000","last_became_active":"2024-09-21T02:22:22.453307+0000","last_became_peered":"2024-09-21T02:22:22.453307+0000","last_unstale":"2024-09-21T02:22:23.433490+0000","last_undegraded":"2024-09-21T02:22:23.433490+0000","last_fullsized":"2024-09-21T02:22:23.433490+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:43.889219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488487+0000","last_change":"2024-09-21T02:22:22.484753+0000","last_active":"2024-09-21T02:22:23.488487+0000","last_peered":"2024-09-21T02:22:23.488487+0000","last_clean":"2024-09-21T02:22:23.488487+0000","last_became_active":"2024-09-21T02:22:22.484443+0000","last_became_peered":"2024-09-21T02:22:22.484443+0000","last_unstale":"2024-09-21T02:22:23.488487+0000","last_undegraded":"2024-09-21T02:22:23.488487+0000","last_fullsized":"2024-09-21T02:22:23.488487+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:30.654175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489181+0000","last_change":"2024-09-21T02:22:22.496770+0000","last_active":"2024-09-21T02:22:23.489181+0000","last_peered":"2024-09-21T02:22:23.489181+0000","last_clean":"2024-09-21T02:22:23.489181+0000","last_became_active":"2024-09-21T02:22:22.495967+0000","last_became_peered":"2024-09-21T02:22:22.495967+0000","last_unstale":"2024-09-21T02:22:23.489181+0000","last_undegraded":"2024-09-21T02:22:23.489181+0000","last_fullsized":"2024-09-21T02:22:23.489181+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:21.738039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488586+0000","last_change":"2024-09-21T02:22:22.479588+0000","last_active":"2024-09-21T02:22:23.488586+0000","last_peered":"2024-09-21T02:22:23.488586+0000","last_clean":"2024-09-21T02:22:23.488586+0000","last_became_active":"2024-09-21T02:22:22.478636+0000","last_became_peered":"2024-09-21T02:22:22.478636+0000","last_unstale":"2024-09-21T02:22:23.488586+0000","last_undegraded":"2024-09-21T02:22:23.488586+0000","last_fullsized":"2024-09-21T02:22:23.488586+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:11.247218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486768+0000","last_change":"2024-09-21T02:22:22.468975+0000","last_active":"2024-09-21T02:22:23.486768+0000","last_peered":"2024-09-21T02:22:23.486768+0000","last_clean":"2024-09-21T02:22:23.486768+0000","last_became_active":"2024-09-21T02:22:22.468642+0000","last_became_peered":"2024-09-21T02:22:22.468642+0000","last_unstale":"2024-09-21T02:22:23.486768+0000","last_undegraded":"2024-09-21T02:22:23.486768+0000","last_fullsized":"2024-09-21T02:22:23.486768+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:59:32.303310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"165'4","reported_seq":19,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486308+0000","last_change":"2024-09-21T02:22:21.481003+0000","last_active":"2024-09-21T02:22:23.486308+0000","last_peered":"2024-09-21T02:22:23.486308+0000","last_clean":"2024-09-21T02:22:23.486308+0000","last_became_active":"2024-09-21T02:22:21.480530+0000","last_became_peered":"2024-09-21T02:22:21.480530+0000","last_unstale":"2024-09-21T02:22:23.486308+0000","last_undegraded":"2024-09-21T02:22:23.486308+0000","last_fullsized":"2024-09-21T02:22:23.486308+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:21.830021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488690+0000","last_change":"2024-09-21T02:22:22.475603+0000","last_active":"2024-09-21T02:22:23.488690+0000","last_peered":"2024-09-21T02:22:23.488690+0000","last_clean":"2024-09-21T02:22:23.488690+0000","last_became_active":"2024-09-21T02:22:22.475158+0000","last_became_peered":"2024-09-21T02:22:22.475158+0000","last_unstale":"2024-09-21T02:22:23.488690+0000","last_undegraded":"2024-09-21T02:22:23.488690+0000","last_fullsized":"2024-09-21T02:22:23.488690+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:18.517560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433538+0000","last_change":"2024-09-21T02:22:21.450012+0000","last_active":"2024-09-21T02:22:23.433538+0000","last_peered":"2024-09-21T02:22:23.433538+0000","last_clean":"2024-09-21T02:22:23.433538+0000","last_became_active":"2024-09-21T02:22:21.449316+0000","last_became_peered":"2024-09-21T02:22:21.449316+0000","last_unstale":"2024-09-21T02:22:23.433538+0000","last_undegraded":"2024-09-21T02:22:23.433538+0000","last_fullsized":"2024-09-21T02:22:23.433538+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:19.975657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490364+0000","last_change":"2024-09-21T02:22:22.501226+0000","last_active":"2024-09-21T02:22:23.490364+0000","last_peered":"2024-09-21T02:22:23.490364+0000","last_clean":"2024-09-21T02:22:23.490364+0000","last_became_active":"2024-09-21T02:22:22.498228+0000","last_became_peered":"2024-09-21T02:22:22.498228+0000","last_unstale":"2024-09-21T02:22:23.490364+0000","last_undegraded":"2024-09-21T02:22:23.490364+0000","last_fullsized":"2024-09-21T02:22:23.490364+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:09.652755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440551+0000","last_change":"2024-09-21T02:22:21.470893+0000","last_active":"2024-09-21T02:22:23.440551+0000","last_peered":"2024-09-21T02:22:23.440551+0000","last_clean":"2024-09-21T02:22:23.440551+0000","last_became_active":"2024-09-21T02:22:21.470125+0000","last_became_peered":"2024-09-21T02:22:21.470125+0000","last_unstale":"2024-09-21T02:22:23.440551+0000","last_undegraded":"2024-09-21T02:22:23.440551+0000","last_fullsized":"2024-09-21T02:22:23.440551+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:09.343888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492556+0000","last_change":"2024-09-21T02:22:22.504645+0000","last_active":"2024-09-21T02:22:23.492556+0000","last_peered":"2024-09-21T02:22:23.492556+0000","last_clean":"2024-09-21T02:22:23.492556+0000","last_became_active":"2024-09-21T02:22:22.503399+0000","last_became_peered":"2024-09-21T02:22:22.503399+0000","last_unstale":"2024-09-21T02:22:23.492556+0000","last_undegraded":"2024-09-21T02:22:23.492556+0000","last_fullsized":"2024-09-21T02:22:23.492556+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:09.331075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489280+0000","last_change":"2024-09-21T02:22:21.445985+0000","last_active":"2024-09-21T02:22:23.489280+0000","last_peered":"2024-09-21T02:22:23.489280+0000","last_clean":"2024-09-21T02:22:23.489280+0000","last_became_active":"2024-09-21T02:22:21.445023+0000","last_became_peered":"2024-09-21T02:22:21.445023+0000","last_unstale":"2024-09-21T02:22:23.489280+0000","last_undegraded":"2024-09-21T02:22:23.489280+0000","last_fullsized":"2024-09-21T02:22:23.489280+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:27.025700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488811+0000","last_change":"2024-09-21T02:22:22.496514+0000","last_active":"2024-09-21T02:22:23.488811+0000","last_peered":"2024-09-21T02:22:23.488811+0000","last_clean":"2024-09-21T02:22:23.488811+0000","last_became_active":"2024-09-21T02:22:22.496060+0000","last_became_peered":"2024-09-21T02:22:22.496060+0000","last_unstale":"2024-09-21T02:22:23.488811+0000","last_undegraded":"2024-09-21T02:22:23.488811+0000","last_fullsized":"2024-09-21T02:22:23.488811+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:49.474503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439631+0000","last_change":"2024-09-21T02:22:21.479742+0000","last_active":"2024-09-21T02:22:23.439631+0000","last_peered":"2024-09-21T02:22:23.439631+0000","last_clean":"2024-09-21T02:22:23.439631+0000","last_became_active":"2024-09-21T02:22:21.478735+0000","last_became_peered":"2024-09-21T02:22:21.478735+0000","last_unstale":"2024-09-21T02:22:23.439631+0000","last_undegraded":"2024-09-21T02:22:23.439631+0000","last_fullsized":"2024-09-21T02:22:23.439631+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:08.648916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437032+0000","last_change":"2024-09-21T02:22:22.460708+0000","last_active":"2024-09-21T02:22:23.437032+0000","last_peered":"2024-09-21T02:22:23.437032+0000","last_clean":"2024-09-21T02:22:23.437032+0000","last_became_active":"2024-09-21T02:22:22.459772+0000","last_became_peered":"2024-09-21T02:22:22.459772+0000","last_unstale":"2024-09-21T02:22:23.437032+0000","last_undegraded":"2024-09-21T02:22:23.437032+0000","last_fullsized":"2024-09-21T02:22:23.437032+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:06.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438403+0000","last_change":"2024-09-21T02:22:21.476224+0000","last_active":"2024-09-21T02:22:23.438403+0000","last_peered":"2024-09-21T02:22:23.438403+0000","last_clean":"2024-09-21T02:22:23.438403+0000","last_became_active":"2024-09-21T02:22:21.474796+0000","last_became_peered":"2024-09-21T02:22:21.474796+0000","last_unstale":"2024-09-21T02:22:23.438403+0000","last_undegraded":"2024-09-21T02:22:23.438403+0000","last_fullsized":"2024-09-21T02:22:23.438403+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:18.592314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481616+0000","last_change":"2024-09-21T02:22:22.457333+0000","last_active":"2024-09-21T02:22:23.481616+0000","last_peered":"2024-09-21T02:22:23.481616+0000","last_clean":"2024-09-21T02:22:23.481616+0000","last_became_active":"2024-09-21T02:22:22.456486+0000","last_became_peered":"2024-09-21T02:22:22.456486+0000","last_unstale":"2024-09-21T02:22:23.481616+0000","last_undegraded":"2024-09-21T02:22:23.481616+0000","last_fullsized":"2024-09-21T02:22:23.481616+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:38.550121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488018+0000","last_change":"2024-09-21T02:22:21.465616+0000","last_active":"2024-09-21T02:22:23.488018+0000","last_peered":"2024-09-21T02:22:23.488018+0000","last_clean":"2024-09-21T02:22:23.488018+0000","last_became_active":"2024-09-21T02:22:21.465312+0000","last_became_peered":"2024-09-21T02:22:21.465312+0000","last_unstale":"2024-09-21T02:22:23.488018+0000","last_undegraded":"2024-09-21T02:22:23.488018+0000","last_fullsized":"2024-09-21T02:22:23.488018+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:43.898880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488975+0000","last_change":"2024-09-21T02:22:22.485325+0000","last_active":"2024-09-21T02:22:23.488975+0000","last_peered":"2024-09-21T02:22:23.488975+0000","last_clean":"2024-09-21T02:22:23.488975+0000","last_became_active":"2024-09-21T02:22:22.483953+0000","last_became_peered":"2024-09-21T02:22:22.483953+0000","last_unstale":"2024-09-21T02:22:23.488975+0000","last_undegraded":"2024-09-21T02:22:23.488975+0000","last_fullsized":"2024-09-21T02:22:23.488975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:42:59.711639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.492576+0000","last_change":"2024-09-21T02:22:21.474042+0000","last_active":"2024-09-21T02:22:23.492576+0000","last_peered":"2024-09-21T02:22:23.492576+0000","last_clean":"2024-09-21T02:22:23.492576+0000","last_became_active":"2024-09-21T02:22:21.473281+0000","last_became_peered":"2024-09-21T02:22:21.473281+0000","last_unstale":"2024-09-21T02:22:23.492576+0000","last_undegraded":"2024-09-21T02:22:23.492576+0000","last_fullsized":"2024-09-21T02:22:23.492576+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:41.069296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489324+0000","last_change":"2024-09-21T02:22:22.474638+0000","last_active":"2024-09-21T02:22:23.489324+0000","last_peered":"2024-09-21T02:22:23.489324+0000","last_clean":"2024-09-21T02:22:23.489324+0000","last_became_active":"2024-09-21T02:22:22.473231+0000","last_became_peered":"2024-09-21T02:22:22.473231+0000","last_unstale":"2024-09-21T02:22:23.489324+0000","last_undegraded":"2024-09-21T02:22:23.489324+0000","last_fullsized":"2024-09-21T02:22:23.489324+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:12.827287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489184+0000","last_change":"2024-09-21T02:22:21.448461+0000","last_active":"2024-09-21T02:22:23.489184+0000","last_peered":"2024-09-21T02:22:23.489184+0000","last_clean":"2024-09-21T02:22:23.489184+0000","last_became_active":"2024-09-21T02:22:21.447720+0000","last_became_peered":"2024-09-21T02:22:21.447720+0000","last_unstale":"2024-09-21T02:22:23.489184+0000","last_undegraded":"2024-09-21T02:22:23.489184+0000","last_fullsized":"2024-09-21T02:22:23.489184+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:19.892947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481975+0000","last_change":"2024-09-21T02:22:22.468831+0000","last_active":"2024-09-21T02:22:23.481975+0000","last_peered":"2024-09-21T02:22:23.481975+0000","last_clean":"2024-09-21T02:22:23.481975+0000","last_became_active":"2024-09-21T02:22:22.463670+0000","last_became_peered":"2024-09-21T02:22:22.463670+0000","last_unstale":"2024-09-21T02:22:23.481975+0000","last_undegraded":"2024-09-21T02:22:23.481975+0000","last_fullsized":"2024-09-21T02:22:23.481975+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:51.120742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"165'3","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.440895+0000","last_change":"2024-09-21T02:22:21.481467+0000","last_active":"2024-09-21T02:22:23.440895+0000","last_peered":"2024-09-21T02:22:23.440895+0000","last_clean":"2024-09-21T02:22:23.440895+0000","last_became_active":"2024-09-21T02:22:21.480582+0000","last_became_peered":"2024-09-21T02:22:21.480582+0000","last_unstale":"2024-09-21T02:22:23.440895+0000","last_undegraded":"2024-09-21T02:22:23.440895+0000","last_fullsized":"2024-09-21T02:22:23.440895+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:30.217816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439942+0000","last_change":"2024-09-21T02:22:22.467611+0000","last_active":"2024-09-21T02:22:23.439942+0000","last_peered":"2024-09-21T02:22:23.439942+0000","last_clean":"2024-09-21T02:22:23.439942+0000","last_became_active":"2024-09-21T02:22:22.466706+0000","last_became_peered":"2024-09-21T02:22:22.466706+0000","last_unstale":"2024-09-21T02:22:23.439942+0000","last_undegraded":"2024-09-21T02:22:23.439942+0000","last_fullsized":"2024-09-21T02:22:23.439942+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:12:41.744685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"165'6","reported_seq":21,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481728+0000","last_change":"2024-09-21T02:22:21.448858+0000","last_active":"2024-09-21T02:22:23.481728+0000","last_peered":"2024-09-21T02:22:23.481728+0000","last_clean":"2024-09-21T02:22:23.481728+0000","last_became_active":"2024-09-21T02:22:21.448497+0000","last_became_peered":"2024-09-21T02:22:21.448497+0000","last_unstale":"2024-09-21T02:22:23.481728+0000","last_undegraded":"2024-09-21T02:22:23.481728+0000","last_fullsized":"2024-09-21T02:22:23.481728+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:20.373492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491444+0000","last_change":"2024-09-21T02:22:22.481160+0000","last_active":"2024-09-21T02:22:23.491444+0000","last_peered":"2024-09-21T02:22:23.491444+0000","last_clean":"2024-09-21T02:22:23.491444+0000","last_became_active":"2024-09-21T02:22:22.480341+0000","last_became_peered":"2024-09-21T02:22:22.480341+0000","last_unstale":"2024-09-21T02:22:23.491444+0000","last_undegraded":"2024-09-21T02:22:23.491444+0000","last_fullsized":"2024-09-21T02:22:23.491444+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:26.627544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481852+0000","last_change":"2024-09-21T02:22:21.451094+0000","last_active":"2024-09-21T02:22:23.481852+0000","last_peered":"2024-09-21T02:22:23.481852+0000","last_clean":"2024-09-21T02:22:23.481852+0000","last_became_active":"2024-09-21T02:22:21.450270+0000","last_became_peered":"2024-09-21T02:22:21.450270+0000","last_unstale":"2024-09-21T02:22:23.481852+0000","last_undegraded":"2024-09-21T02:22:23.481852+0000","last_fullsized":"2024-09-21T02:22:23.481852+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:14.367736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491626+0000","last_change":"2024-09-21T02:22:22.502467+0000","last_active":"2024-09-21T02:22:23.491626+0000","last_peered":"2024-09-21T02:22:23.491626+0000","last_clean":"2024-09-21T02:22:23.491626+0000","last_became_active":"2024-09-21T02:22:22.501820+0000","last_became_peered":"2024-09-21T02:22:22.501820+0000","last_unstale":"2024-09-21T02:22:23.491626+0000","last_undegraded":"2024-09-21T02:22:23.491626+0000","last_fullsized":"2024-09-21T02:22:23.491626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:47.906300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433515+0000","last_change":"2024-09-21T02:22:21.469222+0000","last_active":"2024-09-21T02:22:23.433515+0000","last_peered":"2024-09-21T02:22:23.433515+0000","last_clean":"2024-09-21T02:22:23.433515+0000","last_became_active":"2024-09-21T02:22:21.468703+0000","last_became_peered":"2024-09-21T02:22:21.468703+0000","last_unstale":"2024-09-21T02:22:23.433515+0000","last_undegraded":"2024-09-21T02:22:23.433515+0000","last_fullsized":"2024-09-21T02:22:23.433515+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:31.233975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489458+0000","last_change":"2024-09-21T02:22:22.497046+0000","last_active":"2024-09-21T02:22:23.489458+0000","last_peered":"2024-09-21T02:22:23.489458+0000","last_clean":"2024-09-21T02:22:23.489458+0000","last_became_active":"2024-09-21T02:22:22.495953+0000","last_became_peered":"2024-09-21T02:22:22.495953+0000","last_unstale":"2024-09-21T02:22:23.489458+0000","last_undegraded":"2024-09-21T02:22:23.489458+0000","last_fullsized":"2024-09-21T02:22:23.489458+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:48.894869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438698+0000","last_change":"2024-09-21T02:22:21.461064+0000","last_active":"2024-09-21T02:22:23.438698+0000","last_peered":"2024-09-21T02:22:23.438698+0000","last_clean":"2024-09-21T02:22:23.438698+0000","last_became_active":"2024-09-21T02:22:21.460127+0000","last_became_peered":"2024-09-21T02:22:21.460127+0000","last_unstale":"2024-09-21T02:22:23.438698+0000","last_undegraded":"2024-09-21T02:22:23.438698+0000","last_fullsized":"2024-09-21T02:22:23.438698+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:58.845230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437145+0000","last_change":"2024-09-21T02:22:22.448417+0000","last_active":"2024-09-21T02:22:23.437145+0000","last_peered":"2024-09-21T02:22:23.437145+0000","last_clean":"2024-09-21T02:22:23.437145+0000","last_became_active":"2024-09-21T02:22:22.447355+0000","last_became_peered":"2024-09-21T02:22:22.447355+0000","last_unstale":"2024-09-21T02:22:23.437145+0000","last_undegraded":"2024-09-21T02:22:23.437145+0000","last_fullsized":"2024-09-21T02:22:23.437145+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:26.961413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"165'8","reported_seq":23,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437721+0000","last_change":"2024-09-21T02:22:21.468968+0000","last_active":"2024-09-21T02:22:23.437721+0000","last_peered":"2024-09-21T02:22:23.437721+0000","last_clean":"2024-09-21T02:22:23.437721+0000","last_became_active":"2024-09-21T02:22:21.468178+0000","last_became_peered":"2024-09-21T02:22:21.468178+0000","last_unstale":"2024-09-21T02:22:23.437721+0000","last_undegraded":"2024-09-21T02:22:23.437721+0000","last_fullsized":"2024-09-21T02:22:23.437721+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:54.424298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486627+0000","last_change":"2024-09-21T02:22:22.486901+0000","last_active":"2024-09-21T02:22:23.486627+0000","last_peered":"2024-09-21T02:22:23.486627+0000","last_clean":"2024-09-21T02:22:23.486627+0000","last_became_active":"2024-09-21T02:22:22.486512+0000","last_became_peered":"2024-09-21T02:22:22.486512+0000","last_unstale":"2024-09-21T02:22:23.486627+0000","last_undegraded":"2024-09-21T02:22:23.486627+0000","last_fullsized":"2024-09-21T02:22:23.486627+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:05.678401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.442557+0000","last_change":"2024-09-21T02:22:21.479020+0000","last_active":"2024-09-21T02:22:23.442557+0000","last_peered":"2024-09-21T02:22:23.442557+0000","last_clean":"2024-09-21T02:22:23.442557+0000","last_became_active":"2024-09-21T02:22:21.475721+0000","last_became_peered":"2024-09-21T02:22:21.475721+0000","last_unstale":"2024-09-21T02:22:23.442557+0000","last_undegraded":"2024-09-21T02:22:23.442557+0000","last_fullsized":"2024-09-21T02:22:23.442557+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:42:47.891029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488927+0000","last_change":"2024-09-21T02:22:22.476386+0000","last_active":"2024-09-21T02:22:23.488927+0000","last_peered":"2024-09-21T02:22:23.488927+0000","last_clean":"2024-09-21T02:22:23.488927+0000","last_became_active":"2024-09-21T02:22:22.475984+0000","last_became_peered":"2024-09-21T02:22:22.475984+0000","last_unstale":"2024-09-21T02:22:23.488927+0000","last_undegraded":"2024-09-21T02:22:23.488927+0000","last_fullsized":"2024-09-21T02:22:23.488927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:22:26.379696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.481861+0000","last_change":"2024-09-21T02:22:21.470128+0000","last_active":"2024-09-21T02:22:23.481861+0000","last_peered":"2024-09-21T02:22:23.481861+0000","last_clean":"2024-09-21T02:22:23.481861+0000","last_became_active":"2024-09-21T02:22:21.469768+0000","last_became_peered":"2024-09-21T02:22:21.469768+0000","last_unstale":"2024-09-21T02:22:23.481861+0000","last_undegraded":"2024-09-21T02:22:23.481861+0000","last_fullsized":"2024-09-21T02:22:23.481861+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:44.008372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491046+0000","last_change":"2024-09-21T02:22:22.508645+0000","last_active":"2024-09-21T02:22:23.491046+0000","last_peered":"2024-09-21T02:22:23.491046+0000","last_clean":"2024-09-21T02:22:23.491046+0000","last_became_active":"2024-09-21T02:22:22.507972+0000","last_became_peered":"2024-09-21T02:22:22.507972+0000","last_unstale":"2024-09-21T02:22:23.491046+0000","last_undegraded":"2024-09-21T02:22:23.491046+0000","last_fullsized":"2024-09-21T02:22:23.491046+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:18.675323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490199+0000","last_change":"2024-09-21T02:22:21.453206+0000","last_active":"2024-09-21T02:22:23.490199+0000","last_peered":"2024-09-21T02:22:23.490199+0000","last_clean":"2024-09-21T02:22:23.490199+0000","last_became_active":"2024-09-21T02:22:21.452175+0000","last_became_peered":"2024-09-21T02:22:21.452175+0000","last_unstale":"2024-09-21T02:22:23.490199+0000","last_undegraded":"2024-09-21T02:22:23.490199+0000","last_fullsized":"2024-09-21T02:22:23.490199+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:03.484364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.441150+0000","last_change":"2024-09-21T02:22:22.473490+0000","last_active":"2024-09-21T02:22:23.441150+0000","last_peered":"2024-09-21T02:22:23.441150+0000","last_clean":"2024-09-21T02:22:23.441150+0000","last_became_active":"2024-09-21T02:22:22.465332+0000","last_became_peered":"2024-09-21T02:22:22.465332+0000","last_unstale":"2024-09-21T02:22:23.441150+0000","last_undegraded":"2024-09-21T02:22:23.441150+0000","last_fullsized":"2024-09-21T02:22:23.441150+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:52.893313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438629+0000","last_change":"2024-09-21T02:22:21.474322+0000","last_active":"2024-09-21T02:22:23.438629+0000","last_peered":"2024-09-21T02:22:23.438629+0000","last_clean":"2024-09-21T02:22:23.438629+0000","last_became_active":"2024-09-21T02:22:21.473688+0000","last_became_peered":"2024-09-21T02:22:21.473688+0000","last_unstale":"2024-09-21T02:22:23.438629+0000","last_undegraded":"2024-09-21T02:22:23.438629+0000","last_fullsized":"2024-09-21T02:22:23.438629+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:28.859889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490966+0000","last_change":"2024-09-21T02:22:22.508860+0000","last_active":"2024-09-21T02:22:23.490966+0000","last_peered":"2024-09-21T02:22:23.490966+0000","last_clean":"2024-09-21T02:22:23.490966+0000","last_became_active":"2024-09-21T02:22:22.508254+0000","last_became_peered":"2024-09-21T02:22:22.508254+0000","last_unstale":"2024-09-21T02:22:23.490966+0000","last_undegraded":"2024-09-21T02:22:23.490966+0000","last_fullsized":"2024-09-21T02:22:23.490966+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:58.267000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488961+0000","last_change":"2024-09-21T02:22:21.463499+0000","last_active":"2024-09-21T02:22:23.488961+0000","last_peered":"2024-09-21T02:22:23.488961+0000","last_clean":"2024-09-21T02:22:23.488961+0000","last_became_active":"2024-09-21T02:22:21.462105+0000","last_became_peered":"2024-09-21T02:22:21.462105+0000","last_unstale":"2024-09-21T02:22:23.488961+0000","last_undegraded":"2024-09-21T02:22:23.488961+0000","last_fullsized":"2024-09-21T02:22:23.488961+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:47.244243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439521+0000","last_change":"2024-09-21T02:22:22.472217+0000","last_active":"2024-09-21T02:22:23.439521+0000","last_peered":"2024-09-21T02:22:23.439521+0000","last_clean":"2024-09-21T02:22:23.439521+0000","last_became_active":"2024-09-21T02:22:22.471470+0000","last_became_peered":"2024-09-21T02:22:22.471470+0000","last_unstale":"2024-09-21T02:22:23.439521+0000","last_undegraded":"2024-09-21T02:22:23.439521+0000","last_fullsized":"2024-09-21T02:22:23.439521+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:49.362864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437790+0000","last_change":"2024-09-21T02:22:21.470671+0000","last_active":"2024-09-21T02:22:23.437790+0000","last_peered":"2024-09-21T02:22:23.437790+0000","last_clean":"2024-09-21T02:22:23.437790+0000","last_became_active":"2024-09-21T02:22:21.469843+0000","last_became_peered":"2024-09-21T02:22:21.469843+0000","last_unstale":"2024-09-21T02:22:23.437790+0000","last_undegraded":"2024-09-21T02:22:23.437790+0000","last_fullsized":"2024-09-21T02:22:23.437790+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:33.235495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488322+0000","last_change":"2024-09-21T02:22:22.476020+0000","last_active":"2024-09-21T02:22:23.488322+0000","last_peered":"2024-09-21T02:22:23.488322+0000","last_clean":"2024-09-21T02:22:23.488322+0000","last_became_active":"2024-09-21T02:22:22.475427+0000","last_became_peered":"2024-09-21T02:22:22.475427+0000","last_unstale":"2024-09-21T02:22:23.488322+0000","last_undegraded":"2024-09-21T02:22:23.488322+0000","last_fullsized":"2024-09-21T02:22:23.488322+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:29.399926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489757+0000","last_change":"2024-09-21T02:22:21.474582+0000","last_active":"2024-09-21T02:22:23.489757+0000","last_peered":"2024-09-21T02:22:23.489757+0000","last_clean":"2024-09-21T02:22:23.489757+0000","last_became_active":"2024-09-21T02:22:21.473601+0000","last_became_peered":"2024-09-21T02:22:21.473601+0000","last_unstale":"2024-09-21T02:22:23.489757+0000","last_undegraded":"2024-09-21T02:22:23.489757+0000","last_fullsized":"2024-09-21T02:22:23.489757+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:50.116902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489550+0000","last_change":"2024-09-21T02:22:22.474853+0000","last_active":"2024-09-21T02:22:23.489550+0000","last_peered":"2024-09-21T02:22:23.489550+0000","last_clean":"2024-09-21T02:22:23.489550+0000","last_became_active":"2024-09-21T02:22:22.462271+0000","last_became_peered":"2024-09-21T02:22:22.462271+0000","last_unstale":"2024-09-21T02:22:23.489550+0000","last_undegraded":"2024-09-21T02:22:23.489550+0000","last_fullsized":"2024-09-21T02:22:23.489550+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:09.589293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438172+0000","last_change":"2024-09-21T02:22:21.481357+0000","last_active":"2024-09-21T02:22:23.438172+0000","last_peered":"2024-09-21T02:22:23.438172+0000","last_clean":"2024-09-21T02:22:23.438172+0000","last_became_active":"2024-09-21T02:22:21.480659+0000","last_became_peered":"2024-09-21T02:22:21.480659+0000","last_unstale":"2024-09-21T02:22:23.438172+0000","last_undegraded":"2024-09-21T02:22:23.438172+0000","last_fullsized":"2024-09-21T02:22:23.438172+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:30.738468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486795+0000","last_change":"2024-09-21T02:22:22.447997+0000","last_active":"2024-09-21T02:22:23.486795+0000","last_peered":"2024-09-21T02:22:23.486795+0000","last_clean":"2024-09-21T02:22:23.486795+0000","last_became_active":"2024-09-21T02:22:22.447015+0000","last_became_peered":"2024-09-21T02:22:22.447015+0000","last_unstale":"2024-09-21T02:22:23.486795+0000","last_undegraded":"2024-09-21T02:22:23.486795+0000","last_fullsized":"2024-09-21T02:22:23.486795+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:38.617086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"165'2","reported_seq":17,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.436812+0000","last_change":"2024-09-21T02:22:21.456293+0000","last_active":"2024-09-21T02:22:23.436812+0000","last_peered":"2024-09-21T02:22:23.436812+0000","last_clean":"2024-09-21T02:22:23.436812+0000","last_became_active":"2024-09-21T02:22:21.455368+0000","last_became_peered":"2024-09-21T02:22:21.455368+0000","last_unstale":"2024-09-21T02:22:23.436812+0000","last_undegraded":"2024-09-21T02:22:23.436812+0000","last_fullsized":"2024-09-21T02:22:23.436812+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:49.978614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.441623+0000","last_change":"2024-09-21T02:22:22.451994+0000","last_active":"2024-09-21T02:22:23.441623+0000","last_peered":"2024-09-21T02:22:23.441623+0000","last_clean":"2024-09-21T02:22:23.441623+0000","last_became_active":"2024-09-21T02:22:22.451174+0000","last_became_peered":"2024-09-21T02:22:22.451174+0000","last_unstale":"2024-09-21T02:22:23.441623+0000","last_undegraded":"2024-09-21T02:22:23.441623+0000","last_fullsized":"2024-09-21T02:22:23.441623+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:07.426362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"166'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.480541+0000","last_change":"2024-09-21T02:22:21.454116+0000","last_active":"2024-09-21T02:22:23.480541+0000","last_peered":"2024-09-21T02:22:23.480541+0000","last_clean":"2024-09-21T02:22:23.480541+0000","last_became_active":"2024-09-21T02:22:21.453802+0000","last_became_peered":"2024-09-21T02:22:21.453802+0000","last_unstale":"2024-09-21T02:22:23.480541+0000","last_undegraded":"2024-09-21T02:22:23.480541+0000","last_fullsized":"2024-09-21T02:22:23.480541+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:51.978522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433413+0000","last_change":"2024-09-21T02:22:22.454925+0000","last_active":"2024-09-21T02:22:23.433413+0000","last_peered":"2024-09-21T02:22:23.433413+0000","last_clean":"2024-09-21T02:22:23.433413+0000","last_became_active":"2024-09-21T02:22:22.454050+0000","last_became_peered":"2024-09-21T02:22:22.454050+0000","last_unstale":"2024-09-21T02:22:23.433413+0000","last_undegraded":"2024-09-21T02:22:23.433413+0000","last_fullsized":"2024-09-21T02:22:23.433413+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:05.433936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.441325+0000","last_change":"2024-09-21T02:22:21.471338+0000","last_active":"2024-09-21T02:22:23.441325+0000","last_peered":"2024-09-21T02:22:23.441325+0000","last_clean":"2024-09-21T02:22:23.441325+0000","last_became_active":"2024-09-21T02:22:21.470418+0000","last_became_peered":"2024-09-21T02:22:21.470418+0000","last_unstale":"2024-09-21T02:22:23.441325+0000","last_undegraded":"2024-09-21T02:22:23.441325+0000","last_fullsized":"2024-09-21T02:22:23.441325+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:19:17.671483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486612+0000","last_change":"2024-09-21T02:22:22.468019+0000","last_active":"2024-09-21T02:22:23.486612+0000","last_peered":"2024-09-21T02:22:23.486612+0000","last_clean":"2024-09-21T02:22:23.486612+0000","last_became_active":"2024-09-21T02:22:22.467677+0000","last_became_peered":"2024-09-21T02:22:22.467677+0000","last_unstale":"2024-09-21T02:22:23.486612+0000","last_undegraded":"2024-09-21T02:22:23.486612+0000","last_fullsized":"2024-09-21T02:22:23.486612+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:03:26.561468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.438739+0000","last_change":"2024-09-21T02:22:21.453344+0000","last_active":"2024-09-21T02:22:23.438739+0000","last_peered":"2024-09-21T02:22:23.438739+0000","last_clean":"2024-09-21T02:22:23.438739+0000","last_became_active":"2024-09-21T02:22:21.452601+0000","last_became_peered":"2024-09-21T02:22:21.452601+0000","last_unstale":"2024-09-21T02:22:23.438739+0000","last_undegraded":"2024-09-21T02:22:23.438739+0000","last_fullsized":"2024-09-21T02:22:23.438739+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:30:20.734200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.439440+0000","last_change":"2024-09-21T02:22:22.461537+0000","last_active":"2024-09-21T02:22:23.439440+0000","last_peered":"2024-09-21T02:22:23.439440+0000","last_clean":"2024-09-21T02:22:23.439440+0000","last_became_active":"2024-09-21T02:22:22.459891+0000","last_became_peered":"2024-09-21T02:22:22.459891+0000","last_unstale":"2024-09-21T02:22:23.439440+0000","last_undegraded":"2024-09-21T02:22:23.439440+0000","last_fullsized":"2024-09-21T02:22:23.439440+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:08.668744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488930+0000","last_change":"2024-09-21T02:22:21.462431+0000","last_active":"2024-09-21T02:22:23.488930+0000","last_peered":"2024-09-21T02:22:23.488930+0000","last_clean":"2024-09-21T02:22:23.488930+0000","last_became_active":"2024-09-21T02:22:21.461925+0000","last_became_peered":"2024-09-21T02:22:21.461925+0000","last_unstale":"2024-09-21T02:22:23.488930+0000","last_undegraded":"2024-09-21T02:22:23.488930+0000","last_fullsized":"2024-09-21T02:22:23.488930+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:20.461260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433724+0000","last_change":"2024-09-21T02:22:22.471896+0000","last_active":"2024-09-21T02:22:23.433724+0000","last_peered":"2024-09-21T02:22:23.433724+0000","last_clean":"2024-09-21T02:22:23.433724+0000","last_became_active":"2024-09-21T02:22:22.471020+0000","last_became_peered":"2024-09-21T02:22:22.471020+0000","last_unstale":"2024-09-21T02:22:23.433724+0000","last_undegraded":"2024-09-21T02:22:23.433724+0000","last_fullsized":"2024-09-21T02:22:23.433724+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:28.861772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.491265+0000","last_change":"2024-09-21T02:22:21.476491+0000","last_active":"2024-09-21T02:22:23.491265+0000","last_peered":"2024-09-21T02:22:23.491265+0000","last_clean":"2024-09-21T02:22:23.491265+0000","last_became_active":"2024-09-21T02:22:21.475140+0000","last_became_peered":"2024-09-21T02:22:21.475140+0000","last_unstale":"2024-09-21T02:22:23.491265+0000","last_undegraded":"2024-09-21T02:22:23.491265+0000","last_fullsized":"2024-09-21T02:22:23.491265+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:56.225401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.437378+0000","last_change":"2024-09-21T02:22:22.452173+0000","last_active":"2024-09-21T02:22:23.437378+0000","last_peered":"2024-09-21T02:22:23.437378+0000","last_clean":"2024-09-21T02:22:23.437378+0000","last_became_active":"2024-09-21T02:22:22.450075+0000","last_became_peered":"2024-09-21T02:22:22.450075+0000","last_unstale":"2024-09-21T02:22:23.437378+0000","last_undegraded":"2024-09-21T02:22:23.437378+0000","last_fullsized":"2024-09-21T02:22:23.437378+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:25.144752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.486415+0000","last_change":"2024-09-21T02:22:21.454568+0000","last_active":"2024-09-21T02:22:23.486415+0000","last_peered":"2024-09-21T02:22:23.486415+0000","last_clean":"2024-09-21T02:22:23.486415+0000","last_became_active":"2024-09-21T02:22:21.454228+0000","last_became_peered":"2024-09-21T02:22:21.454228+0000","last_unstale":"2024-09-21T02:22:23.486415+0000","last_undegraded":"2024-09-21T02:22:23.486415+0000","last_fullsized":"2024-09-21T02:22:23.486415+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:31:09.390858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.489013+0000","last_change":"2024-09-21T02:22:22.475972+0000","last_active":"2024-09-21T02:22:23.489013+0000","last_peered":"2024-09-21T02:22:23.489013+0000","last_clean":"2024-09-21T02:22:23.489013+0000","last_became_active":"2024-09-21T02:22:22.475470+0000","last_became_peered":"2024-09-21T02:22:22.475470+0000","last_unstale":"2024-09-21T02:22:23.489013+0000","last_undegraded":"2024-09-21T02:22:23.489013+0000","last_fullsized":"2024-09-21T02:22:23.489013+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:33:29.730779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488129+0000","last_change":"2024-09-21T02:22:21.465287+0000","last_active":"2024-09-21T02:22:23.488129+0000","last_peered":"2024-09-21T02:22:23.488129+0000","last_clean":"2024-09-21T02:22:23.488129+0000","last_became_active":"2024-09-21T02:22:21.464854+0000","last_became_peered":"2024-09-21T02:22:21.464854+0000","last_unstale":"2024-09-21T02:22:23.488129+0000","last_undegraded":"2024-09-21T02:22:23.488129+0000","last_fullsized":"2024-09-21T02:22:23.488129+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:39.671206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.490863+0000","last_change":"2024-09-21T02:22:22.501874+0000","last_active":"2024-09-21T02:22:23.490863+0000","last_peered":"2024-09-21T02:22:23.490863+0000","last_clean":"2024-09-21T02:22:23.490863+0000","last_became_active":"2024-09-21T02:22:22.500983+0000","last_became_peered":"2024-09-21T02:22:22.500983+0000","last_unstale":"2024-09-21T02:22:23.490863+0000","last_undegraded":"2024-09-21T02:22:23.490863+0000","last_fullsized":"2024-09-21T02:22:23.490863+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:21.414568+0000","last_clean_scrub_stamp":"2024-09-21T02:22:21.414568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:17.715335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"165'1","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.433656+0000","last_change":"2024-09-21T02:22:21.453719+0000","last_active":"2024-09-21T02:22:23.433656+0000","last_peered":"2024-09-21T02:22:23.433656+0000","last_clean":"2024-09-21T02:22:23.433656+0000","last_became_active":"2024-09-21T02:22:21.453306+0000","last_became_peered":"2024-09-21T02:22:21.453306+0000","last_unstale":"2024-09-21T02:22:23.433656+0000","last_undegraded":"2024-09-21T02:22:23.433656+0000","last_fullsized":"2024-09-21T02:22:23.433656+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:22:20.410729+0000","last_clean_scrub_stamp":"2024-09-21T02:22:20.410729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:33:44.439955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":401,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T02:22:23.488207+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:22:23.488207+0000","last_peered":"2024-09-21T02:22:23.488207+0000","last_clean":"2024-09-21T02:22:23.488207+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:22:23.488207+0000","last_undegraded":"2024-09-21T02:22:23.488207+0000","last_fullsized":"2024-09-21T02:22:23.488207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2093056,"data_stored":924035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509552,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2093056,"data_stored":924083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2699264,"data_stored":1515109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542258,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2093056,"data_stored":924103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2101248,"data_stored":925625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41572,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330268,"statfs":{"total":96636764160,"available":96594194432,"internally_reserved":0,"allocated":2109440,"data_stored":925729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509553,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42132,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329708,"statfs":{"total":96636764160,"available":96593620992,"internally_reserved":0,"allocated":2682880,"data_stored":1514381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41552,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330288,"statfs":{"total":96636764160,"available":96594214912,"internally_reserved":0,"allocated":2093056,"data_stored":924595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:22:27.025 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:22:27.026 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T02:22:27.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 -- 172.21.15.170:0/3848746383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440cf7d0 msgr2=0x7f8744073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3848746383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440cf7d0 0x7f8744073b60 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8740009840 tx=0x7f874001c820 comp rx=0 tx=0).stop 2024-09-21T02:22:27.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 -- 172.21.15.170:0/3848746383 shutdown_connections 2024-09-21T02:22:27.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3848746383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440cf7d0 0x7f8744073b60 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 -- 172.21.15.170:0/3848746383 >> 172.21.15.170:0/3848746383 conn(0x7f87440d0170 msgr2=0x7f87440d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:27.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 -- 172.21.15.170:0/3848746383 shutdown_connections 2024-09-21T02:22:27.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.223+0000 7f874b8dd640 1 -- 172.21.15.170:0/3848746383 wait complete. 2024-09-21T02:22:27.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 Processor -- start 2024-09-21T02:22:27.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- start start 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87440740a0 0x7f87440c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 0x7f87440c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87440cf7d0 0x7f87441f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8744169ed0 con 0x7f87440744b0 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87440b5c90 con 0x7f87440cf7d0 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f874416a050 con 0x7f87440740a0 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 0x7f87440c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 0x7f87440c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44050/0 (socket says 172.21.15.170:44050) 2024-09-21T02:22:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 -- 172.21.15.170:0/3669728873 learned_addr learned my addr 172.21.15.170:0/3669728873 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8749679640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87440740a0 0x7f87440c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 -- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87440740a0 msgr2=0x7f87440c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87440740a0 0x7f87440c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 -- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87440cf7d0 msgr2=0x7f87441f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87440cf7d0 0x7f87441f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 -- 172.21.15.170:0/3669728873 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8744152e50 con 0x7f87440744b0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8748e78640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 0x7f87440c7760 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f873400b410 tx=0x7f873400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734015020 con 0x7f87440744b0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8734011920 con 0x7f87440744b0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87440c9190 con 0x7f87440744b0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734011d70 con 0x7f87440744b0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87440c9910 con 0x7f87440744b0 2024-09-21T02:22:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.227+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8744073b60 con 0x7f87440744b0 2024-09-21T02:22:27.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.231+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f8734018020 con 0x7f87440744b0 2024-09-21T02:22:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.231+0000 7f8739ffb640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8728073c20 0x7f8728076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.231+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f8734093240 con 0x7f87440744b0 2024-09-21T02:22:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.231+0000 7f8749679640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8728073c20 0x7f8728076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.231+0000 7f8749679640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8728073c20 0x7f8728076130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f87400af000 tx=0x7f874001b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.235+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f873401f020 con 0x7f87440744b0 2024-09-21T02:22:27.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.455+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f87440c93c0 con 0x7f87440744b0 2024-09-21T02:22:27.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.459+0000 7f8739ffb640 1 -- 172.21.15.170:0/3669728873 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7f873405fad0 con 0x7f87440744b0 2024-09-21T02:22:27.461 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T02:22:27.461 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCKK+5mm71QERAAhOFK5lAjMzYLylvNZuy4uA== 2024-09-21T02:22:27.462 INFO:teuthology.orchestra.run.smithi170.stdout: caps mds = "allow" 2024-09-21T02:22:27.462 INFO:teuthology.orchestra.run.smithi170.stdout: caps mon = "allow r" 2024-09-21T02:22:27.462 INFO:teuthology.orchestra.run.smithi170.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T02:22:27.462 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8728073c20 msgr2=0x7f8728076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8728073c20 0x7f8728076130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f87400af000 tx=0x7f874001b060 comp rx=0 tx=0).stop 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 msgr2=0x7f87440c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 0x7f87440c7760 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f873400b410 tx=0x7f873400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 shutdown_connections 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f8728073c20 0x7f8728076130 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87440cf7d0 0x7f87441f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87440744b0 0x7f87440c7760 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 --2- 172.21.15.170:0/3669728873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87440740a0 0x7f87440c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 >> 172.21.15.170:0/3669728873 conn(0x7f87440d0170 msgr2=0x7f87440b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 shutdown_connections 2024-09-21T02:22:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.463+0000 7f874b8dd640 1 -- 172.21.15.170:0/3669728873 wait complete. 2024-09-21T02:22:27.485 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:27.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 -- 172.21.15.170:0/2169126294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cbc50 msgr2=0x7f2b9c0cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 --2- 172.21.15.170:0/2169126294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cbc50 0x7f2b9c0cc060 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0d0260 tx=0x7f2b9801c750 comp rx=0 tx=0).stop 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 -- 172.21.15.170:0/2169126294 shutdown_connections 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 --2- 172.21.15.170:0/2169126294 >> v2:172.21.15.191:3300/0 conn(0x7f2b9c0cc5a0 0x7f2b9c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 --2- 172.21.15.170:0/2169126294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cbc50 0x7f2b9c0cc060 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 -- 172.21.15.170:0/2169126294 >> 172.21.15.170:0/2169126294 conn(0x7f2b9c09a9c0 msgr2=0x7f2b9c0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 -- 172.21.15.170:0/2169126294 shutdown_connections 2024-09-21T02:22:27.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.635+0000 7f2ba4728640 1 -- 172.21.15.170:0/2169126294 wait complete. 2024-09-21T02:22:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 Processor -- start 2024-09-21T02:22:27.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 -- start start 2024-09-21T02:22:27.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c0b25f0 0x7f2b9c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b9c0cbc50 0x7f2b9c153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 0x7f2b9c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c168f40 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c168dc0 con 0x7f2b9c0b25f0 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba4728640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c1690c0 con 0x7f2b9c0cbc50 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 0x7f2b9c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 0x7f2b9c14dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44064/0 (socket says 172.21.15.170:44064) 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 -- 172.21.15.170:0/1558979577 learned_addr learned my addr 172.21.15.170:0/1558979577 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba1cc3640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b9c0cbc50 0x7f2b9c153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba24c4640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c0b25f0 0x7f2b9c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 -- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b9c0cbc50 msgr2=0x7f2b9c153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b9c0cbc50 0x7f2b9c153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 -- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c0b25f0 msgr2=0x7f2b9c14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c0b25f0 0x7f2b9c14c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.639+0000 7f2ba2cc5640 1 -- 172.21.15.170:0/1558979577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c14c930 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2ba2cc5640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 0x7f2b9c14dba0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400b9d0 tx=0x7f2b9400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94019070 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b9400cce0 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c156250 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940146a0 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c156810 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c0cc060 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f2b9401c070 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2b8effd640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b80073c50 0x7f2b80076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f2b94092ed0 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2ba24c4640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b80073c50 0x7f2b80076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.643+0000 7f2ba24c4640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b80073c50 0x7f2b80076160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b900017b0 tx=0x7f2b90009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:27.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.647+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f2b94023020 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.847+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b9c0ce9d0 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.847+0000 7f2b8effd640 1 -- 172.21.15.170:0/1558979577 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5245 (secure 0 0 0) 0x7f2b9405f7f0 con 0x7f2b9c0cc5a0 2024-09-21T02:22:27.850 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:27.850 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":120,"btime":"2024-09-21T02:22:25:485673+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:22:21.863736+0000","modified":"2024-09-21T02:22:24.886454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9303":{"gid":9303,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1712756729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1712756729},{"type":"v1","addr":"172.21.15.170:6835","nonce":1712756729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9303,"qdb_cluster":[9303]},"id":13}]} 2024-09-21T02:22:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 120 2024-09-21T02:22:27.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b80073c50 msgr2=0x7f2b80076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b80073c50 0x7f2b80076160 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b900017b0 tx=0x7f2b90009040 comp rx=0 tx=0).stop 2024-09-21T02:22:27.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 msgr2=0x7f2b9c14dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:27.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 0x7f2b9c14dba0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400b9d0 tx=0x7f2b9400be90 comp rx=0 tx=0).stop 2024-09-21T02:22:27.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 shutdown_connections 2024-09-21T02:22:27.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2b80073c50 0x7f2b80076160 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b9c0cbc50 0x7f2b9c153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b9c0cc5a0 0x7f2b9c14dba0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 --2- 172.21.15.170:0/1558979577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b9c0b25f0 0x7f2b9c14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 >> 172.21.15.170:0/1558979577 conn(0x7f2b9c09a9c0 msgr2=0x7f2b9c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 shutdown_connections 2024-09-21T02:22:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:27.851+0000 7f2ba4728640 1 -- 172.21.15.170:0/1558979577 wait complete. 2024-09-21T02:22:28.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T02:22:21.863736+0000', 'modified': '2024-09-21T02:22:24.886454+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9303': {'gid': 9303, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6835/1712756729', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1712756729}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1712756729}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9303, 'qdb_cluster': [9303]} 2024-09-21T02:22:28.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:22:28.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 -- 172.21.15.170:0/3894912462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d8004810 msgr2=0x7f09d8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3894912462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d8004810 0x7f09d8004c20 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f09d40016f0 tx=0x7f09d401c750 comp rx=0 tx=0).stop 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 -- 172.21.15.170:0/3894912462 shutdown_connections 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3894912462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09d8004810 0x7f09d8004c20 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 -- 172.21.15.170:0/3894912462 >> 172.21.15.170:0/3894912462 conn(0x7f09ec0b1c40 msgr2=0x7f09ec0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 -- 172.21.15.170:0/3894912462 shutdown_connections 2024-09-21T02:22:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 -- 172.21.15.170:0/3894912462 wait complete. 2024-09-21T02:22:28.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 Processor -- start 2024-09-21T02:22:28.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.439+0000 7f09f21c4640 1 -- start start 2024-09-21T02:22:28.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d8004810 0x7f09ec14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09ec14ae20 0x7f09ec1f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 0x7f09ec1f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0cf760 con 0x7f09ec1f8d80 2024-09-21T02:22:28.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0c3a10 con 0x7f09ec14ae20 2024-09-21T02:22:28.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec0cf8e0 con 0x7f09d8004810 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 0x7f09ec1f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 0x7f09ec1f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44086/0 (socket says 172.21.15.170:44086) 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 -- 172.21.15.170:0/3133826266 learned_addr learned my addr 172.21.15.170:0/3133826266 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09ec14ae20 0x7f09ec1f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d8004810 0x7f09ec14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 -- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d8004810 msgr2=0x7f09ec14e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d8004810 0x7f09ec14e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 -- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09ec14ae20 msgr2=0x7f09ec1f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09ec14ae20 0x7f09ec1f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 -- 172.21.15.170:0/3133826266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec1f9400 con 0x7f09ec1f8d80 2024-09-21T02:22:28.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d8004810 0x7f09ec14e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:28.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09ebfff640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 0x7f09ec1f5250 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0cd8e0 tx=0x7f09e4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e8ff9640 1 -- 172.21.15.170:0/3133826266 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e4004b30 con 0x7f09ec1f8d80 2024-09-21T02:22:28.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e8ff9640 1 -- 172.21.15.170:0/3133826266 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e4004dc0 con 0x7f09ec1f8d80 2024-09-21T02:22:28.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec1f96e0 con 0x7f09ec1f8d80 2024-09-21T02:22:28.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e8ff9640 1 -- 172.21.15.170:0/3133826266 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e4010900 con 0x7f09ec1f8d80 2024-09-21T02:22:28.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec14dae0 con 0x7f09ec1f8d80 2024-09-21T02:22:28.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e8ff9640 1 -- 172.21.15.170:0/3133826266 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f09e401a020 con 0x7f09ec1f8d80 2024-09-21T02:22:28.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e8ff9640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09cc073c20 0x7f09cc076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09e8ff9640 1 -- 172.21.15.170:0/3133826266 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f09e4092d90 con 0x7f09ec1f8d80 2024-09-21T02:22:28.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.443+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09cc073c20 0x7f09cc076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09cc073c20 0x7f09cc076130 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f09dc000fc0 tx=0x7f09dc004610 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 Processor -- start 2024-09-21T02:22:28.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 -- start start 2024-09-21T02:22:28.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce77e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce77e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce7678ca0 0x55bce77e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bce76b5410 con 0x55bce768f950 2024-09-21T02:22:28.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bce75f2b40 con 0x55bce729f460 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55bce75e1f70 con 0x55bce7678ca0 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce77e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce77e0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44092/0 (socket says 172.21.15.170:44092) 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 -- 172.21.15.170:0/606721820 learned_addr learned my addr 172.21.15.170:0/606721820 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce77e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09ebfff640 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce7678ca0 0x55bce77e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 -- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce7678ca0 msgr2=0x55bce77e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce7678ca0 0x55bce77e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 -- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 msgr2=0x55bce77e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce77e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 -- 172.21.15.170:0/606721820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce75a95b0 con 0x55bce768f950 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09ebfff640 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce7678ca0 0x55bce77e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:28.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce77e0be0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f09d40016f0 tx=0x7f09d4019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3090884dff8805a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e17fa640 1 -- 172.21.15.170:0/606721820 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4006350 con 0x55bce768f950 2024-09-21T02:22:28.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e17fa640 1 -- 172.21.15.170:0/606721820 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09d40a9070 con 0x55bce768f950 2024-09-21T02:22:28.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09e17fa640 1 -- 172.21.15.170:0/606721820 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4003600 con 0x55bce768f950 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 -- 172.21.15.170:0/606721820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 msgr2=0x55bce77e0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce77e0be0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f09d40016f0 tx=0x7f09d4019540 comp rx=0 tx=0).stop 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 -- 172.21.15.170:0/606721820 shutdown_connections 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.447+0000 7f09f948f000 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce7678ca0 0x55bce77e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce77e0be0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 --2- 172.21.15.170:0/606721820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce77e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- 172.21.15.170:0/606721820 >> 172.21.15.170:0/606721820 conn(0x55bce7765d90 msgr2=0x55bce7795690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- 172.21.15.170:0/606721820 shutdown_connections 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- 172.21.15.170:0/606721820 wait complete. 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- start start 2024-09-21T02:22:28.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 Processor -- start 2024-09-21T02:22:28.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 0x55bce77e7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce77e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bce757c5f0 con 0x55bce729f460 2024-09-21T02:22:28.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bce75e5d80 con 0x55bce7678ca0 2024-09-21T02:22:28.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55bce7588520 con 0x55bce768f950 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 10 client.0 _ll_register_callbacks cb 0x55bce758c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 0x55bce77e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 0x55bce77e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44108/0 (socket says 172.21.15.170:44108) 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 -- 172.21.15.170:0/988577095 learned_addr learned my addr 172.21.15.170:0/988577095 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ebfff640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce77e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 -- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 msgr2=0x55bce77e83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce77e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 -- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 msgr2=0x55bce77e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 -- 172.21.15.170:0/988577095 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce7580190 con 0x55bce729f460 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 0x55bce77e7920 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00fd70 tx=0x7f09dc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f9ae12c6437a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:22:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ca7fc640 1 -- 172.21.15.170:0/988577095 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc01b840 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bce757a110 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ca7fc640 1 -- 172.21.15.170:0/988577095 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f09dc01b9e0 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ca7fc640 1 -- 172.21.15.170:0/988577095 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7f09dc02e050 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 10 client.9441 fetch_fsmap learned FSMap version 120 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) -- 0x55bce7666f10 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ca7fc640 1 -- 172.21.15.170:0/988577095 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55bce757a110 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09ca7fc640 1 -- 172.21.15.170:0/988577095 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 120) ==== 3502+0+0 (secure 0 0 0) 0x7f09dc031020 con 0x55bce729f460 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 10 client.9441 fetch_fsmap finished waiting for FSMap version 120 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 10 client.9441 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 20 client.9441 populate_metadata read hostname 'smithi170' 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce72847d0 0x55bce77f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 4 client.9441 mds_command: new command op to 9303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 --> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bce75f4a90 con 0x55bce72847d0 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.451+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce72847d0 0x55bce77f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09ca7fc640 10 client.9441 ms_handle_connect on v2:172.21.15.170:6834/1712756729 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce72847d0 0x55bce77f1d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb43b5644767a9d4 server_cookie=cc81ba417fd382b in_seq=0 out_seq=0 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09ca7fc640 1 -- 172.21.15.170:0/988577095 <== mds.0 v2:172.21.15.170:6834/1712756729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bce75f4a90 con 0x55bce72847d0 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09ca7fc640 10 client.9441 handle_command_reply: tid=0 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 client.9441 shutdown 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 msgr2=0x55bce77e7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 0x55bce77e7920 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00fd70 tx=0x7f09dc001a00 comp rx=0 tx=0).stop 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 shutdown_connections 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce72847d0 0x55bce77f1d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce77e83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09ca7fc640 0 client.9441 ms_handle_reset on v2:172.21.15.170:6834/1712756729 2024-09-21T02:22:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 --2- 172.21.15.170:0/988577095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce729f460 0x55bce77e7920 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 >> 172.21.15.170:0/988577095 conn(0x55bce7765d90 msgr2=0x55bce779c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 shutdown_connections 2024-09-21T02:22:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 1 -- 172.21.15.170:0/988577095 wait complete. 2024-09-21T02:22:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.455+0000 7f09f948f000 20 client.9441 trim_cache size 0 max 16384 2024-09-21T02:22:28.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.471+0000 7f09f948f000 1 Processor -- start 2024-09-21T02:22:28.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.471+0000 7f09f948f000 1 -- start start 2024-09-21T02:22:28.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce729f460 0x55bce77df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce7794cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec202560 con 0x55bce768f950 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec201580 con 0x55bce7678ca0 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec0b4780 con 0x55bce729f460 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce7794cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce7794cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44110/0 (socket says 172.21.15.170:44110) 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 -- 172.21.15.170:0/2250616861 learned_addr learned my addr 172.21.15.170:0/2250616861 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 -- 172.21.15.170:0/2250616861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce729f460 msgr2=0x55bce77df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce729f460 0x55bce77df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 -- 172.21.15.170:0/2250616861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 msgr2=0x55bce77dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 -- 172.21.15.170:0/2250616861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce763ddb0 con 0x55bce768f950 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09ebfff640 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce7794cd0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f09e40930b0 tx=0x7f09e405eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69015a020b74be88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09e0ff9640 1 -- 172.21.15.170:0/2250616861 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e405f030 con 0x55bce768f950 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09e0ff9640 1 -- 172.21.15.170:0/2250616861 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e4063b10 con 0x55bce768f950 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09e0ff9640 1 -- 172.21.15.170:0/2250616861 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e405e4f0 con 0x55bce768f950 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- 172.21.15.170:0/2250616861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 msgr2=0x55bce7794cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce7794cd0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f09e40930b0 tx=0x7f09e405eab0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- 172.21.15.170:0/2250616861 shutdown_connections 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce768f950 0x55bce7794cd0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce7678ca0 0x55bce77dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 --2- 172.21.15.170:0/2250616861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce729f460 0x55bce77df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- 172.21.15.170:0/2250616861 >> 172.21.15.170:0/2250616861 conn(0x55bce7765d90 msgr2=0x55bce77f0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- 172.21.15.170:0/2250616861 shutdown_connections 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- 172.21.15.170:0/2250616861 wait complete. 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 -- start start 2024-09-21T02:22:28.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.475+0000 7f09f948f000 1 Processor -- start 2024-09-21T02:22:28.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce7797cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 0x55bce7798210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce7812220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bce757c5f0 con 0x55bce7678ca0 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec200f80 con 0x55bce729f460 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x55bce75ea3b0 con 0x55bce768f950 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 10 client.0 _ll_register_callbacks cb 0x7f09dc001770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 0x55bce7798210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 0x55bce7798210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44124/0 (socket says 172.21.15.170:44124) 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 -- 172.21.15.170:0/2275043058 learned_addr learned my addr 172.21.15.170:0/2275043058 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 -- 172.21.15.170:0/2275043058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 msgr2=0x55bce7812220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce7812220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 -- 172.21.15.170:0/2275043058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 msgr2=0x55bce7797cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce7797cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 -- 172.21.15.170:0/2275043058 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce75a8ab0 con 0x55bce7678ca0 2024-09-21T02:22:28.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09e2ffd640 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 0x55bce7798210 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f09d4007010 tx=0x7f09d401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd48d8ef9e37456 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 1 -- 172.21.15.170:0/2275043058 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d40065d0 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 1 -- 172.21.15.170:0/2275043058 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f09d40afb00 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09e4063b10 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 1 -- 172.21.15.170:0/2275043058 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7f09d40afd00 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 10 client.9447 fetch_fsmap learned FSMap version 120 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) -- 0x55bce7587070 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 1 -- 172.21.15.170:0/2275043058 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e4063b10 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 1 -- 172.21.15.170:0/2275043058 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 120) ==== 3502+0+0 (secure 0 0 0) 0x7f09d4003600 con 0x55bce7678ca0 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 10 client.9447 fetch_fsmap finished waiting for FSMap version 120 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 10 client.9447 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 20 client.9447 populate_metadata read hostname 'smithi170' 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce7272830 0x55bce7812af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 4 client.9447 mds_command: new command op to 9303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:22:28.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 --> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] -- command(tid 0: {"prefix": "status"}) -- 0x55bce75f0540 con 0x55bce7272830 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce7272830 0x55bce7812af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 10 client.9447 ms_handle_connect on v2:172.21.15.170:6834/1712756729 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09eb7fe640 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce7272830 0x55bce7812af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd78be344e3f4c57 server_cookie=471e0858ecf7d4d in_seq=0 out_seq=0 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 1 -- 172.21.15.170:0/2275043058 <== mds.0 v2:172.21.15.170:6834/1712756729 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55bce75f0540 con 0x55bce7272830 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 10 client.9447 handle_command_reply: tid=0 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 client.9447 shutdown 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 msgr2=0x55bce7798210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 0x55bce7798210 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f09d4007010 tx=0x7f09d401c9a0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 shutdown_connections 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:6834/1712756729,v1:172.21.15.170:6835/1712756729] conn(0x55bce7272830 0x55bce7812af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55bce768f950 0x55bce7812220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bce7678ca0 0x55bce7798210 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 --2- 172.21.15.170:0/2275043058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bce729f460 0x55bce7797cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 >> 172.21.15.170:0/2275043058 conn(0x55bce7765d90 msgr2=0x55bce77f18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09ca7fc640 0 client.9447 ms_handle_reset on v2:172.21.15.170:6834/1712756729 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 shutdown_connections 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 1 -- 172.21.15.170:0/2275043058 wait complete. 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f948f000 20 client.9447 trim_cache size 0 max 16384 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "75527983-1c6c-4a94-aa07-cb5c10ea5972", 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 9303, 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2024-09-21T02:22:28.487 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 6.0131677630000002, 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 120, 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 166, 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 166, 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.567480606 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09cc073c20 msgr2=0x7f09cc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09cc073c20 0x7f09cc076130 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f09dc000fc0 tx=0x7f09dc004610 comp rx=0 tx=0).stop 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 msgr2=0x7f09ec1f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 0x7f09ec1f5250 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0cd8e0 tx=0x7f09e4009b00 comp rx=0 tx=0).stop 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 shutdown_connections 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f09cc073c20 0x7f09cc076130 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09ec1f8d80 0x7f09ec1f5250 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09ec14ae20 0x7f09ec1f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 --2- 172.21.15.170:0/3133826266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09d8004810 0x7f09ec14e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 >> 172.21.15.170:0/3133826266 conn(0x7f09ec0b1c40 msgr2=0x7f09ec0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 shutdown_connections 2024-09-21T02:22:28.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.479+0000 7f09f21c4640 1 -- 172.21.15.170:0/3133826266 wait complete. 2024-09-21T02:22:28.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/3865496940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b3fa0 msgr2=0x7fe4f00cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/3865496940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b3fa0 0x7fe4f00cf970 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009900 tx=0x7fe4ec01c850 comp rx=0 tx=0).stop 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/3865496940 shutdown_connections 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/3865496940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f00b1230 0x7fe4f00b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/3865496940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b3fa0 0x7fe4f00cf970 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/3865496940 >> 172.21.15.170:0/3865496940 conn(0x7fe4f008bf50 msgr2=0x7fe4f009d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/3865496940 shutdown_connections 2024-09-21T02:22:28.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.647+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/3865496940 wait complete. 2024-09-21T02:22:28.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 Processor -- start 2024-09-21T02:22:28.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 -- start start 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 0x7fe4f00cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00d0e50 0x7fe4f00cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f01fc2b0 0x7fe4f01fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f016bf90 con 0x7fe4f00b1230 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f016be10 con 0x7fe4f00d0e50 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f016c110 con 0x7fe4f01fc2b0 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 0x7fe4f00cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 0x7fe4f00cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44150/0 (socket says 172.21.15.170:44150) 2024-09-21T02:22:28.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 -- 172.21.15.170:0/1129144877 learned_addr learned my addr 172.21.15.170:0/1129144877 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5438640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00d0e50 0x7fe4f00cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f643a640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f01fc2b0 0x7fe4f01fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 -- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f01fc2b0 msgr2=0x7fe4f01fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f01fc2b0 0x7fe4f01fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 -- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00d0e50 msgr2=0x7fe4f00cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00d0e50 0x7fe4f00cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 -- 172.21.15.170:0/1129144877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f00cb0c0 con 0x7fe4f00b1230 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5438640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00d0e50 0x7fe4f00cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f5c39640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 0x7fe4f00cab80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0b0040 tx=0x7fe4ec0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0ba020 con 0x7fe4f00b1230 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4ec004430 con 0x7fe4f00b1230 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f01fcc00 con 0x7fe4f00b1230 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0b2040 con 0x7fe4f00b1230 2024-09-21T02:22:28.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.651+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f0158bc0 con 0x7fe4f00b1230 2024-09-21T02:22:28.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.655+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f00cf970 con 0x7fe4f00b1230 2024-09-21T02:22:28.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.655+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fe4ec005290 con 0x7fe4f00b1230 2024-09-21T02:22:28.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.655+0000 7fe4e67fc640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe4d4073c50 0x7fe4d4076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:28.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.655+0000 7fe4f5438640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe4d4073c50 0x7fe4d4076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:28.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.655+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fe4ec12cd30 con 0x7fe4f00b1230 2024-09-21T02:22:28.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.655+0000 7fe4f5438640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe4d4073c50 0x7fe4d4076160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000f70 tx=0x7fe4e0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:28.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.659+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe4ec0c4060 con 0x7fe4f00b1230 2024-09-21T02:22:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.859+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe4f015bcc0 con 0x7fe4f00b1230 2024-09-21T02:22:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.859+0000 7fe4e67fc640 1 -- 172.21.15.170:0/1129144877 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v120) ==== 53+0+73 (secure 0 0 0) 0x7fe4f01fcc00 con 0x7fe4f00b1230 2024-09-21T02:22:28.861 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe4d4073c50 msgr2=0x7fe4d4076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe4d4073c50 0x7fe4d4076160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000f70 tx=0x7fe4e0009040 comp rx=0 tx=0).stop 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 msgr2=0x7fe4f00cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 0x7fe4f00cab80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0b0040 tx=0x7fe4ec0a5710 comp rx=0 tx=0).stop 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 shutdown_connections 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe4d4073c50 0x7fe4d4076160 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f01fc2b0 0x7fe4f01fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00d0e50 0x7fe4f00cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 --2- 172.21.15.170:0/1129144877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4f00b1230 0x7fe4f00cab80 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 >> 172.21.15.170:0/1129144877 conn(0x7fe4f008bf50 msgr2=0x7fe4f015c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 shutdown_connections 2024-09-21T02:22:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:28.863+0000 7fe4f7e9d640 1 -- 172.21.15.170:0/1129144877 wait complete. 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:22:28.883 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:22:28.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:28.884 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2024-09-21T02:22:28.889 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:22:28.889 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:22:28.889 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:22:28.890 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41967sec preferred_lft 41967sec 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 02:42:0d:50:21:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:22:28.891 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:28.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:28.892 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:22:28.940 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T02:22:28.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:28.942 DEBUG:teuthology.orchestra.run.smithi170:> route 2024-09-21T02:22:28.991 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2024-09-21T02:22:28.991 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:22:28.991 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:22:28.991 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:22:28.991 INFO:teuthology.orchestra.run.smithi170.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:22:28.991 INFO:teuthology.orchestra.run.smithi170.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:22:28.992 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:22:28.992 INFO:teuthology.orchestra.run.smithi170.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:28.992 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:28.992 INFO:teuthology.orchestra.run.smithi170.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:28.992 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:22:28.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:22:28.992 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2024-09-21T02:22:28.992 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:22:28.992 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:22:28.992 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:22:28.993 DEBUG:teuthology.orchestra.run.smithi170:> ') 2024-09-21T02:22:29.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:22:29.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:29.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:29.175 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:22:29.225 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:22:29.225 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:22:29.226 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:22:29.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:29.226 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2024-09-21T02:22:29.283 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T02:22:29.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:22:29.283 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:29.332 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T02:22:29.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:22:29.333 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:29.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:22:29.437 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:29.484 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:29.485 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:22:29.486 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:22:29.487 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:22:29.487 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:29.487 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:29.487 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:29.487 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:29.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:29.488 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:22:29.540 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:22:29.540 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T02:22:29.585 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:22:29.600 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:29.648 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:22:29.648 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:22:29.648 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:22:29.648 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:22:29.648 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:22:29.649 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:22:29.650 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:29.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:29.651 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:22:29.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:2024-09-21T02:22:29.691+0000 7f3ced5df580 -1 init, newargv = 0x55559ed7efd0 newargc=15 2024-09-21T02:22:29.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[53734]: starting ceph client 2024-09-21T02:22:29.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[53734]: starting fuse 2024-09-21T02:22:30.800 DEBUG:teuthology.orchestra.run.smithi170:> sudo modprobe fuse 2024-09-21T02:22:30.816 DEBUG:teuthology.orchestra.run.smithi170:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:30.870 INFO:teuthology.orchestra.run.smithi170.stdout:/sys 2024-09-21T02:22:30.870 INFO:teuthology.orchestra.run.smithi170.stdout:/proc 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/dev 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/pts 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/run 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/ 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/security 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/shm 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/run/lock 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/cgroup 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/pstore 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/bpf 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/hugepages 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/mqueue 2024-09-21T02:22:30.871 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/debug 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/tracing 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/proc/fs/nfsd 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/sys/kernel/config 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/run/rpc_pipefs 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/run/user/1000 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:30.872 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:30.873 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:30.873 INFO:teuthology.orchestra.run.smithi170.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:30.873 INFO:teuthology.orchestra.run.smithi170.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:30.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:30.874 DEBUG:teuthology.orchestra.run.smithi170:> ls /sys/fs/fuse/connections 2024-09-21T02:22:30.920 INFO:teuthology.orchestra.run.smithi170.stdout:45 2024-09-21T02:22:30.920 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:22:30.920 DEBUG:teuthology.orchestra.run.smithi170:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:22:30.920 DEBUG:teuthology.orchestra.run.smithi170:> import glob 2024-09-21T02:22:30.920 DEBUG:teuthology.orchestra.run.smithi170:> import re 2024-09-21T02:22:30.920 DEBUG:teuthology.orchestra.run.smithi170:> import os 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> import subprocess 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> def _find_admin_socket(client_name): 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> files = glob.glob(asok_path) 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> # Given a non-glob path, it better be there 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> if "*" not in asok_path: 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> assert(len(files) == 1) 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> return files[0] 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> for f in files: 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:22:30.921 DEBUG:teuthology.orchestra.run.smithi170:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> contents = proc_f.read() 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> if mountpoint in contents: 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> return f 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> print(_find_admin_socket("client.0")) 2024-09-21T02:22:30.922 DEBUG:teuthology.orchestra.run.smithi170:> ' 2024-09-21T02:22:31.046 INFO:teuthology.orchestra.run.smithi170.stdout:/var/run/ceph/ceph-client.0.53734.asok 2024-09-21T02:22:31.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:31.057 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53734.asok 2024-09-21T02:22:31.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53734.asok status 2024-09-21T02:22:31.236 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:31.236 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": { 2024-09-21T02:22:31.236 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:22:31.236 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:22:31.236 INFO:teuthology.orchestra.run.smithi170.stdout: "entity_id": "0", 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "hostname": "smithi170", 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "pid": "53734", 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "root": "/" 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_count": 0, 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "dentry_pinned_count": 0, 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 9459, 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "inst": { 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "name": { 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "client", 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "num": 9459 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:31.237 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 978788915 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": { 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "type": "v1", 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "nonce": 978788915 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "inst_str": "client.9459 192.168.0.1:0/978788915", 2024-09-21T02:22:31.238 INFO:teuthology.orchestra.run.smithi170.stdout: "addr_str": "192.168.0.1:0/978788915", 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout: "inode_count": 1, 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_epoch": 120, 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch": 166, 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout: "blocklisted": false, 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs" 2024-09-21T02:22:31.239 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:31.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.252 DEBUG:teuthology.orchestra.run.smithi170:> stat --file-system '--printf=%T 2024-09-21T02:22:31.252 DEBUG:teuthology.orchestra.run.smithi170:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:31.258 INFO:teuthology.orchestra.run.smithi170.stdout:fuseblk 2024-09-21T02:22:31.258 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:31.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.259 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:31.312 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T02:22:31.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 -- 172.21.15.170:0/1457829004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b5410 msgr2=0x7faf5806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/1457829004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b5410 0x7faf5806fa30 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7faf580b3a10 tx=0x7faf4001cba0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 -- 172.21.15.170:0/1457829004 shutdown_connections 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/1457829004 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf580b2f40 0x7faf580b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/1457829004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b5410 0x7faf5806fa30 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 -- 172.21.15.170:0/1457829004 >> 172.21.15.170:0/1457829004 conn(0x7faf580b1c40 msgr2=0x7faf580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 -- 172.21.15.170:0/1457829004 shutdown_connections 2024-09-21T02:22:31.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 -- 172.21.15.170:0/1457829004 wait complete. 2024-09-21T02:22:31.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 Processor -- start 2024-09-21T02:22:31.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.511+0000 7faf5ec4c640 1 -- start start 2024-09-21T02:22:31.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 0x7faf581586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf580b5410 0x7faf58159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf580ccb00 0x7faf58159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faf5816a060 con 0x7faf580b2f40 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faf580701b0 con 0x7faf580b5410 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7faf5816a1e0 con 0x7faf580ccb00 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 0x7faf581586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 0x7faf581586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44168/0 (socket says 172.21.15.170:44168) 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 -- 172.21.15.170:0/3257902481 learned_addr learned my addr 172.21.15.170:0/3257902481 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5d1e9640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf580ccb00 0x7faf58159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf57fff640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf580b5410 0x7faf58159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 -- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf580ccb00 msgr2=0x7faf58159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf580ccb00 0x7faf58159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 -- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf580b5410 msgr2=0x7faf58159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf580b5410 0x7faf58159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 -- 172.21.15.170:0/3257902481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5809b340 con 0x7faf580b2f40 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf57fff640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf580b5410 0x7faf58159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5c9e8640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 0x7faf581586e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7faf40000f90 tx=0x7faf400092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400bb070 con 0x7faf580b2f40 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf40004e00 con 0x7faf580b2f40 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5815e070 con 0x7faf580b2f40 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400b2040 con 0x7faf580b2f40 2024-09-21T02:22:31.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5815e630 con 0x7faf580b2f40 2024-09-21T02:22:31.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf580b3350 con 0x7faf580b2f40 2024-09-21T02:22:31.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.515+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7faf40004200 con 0x7faf580b2f40 2024-09-21T02:22:31.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.519+0000 7faf557fa640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf38073c50 0x7faf38076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:31.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.519+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7faf4012d130 con 0x7faf580b2f40 2024-09-21T02:22:31.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.519+0000 7faf57fff640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf38073c50 0x7faf38076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:31.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.519+0000 7faf57fff640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf38073c50 0x7faf38076160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faf50005fd0 tx=0x7faf50005de0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:31.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.523+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7faf40016790 con 0x7faf580b2f40 2024-09-21T02:22:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faf580c7fe0 con 0x7faf580b2f40 2024-09-21T02:22:31.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf557fa640 1 -- 172.21.15.170:0/3257902481 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v120) ==== 53+0+73 (secure 0 0 0) 0x7faf400f9420 con 0x7faf580b2f40 2024-09-21T02:22:31.732 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T02:22:31.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf38073c50 msgr2=0x7faf38076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:31.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf38073c50 0x7faf38076160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faf50005fd0 tx=0x7faf50005de0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 msgr2=0x7faf581586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 0x7faf581586e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7faf40000f90 tx=0x7faf400092c0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 shutdown_connections 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7faf38073c50 0x7faf38076160 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faf580ccb00 0x7faf58159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faf580b5410 0x7faf58159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 --2- 172.21.15.170:0/3257902481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf580b2f40 0x7faf581586e0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 >> 172.21.15.170:0/3257902481 conn(0x7faf580b1c40 msgr2=0x7faf580c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 shutdown_connections 2024-09-21T02:22:31.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:31.731+0000 7faf5ec4c640 1 -- 172.21.15.170:0/3257902481 wait complete. 2024-09-21T02:22:31.752 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T02:22:31.752 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T02:22:31.752 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-09-21T02:22:31.752 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T02:22:31.753 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:31.753 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T02:22:31.753 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T02:22:31.753 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T02:22:31.753 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:22:31.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.753 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-09-21T02:22:31.760 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-09-21T02:22:31.761 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41957sec preferred_lft 41957sec 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:42:d3:60:71:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-21T02:22:31.762 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T02:22:31.763 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:31.764 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-21T02:22:31.764 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-09-21T02:22:31.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.765 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T02:22:31.817 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-21T02:22:31.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.819 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-09-21T02:22:31.869 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-21T02:22:31.870 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T02:22:31.871 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T02:22:31.871 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-09-21T02:22:31.871 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T02:22:31.871 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T02:22:31.871 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T02:22:31.871 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-09-21T02:22:31.960 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:22:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T02:22:31.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:22:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:31.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:31.997 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:22:32.003 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:22:32.003 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:22:32.003 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:22:32.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:32.003 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-09-21T02:22:32.056 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T02:22:32.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:22:32.056 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:32.101 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T02:22:32.102 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T02:22:32.102 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:32.149 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:22:32.207 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:32.254 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:22:32.255 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:22:32.256 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:22:32.257 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:22:32.257 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:22:32.257 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:32.257 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:32.257 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:32.257 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:32.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:32.258 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:22:32.305 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T02:22:32.305 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T02:22:32.350 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:22:32.363 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:32.410 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:22:32.410 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:22:32.410 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:22:32.410 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:22:32.410 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:22:32.410 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:22:32.411 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:32.412 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:32.413 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:32.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:32.414 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:22:32.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:2024-09-21T02:22:32.455+0000 7f0c5cb72580 -1 init, newargv = 0x7f0c4c0ad020 newargc=15 2024-09-21T02:22:32.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[36023]: starting ceph client 2024-09-21T02:22:32.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[36023]: starting fuse 2024-09-21T02:22:33.561 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2024-09-21T02:22:33.577 DEBUG:teuthology.orchestra.run.smithi191:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/sys 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/proc 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/dev 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/pts 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/run 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/ 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/security 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/shm 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/run/lock 2024-09-21T02:22:33.627 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/cgroup 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/pstore 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/bpf 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/hugepages 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/mqueue 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/debug 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/tracing 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/proc/fs/nfsd 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/fs/fuse/connections 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/sys/kernel/config 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/run/credentials/systemd-sysusers.service 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/run/rpc_pipefs 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/run/user/1000 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph 2024-09-21T02:22:33.628 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-4 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-5 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-6 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/var/lib/ceph/osd/ceph-7 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:33.629 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:33.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:33.630 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2024-09-21T02:22:33.678 INFO:teuthology.orchestra.run.smithi191.stdout:45 2024-09-21T02:22:33.678 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo stdin-killer -- python3 -c ' 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> import re 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> import os 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:33.678 DEBUG:teuthology.orchestra.run.smithi191:> def _find_admin_socket(client_name): 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> contents = proc_f.read() 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> if mountpoint in contents: 2024-09-21T02:22:33.679 DEBUG:teuthology.orchestra.run.smithi191:> return f 2024-09-21T02:22:33.680 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T02:22:33.680 DEBUG:teuthology.orchestra.run.smithi191:> 2024-09-21T02:22:33.680 DEBUG:teuthology.orchestra.run.smithi191:> print(_find_admin_socket("client.1")) 2024-09-21T02:22:33.680 DEBUG:teuthology.orchestra.run.smithi191:> ' 2024-09-21T02:22:33.800 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.1.36023.asok 2024-09-21T02:22:33.803 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-21T02:22:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:33.811 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36023.asok 2024-09-21T02:22:33.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:33.812 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36023.asok status 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "1", 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "36023", 2024-09-21T02:22:33.992 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 7339, 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 7339 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2667962227 2024-09-21T02:22:33.993 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "192.168.0.1:0", 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2667962227 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.7339 192.168.0.1:0/2667962227", 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "192.168.0.1:0/2667962227", 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 120, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 166, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "blocklisted": false, 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2024-09-21T02:22:33.994 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-21T02:22:34.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:34.010 DEBUG:teuthology.orchestra.run.smithi191:> stat --file-system '--printf=%T 2024-09-21T02:22:34.010 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:34.016 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2024-09-21T02:22:34.016 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:34.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:34.017 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:34.072 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T02:22:34.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 -- 172.21.15.170:0/1338051607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8268003a10 msgr2=0x7f8268003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:34.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 --2- 172.21.15.170:0/1338051607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8268003a10 0x7f8268003e80 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f827c075300 tx=0x7f826401cc20 comp rx=0 tx=0).stop 2024-09-21T02:22:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f827b59e640 1 -- 172.21.15.170:0/1338051607 reap_dead start 2024-09-21T02:22:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 -- 172.21.15.170:0/1338051607 shutdown_connections 2024-09-21T02:22:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 -- 172.21.15.170:0/1338051607 >> 172.21.15.170:0/1338051607 conn(0x7f827c0b1c40 msgr2=0x7f827c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 -- 172.21.15.170:0/1338051607 shutdown_connections 2024-09-21T02:22:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 -- 172.21.15.170:0/1338051607 wait complete. 2024-09-21T02:22:34.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 Processor -- start 2024-09-21T02:22:34.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.243+0000 7f8281180640 1 -- start start 2024-09-21T02:22:34.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8268003a10 0x7f827c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:34.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 0x7f827c09c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:34.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f827c1575d0 0x7f827c09c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:34.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f827c166d80 con 0x7f827c09d770 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f827c166c00 con 0x7f8268003a10 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f827c166f00 con 0x7f827c1575d0 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 0x7f827c09c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 0x7f827c09c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44188/0 (socket says 172.21.15.170:44188) 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 -- 172.21.15.170:0/139079111 learned_addr learned my addr 172.21.15.170:0/139079111 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827ad9d640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8268003a10 0x7f827c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827b59e640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f827c1575d0 0x7f827c09c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 -- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f827c1575d0 msgr2=0x7f827c09c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f827c1575d0 0x7f827c09c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 -- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8268003a10 msgr2=0x7f827c156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8268003a10 0x7f827c156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 -- 172.21.15.170:0/139079111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c09cfb0 con 0x7f827c09d770 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827b59e640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f827c1575d0 0x7f827c09c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f827a59c640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 0x7f827c09c350 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f826c00d980 tx=0x7f826c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:34.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c009c30 con 0x7f827c09d770 2024-09-21T02:22:34.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f826c004030 con 0x7f827c09d770 2024-09-21T02:22:34.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004480 con 0x7f827c09d770 2024-09-21T02:22:34.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c09aa60 con 0x7f827c09d770 2024-09-21T02:22:34.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c09ad40 con 0x7f827c09d770 2024-09-21T02:22:34.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c1573b0 con 0x7f827c09d770 2024-09-21T02:22:34.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.247+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f826c003600 con 0x7f827c09d770 2024-09-21T02:22:34.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.251+0000 7f82737fe640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f825c073c50 0x7f825c076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:34.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.251+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f826c0932d0 con 0x7f827c09d770 2024-09-21T02:22:34.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.251+0000 7f827ad9d640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f825c073c50 0x7f825c076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:34.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.251+0000 7f827ad9d640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f825c073c50 0x7f825c076160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f82640af000 tx=0x7f82640a7040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:34.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.255+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f826c020030 con 0x7f827c09d770 2024-09-21T02:22:34.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:34.463+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f827c09cdd0 con 0x7f827c09d770 2024-09-21T02:22:35.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.011+0000 7f82737fe640 1 -- 172.21.15.170:0/139079111 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v121) ==== 136+0+0 (secure 0 0 0) 0x7f827c09cdd0 con 0x7f827c09d770 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f825c073c50 msgr2=0x7f825c076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f825c073c50 0x7f825c076160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f82640af000 tx=0x7f82640a7040 comp rx=0 tx=0).stop 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 msgr2=0x7f827c09c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 0x7f827c09c350 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f826c00d980 tx=0x7f826c00de40 comp rx=0 tx=0).stop 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 shutdown_connections 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f825c073c50 0x7f825c076160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f827c1575d0 0x7f827c09c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f827c09d770 0x7f827c09c350 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 --2- 172.21.15.170:0/139079111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8268003a10 0x7f827c156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 >> 172.21.15.170:0/139079111 conn(0x7f827c0b1c40 msgr2=0x7f827c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:35.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 shutdown_connections 2024-09-21T02:22:35.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.015+0000 7f8281180640 1 -- 172.21.15.170:0/139079111 wait complete. 2024-09-21T02:22:35.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T02:22:35.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 -- 172.21.15.170:0/3960924423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840b7ab0 msgr2=0x7f788415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 --2- 172.21.15.170:0/3960924423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840b7ab0 0x7f788415c850 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7884075080 tx=0x7f787c01c800 comp rx=0 tx=0).stop 2024-09-21T02:22:35.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 -- 172.21.15.170:0/3960924423 shutdown_connections 2024-09-21T02:22:35.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 --2- 172.21.15.170:0/3960924423 >> v2:172.21.15.191:3300/0 conn(0x7f78840d0aa0 0x7f78840d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 --2- 172.21.15.170:0/3960924423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840b7ab0 0x7f788415c850 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 -- 172.21.15.170:0/3960924423 >> 172.21.15.170:0/3960924423 conn(0x7f78840cee30 msgr2=0x7f78840cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:35.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 -- 172.21.15.170:0/3960924423 shutdown_connections 2024-09-21T02:22:35.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.199+0000 7f788c559640 1 -- 172.21.15.170:0/3960924423 wait complete. 2024-09-21T02:22:35.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 Processor -- start 2024-09-21T02:22:35.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 -- start start 2024-09-21T02:22:35.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78840b3e30 0x7f7884155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78840b7ab0 0x7f78841566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 0x7f78840c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f788416a510 con 0x7f78840d0aa0 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f788416a390 con 0x7f78840b7ab0 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f788416a690 con 0x7f78840b3e30 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 0x7f78840c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 0x7f78840c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44210/0 (socket says 172.21.15.170:44210) 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 -- 172.21.15.170:0/3835939462 learned_addr learned my addr 172.21.15.170:0/3835939462 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788a2f5640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78840b3e30 0x7f7884155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f7889af4640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78840b7ab0 0x7f78841566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 -- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78840b3e30 msgr2=0x7f7884155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78840b3e30 0x7f7884155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 -- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78840b7ab0 msgr2=0x7f78841566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78840b7ab0 0x7f78841566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 -- 172.21.15.170:0/3835939462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7884155930 con 0x7f78840d0aa0 2024-09-21T02:22:35.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f7889af4640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78840b7ab0 0x7f78841566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788aaf6640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 0x7f78840c7070 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f787400b440 tx=0x7f787400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874015020 con 0x7f78840d0aa0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7874011920 con 0x7f78840d0aa0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841f7dd0 con 0x7f78840d0aa0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874011d70 con 0x7f78840d0aa0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.203+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7884155eb0 con 0x7f78840d0aa0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.207+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78841f7b20 con 0x7f78840d0aa0 2024-09-21T02:22:35.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.207+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f7874018020 con 0x7f78840d0aa0 2024-09-21T02:22:35.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.207+0000 7f7882ffd640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7870073c50 0x7f7870076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.207+0000 7f788a2f5640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7870073c50 0x7f7870076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.207+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f7874092ca0 con 0x7f78840d0aa0 2024-09-21T02:22:35.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.207+0000 7f788a2f5640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7870073c50 0x7f7870076160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f787c0af000 tx=0x7f787c01b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:35.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.211+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f787401f020 con 0x7f78840d0aa0 2024-09-21T02:22:35.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.407+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f788406a270 con 0x7f7870073c50 2024-09-21T02:22:35.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f7882ffd640 1 -- 172.21.15.170:0/3835939462 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f787c0b3040 con 0x7f7870073c50 2024-09-21T02:22:35.412 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2024-09-21T02:22:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T02:22:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2024-09-21T02:22:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T02:22:35.412 INFO:teuthology.orchestra.run.smithi170.stdout:] 2024-09-21T02:22:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7870073c50 msgr2=0x7f7870076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7870073c50 0x7f7870076160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f787c0af000 tx=0x7f787c01b040 comp rx=0 tx=0).stop 2024-09-21T02:22:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 msgr2=0x7f78840c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 0x7f78840c7070 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f787400b440 tx=0x7f787400b900 comp rx=0 tx=0).stop 2024-09-21T02:22:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 shutdown_connections 2024-09-21T02:22:35.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f7870073c50 0x7f7870076160 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78840d0aa0 0x7f78840c7070 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78840b7ab0 0x7f78841566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 --2- 172.21.15.170:0/3835939462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78840b3e30 0x7f7884155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 >> 172.21.15.170:0/3835939462 conn(0x7f78840cee30 msgr2=0x7f78840b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 shutdown_connections 2024-09-21T02:22:35.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.411+0000 7f788c559640 1 -- 172.21.15.170:0/3835939462 wait complete. 2024-09-21T02:22:35.433 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 -- 172.21.15.170:0/1389529360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740074270 msgr2=0x7fb7400cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 --2- 172.21.15.170:0/1389529360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740074270 0x7fb7400cb280 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb7400b61b0 tx=0x7fb73001c8c0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 -- 172.21.15.170:0/1389529360 shutdown_connections 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 --2- 172.21.15.170:0/1389529360 >> v2:172.21.15.191:3300/0 conn(0x7fb740074680 0x7fb7400cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 --2- 172.21.15.170:0/1389529360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740074270 0x7fb7400cb280 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 -- 172.21.15.170:0/1389529360 >> 172.21.15.170:0/1389529360 conn(0x7fb7400d0170 msgr2=0x7fb7400d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 -- 172.21.15.170:0/1389529360 shutdown_connections 2024-09-21T02:22:35.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.587+0000 7fb745092640 1 -- 172.21.15.170:0/1389529360 wait complete. 2024-09-21T02:22:35.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 Processor -- start 2024-09-21T02:22:35.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- start start 2024-09-21T02:22:35.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740074270 0x7fb74009b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb740074680 0x7fb7400c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 0x7fb7400c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb740169040 con 0x7fb7400cf7d0 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb740168ec0 con 0x7fb740074270 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fb7401691c0 con 0x7fb740074680 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 0x7fb7400c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 0x7fb7400c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43052/0 (socket says 172.21.15.170:43052) 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 -- 172.21.15.170:0/2367873839 learned_addr learned my addr 172.21.15.170:0/2367873839 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73ed9d640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740074270 0x7fb74009b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 -- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb740074680 msgr2=0x7fb7400c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73e59c640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb740074680 0x7fb7400c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb740074680 0x7fb7400c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 -- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740074270 msgr2=0x7fb74009b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740074270 0x7fb74009b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 -- 172.21.15.170:0/2367873839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7400b1570 con 0x7fb7400cf7d0 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73ed9d640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740074270 0x7fb74009b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb73f59e640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 0x7fb7400c50e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fb73800cce0 tx=0x7fb73800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73800f450 con 0x7fb7400cf7d0 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb73800f6e0 con 0x7fb7400cf7d0 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7400c6430 con 0x7fb7400cf7d0 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73800fb30 con 0x7fb7400cf7d0 2024-09-21T02:22:35.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7400c4680 con 0x7fb7400cf7d0 2024-09-21T02:22:35.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.591+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb740073b60 con 0x7fb7400cf7d0 2024-09-21T02:22:35.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.595+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fb73800fcd0 con 0x7fb7400cf7d0 2024-09-21T02:22:35.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.595+0000 7fb7277fe640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb72c073c20 0x7fb72c076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:35.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.595+0000 7fb73ed9d640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb72c073c20 0x7fb72c076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:35.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.595+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fb738093a80 con 0x7fb7400cf7d0 2024-09-21T02:22:35.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.595+0000 7fb73ed9d640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb72c073c20 0x7fb72c076130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb73400a850 tx=0x7fb734008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:35.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.599+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fb738014030 con 0x7fb7400cf7d0 2024-09-21T02:22:35.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.759+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb74015e5e0 con 0x7fb7400cf7d0 2024-09-21T02:22:35.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.759+0000 7fb7277fe640 1 -- 172.21.15.170:0/2367873839 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7fb738060290 con 0x7fb7400cf7d0 2024-09-21T02:22:35.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb72c073c20 msgr2=0x7fb72c076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb72c073c20 0x7fb72c076130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb73400a850 tx=0x7fb734008040 comp rx=0 tx=0).stop 2024-09-21T02:22:35.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 msgr2=0x7fb7400c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:35.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 0x7fb7400c50e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fb73800cce0 tx=0x7fb73800a620 comp rx=0 tx=0).stop 2024-09-21T02:22:35.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 shutdown_connections 2024-09-21T02:22:35.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fb72c073c20 0x7fb72c076130 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb740074680 0x7fb7400c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7400cf7d0 0x7fb7400c50e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 --2- 172.21.15.170:0/2367873839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740074270 0x7fb74009b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:35.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 >> 172.21.15.170:0/2367873839 conn(0x7fb7400d0170 msgr2=0x7fb7400ad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:35.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 shutdown_connections 2024-09-21T02:22:35.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:35.763+0000 7fb745092640 1 -- 172.21.15.170:0/2367873839 wait complete. 2024-09-21T02:22:35.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:35.784 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp3460/subvol71190) 2024-09-21T02:22:35.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:35.806 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T02:22:35.858 INFO:teuthology.orchestra.run.smithi170.stdout:cephfs_data 2024-09-21T02:22:35.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:35.858 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp3460/subvol71190) 2024-09-21T02:22:35.917 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:35.917 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp3460/subvol71190/.meta 2024-09-21T02:22:35.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol71190 subvol_grp3460 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 -- 172.21.15.170:0/1459521238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 msgr2=0x7f928806f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 --2- 172.21.15.170:0/1459521238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f928806f6e0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f92880ce1d0 tx=0x7f928001ccf0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 -- 172.21.15.170:0/1459521238 shutdown_connections 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 --2- 172.21.15.170:0/1459521238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f928806f6e0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 --2- 172.21.15.170:0/1459521238 >> v2:172.21.15.191:3300/0 conn(0x7f92880ca720 0x7f92880cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 -- 172.21.15.170:0/1459521238 >> 172.21.15.170:0/1459521238 conn(0x7f92880b1c40 msgr2=0x7f92880b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 -- 172.21.15.170:0/1459521238 shutdown_connections 2024-09-21T02:22:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.187+0000 7f928cbb5640 1 -- 172.21.15.170:0/1459521238 wait complete. 2024-09-21T02:22:36.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 Processor -- start 2024-09-21T02:22:36.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 -- start start 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f92881f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f928809dc00 0x7f9288156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92880ca720 0x7f92881f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9288167d70 con 0x7f928806f2d0 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9288070600 con 0x7f928809dc00 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928cbb5640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f9288167ef0 con 0x7f92880ca720 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f92881f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f92881f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43070/0 (socket says 172.21.15.170:43070) 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.191+0000 7f928659c640 1 -- 172.21.15.170:0/2099863700 learned_addr learned my addr 172.21.15.170:0/2099863700 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f927dd9b640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f928809dc00 0x7f9288156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f9286d9d640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92880ca720 0x7f92881f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928659c640 1 -- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92880ca720 msgr2=0x7f92881f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928659c640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92880ca720 0x7f92881f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928659c640 1 -- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f928809dc00 msgr2=0x7f9288156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928659c640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f928809dc00 0x7f9288156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928659c640 1 -- 172.21.15.170:0/2099863700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9288156ec0 con 0x7f928806f2d0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f9286d9d640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92880ca720 0x7f92881f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928659c640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f92881f8190 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f927000c3a0 tx=0x7f927000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270017070 con 0x7f928806f2d0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92700053c0 con 0x7f928806f2d0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92881f89c0 con 0x7f928806f2d0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700038b0 con 0x7f928806f2d0 2024-09-21T02:22:36.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92881f8f80 con 0x7f928806f2d0 2024-09-21T02:22:36.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928806f6e0 con 0x7f928806f2d0 2024-09-21T02:22:36.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.195+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7f927001a070 con 0x7f928806f2d0 2024-09-21T02:22:36.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.199+0000 7f927f7fe640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9268073c20 0x7f9268076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.199+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f92700924f0 con 0x7f928806f2d0 2024-09-21T02:22:36.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.199+0000 7f927dd9b640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9268073c20 0x7f9268076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.199+0000 7f927dd9b640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9268073c20 0x7f9268076130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f928806fa30 tx=0x7f9278011040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:36.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.203+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f9270021020 con 0x7f928806f2d0 2024-09-21T02:22:36.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.395+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol71190", "group_name": "subvol_grp3460", "target": ["mon-mgr", ""]}) -- 0x7f92880cb3c0 con 0x7f9268073c20 2024-09-21T02:22:36.410 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:36.407+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:36.410 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:36.407+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:36.411 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:36.407+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:36.411 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:36.407+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:36.411 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:36.407+0000 7efb9f2c4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:36.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f927f7fe640 1 -- 172.21.15.170:0/2099863700 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f92880cb3c0 con 0x7f9268073c20 2024-09-21T02:22:36.464 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp3460/subvol71190 2024-09-21T02:22:36.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9268073c20 msgr2=0x7f9268076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9268073c20 0x7f9268076130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f928806fa30 tx=0x7f9278011040 comp rx=0 tx=0).stop 2024-09-21T02:22:36.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 msgr2=0x7f92881f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f92881f8190 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f927000c3a0 tx=0x7f927000c860 comp rx=0 tx=0).stop 2024-09-21T02:22:36.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 shutdown_connections 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9268073c20 0x7f9268076130 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92880ca720 0x7f92881f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f928809dc00 0x7f9288156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 --2- 172.21.15.170:0/2099863700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f928806f2d0 0x7f92881f8190 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 >> 172.21.15.170:0/2099863700 conn(0x7f92880b1c40 msgr2=0x7f92880b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 shutdown_connections 2024-09-21T02:22:36.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.463+0000 7f928cbb5640 1 -- 172.21.15.170:0/2099863700 wait complete. 2024-09-21T02:22:36.485 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol71190 subvol_grp3460 2024-09-21T02:22:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 -- 172.21.15.170:0/1493566816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac4c004b20 msgr2=0x7fac4c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 --2- 172.21.15.170:0/1493566816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac4c004b20 0x7fac4c004f90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fac6000bb20 tx=0x7fac6001cdc0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac69614640 1 -- 172.21.15.170:0/1493566816 reap_dead start 2024-09-21T02:22:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 -- 172.21.15.170:0/1493566816 shutdown_connections 2024-09-21T02:22:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 -- 172.21.15.170:0/1493566816 >> 172.21.15.170:0/1493566816 conn(0x7fac640b1c40 msgr2=0x7fac640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 -- 172.21.15.170:0/1493566816 shutdown_connections 2024-09-21T02:22:36.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 -- 172.21.15.170:0/1493566816 wait complete. 2024-09-21T02:22:36.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 Processor -- start 2024-09-21T02:22:36.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.635+0000 7fac6b077640 1 -- start start 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac641fd490 0x7fac64074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac64156da0 0x7fac640c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 0x7fac64158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fac64165a30 con 0x7fac64158a80 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fac6415b340 con 0x7fac64156da0 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fac64165bb0 con 0x7fac641fd490 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 0x7fac64158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 0x7fac64158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43112/0 (socket says 172.21.15.170:43112) 2024-09-21T02:22:36.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 -- 172.21.15.170:0/2054441651 learned_addr learned my addr 172.21.15.170:0/2054441651 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac68e13640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac641fd490 0x7fac64074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac5bfff640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac64156da0 0x7fac640c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 -- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac641fd490 msgr2=0x7fac64074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac641fd490 0x7fac64074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 -- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac64156da0 msgr2=0x7fac640c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac64156da0 0x7fac640c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 -- 172.21.15.170:0/2054441651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac641593d0 con 0x7fac64158a80 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac69614640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 0x7fac64158e90 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000f90 tx=0x7fac5c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00b7a0 con 0x7fac64158a80 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac5c015070 con 0x7fac64158a80 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac641fc9f0 con 0x7fac64158a80 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c00bcb0 con 0x7fac64158a80 2024-09-21T02:22:36.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac641fcc90 con 0x7fac64158a80 2024-09-21T02:22:36.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac640b2f40 con 0x7fac64158a80 2024-09-21T02:22:36.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) ==== 98152+0+0 (secure 0 0 0) 0x7fac5c006070 con 0x7fac64158a80 2024-09-21T02:22:36.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.639+0000 7fac597fa640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac50073c20 0x7fac50076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:36.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.643+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fac5c092f80 con 0x7fac64158a80 2024-09-21T02:22:36.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.643+0000 7fac68e13640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac50073c20 0x7fac50076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:36.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.643+0000 7fac68e13640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac50073c20 0x7fac50076130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac600b0000 tx=0x7fac600a7040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:36.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.647+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fac641593d0 con 0x7fac64158a80 2024-09-21T02:22:36.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.847+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol71190", "group_name": "subvol_grp3460", "target": ["mon-mgr", ""]}) -- 0x7fac640b1940 con 0x7fac50073c20 2024-09-21T02:22:36.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.859+0000 7fac597fa640 1 -- 172.21.15.170:0/2054441651 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fac640b1940 con 0x7fac50073c20 2024-09-21T02:22:36.862 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/subvol_grp3460/subvol71190 2024-09-21T02:22:36.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac50073c20 msgr2=0x7fac50076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac50073c20 0x7fac50076130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac600b0000 tx=0x7fac600a7040 comp rx=0 tx=0).stop 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 msgr2=0x7fac64158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 0x7fac64158e90 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000f90 tx=0x7fac5c00dcd0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 shutdown_connections 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fac50073c20 0x7fac50076130 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac64158a80 0x7fac64158e90 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac64156da0 0x7fac640c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 --2- 172.21.15.170:0/2054441651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac641fd490 0x7fac64074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 >> 172.21.15.170:0/2054441651 conn(0x7fac640b1c40 msgr2=0x7fac641647c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 shutdown_connections 2024-09-21T02:22:36.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:36.863+0000 7fac6b077640 1 -- 172.21.15.170:0/2054441651 wait complete. 2024-09-21T02:22:36.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:36.884 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/90fcba56d888744a7423bcd9b60498bb.meta) 2024-09-21T02:22:37.926 INFO:teuthology.orchestra.run.smithi170.stdout:[GLOBAL] 2024-09-21T02:22:37.926 INFO:teuthology.orchestra.run.smithi170.stdout:version = 1 2024-09-21T02:22:37.926 INFO:teuthology.orchestra.run.smithi170.stdout:type = subvolume 2024-09-21T02:22:37.926 INFO:teuthology.orchestra.run.smithi170.stdout:path = /volumes/subvol_grp3460/subvol71190 2024-09-21T02:22:37.926 INFO:teuthology.orchestra.run.smithi170.stdout:state = complete 2024-09-21T02:22:37.927 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:37.929 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol71190 subvol_grp3460 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 -- 172.21.15.170:0/3606773736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac15eb10 msgr2=0x7fdfac15ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/3606773736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac15eb10 0x7fdfac15ef20 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009bc0 tx=0x7fdfa401cad0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 -- 172.21.15.170:0/3606773736 shutdown_connections 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/3606773736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac15eb10 0x7fdfac15ef20 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/3606773736 >> v2:172.21.15.191:3300/0 conn(0x7fdfac0b5f60 0x7fdfac0b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 -- 172.21.15.170:0/3606773736 >> 172.21.15.170:0/3606773736 conn(0x7fdfac0d44e0 msgr2=0x7fdfac0d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 -- 172.21.15.170:0/3606773736 shutdown_connections 2024-09-21T02:22:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.103+0000 7fdfb2f28640 1 -- 172.21.15.170:0/3606773736 wait complete. 2024-09-21T02:22:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 Processor -- start 2024-09-21T02:22:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 -- start start 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 0x7fdfac150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac15eb10 0x7fdfac0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac1601d0 0x7fdfac1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac16c3b0 con 0x7fdfac0b5f60 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac16c230 con 0x7fdfac15eb10 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac16c530 con 0x7fdfac1601d0 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 0x7fdfac150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 0x7fdfac150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43144/0 (socket says 172.21.15.170:43144) 2024-09-21T02:22:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 -- 172.21.15.170:0/2086959356 learned_addr learned my addr 172.21.15.170:0/2086959356 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfabfff640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac15eb10 0x7fdfac0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb14c5640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac1601d0 0x7fdfac1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 -- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac1601d0 msgr2=0x7fdfac1f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac1601d0 0x7fdfac1f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 -- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac15eb10 msgr2=0x7fdfac0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac15eb10 0x7fdfac0c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 -- 172.21.15.170:0/2086959356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac0c92c0 con 0x7fdfac0b5f60 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb14c5640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac1601d0 0x7fdfac1f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb0cc4640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 0x7fdfac150450 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00ca20 tx=0x7fdf9c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c007c70 con 0x7fdfac0b5f60 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf9c00f450 con 0x7fdfac0b5f60 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00f8a0 con 0x7fdfac0b5f60 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac1f9c70 con 0x7fdfac0b5f60 2024-09-21T02:22:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.107+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac150d00 con 0x7fdfac0b5f60 2024-09-21T02:22:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.111+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac150af0 con 0x7fdfac0b5f60 2024-09-21T02:22:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.111+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) ==== 98203+0+0 (secure 0 0 0) 0x7fdf9c018400 con 0x7fdfac0b5f60 2024-09-21T02:22:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.111+0000 7fdfa97fa640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdf98073cb0 0x7fdf980761c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.111+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fdf9c093980 con 0x7fdfac0b5f60 2024-09-21T02:22:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.111+0000 7fdfabfff640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdf98073cb0 0x7fdf980761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.111+0000 7fdfabfff640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdf98073cb0 0x7fdf980761c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdfac15b880 tx=0x7fdfa0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:38.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.115+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fdf9c026020 con 0x7fdfac0b5f60 2024-09-21T02:22:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.311+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol71190", "group_name": "subvol_grp3460", "target": ["mon-mgr", ""]}) -- 0x7fdfac0b6370 con 0x7fdf98073cb0 2024-09-21T02:22:38.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.343+0000 7fdfa97fa640 1 -- 172.21.15.170:0/2086959356 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdfac0b6370 con 0x7fdf98073cb0 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdf98073cb0 msgr2=0x7fdf980761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdf98073cb0 0x7fdf980761c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdfac15b880 tx=0x7fdfa0009040 comp rx=0 tx=0).stop 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 msgr2=0x7fdfac150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 0x7fdfac150450 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00ca20 tx=0x7fdf9c00cee0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 shutdown_connections 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fdf98073cb0 0x7fdf980761c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac1601d0 0x7fdfac1f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac15eb10 0x7fdfac0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 --2- 172.21.15.170:0/2086959356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac0b5f60 0x7fdfac150450 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 >> 172.21.15.170:0/2086959356 conn(0x7fdfac0d44e0 msgr2=0x7fdfac15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 shutdown_connections 2024-09-21T02:22:38.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.347+0000 7fdfb2f28640 1 -- 172.21.15.170:0/2086959356 wait complete. 2024-09-21T02:22:38.358 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.355+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.358 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.355+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.358 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.355+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.358 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.355+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.358 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.355+0000 7efba22ca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.373 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T02:22:38.375 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.375+0000 7efba1ac9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.376 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.375+0000 7efba1ac9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.376 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.375+0000 7efba1ac9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.376 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.375+0000 7efba1ac9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.376 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-09-21T02:22:38.375+0000 7efba1ac9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:22:38.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T02:22:38.419 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T02:22:38.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T02:22:38.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp3460 2024-09-21T02:22:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 -- 172.21.15.170:0/2501237176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 msgr2=0x7ff44c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 --2- 172.21.15.170:0/2501237176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c0cb280 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009840 tx=0x7ff43c01c820 comp rx=0 tx=0).stop 2024-09-21T02:22:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 -- 172.21.15.170:0/2501237176 shutdown_connections 2024-09-21T02:22:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 --2- 172.21.15.170:0/2501237176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c0cb280 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 -- 172.21.15.170:0/2501237176 >> 172.21.15.170:0/2501237176 conn(0x7ff44c0d0170 msgr2=0x7ff44c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:38.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 -- 172.21.15.170:0/2501237176 shutdown_connections 2024-09-21T02:22:38.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 -- 172.21.15.170:0/2501237176 wait complete. 2024-09-21T02:22:38.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 Processor -- start 2024-09-21T02:22:38.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.631+0000 7ff453d8c640 1 -- start start 2024-09-21T02:22:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff44c074680 0x7ff44c0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff44c0cf7d0 0x7ff44c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c16a2b0 con 0x7ff44c074270 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c16a130 con 0x7ff44c074680 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7ff44c16a430 con 0x7ff44c0cf7d0 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43190/0 (socket says 172.21.15.170:43190) 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 -- 172.21.15.170:0/4130889233 learned_addr learned my addr 172.21.15.170:0/4130889233 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451327640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff44c074680 0x7ff44c0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff452329640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff44c0cf7d0 0x7ff44c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 -- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff44c0cf7d0 msgr2=0x7ff44c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff44c0cf7d0 0x7ff44c0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 -- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff44c074680 msgr2=0x7ff44c0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff44c074680 0x7ff44c0b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 -- 172.21.15.170:0/4130889233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c1f6e80 con 0x7ff44c074270 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff451b28640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c15bfc0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7ff44800cb70 tx=0x7ff44800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448016020 con 0x7ff44c074270 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff44800ab00 con 0x7ff44c074270 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c09dca0 con 0x7ff44c074270 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448012420 con 0x7ff44c074270 2024-09-21T02:22:38.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c09e220 con 0x7ff44c074270 2024-09-21T02:22:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.635+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44c073b60 con 0x7ff44c074270 2024-09-21T02:22:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.639+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) ==== 98203+0+0 (secure 0 0 0) 0x7ff448019020 con 0x7ff44c074270 2024-09-21T02:22:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.639+0000 7ff4427fc640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff430073cb0 0x7ff4300761c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.639+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7ff448093340 con 0x7ff44c074270 2024-09-21T02:22:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.639+0000 7ff451327640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff430073cb0 0x7ff4300761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.639+0000 7ff451327640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff430073cb0 0x7ff4300761c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0af000 tx=0x7ff43c01b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.643+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7ff448020020 con 0x7ff44c074270 2024-09-21T02:22:38.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.839+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp3460", "target": ["mon-mgr", ""]}) -- 0x7ff44c155db0 con 0x7ff430073cb0 2024-09-21T02:22:38.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.843+0000 7ff4427fc640 1 -- 172.21.15.170:0/4130889233 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff44c155db0 con 0x7ff430073cb0 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff430073cb0 msgr2=0x7ff4300761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff430073cb0 0x7ff4300761c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0af000 tx=0x7ff43c01b060 comp rx=0 tx=0).stop 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 msgr2=0x7ff44c15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c15bfc0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7ff44800cb70 tx=0x7ff44800cc60 comp rx=0 tx=0).stop 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 shutdown_connections 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7ff430073cb0 0x7ff4300761c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff44c0cf7d0 0x7ff44c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff44c074680 0x7ff44c0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 --2- 172.21.15.170:0/4130889233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff44c074270 0x7ff44c15bfc0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 >> 172.21.15.170:0/4130889233 conn(0x7ff44c0d0170 msgr2=0x7ff44c154140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 shutdown_connections 2024-09-21T02:22:38.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:38.847+0000 7ff453d8c640 1 -- 172.21.15.170:0/4130889233 wait complete. 2024-09-21T02:22:38.869 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T02:22:38.869 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:22:38.928 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T02:22:39.014 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T02:22:39.015 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:39.015 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:39.015 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:39.021 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:39.021 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:39.069 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 -- 172.21.15.170:0/4228420598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cf7e0 msgr2=0x7f70e8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 --2- 172.21.15.170:0/4228420598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cf7e0 0x7f70e8070340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009840 tx=0x7f70d801c820 comp rx=0 tx=0).stop 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 -- 172.21.15.170:0/4228420598 shutdown_connections 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 --2- 172.21.15.170:0/4228420598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70e80cb460 0x7f70e80cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 --2- 172.21.15.170:0/4228420598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cf7e0 0x7f70e8070340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 -- 172.21.15.170:0/4228420598 >> 172.21.15.170:0/4228420598 conn(0x7f70e80b1c40 msgr2=0x7f70e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 -- 172.21.15.170:0/4228420598 shutdown_connections 2024-09-21T02:22:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 -- 172.21.15.170:0/4228420598 wait complete. 2024-09-21T02:22:39.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 Processor -- start 2024-09-21T02:22:39.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.275+0000 7f70ed235640 1 -- start start 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cb460 0x7f70e81658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 0x7f70e80c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70e80cf7e0 0x7f70e806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81690d0 con 0x7f70e80cc8c0 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8168f50 con 0x7f70e80cb460 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f70e8169250 con 0x7f70e80cf7e0 2024-09-21T02:22:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 0x7f70e80c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 0x7f70e80c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43200/0 (socket says 172.21.15.170:43200) 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 -- 172.21.15.170:0/2230522163 learned_addr learned my addr 172.21.15.170:0/2230522163 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70e6d9d640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cb460 0x7f70e81658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70e759e640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70e80cf7e0 0x7f70e806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 -- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70e80cf7e0 msgr2=0x7f70e806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70e80cf7e0 0x7f70e806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 -- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cb460 msgr2=0x7f70e81658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cb460 0x7f70e81658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 -- 172.21.15.170:0/2230522163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e80b2f40 con 0x7f70e80cc8c0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70dffff640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 0x7f70e80c80f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e7c0 tx=0x7f70d000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000cc60 con 0x7f70e80cc8c0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70d000ce20 con 0x7f70e80cc8c0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e80b0fb0 con 0x7f70e80cc8c0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0005280 con 0x7f70e80cc8c0 2024-09-21T02:22:39.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81f71c0 con 0x7f70e80cc8c0 2024-09-21T02:22:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e80cb870 con 0x7f70e80cc8c0 2024-09-21T02:22:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.279+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) ==== 98203+0+0 (secure 0 0 0) 0x7f70d0005420 con 0x7f70e80cc8c0 2024-09-21T02:22:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.283+0000 7f70df7fe640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70c8073cb0 0x7f70c80761c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.283+0000 7f70e6d9d640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70c8073cb0 0x7f70c80761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.283+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f70d0093c20 con 0x7f70e80cc8c0 2024-09-21T02:22:39.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.283+0000 7f70e6d9d640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70c8073cb0 0x7f70c80761c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f70d80af000 tx=0x7f70d801b060 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:39.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.283+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f70d00604a0 con 0x7f70e80cc8c0 2024-09-21T02:22:39.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.471+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f70e806de10 con 0x7f70e80cc8c0 2024-09-21T02:22:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.471+0000 7f70df7fe640 1 -- 172.21.15.170:0/2230522163 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v166) ==== 80+0+0 (secure 0 0 0) 0x7f70d005fda0 con 0x7f70e80cc8c0 2024-09-21T02:22:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:22:39.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70c8073cb0 msgr2=0x7f70c80761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70c8073cb0 0x7f70c80761c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f70d80af000 tx=0x7f70d801b060 comp rx=0 tx=0).stop 2024-09-21T02:22:39.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 msgr2=0x7f70e80c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 0x7f70e80c80f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e7c0 tx=0x7f70d000ec80 comp rx=0 tx=0).stop 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 shutdown_connections 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f70c8073cb0 0x7f70c80761c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70e80cf7e0 0x7f70e806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80cc8c0 0x7f70e80c80f0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 --2- 172.21.15.170:0/2230522163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70e80cb460 0x7f70e81658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 >> 172.21.15.170:0/2230522163 conn(0x7f70e80b1c40 msgr2=0x7f70e80ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 shutdown_connections 2024-09-21T02:22:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.475+0000 7f70ed235640 1 -- 172.21.15.170:0/2230522163 wait complete. 2024-09-21T02:22:39.494 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi170.front.sepia.ceph.com... 2024-09-21T02:22:39.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:39.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T02:22:39.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi170.stderr:ceph-fuse[53734]: fuse finished with error 0 and tester_r 0 2024-09-21T02:22:39.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:22:39.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:39.507 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T02:22:39.664 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:39.665 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:39.665 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:39.671 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:39.671 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:39.722 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- 172.21.15.170:0/2363222497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200ce1c0 msgr2=0x7fd6200ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 --2- 172.21.15.170:0/2363222497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200ce1c0 0x7fd6200ce5d0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd6200d1160 tx=0x7fd61801cb80 comp rx=0 tx=0).stop 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- 172.21.15.170:0/2363222497 shutdown_connections 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 --2- 172.21.15.170:0/2363222497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200ce1c0 0x7fd6200ce5d0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 --2- 172.21.15.170:0/2363222497 >> v2:172.21.15.191:3300/0 conn(0x7fd6200cf7e0 0x7fd620070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- 172.21.15.170:0/2363222497 >> 172.21.15.170:0/2363222497 conn(0x7fd6200b1c40 msgr2=0x7fd6200b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- 172.21.15.170:0/2363222497 shutdown_connections 2024-09-21T02:22:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- 172.21.15.170:0/2363222497 wait complete. 2024-09-21T02:22:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 Processor -- start 2024-09-21T02:22:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- start start 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 0x7fd620154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6200ce1c0 0x7fd620155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6200cf7e0 0x7fd620158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd62006f240 con 0x7fd6200cb460 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd62006f0c0 con 0x7fd6200cf7e0 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd627eda640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd62006f3c0 con 0x7fd6200ce1c0 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd625c76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 0x7fd620154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd625c76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 0x7fd620154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43230/0 (socket says 172.21.15.170:43230) 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.875+0000 7fd625c76640 1 -- 172.21.15.170:0/1472661471 learned_addr learned my addr 172.21.15.170:0/1472661471 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd626477640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6200cf7e0 0x7fd620158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625475640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6200ce1c0 0x7fd620155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625c76640 1 -- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6200ce1c0 msgr2=0x7fd620155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625c76640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6200ce1c0 0x7fd620155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625c76640 1 -- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6200cf7e0 msgr2=0x7fd620158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625c76640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6200cf7e0 0x7fd620158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625c76640 1 -- 172.21.15.170:0/1472661471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd620140e50 con 0x7fd6200cb460 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625c76640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 0x7fd620154510 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c970 tx=0x7fd61c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c004220 con 0x7fd6200cb460 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd61c0079b0 con 0x7fd6200cb460 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0044b0 con 0x7fd6200cb460 2024-09-21T02:22:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd620141130 con 0x7fd6200cb460 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201416f0 con 0x7fd6200cb460 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6200ce5d0 con 0x7fd6200cb460 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) ==== 98203+0+0 (secure 0 0 0) 0x7fd61c003600 con 0x7fd6200cb460 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd6167fc640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd604073cb0 0x7fd6040761c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625475640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd604073cb0 0x7fd6040761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fd61c092be0 con 0x7fd6200cb460 2024-09-21T02:22:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.879+0000 7fd625475640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd604073cb0 0x7fd6040761c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd620143dd0 tx=0x7fd610009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:39.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:39.883+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd61c014030 con 0x7fd6200cb460 2024-09-21T02:22:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.063+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fd61c05b2b0 con 0x7fd6200cb460 2024-09-21T02:22:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.067+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd6201662b0 con 0x7fd6200cb460 2024-09-21T02:22:40.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd6167fc640 1 -- 172.21.15.170:0/1472661471 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v166) ==== 80+0+0 (secure 0 0 0) 0x7fd61c05f740 con 0x7fd6200cb460 2024-09-21T02:22:40.071 INFO:teuthology.orchestra.run.smithi170.stderr:listed 0 entries 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd604073cb0 msgr2=0x7fd6040761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd604073cb0 0x7fd6040761c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd620143dd0 tx=0x7fd610009040 comp rx=0 tx=0).stop 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 msgr2=0x7fd620154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 0x7fd620154510 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c970 tx=0x7fd61c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 shutdown_connections 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd604073cb0 0x7fd6040761c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6200ce1c0 0x7fd620155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6200cf7e0 0x7fd620158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 --2- 172.21.15.170:0/1472661471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6200cb460 0x7fd620154510 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 >> 172.21.15.170:0/1472661471 conn(0x7fd6200b1c40 msgr2=0x7fd6200b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 shutdown_connections 2024-09-21T02:22:40.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.071+0000 7fd627eda640 1 -- 172.21.15.170:0/1472661471 wait complete. 2024-09-21T02:22:40.093 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2024-09-21T02:22:40.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:40.093 DEBUG:teuthology.orchestra.run.smithi191:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-21T02:22:40.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi191.stderr:ceph-fuse[36023]: fuse finished with error 0 and tester_r 0 2024-09-21T02:22:40.107 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:22:40.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:40.107 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T02:22:40.234 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:40.234 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:40.234 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:40.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:22:40.241 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T02:22:40.241 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:40.241 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:40.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:22:40.247 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 -- 172.21.15.170:0/3165260710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80ccb00 msgr2=0x7fc2d80ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 --2- 172.21.15.170:0/3165260710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80ccb00 0x7fc2d80ccf10 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8070e90 tx=0x7fc2d001cba0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 -- 172.21.15.170:0/3165260710 shutdown_connections 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 --2- 172.21.15.170:0/3165260710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80ccb00 0x7fc2d80ccf10 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 --2- 172.21.15.170:0/3165260710 >> v2:172.21.15.191:3300/0 conn(0x7fc2d80b2f40 0x7fc2d80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 -- 172.21.15.170:0/3165260710 >> 172.21.15.170:0/3165260710 conn(0x7fc2d80b1c40 msgr2=0x7fc2d80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 -- 172.21.15.170:0/3165260710 shutdown_connections 2024-09-21T02:22:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.439+0000 7fc2e0205640 1 -- 172.21.15.170:0/3165260710 wait complete. 2024-09-21T02:22:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 Processor -- start 2024-09-21T02:22:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 -- start start 2024-09-21T02:22:40.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 0x7fc2d81fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2d80b5410 0x7fc2d809bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2d80ccb00 0x7fc2d809c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8169e50 con 0x7fc2d80b2f40 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d80cf970 con 0x7fc2d80ccb00 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d8169fd0 con 0x7fc2d80b5410 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 0x7fc2d81fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 0x7fc2d81fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43248/0 (socket says 172.21.15.170:43248) 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 -- 172.21.15.170:0/1263597295 learned_addr learned my addr 172.21.15.170:0/1263597295 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2de7a2640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2d80ccb00 0x7fc2d809c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2dd7a0640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2d80b5410 0x7fc2d809bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 -- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2d80b5410 msgr2=0x7fc2d809bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2d80b5410 0x7fc2d809bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 -- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2d80ccb00 msgr2=0x7fc2d809c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2d80ccb00 0x7fc2d809c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 -- 172.21.15.170:0/1263597295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d81f67d0 con 0x7fc2d80b2f40 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2de7a2640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2d80ccb00 0x7fc2d809c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ddfa1640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 0x7fc2d81fa820 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800b3e0 tx=0x7fc2c800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8015020 con 0x7fc2d80b2f40 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2c8011940 con 0x7fc2d80b2f40 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d81f6c50 con 0x7fc2d80b2f40 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8011d90 con 0x7fc2d80b2f40 2024-09-21T02:22:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.443+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d81f7210 con 0x7fc2d80b2f40 2024-09-21T02:22:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.447+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d80b3350 con 0x7fc2d80b2f40 2024-09-21T02:22:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.447+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fc2c8018020 con 0x7fc2d80b2f40 2024-09-21T02:22:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.447+0000 7fc2ce7fc640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc2bc073e00 0x7fc2bc076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.447+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fc2c8092750 con 0x7fc2d80b2f40 2024-09-21T02:22:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.447+0000 7fc2dd7a0640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc2bc073e00 0x7fc2bc076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.447+0000 7fc2dd7a0640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc2bc073e00 0x7fc2bc076310 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc2d809c500 tx=0x7fc2d4005bf0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.451+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc2c801f070 con 0x7fc2d80b2f40 2024-09-21T02:22:40.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.647+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2d80cf3a0 con 0x7fc2d80b2f40 2024-09-21T02:22:40.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.647+0000 7fc2ce7fc640 1 -- 172.21.15.170:0/1263597295 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5245 (secure 0 0 0) 0x7fc2c805f050 con 0x7fc2d80b2f40 2024-09-21T02:22:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:40.651 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":121,"btime":"2024-09-21T02:22:35:005454+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:22:21.863736+0000","modified":"2024-09-21T02:22:24.886454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9303":{"gid":9303,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1712756729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1712756729},{"type":"v1","addr":"172.21.15.170:6835","nonce":1712756729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9303,"qdb_cluster":[9303]},"id":13}]} 2024-09-21T02:22:40.651 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 121 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc2bc073e00 msgr2=0x7fc2bc076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc2bc073e00 0x7fc2bc076310 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc2d809c500 tx=0x7fc2d4005bf0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 msgr2=0x7fc2d81fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 0x7fc2d81fa820 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800b3e0 tx=0x7fc2c800b8a0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 shutdown_connections 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc2bc073e00 0x7fc2bc076310 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2d80b5410 0x7fc2d809bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2d80ccb00 0x7fc2d809c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 --2- 172.21.15.170:0/1263597295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2d80b2f40 0x7fc2d81fa820 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 >> 172.21.15.170:0/1263597295 conn(0x7fc2d80b1c40 msgr2=0x7fc2d80c4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 shutdown_connections 2024-09-21T02:22:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.651+0000 7fc2e0205640 1 -- 172.21.15.170:0/1263597295 wait complete. 2024-09-21T02:22:40.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:40.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 -- 172.21.15.170:0/3487140494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cf7e0 msgr2=0x7f64a0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3487140494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cf7e0 0x7f64a0070340 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f64a0072de0 tx=0x7f649c01c8c0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 -- 172.21.15.170:0/3487140494 shutdown_connections 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3487140494 >> v2:172.21.15.191:3300/0 conn(0x7f64a00cc8c0 0x7f64a00cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3487140494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64a00cb460 0x7f64a00cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3487140494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cf7e0 0x7f64a0070340 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 -- 172.21.15.170:0/3487140494 >> 172.21.15.170:0/3487140494 conn(0x7f64a00b1c40 msgr2=0x7f64a00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 -- 172.21.15.170:0/3487140494 shutdown_connections 2024-09-21T02:22:40.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 -- 172.21.15.170:0/3487140494 wait complete. 2024-09-21T02:22:40.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 Processor -- start 2024-09-21T02:22:40.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.819+0000 7f64a87e8640 1 -- start start 2024-09-21T02:22:40.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64a00cb460 0x7f64a009cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 0x7f64a015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64a00cf7e0 0x7f64a006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00ced60 con 0x7f64a00cc8c0 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00cebe0 con 0x7f64a00cf7e0 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f64a00ceee0 con 0x7f64a00cb460 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 0x7f64a015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 0x7f64a015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43260/0 (socket says 172.21.15.170:43260) 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 -- 172.21.15.170:0/3046274669 learned_addr learned my addr 172.21.15.170:0/3046274669 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a6584640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64a00cb460 0x7f64a009cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a6d85640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64a00cf7e0 0x7f64a006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 -- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64a00cb460 msgr2=0x7f64a009cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64a00cb460 0x7f64a009cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 -- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64a00cf7e0 msgr2=0x7f64a006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64a00cf7e0 0x7f64a006eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 -- 172.21.15.170:0/3046274669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a009d0d0 con 0x7f64a00cc8c0 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a6d85640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64a00cf7e0 0x7f64a006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a5d83640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 0x7f64a015e720 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f649000b410 tx=0x7f649000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490015020 con 0x7f64a00cc8c0 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6490011920 con 0x7f64a00cc8c0 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490011d70 con 0x7f64a00cc8c0 2024-09-21T02:22:40.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a009d5e0 con 0x7f64a00cc8c0 2024-09-21T02:22:40.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a015c1c0 con 0x7f64a00cc8c0 2024-09-21T02:22:40.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a009d350 con 0x7f64a00cc8c0 2024-09-21T02:22:40.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f6490018020 con 0x7f64a00cc8c0 2024-09-21T02:22:40.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f6496ffd640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6484073e00 0x7f6484076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:40.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f6490092c90 con 0x7f64a00cc8c0 2024-09-21T02:22:40.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.823+0000 7f64a6584640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6484073e00 0x7f6484076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:40.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.827+0000 7f64a6584640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6484073e00 0x7f6484076310 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f649c0af000 tx=0x7f649c01b100 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:40.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:40.831+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f649001f020 con 0x7f64a00cc8c0 2024-09-21T02:22:41.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.039+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64a015ad50 con 0x7f64a00cc8c0 2024-09-21T02:22:41.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.039+0000 7f6496ffd640 1 -- 172.21.15.170:0/3046274669 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5245 (secure 0 0 0) 0x7f64a015ad50 con 0x7f64a00cc8c0 2024-09-21T02:22:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:41.042 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":121,"btime":"2024-09-21T02:22:35:005454+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:22:21.863736+0000","modified":"2024-09-21T02:22:24.886454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9303":{"gid":9303,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1712756729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1712756729},{"type":"v1","addr":"172.21.15.170:6835","nonce":1712756729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9303,"qdb_cluster":[9303]},"id":13}]} 2024-09-21T02:22:41.043 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 121 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6484073e00 msgr2=0x7f6484076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6484073e00 0x7f6484076310 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f649c0af000 tx=0x7f649c01b100 comp rx=0 tx=0).stop 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 msgr2=0x7f64a015e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 0x7f64a015e720 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f649000b410 tx=0x7f649000b8d0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 shutdown_connections 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f6484073e00 0x7f6484076310 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64a00cf7e0 0x7f64a006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64a00cc8c0 0x7f64a015e720 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 --2- 172.21.15.170:0/3046274669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64a00cb460 0x7f64a009cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 >> 172.21.15.170:0/3046274669 conn(0x7f64a00b1c40 msgr2=0x7f64a00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:41.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 shutdown_connections 2024-09-21T02:22:41.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.043+0000 7f64a87e8640 1 -- 172.21.15.170:0/3046274669 wait complete. 2024-09-21T02:22:41.065 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 -- 172.21.15.170:0/752556359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b2f40 msgr2=0x7fa2340b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/752556359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b2f40 0x7fa2340b3350 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fa234071d80 tx=0x7fa22401cb80 comp rx=0 tx=0).stop 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 -- 172.21.15.170:0/752556359 shutdown_connections 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/752556359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b2f40 0x7fa2340b3350 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/752556359 >> v2:172.21.15.191:3300/0 conn(0x7fa2340b5410 0x7fa23406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 -- 172.21.15.170:0/752556359 >> 172.21.15.170:0/752556359 conn(0x7fa2340b1c40 msgr2=0x7fa2340b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 -- 172.21.15.170:0/752556359 shutdown_connections 2024-09-21T02:22:41.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.211+0000 7fa23a0d8640 1 -- 172.21.15.170:0/752556359 wait complete. 2024-09-21T02:22:41.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 Processor -- start 2024-09-21T02:22:41.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- start start 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2340b2f40 0x7fa23414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 0x7fa2341fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2340ccb00 0x7fa2340c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa234169fd0 con 0x7fa2340b5410 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa234169e50 con 0x7fa2340ccb00 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa23416a150 con 0x7fa2340b2f40 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 0x7fa2341fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 0x7fa2341fa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43282/0 (socket says 172.21.15.170:43282) 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 -- 172.21.15.170:0/2768636652 learned_addr learned my addr 172.21.15.170:0/2768636652 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa2337fe640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2340b2f40 0x7fa23414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 -- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2340b2f40 msgr2=0x7fa23414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2340b2f40 0x7fa23414fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 -- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2340ccb00 msgr2=0x7fa2340c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2340ccb00 0x7fa2340c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 -- 172.21.15.170:0/2768636652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341faae0 con 0x7fa2340b5410 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa232ffd640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 0x7fa2341fa5a0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fa224000f90 tx=0x7fa224003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240bb070 con 0x7fa2340b5410 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa224004d40 con 0x7fa2340b5410 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2341517e0 con 0x7fa2340b5410 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240b2040 con 0x7fa2340b5410 2024-09-21T02:22:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa234150a20 con 0x7fa2340b5410 2024-09-21T02:22:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.215+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2340b3350 con 0x7fa2340b5410 2024-09-21T02:22:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.219+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fa2240be070 con 0x7fa2340b5410 2024-09-21T02:22:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.219+0000 7fa21bfff640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa220073dd0 0x7fa2200762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.219+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fa22412d650 con 0x7fa2340b5410 2024-09-21T02:22:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.219+0000 7fa2337fe640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa220073dd0 0x7fa2200762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.219+0000 7fa2337fe640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa220073dd0 0x7fa2200762e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa2280058d0 tx=0x7fa228005bd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:41.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.223+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa2240c5070 con 0x7fa2340b5410 2024-09-21T02:22:41.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.383+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2340b2390 con 0x7fa2340b5410 2024-09-21T02:22:41.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.383+0000 7fa21bfff640 1 -- 172.21.15.170:0/2768636652 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16767 (secure 0 0 0) 0x7fa2341faae0 con 0x7fa2340b5410 2024-09-21T02:22:41.387 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:41.387 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":166,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:23.429025+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:22:19.529466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T02:22:20.819055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:41.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa220073dd0 msgr2=0x7fa2200762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa220073dd0 0x7fa2200762e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa2280058d0 tx=0x7fa228005bd0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 msgr2=0x7fa2341fa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 0x7fa2341fa5a0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fa224000f90 tx=0x7fa224003bf0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 shutdown_connections 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa220073dd0 0x7fa2200762e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2340ccb00 0x7fa2340c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2340b5410 0x7fa2341fa5a0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 --2- 172.21.15.170:0/2768636652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2340b2f40 0x7fa23414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 >> 172.21.15.170:0/2768636652 conn(0x7fa2340b1c40 msgr2=0x7fa2340b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 shutdown_connections 2024-09-21T02:22:41.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.387+0000 7fa23a0d8640 1 -- 172.21.15.170:0/2768636652 wait complete. 2024-09-21T02:22:41.408 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:22:41.408 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:41.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 -- 172.21.15.170:0/1141699726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 msgr2=0x7f28840cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 --2- 172.21.15.170:0/1141699726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cf950 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f28800098f0 tx=0x7f288001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 -- 172.21.15.170:0/1141699726 shutdown_connections 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 --2- 172.21.15.170:0/1141699726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2884160610 0x7f28840b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 --2- 172.21.15.170:0/1141699726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cf950 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 -- 172.21.15.170:0/1141699726 >> 172.21.15.170:0/1141699726 conn(0x7f288409a9c0 msgr2=0x7f28840d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 -- 172.21.15.170:0/1141699726 shutdown_connections 2024-09-21T02:22:41.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.555+0000 7f288b272640 1 -- 172.21.15.170:0/1141699726 wait complete. 2024-09-21T02:22:41.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 Processor -- start 2024-09-21T02:22:41.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 -- start start 2024-09-21T02:22:41.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288415faf0 0x7f28840ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2884160610 0x7f288406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2884168f40 con 0x7f28840b25f0 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2884168dc0 con 0x7f288415faf0 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288b272640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f28841690c0 con 0x7f2884160610 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43304/0 (socket says 172.21.15.170:43304) 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 -- 172.21.15.170:0/1199596892 learned_addr learned my addr 172.21.15.170:0/1199596892 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288980f640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2884160610 0x7f288406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288880d640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288415faf0 0x7f28840ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 -- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2884160610 msgr2=0x7f288406ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2884160610 0x7f288406ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 -- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288415faf0 msgr2=0x7f28840ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288415faf0 0x7f28840ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.559+0000 7f288900e640 1 -- 172.21.15.170:0/1199596892 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28840ca560 con 0x7f28840b25f0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f288900e640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cac80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c10 tx=0x7f28800a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28800bb070 con 0x7f28840b25f0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2880004470 con 0x7f28840b25f0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28800b2040 con 0x7f28840b25f0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28841fa680 con 0x7f28840b25f0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28841fac40 con 0x7f28840b25f0 2024-09-21T02:22:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28840cf950 con 0x7f28840b25f0 2024-09-21T02:22:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f2880005270 con 0x7f28840b25f0 2024-09-21T02:22:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f2879ffb640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2868073dd0 0x7f28680762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f288880d640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2868073dd0 0x7f28680762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f288012cd90 con 0x7f28840b25f0 2024-09-21T02:22:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.563+0000 7f288880d640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2868073dd0 0x7f28680762e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2884159670 tx=0x7f2874008040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:41.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.567+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f28800c4020 con 0x7f28840b25f0 2024-09-21T02:22:41.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.759+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28840ce9d0 con 0x7f28840b25f0 2024-09-21T02:22:41.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.759+0000 7f2879ffb640 1 -- 172.21.15.170:0/1199596892 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5245 (secure 0 0 0) 0x7f288012e070 con 0x7f28840b25f0 2024-09-21T02:22:41.762 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:41.762 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":121,"btime":"2024-09-21T02:22:35:005454+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:22:21.863736+0000","modified":"2024-09-21T02:22:24.886454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9303":{"gid":9303,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1712756729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1712756729},{"type":"v1","addr":"172.21.15.170:6835","nonce":1712756729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9303,"qdb_cluster":[9303]},"id":13}]} 2024-09-21T02:22:41.762 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 121 2024-09-21T02:22:41.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2868073dd0 msgr2=0x7f28680762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2868073dd0 0x7f28680762e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2884159670 tx=0x7f2874008040 comp rx=0 tx=0).stop 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 msgr2=0x7f28840cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cac80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c10 tx=0x7f28800a56b0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 shutdown_connections 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2868073dd0 0x7f28680762e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2884160610 0x7f288406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f288415faf0 0x7f28840ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 --2- 172.21.15.170:0/1199596892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28840b25f0 0x7f28840cac80 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 >> 172.21.15.170:0/1199596892 conn(0x7f288409a9c0 msgr2=0x7f2884070020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 shutdown_connections 2024-09-21T02:22:41.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.763+0000 7f288b272640 1 -- 172.21.15.170:0/1199596892 wait complete. 2024-09-21T02:22:41.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:41.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 -- 172.21.15.170:0/370848269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 msgr2=0x7fe2b006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/370848269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b006fa30 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0070e90 tx=0x7fe2ac01c800 comp rx=0 tx=0).stop 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 -- 172.21.15.170:0/370848269 shutdown_connections 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/370848269 >> v2:172.21.15.191:3300/0 conn(0x7fe2b00b2f40 0x7fe2b00b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/370848269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b006fa30 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 -- 172.21.15.170:0/370848269 >> 172.21.15.170:0/370848269 conn(0x7fe2b00b1c40 msgr2=0x7fe2b00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 -- 172.21.15.170:0/370848269 shutdown_connections 2024-09-21T02:22:41.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.939+0000 7fe2b7c85640 1 -- 172.21.15.170:0/370848269 wait complete. 2024-09-21T02:22:41.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 Processor -- start 2024-09-21T02:22:41.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 -- start start 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2b00b2f40 0x7fe2b009dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b00af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2b00ccb00 0x7fe2b0148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0169fd0 con 0x7fe2b00b5410 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0169e50 con 0x7fe2b00b2f40 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b016a150 con 0x7fe2b00ccb00 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b00af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b00af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43318/0 (socket says 172.21.15.170:43318) 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 -- 172.21.15.170:0/1240411416 learned_addr learned my addr 172.21.15.170:0/1240411416 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5a21640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2b00b2f40 0x7fe2b009dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b6222640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2b00ccb00 0x7fe2b0148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 -- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2b00ccb00 msgr2=0x7fe2b0148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2b00ccb00 0x7fe2b0148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 -- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2b00b2f40 msgr2=0x7fe2b009dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2b00b2f40 0x7fe2b009dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 -- 172.21.15.170:0/1240411416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00afaa0 con 0x7fe2b00b5410 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b5220640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b00af560 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c940 tx=0x7fe2a000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0004220 con 0x7fe2b00b5410 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a00079b0 con 0x7fe2b00b5410 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b009c6d0 con 0x7fe2b00b5410 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00044b0 con 0x7fe2b00b5410 2024-09-21T02:22:41.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.943+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b009cc90 con 0x7fe2b00b5410 2024-09-21T02:22:41.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.947+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b00b3350 con 0x7fe2b00b5410 2024-09-21T02:22:41.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.947+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fe2a0005020 con 0x7fe2b00b5410 2024-09-21T02:22:41.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.947+0000 7fe2a67fc640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe294073e00 0x7fe294076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:41.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.947+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7fe2a00930f0 con 0x7fe2b00b5410 2024-09-21T02:22:41.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.947+0000 7fe2b5a21640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe294073e00 0x7fe294076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:41.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.947+0000 7fe2b5a21640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe294073e00 0x7fe294076310 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0af000 tx=0x7fe2ac01b040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:41.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:41.951+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe2a0014030 con 0x7fe2b00b5410 2024-09-21T02:22:42.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.143+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b00ce330 con 0x7fe2b00b5410 2024-09-21T02:22:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2a67fc640 1 -- 172.21.15.170:0/1240411416 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5245 (secure 0 0 0) 0x7fe2a005fc30 con 0x7fe2b00b5410 2024-09-21T02:22:42.148 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:42.149 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":121,"btime":"2024-09-21T02:22:35:005454+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T02:22:21.863736+0000","modified":"2024-09-21T02:22:24.886454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9303":{"gid":9303,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6835/1712756729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1712756729},{"type":"v1","addr":"172.21.15.170:6835","nonce":1712756729}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9303,"qdb_cluster":[9303]},"id":13}]} 2024-09-21T02:22:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 121 2024-09-21T02:22:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe294073e00 msgr2=0x7fe294076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe294073e00 0x7fe294076310 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0af000 tx=0x7fe2ac01b040 comp rx=0 tx=0).stop 2024-09-21T02:22:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 msgr2=0x7fe2b00af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b00af560 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c940 tx=0x7fe2a000ce00 comp rx=0 tx=0).stop 2024-09-21T02:22:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 shutdown_connections 2024-09-21T02:22:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe294073e00 0x7fe294076310 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2b00ccb00 0x7fe2b0148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b00b5410 0x7fe2b00af560 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 --2- 172.21.15.170:0/1240411416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2b00b2f40 0x7fe2b009dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 >> 172.21.15.170:0/1240411416 conn(0x7fe2b00b1c40 msgr2=0x7fe2b00ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 shutdown_connections 2024-09-21T02:22:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.147+0000 7fe2b7c85640 1 -- 172.21.15.170:0/1240411416 wait complete. 2024-09-21T02:22:42.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 -- 172.21.15.170:0/1727114838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80cc0a0 msgr2=0x7f40f80afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 --2- 172.21.15.170:0/1727114838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80cc0a0 0x7f40f80afba0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f40f8069610 tx=0x7f40f001cb90 comp rx=0 tx=0).stop 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 -- 172.21.15.170:0/1727114838 shutdown_connections 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 --2- 172.21.15.170:0/1727114838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80cc0a0 0x7f40f80afba0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 --2- 172.21.15.170:0/1727114838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f8074340 0x7f40f8070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 -- 172.21.15.170:0/1727114838 >> 172.21.15.170:0/1727114838 conn(0x7f40f80cd3c0 msgr2=0x7f40f80cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 -- 172.21.15.170:0/1727114838 shutdown_connections 2024-09-21T02:22:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 -- 172.21.15.170:0/1727114838 wait complete. 2024-09-21T02:22:42.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 Processor -- start 2024-09-21T02:22:42.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.315+0000 7f40fc82c640 1 -- start start 2024-09-21T02:22:42.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 0x7f40f81523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8074340 0x7f40f815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f80cc0a0 0x7f40f8158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40f815d3e0 con 0x7f40f80706c0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40f815d260 con 0x7f40f8074340 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f40f815d560 con 0x7f40f80cc0a0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 0x7f40f81523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 0x7f40f81523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43336/0 (socket says 172.21.15.170:43336) 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 -- 172.21.15.170:0/693968100 learned_addr learned my addr 172.21.15.170:0/693968100 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 -- 172.21.15.170:0/693968100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f80cc0a0 msgr2=0x7f40f8158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f80cc0a0 0x7f40f8158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 -- 172.21.15.170:0/693968100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8074340 msgr2=0x7f40f815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8074340 0x7f40f815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 -- 172.21.15.170:0/693968100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f815a030 con 0x7f40f80706c0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f659c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 0x7f40f81523a0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c970 tx=0x7f40e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004220 con 0x7f40f80706c0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40e80079b0 con 0x7f40f80706c0 2024-09-21T02:22:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80044b0 con 0x7f40f80706c0 2024-09-21T02:22:42.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f815b0d0 con 0x7f40f80706c0 2024-09-21T02:22:42.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f815a5f0 con 0x7f40f80706c0 2024-09-21T02:22:42.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f815daa0 con 0x7f40f80706c0 2024-09-21T02:22:42.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f40e8005020 con 0x7f40f80706c0 2024-09-21T02:22:42.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40e6ffd640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f40d8073dd0 0x7f40d80762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f40e8093110 con 0x7f40f80706c0 2024-09-21T02:22:42.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.319+0000 7f40f5d9b640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f40d8073dd0 0x7f40d80762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:42.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.323+0000 7f40f5d9b640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f40d8073dd0 0x7f40d80762e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f40f80cbef0 tx=0x7f40ec006bf0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:42.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.327+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f40e8014030 con 0x7f40f80706c0 2024-09-21T02:22:42.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.487+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40f80cbc40 con 0x7f40f80706c0 2024-09-21T02:22:42.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40e6ffd640 1 -- 172.21.15.170:0/693968100 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16767 (secure 0 0 0) 0x7f40e805fc70 con 0x7f40f80706c0 2024-09-21T02:22:42.492 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:42.493 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":166,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:23.429025+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:22:19.529466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T02:22:20.819055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:42.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f40d8073dd0 msgr2=0x7f40d80762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f40d8073dd0 0x7f40d80762e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f40f80cbef0 tx=0x7f40ec006bf0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 msgr2=0x7f40f81523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 0x7f40f81523a0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c970 tx=0x7f40e800ce30 comp rx=0 tx=0).stop 2024-09-21T02:22:42.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 shutdown_connections 2024-09-21T02:22:42.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f40d8073dd0 0x7f40d80762e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f80cc0a0 0x7f40f8158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8074340 0x7f40f815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 --2- 172.21.15.170:0/693968100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f80706c0 0x7f40f81523a0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 >> 172.21.15.170:0/693968100 conn(0x7f40f80cd3c0 msgr2=0x7f40f815e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:42.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 shutdown_connections 2024-09-21T02:22:42.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.491+0000 7f40fc82c640 1 -- 172.21.15.170:0/693968100 wait complete. 2024-09-21T02:22:42.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- 172.21.15.170:0/4072744166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5538004810 msgr2=0x7f5538004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 --2- 172.21.15.170:0/4072744166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5538004810 0x7f5538004c20 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f55480016f0 tx=0x7f554801c7b0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f55519b2640 1 -- 172.21.15.170:0/4072744166 reap_dead start 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- 172.21.15.170:0/4072744166 shutdown_connections 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- 172.21.15.170:0/4072744166 >> 172.21.15.170:0/4072744166 conn(0x7f554c0b1c40 msgr2=0x7f554c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- 172.21.15.170:0/4072744166 shutdown_connections 2024-09-21T02:22:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- 172.21.15.170:0/4072744166 wait complete. 2024-09-21T02:22:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 Processor -- start 2024-09-21T02:22:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- start start 2024-09-21T02:22:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5538004810 0x7f554c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f554c09be40 0x7f554c09c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 0x7f554c1f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0cf8e0 con 0x7f554c1f72e0 2024-09-21T02:22:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0cf760 con 0x7f554c09be40 2024-09-21T02:22:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f5553415640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f554c0cd210 con 0x7f5538004810 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f55519b2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 0x7f554c1f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f55519b2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 0x7f554c1f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43362/0 (socket says 172.21.15.170:43362) 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.663+0000 7f55519b2640 1 -- 172.21.15.170:0/31869889 learned_addr learned my addr 172.21.15.170:0/31869889 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55511b1640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5538004810 0x7f554c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55509b0640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f554c09be40 0x7f554c09c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55519b2640 1 -- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5538004810 msgr2=0x7f554c1f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55519b2640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5538004810 0x7f554c1f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55519b2640 1 -- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f554c09be40 msgr2=0x7f554c09c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55519b2640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f554c09be40 0x7f554c09c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55519b2640 1 -- 172.21.15.170:0/31869889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c1f3c10 con 0x7f554c1f72e0 2024-09-21T02:22:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55519b2640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 0x7f554c1f76f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f5538004790 tx=0x7f554000ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540016030 con 0x7f554c1f72e0 2024-09-21T02:22:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5540005130 con 0x7f554c1f72e0 2024-09-21T02:22:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540005580 con 0x7f554c1f72e0 2024-09-21T02:22:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c1f7c30 con 0x7f554c1f72e0 2024-09-21T02:22:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c09cec0 con 0x7f554c1f72e0 2024-09-21T02:22:42.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c0b3350 con 0x7f554c1f72e0 2024-09-21T02:22:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f5540019070 con 0x7f554c1f72e0 2024-09-21T02:22:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f553dffb640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5530073e00 0x7f5530076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55511b1640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5530073e00 0x7f5530076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5737+0+0 (secure 0 0 0) 0x7f5540092a00 con 0x7f554c1f72e0 2024-09-21T02:22:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.667+0000 7f55511b1640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5530073e00 0x7f5530076310 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5544003a30 tx=0x7f55440018e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.671+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f554c1f7c30 con 0x7f554c1f72e0 2024-09-21T02:22:42.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.871+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f554c09c970 con 0x7f554c1f72e0 2024-09-21T02:22:42.875 INFO:tasks.ceph.mon.a.smithi170.stderr:2024-09-21T02:22:42.875+0000 7fa93d68b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T02:22:42.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.875+0000 7f553dffb640 1 -- 172.21.15.170:0/31869889 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v122) ==== 161+0+0 (secure 0 0 0) 0x7f5540005720 con 0x7f554c1f72e0 2024-09-21T02:22:42.878 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T02:22:42.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5530073e00 msgr2=0x7f5530076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5530073e00 0x7f5530076310 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5544003a30 tx=0x7f55440018e0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 msgr2=0x7f554c1f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:42.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 0x7f554c1f76f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f5538004790 tx=0x7f554000ea30 comp rx=0 tx=0).stop 2024-09-21T02:22:42.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 shutdown_connections 2024-09-21T02:22:42.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5530073e00 0x7f5530076310 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f554c1f72e0 0x7f554c1f76f0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f554c09be40 0x7f554c09c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 --2- 172.21.15.170:0/31869889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5538004810 0x7f554c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:42.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 >> 172.21.15.170:0/31869889 conn(0x7f554c0b1c40 msgr2=0x7f554c0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:42.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 shutdown_connections 2024-09-21T02:22:42.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:42.879+0000 7f5553415640 1 -- 172.21.15.170:0/31869889 wait complete. 2024-09-21T02:22:42.902 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T02:22:42.977 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 -- 172.21.15.170:0/2314146609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa61806fb40 msgr2=0x7fa61806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/2314146609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa61806fb40 0x7fa61806ff50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa600009900 tx=0x7fa60001c850 comp rx=0 tx=0).stop 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 -- 172.21.15.170:0/2314146609 shutdown_connections 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/2314146609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa61806fb40 0x7fa61806ff50 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 -- 172.21.15.170:0/2314146609 >> 172.21.15.170:0/2314146609 conn(0x7fa6180cd5d0 msgr2=0x7fa6180cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 -- 172.21.15.170:0/2314146609 shutdown_connections 2024-09-21T02:22:43.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.043+0000 7fa61e4e2640 1 -- 172.21.15.170:0/2314146609 wait complete. 2024-09-21T02:22:43.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 Processor -- start 2024-09-21T02:22:43.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 -- start start 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa61806fb40 0x7fa61814b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa618070490 0x7fa61816a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 0x7fa61816a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa618169e50 con 0x7fa6180b2400 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa618169cd0 con 0x7fa618070490 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa618169fd0 con 0x7fa61806fb40 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 0x7fa61816a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 0x7fa61816a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43394/0 (socket says 172.21.15.170:43394) 2024-09-21T02:22:43.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 -- 172.21.15.170:0/3888558863 learned_addr learned my addr 172.21.15.170:0/3888558863 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 -- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa61806fb40 msgr2=0x7fa61814b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa6177fe640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa618070490 0x7fa61816a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa617fff640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa61806fb40 0x7fa61814b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa61806fb40 0x7fa61814b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 -- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa618070490 msgr2=0x7fa61816a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa618070490 0x7fa61816a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 -- 172.21.15.170:0/3888558863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61814bd00 con 0x7fa6180b2400 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61ca7f640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 0x7fa61816a9a0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa61000c3c0 tx=0x7fa61000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610017070 con 0x7fa6180b2400 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6100053c0 con 0x7fa6180b2400 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61815c4e0 con 0x7fa6180b2400 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610003860 con 0x7fa6180b2400 2024-09-21T02:22:43.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.047+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61815caa0 con 0x7fa6180b2400 2024-09-21T02:22:43.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.051+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61806ff50 con 0x7fa6180b2400 2024-09-21T02:22:43.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.051+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fa61001a070 con 0x7fa6180b2400 2024-09-21T02:22:43.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.051+0000 7fa614ff9640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa5f8073e00 0x7fa5f8076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:43.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.051+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5823+0+0 (secure 0 0 0) 0x7fa6100925d0 con 0x7fa6180b2400 2024-09-21T02:22:43.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.051+0000 7fa617fff640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa5f8073e00 0x7fa5f8076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:43.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.051+0000 7fa617fff640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa5f8073e00 0x7fa5f8076310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa608007bf0 tx=0x7fa608007240 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:43.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.055+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa610021020 con 0x7fa6180b2400 2024-09-21T02:22:43.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.251+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa6180cc240 con 0x7fa6180b2400 2024-09-21T02:22:43.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.887+0000 7fa614ff9640 1 -- 172.21.15.170:0/3888558863 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v123) ==== 104+0+0 (secure 0 0 0) 0x7fa61005eea0 con 0x7fa6180b2400 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.887+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa5f8073e00 msgr2=0x7fa5f8076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.887+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa5f8073e00 0x7fa5f8076310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa608007bf0 tx=0x7fa608007240 comp rx=0 tx=0).stop 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.887+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 msgr2=0x7fa61816a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.887+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 0x7fa61816a9a0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa61000c3c0 tx=0x7fa61000c880 comp rx=0 tx=0).stop 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 shutdown_connections 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa5f8073e00 0x7fa5f8076310 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6180b2400 0x7fa61816a9a0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa618070490 0x7fa61816a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 --2- 172.21.15.170:0/3888558863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa61806fb40 0x7fa61814b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:43.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 >> 172.21.15.170:0/3888558863 conn(0x7fa6180cd5d0 msgr2=0x7fa618076120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:43.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 shutdown_connections 2024-09-21T02:22:43.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:43.891+0000 7fa61e4e2640 1 -- 172.21.15.170:0/3888558863 wait complete. 2024-09-21T02:22:43.916 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T02:22:44.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.079+0000 7f504a846640 1 -- 172.21.15.170:0/3012124688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440b4d80 msgr2=0x7f50440ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:44.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.079+0000 7f504a846640 1 --2- 172.21.15.170:0/3012124688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440b4d80 0x7f50440ccae0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f5044072900 tx=0x7f503801c830 comp rx=0 tx=0).stop 2024-09-21T02:22:44.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- 172.21.15.170:0/3012124688 shutdown_connections 2024-09-21T02:22:44.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 --2- 172.21.15.170:0/3012124688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440b4d80 0x7f50440ccae0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:44.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 --2- 172.21.15.170:0/3012124688 >> v2:172.21.15.191:3300/0 conn(0x7f50440b52d0 0x7f50440af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:44.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- 172.21.15.170:0/3012124688 >> 172.21.15.170:0/3012124688 conn(0x7f50440b1c40 msgr2=0x7f50440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:44.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- 172.21.15.170:0/3012124688 shutdown_connections 2024-09-21T02:22:44.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- 172.21.15.170:0/3012124688 wait complete. 2024-09-21T02:22:44.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 Processor -- start 2024-09-21T02:22:44.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- start start 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50440b4d80 0x7f504414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50440b52d0 0x7f5044155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 0x7f5044158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5044169d30 con 0x7f50440cd020 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5044169bb0 con 0x7f50440b52d0 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f504a846640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f5044169eb0 con 0x7f50440b4d80 2024-09-21T02:22:44.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f5048de3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 0x7f5044158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f5048de3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 0x7f5044158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43412/0 (socket says 172.21.15.170:43412) 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f5048de3640 1 -- 172.21.15.170:0/2808931843 learned_addr learned my addr 172.21.15.170:0/2808931843 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f5043fff640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50440b4d80 0x7f504414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.083+0000 7f50437fe640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50440b52d0 0x7f5044155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5048de3640 1 -- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50440b4d80 msgr2=0x7f504414d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5048de3640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50440b4d80 0x7f504414d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5048de3640 1 -- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50440b52d0 msgr2=0x7f5044155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5048de3640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50440b52d0 0x7f5044155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5048de3640 1 -- 172.21.15.170:0/2808931843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504414d590 con 0x7f50440cd020 2024-09-21T02:22:44.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5048de3640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 0x7f5044158810 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f503400b3e0 tx=0x7f503400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:44.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034015020 con 0x7f50440cd020 2024-09-21T02:22:44.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5034011940 con 0x7f50440cd020 2024-09-21T02:22:44.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50441facb0 con 0x7f50440cd020 2024-09-21T02:22:44.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034011d90 con 0x7f50440cd020 2024-09-21T02:22:44.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504414db00 con 0x7f50440cd020 2024-09-21T02:22:44.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.087+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50440ccae0 con 0x7f50440cd020 2024-09-21T02:22:44.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.091+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f5034018020 con 0x7f50440cd020 2024-09-21T02:22:44.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.091+0000 7f5040ff9640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5030073dd0 0x7f50300762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:44.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.091+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5823+0+0 (secure 0 0 0) 0x7f5034092c30 con 0x7f50440cd020 2024-09-21T02:22:44.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.091+0000 7f5043fff640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5030073dd0 0x7f50300762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:44.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.091+0000 7f5043fff640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5030073dd0 0x7f50300762e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f503c009e00 tx=0x7f503c009390 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:44.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.095+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f503401f020 con 0x7f50440cd020 2024-09-21T02:22:44.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:44.267+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f504406de10 con 0x7f50440cd020 2024-09-21T02:22:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.095+0000 7f5040ff9640 1 -- 172.21.15.170:0/2808931843 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v168) ==== 181+0+0 (secure 0 0 0) 0x7f50441facb0 con 0x7f50440cd020 2024-09-21T02:22:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5030073dd0 msgr2=0x7f50300762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5030073dd0 0x7f50300762e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f503c009e00 tx=0x7f503c009390 comp rx=0 tx=0).stop 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 msgr2=0x7f5044158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 0x7f5044158810 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f503400b3e0 tx=0x7f503400b8a0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 shutdown_connections 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5030073dd0 0x7f50300762e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50440cd020 0x7f5044158810 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50440b52d0 0x7f5044155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 --2- 172.21.15.170:0/2808931843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50440b4d80 0x7f504414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 >> 172.21.15.170:0/2808931843 conn(0x7f50440b1c40 msgr2=0x7f50441535e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 shutdown_connections 2024-09-21T02:22:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.099+0000 7f504a846640 1 -- 172.21.15.170:0/2808931843 wait complete. 2024-09-21T02:22:45.127 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 -- 172.21.15.170:0/2939375934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a40ca720 msgr2=0x7fd0a40cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 --2- 172.21.15.170:0/2939375934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a40ca720 0x7fd0a40cab30 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40afa20 tx=0x7fd09401d320 comp rx=0 tx=0).stop 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 -- 172.21.15.170:0/2939375934 shutdown_connections 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 --2- 172.21.15.170:0/2939375934 >> v2:172.21.15.191:3300/0 conn(0x7fd0a415e9f0 0x7fd0a415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 --2- 172.21.15.170:0/2939375934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a40ca720 0x7fd0a40cab30 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 --2- 172.21.15.170:0/2939375934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a409dc00 0x7fd0a415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 -- 172.21.15.170:0/2939375934 >> 172.21.15.170:0/2939375934 conn(0x7fd0a40b1c40 msgr2=0x7fd0a40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 -- 172.21.15.170:0/2939375934 shutdown_connections 2024-09-21T02:22:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.287+0000 7fd0ab333640 1 -- 172.21.15.170:0/2939375934 wait complete. 2024-09-21T02:22:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 Processor -- start 2024-09-21T02:22:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 -- start start 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 0x7fd0a41f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a40ca720 0x7fd0a406f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a415e9f0 0x7fd0a406f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4168100 con 0x7fd0a409dc00 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4167f80 con 0x7fd0a415e9f0 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a4168280 con 0x7fd0a40ca720 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 0x7fd0a41f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:45.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 0x7fd0a41f9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43424/0 (socket says 172.21.15.170:43424) 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 -- 172.21.15.170:0/3377372379 learned_addr learned my addr 172.21.15.170:0/3377372379 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 -- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a40ca720 msgr2=0x7fd0a406f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a40ca720 0x7fd0a406f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 -- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a415e9f0 msgr2=0x7fd0a406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a98d0640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a415e9f0 0x7fd0a406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a415e9f0 0x7fd0a406f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 -- 172.21.15.170:0/3377372379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41fa3e0 con 0x7fd0a409dc00 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0a90cf640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 0x7fd0a41f9ea0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000b440 tx=0x7fd0a000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0015030 con 0x7fd0a409dc00 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0a0005980 con 0x7fd0a409dc00 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a4157350 con 0x7fd0a409dc00 2024-09-21T02:22:45.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0005dd0 con 0x7fd0a409dc00 2024-09-21T02:22:45.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.291+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a4157910 con 0x7fd0a409dc00 2024-09-21T02:22:45.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.295+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a40cab30 con 0x7fd0a409dc00 2024-09-21T02:22:45.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.295+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fd0a0018070 con 0x7fd0a409dc00 2024-09-21T02:22:45.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.295+0000 7fd099ffb640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd088073e00 0x7fd088076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:45.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.295+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5385+0+0 (secure 0 0 0) 0x7fd0a0092fb0 con 0x7fd0a409dc00 2024-09-21T02:22:45.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.295+0000 7fd0a88ce640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd088073e00 0x7fd088076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:45.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.295+0000 7fd0a88ce640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd088073e00 0x7fd088076310 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0940af000 tx=0x7fd094019660 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:45.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.299+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd0a41fa3e0 con 0x7fd0a409dc00 2024-09-21T02:22:45.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:45.479+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0a40cb3c0 con 0x7fd0a409dc00 2024-09-21T02:22:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.103+0000 7fd099ffb640 1 -- 172.21.15.170:0/3377372379 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v169) ==== 169+0+0 (secure 0 0 0) 0x7fd0a000d040 con 0x7fd0a409dc00 2024-09-21T02:22:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2024-09-21T02:22:46.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.111+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd088073e00 msgr2=0x7fd088076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.111+0000 7fd0ab333640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd088073e00 0x7fd088076310 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0940af000 tx=0x7fd094019660 comp rx=0 tx=0).stop 2024-09-21T02:22:46.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.111+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 msgr2=0x7fd0a41f9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.111+0000 7fd0ab333640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 0x7fd0a41f9ea0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000b440 tx=0x7fd0a000b900 comp rx=0 tx=0).stop 2024-09-21T02:22:46.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 shutdown_connections 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd088073e00 0x7fd088076310 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0940af000 tx=0x7fd094019660 comp rx=0 tx=0).stop 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0a40ca720 0x7fd0a406f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0a415e9f0 0x7fd0a406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 --2- 172.21.15.170:0/3377372379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0a409dc00 0x7fd0a41f9ea0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 >> 172.21.15.170:0/3377372379 conn(0x7fd0a40b1c40 msgr2=0x7fd0a40afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 shutdown_connections 2024-09-21T02:22:46.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.115+0000 7fd0ab333640 1 -- 172.21.15.170:0/3377372379 wait complete. 2024-09-21T02:22:46.145 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3645405301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774004810 msgr2=0x7fa774004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3645405301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa774004810 0x7fa774004c20 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fa77800bb10 tx=0x7fa77801cdc0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78e438640 1 -- 172.21.15.170:0/3645405301 reap_dead start 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3645405301 shutdown_connections 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3645405301 >> 172.21.15.170:0/3645405301 conn(0x7fa7880b1c40 msgr2=0x7fa7880b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3645405301 shutdown_connections 2024-09-21T02:22:46.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3645405301 wait complete. 2024-09-21T02:22:46.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 Processor -- start 2024-09-21T02:22:46.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.303+0000 7fa78fe9b640 1 -- start start 2024-09-21T02:22:46.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7881e8590 0x7fa7881f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 0x7fa78815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78815b130 0x7fa78815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880cb9f0 con 0x7fa7880afcf0 2024-09-21T02:22:46.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880cb870 con 0x7fa78815b130 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa7880cbb70 con 0x7fa7881e8590 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 0x7fa78815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 0x7fa78815a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39934/0 (socket says 172.21.15.170:39934) 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 -- 172.21.15.170:0/3708212748 learned_addr learned my addr 172.21.15.170:0/3708212748 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78e438640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78815b130 0x7fa78815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78dc37640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7881e8590 0x7fa7881f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 -- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7881e8590 msgr2=0x7fa7881f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7881e8590 0x7fa7881f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 -- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78815b130 msgr2=0x7fa78815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78815b130 0x7fa78815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 -- 172.21.15.170:0/3708212748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78815ba80 con 0x7fa7880afcf0 2024-09-21T02:22:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78d436640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 0x7fa78815a520 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fa77800ba60 tx=0x7fa77800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7780be070 con 0x7fa7880afcf0 2024-09-21T02:22:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa778005330 con 0x7fa7880afcf0 2024-09-21T02:22:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7780ab050 con 0x7fa7880afcf0 2024-09-21T02:22:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7880c9760 con 0x7fa7880afcf0 2024-09-21T02:22:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78806f440 con 0x7fa7880afcf0 2024-09-21T02:22:46.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.307+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7880cf010 con 0x7fa7880afcf0 2024-09-21T02:22:46.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.311+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fa7780b6070 con 0x7fa7880afcf0 2024-09-21T02:22:46.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.311+0000 7fa77e7fc640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa76c073dd0 0x7fa76c0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.311+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fa77812cfb0 con 0x7fa7880afcf0 2024-09-21T02:22:46.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.311+0000 7fa78dc37640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa76c073dd0 0x7fa76c0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.311+0000 7fa78dc37640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa76c073dd0 0x7fa76c0762e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa784001530 tx=0x7fa7840014c0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.315+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa78815ba80 con 0x7fa7880afcf0 2024-09-21T02:22:46.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.495+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fa7880caea0 con 0x7fa7880afcf0 2024-09-21T02:22:46.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.499+0000 7fa77e7fc640 1 -- 172.21.15.170:0/3708212748 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7fa778016ad0 con 0x7fa7880afcf0 2024-09-21T02:22:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa76c073dd0 msgr2=0x7fa76c0762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa76c073dd0 0x7fa76c0762e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa784001530 tx=0x7fa7840014c0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 msgr2=0x7fa78815a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 0x7fa78815a520 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fa77800ba60 tx=0x7fa77800b6f0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 shutdown_connections 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa76c073dd0 0x7fa76c0762e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa78815b130 0x7fa78815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7880afcf0 0x7fa78815a520 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 --2- 172.21.15.170:0/3708212748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7881e8590 0x7fa7881f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 >> 172.21.15.170:0/3708212748 conn(0x7fa7880b1c40 msgr2=0x7fa78815e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 shutdown_connections 2024-09-21T02:22:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.503+0000 7fa78fe9b640 1 -- 172.21.15.170:0/3708212748 wait complete. 2024-09-21T02:22:46.526 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-21T02:22:46.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 -- 172.21.15.170:0/2181226040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 msgr2=0x7fd9a00b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 --2- 172.21.15.170:0/2181226040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 0x7fd9a00b3350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd990009840 tx=0x7fd99001c820 comp rx=0 tx=0).stop 2024-09-21T02:22:46.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 -- 172.21.15.170:0/2181226040 shutdown_connections 2024-09-21T02:22:46.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 --2- 172.21.15.170:0/2181226040 >> v2:172.21.15.191:3300/0 conn(0x7fd9a00ccb00 0x7fd9a00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 --2- 172.21.15.170:0/2181226040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 0x7fd9a00b3350 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 -- 172.21.15.170:0/2181226040 >> 172.21.15.170:0/2181226040 conn(0x7fd9a00b1c40 msgr2=0x7fd9a00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:46.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 -- 172.21.15.170:0/2181226040 shutdown_connections 2024-09-21T02:22:46.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.675+0000 7fd9a8001640 1 -- 172.21.15.170:0/2181226040 wait complete. 2024-09-21T02:22:46.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 Processor -- start 2024-09-21T02:22:46.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 -- start start 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 0x7fd9a00c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 0x7fd9a01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9a00ccb00 0x7fd9a0072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0169e50 con 0x7fd9a00b5410 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00cf970 con 0x7fd9a00ccb00 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a8001640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a0169fd0 con 0x7fd9a00b2f40 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 0x7fd9a01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 0x7fd9a01548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39948/0 (socket says 172.21.15.170:39948) 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 -- 172.21.15.170:0/3103455710 learned_addr learned my addr 172.21.15.170:0/3103455710 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a659e640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9a00ccb00 0x7fd9a0072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a5d9d640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 0x7fd9a00c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 -- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 msgr2=0x7fd9a00c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 0x7fd9a00c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 -- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9a00ccb00 msgr2=0x7fd9a0072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9a00ccb00 0x7fd9a0072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 -- 172.21.15.170:0/3103455710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a00c4820 con 0x7fd9a00b5410 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.679+0000 7fd9a559c640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 0x7fd9a01548e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fd990009790 tx=0x7fd9900a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900bb070 con 0x7fd9a00b5410 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd990004470 con 0x7fd9a00b5410 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a00c3d40 con 0x7fd9a00b5410 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900b2040 con 0x7fd9a00b5410 2024-09-21T02:22:46.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a0156ba0 con 0x7fd9a00b5410 2024-09-21T02:22:46.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a00b3350 con 0x7fd9a00b5410 2024-09-21T02:22:46.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fd990005290 con 0x7fd9a00b5410 2024-09-21T02:22:46.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9967fc640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073e00 0x7fd984076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:46.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fd99012c7c0 con 0x7fd9a00b5410 2024-09-21T02:22:46.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9a5d9d640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073e00 0x7fd984076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:46.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.683+0000 7fd9a5d9d640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073e00 0x7fd984076310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd99c001650 tx=0x7fd99c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:46.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.687+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd990016410 con 0x7fd9a00b5410 2024-09-21T02:22:46.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:46.923+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fd9a00c4390 con 0x7fd9a00b5410 2024-09-21T02:22:47.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.107+0000 7fd9967fc640 1 -- 172.21.15.170:0/3103455710 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v498) ==== 163+0+0 (secure 0 0 0) 0x7fd9900f9040 con 0x7fd9a00b5410 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073e00 msgr2=0x7fd984076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073e00 0x7fd984076310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd99c001650 tx=0x7fd99c009040 comp rx=0 tx=0).stop 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 msgr2=0x7fd9a01548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 0x7fd9a01548e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fd990009790 tx=0x7fd9900a5650 comp rx=0 tx=0).stop 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 shutdown_connections 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd984073e00 0x7fd984076310 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9a00ccb00 0x7fd9a0072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9a00b5410 0x7fd9a01548e0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:47.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 --2- 172.21.15.170:0/3103455710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9a00b2f40 0x7fd9a00c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:47.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 >> 172.21.15.170:0/3103455710 conn(0x7fd9a00b1c40 msgr2=0x7fd9a00c83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:47.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 shutdown_connections 2024-09-21T02:22:47.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:47.111+0000 7fd9a8001640 1 -- 172.21.15.170:0/3103455710 wait complete. 2024-09-21T02:22:47.137 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-21T02:22:47.138 INFO:tasks.cephfs_test_runner: 2024-09-21T02:22:47.138 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T02:22:47.138 INFO:tasks.cephfs_test_runner:Ran 11 tests in 531.815s 2024-09-21T02:22:47.138 INFO:tasks.cephfs_test_runner: 2024-09-21T02:22:47.138 INFO:tasks.cephfs_test_runner:OK 2024-09-21T02:22:47.138 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T02:22:47.147 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-21T02:22:47.168 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-21T02:22:47.169 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:47.169 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:47.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:22:47.175 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T02:22:47.175 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:47.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19091 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T02:22:47.224 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:47.225 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:47.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:22:47.231 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-21T02:22:47.231 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T02:22:47.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336492k,nr_inodes=4084123,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16212 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T02:22:47.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.279 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2024-09-21T02:22:47.285 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T02:22:47.285 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T02:22:47.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.285 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T02:22:47.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T02:22:47.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2024-09-21T02:22:47.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.553 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-09-21T02:22:47.559 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T02:22:47.559 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T02:22:47.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.560 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T02:22:47.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.615 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T02:22:47.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T02:22:47.695 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-09-21T02:22:47.874 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T02:22:47.883 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T02:22:47.883 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:22:48.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd503bfc640 1 -- 172.21.15.170:0/2004242070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e8004d50 msgr2=0x7fd4e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd503bfc640 1 --2- 172.21.15.170:0/2004242070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e8004d50 0x7fd4e8005200 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0b2170 tx=0x7fd4f401caf0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd502199640 1 -- 172.21.15.170:0/2004242070 reap_dead start 2024-09-21T02:22:48.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd503bfc640 1 -- 172.21.15.170:0/2004242070 shutdown_connections 2024-09-21T02:22:48.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd503bfc640 1 -- 172.21.15.170:0/2004242070 >> 172.21.15.170:0/2004242070 conn(0x7fd4fc09a9c0 msgr2=0x7fd4fc0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:48.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd503bfc640 1 -- 172.21.15.170:0/2004242070 shutdown_connections 2024-09-21T02:22:48.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.047+0000 7fd503bfc640 1 -- 172.21.15.170:0/2004242070 wait complete. 2024-09-21T02:22:48.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 Processor -- start 2024-09-21T02:22:48.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 -- start start 2024-09-21T02:22:48.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc154240 0x7fd4fc1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 0x7fd4fc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc159460 0x7fd4fc1e7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cfdf0 con 0x7fd4fc152e40 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cfc70 con 0x7fd4fc159460 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd503bfc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd4fc0cff70 con 0x7fd4fc154240 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 0x7fd4fc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 0x7fd4fc158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39956/0 (socket says 172.21.15.170:39956) 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 -- 172.21.15.170:0/1887010796 learned_addr learned my addr 172.21.15.170:0/1887010796 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501998640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc154240 0x7fd4fc1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd502199640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc159460 0x7fd4fc1e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 -- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc154240 msgr2=0x7fd4fc1f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc154240 0x7fd4fc1f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 -- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc159460 msgr2=0x7fd4fc1e7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc159460 0x7fd4fc1e7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.051+0000 7fd501197640 1 -- 172.21.15.170:0/1887010796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc1f4a60 con 0x7fd4fc152e40 2024-09-21T02:22:48.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd501197640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 0x7fd4fc158f20 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8003680 tx=0x7fd4f800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8015070 con 0x7fd4fc152e40 2024-09-21T02:22:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4f8004310 con 0x7fd4fc152e40 2024-09-21T02:22:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800c840 con 0x7fd4fc152e40 2024-09-21T02:22:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc157490 con 0x7fd4fc152e40 2024-09-21T02:22:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc1e6c60 con 0x7fd4fc152e40 2024-09-21T02:22:48.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4fc0cdcb0 con 0x7fd4fc152e40 2024-09-21T02:22:48.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fd4f800c9e0 con 0x7fd4fc152e40 2024-09-21T02:22:48.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd4f27fc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd4e0073dd0 0x7fd4e00762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fd4f80936e0 con 0x7fd4fc152e40 2024-09-21T02:22:48.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd501998640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd4e0073dd0 0x7fd4e00762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.055+0000 7fd501998640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd4e0073dd0 0x7fd4e00762e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc1e8840 tx=0x7fd4ec005f40 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:48.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.059+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd4fc1f4a60 con 0x7fd4fc152e40 2024-09-21T02:22:48.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.231+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4fc0ce9d0 con 0x7fd4e0073dd0 2024-09-21T02:22:48.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.231+0000 7fd4f27fc640 1 -- 172.21.15.170:0/1887010796 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13451 (secure 0 0 0) 0x7fd4fc0ce9d0 con 0x7fd4e0073dd0 2024-09-21T02:22:48.234 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:48.234 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd4e0073dd0 msgr2=0x7fd4e00762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd4e0073dd0 0x7fd4e00762e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc1e8840 tx=0x7fd4ec005f40 comp rx=0 tx=0).stop 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 msgr2=0x7fd4fc158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 0x7fd4fc158f20 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8003680 tx=0x7fd4f800eb80 comp rx=0 tx=0).stop 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 shutdown_connections 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd4e0073dd0 0x7fd4e00762e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc159460 0x7fd4fc1e7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc152e40 0x7fd4fc158f20 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 --2- 172.21.15.170:0/1887010796 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc154240 0x7fd4fc1f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 >> 172.21.15.170:0/1887010796 conn(0x7fd4fc09a9c0 msgr2=0x7fd4fc0705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 shutdown_connections 2024-09-21T02:22:48.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.235+0000 7fd503bfc640 1 -- 172.21.15.170:0/1887010796 wait complete. 2024-09-21T02:22:48.257 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-21T02:22:46.950816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304688,"kb_used_data":17744,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754670032,"statfs":{"total":773094113280,"available":772782112768,"internally_reserved":0,"allocated":18169856,"data_stored":8728582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":293654065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53146,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-71,"num_write_kb":-77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002448"},"pg_stats":[{"pgid":"1.0","version":"73'79","reported_seq":403,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T02:22:43.258641+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:22:43.258641+0000","last_peered":"2024-09-21T02:22:43.258641+0000","last_clean":"2024-09-21T02:22:43.258641+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:22:43.258641+0000","last_undegraded":"2024-09-21T02:22:43.258641+0000","last_fullsized":"2024-09-21T02:22:43.258641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":930144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":930192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509557,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2711552,"data_stored":1521218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542262,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37404,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334436,"statfs":{"total":96636764160,"available":96598462464,"internally_reserved":0,"allocated":2105344,"data_stored":930212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41620,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2162688,"data_stored":982456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41628,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330212,"statfs":{"total":96636764160,"available":96594137088,"internally_reserved":0,"allocated":2170880,"data_stored":982560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42144,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329696,"statfs":{"total":96636764160,"available":96593608704,"internally_reserved":0,"allocated":2699264,"data_stored":1520793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2109440,"data_stored":931007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:22:48.258 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:22:48.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277c698640 1 -- 172.21.15.170:0/2373310288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27600051b0 msgr2=0x7f276000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277c698640 1 --2- 172.21.15.170:0/2373310288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27600051b0 0x7f276000b9d0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f276c009ee0 tx=0x7f276c01d630 comp rx=0 tx=0).stop 2024-09-21T02:22:48.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277ac35640 1 -- 172.21.15.170:0/2373310288 reap_dead start 2024-09-21T02:22:48.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277c698640 1 -- 172.21.15.170:0/2373310288 shutdown_connections 2024-09-21T02:22:48.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277c698640 1 -- 172.21.15.170:0/2373310288 >> 172.21.15.170:0/2373310288 conn(0x7f277408bf50 msgr2=0x7f277409d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:48.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277c698640 1 -- 172.21.15.170:0/2373310288 shutdown_connections 2024-09-21T02:22:48.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.407+0000 7f277c698640 1 -- 172.21.15.170:0/2373310288 wait complete. 2024-09-21T02:22:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 Processor -- start 2024-09-21T02:22:48.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 -- start start 2024-09-21T02:22:48.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27740cd4c0 0x7f27740ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 0x7f2774160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277409e150 0x7f277409e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27740b1b80 con 0x7f2774160ba0 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f27740cfc00 con 0x7f277409e150 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f27740b1d00 con 0x7f27740cd4c0 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 0x7f2774160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 0x7f2774160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39974/0 (socket says 172.21.15.170:39974) 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 -- 172.21.15.170:0/1167779802 learned_addr learned my addr 172.21.15.170:0/1167779802 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277ac35640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277409e150 0x7f277409e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277a434640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27740cd4c0 0x7f27740ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 -- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27740cd4c0 msgr2=0x7f27740ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27740cd4c0 0x7f27740ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 -- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277409e150 msgr2=0x7f277409e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277409e150 0x7f277409e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 -- 172.21.15.170:0/1167779802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27741fa3b0 con 0x7f2774160ba0 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277a434640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27740cd4c0 0x7f27740ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277ac35640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277409e150 0x7f277409e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f2779c33640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 0x7f2774160fb0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f277000f390 tx=0x7f277000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770009cd0 con 0x7f2774160ba0 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27700080d0 con 0x7f2774160ba0 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277409eaa0 con 0x7f2774160ba0 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700084e0 con 0x7f2774160ba0 2024-09-21T02:22:48.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.411+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27741f9df0 con 0x7f2774160ba0 2024-09-21T02:22:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.415+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27740b2260 con 0x7f2774160ba0 2024-09-21T02:22:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.415+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f277000a030 con 0x7f2774160ba0 2024-09-21T02:22:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.415+0000 7f276affd640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2758073dd0 0x7f27580762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.415+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f2770095b60 con 0x7f2774160ba0 2024-09-21T02:22:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.415+0000 7f277a434640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2758073dd0 0x7f27580762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.415+0000 7f277a434640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2758073dd0 0x7f27580762e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2764005d10 tx=0x7f2764005ec0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:48.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.419+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f277409eaa0 con 0x7f2774160ba0 2024-09-21T02:22:48.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f277406fca0 con 0x7f2758073dd0 2024-09-21T02:22:48.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f276affd640 1 -- 172.21.15.170:0/1167779802 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13451 (secure 0 0 0) 0x7f277406fca0 con 0x7f2758073dd0 2024-09-21T02:22:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:48.581 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:22:48.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2758073dd0 msgr2=0x7f27580762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2758073dd0 0x7f27580762e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2764005d10 tx=0x7f2764005ec0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 msgr2=0x7f2774160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 0x7f2774160fb0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f277000f390 tx=0x7f277000f850 comp rx=0 tx=0).stop 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 shutdown_connections 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f2758073dd0 0x7f27580762e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f277409e150 0x7f277409e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2774160ba0 0x7f2774160fb0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 --2- 172.21.15.170:0/1167779802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27740cd4c0 0x7f27740ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 >> 172.21.15.170:0/1167779802 conn(0x7f277408bf50 msgr2=0x7f27740b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 shutdown_connections 2024-09-21T02:22:48.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.579+0000 7f277c698640 1 -- 172.21.15.170:0/1167779802 wait complete. 2024-09-21T02:22:48.602 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-21T02:22:46.950816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304688,"kb_used_data":17744,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754670032,"statfs":{"total":773094113280,"available":772782112768,"internally_reserved":0,"allocated":18169856,"data_stored":8728582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":293654065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53146,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-71,"num_write_kb":-77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002448"},"pg_stats":[{"pgid":"1.0","version":"73'79","reported_seq":403,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T02:22:43.258641+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:22:43.258641+0000","last_peered":"2024-09-21T02:22:43.258641+0000","last_clean":"2024-09-21T02:22:43.258641+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:22:43.258641+0000","last_undegraded":"2024-09-21T02:22:43.258641+0000","last_fullsized":"2024-09-21T02:22:43.258641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":930144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":930192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509557,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2711552,"data_stored":1521218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542262,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37404,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334436,"statfs":{"total":96636764160,"available":96598462464,"internally_reserved":0,"allocated":2105344,"data_stored":930212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41620,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2162688,"data_stored":982456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41628,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330212,"statfs":{"total":96636764160,"available":96594137088,"internally_reserved":0,"allocated":2170880,"data_stored":982560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42144,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329696,"statfs":{"total":96636764160,"available":96593608704,"internally_reserved":0,"allocated":2699264,"data_stored":1520793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2109440,"data_stored":931007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:22:48.602 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T02:22:48.602 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:22:48.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 -- 172.21.15.170:0/3418584040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 msgr2=0x7fc028003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 --2- 172.21.15.170:0/3418584040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc028003eb0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc038006fd0 tx=0x7fc03801cf40 comp rx=0 tx=0).stop 2024-09-21T02:22:48.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc041f8b640 1 -- 172.21.15.170:0/3418584040 reap_dead start 2024-09-21T02:22:48.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 -- 172.21.15.170:0/3418584040 shutdown_connections 2024-09-21T02:22:48.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 -- 172.21.15.170:0/3418584040 >> 172.21.15.170:0/3418584040 conn(0x7fc03c0b1c40 msgr2=0x7fc03c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:48.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 -- 172.21.15.170:0/3418584040 shutdown_connections 2024-09-21T02:22:48.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 -- 172.21.15.170:0/3418584040 wait complete. 2024-09-21T02:22:48.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.755+0000 7fc0439ee640 1 Processor -- start 2024-09-21T02:22:48.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- start start 2024-09-21T02:22:48.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc03c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc03c160320 0x7fc03c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc03c06eeb0 0x7fc03c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c168230 con 0x7fc028003a40 2024-09-21T02:22:48.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1680b0 con 0x7fc03c06eeb0 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc03c1683b0 con 0x7fc03c160320 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc03c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc03c0b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39988/0 (socket says 172.21.15.170:39988) 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 -- 172.21.15.170:0/2040444015 learned_addr learned my addr 172.21.15.170:0/2040444015 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc041f8b640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc03c06eeb0 0x7fc03c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc040f89640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc03c160320 0x7fc03c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 -- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc03c160320 msgr2=0x7fc03c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc03c160320 0x7fc03c158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 -- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc03c06eeb0 msgr2=0x7fc03c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc03c06eeb0 0x7fc03c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 -- 172.21.15.170:0/2040444015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c06f800 con 0x7fc028003a40 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc04178a640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc03c0b1430 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fc03801ca00 tx=0x7fc038001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:48.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038001640 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc038006280 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c0b0b30 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038003600 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c158980 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c0ce5d0 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.759+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fc0380b7030 con 0x7fc028003a40 2024-09-21T02:22:48.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.763+0000 7fc031ffb640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc020073e00 0x7fc020076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:48.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.763+0000 7fc040f89640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc020073e00 0x7fc020076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:48.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.763+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fc03812c4b0 con 0x7fc028003a40 2024-09-21T02:22:48.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.763+0000 7fc040f89640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc020073e00 0x7fc020076310 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06ff40 tx=0x7fc02c0091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:48.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.767+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc03c06f800 con 0x7fc028003a40 2024-09-21T02:22:48.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.935+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc03c06de10 con 0x7fc020073e00 2024-09-21T02:22:48.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.935+0000 7fc031ffb640 1 -- 172.21.15.170:0/2040444015 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13451 (secure 0 0 0) 0x7fc03c06de10 con 0x7fc020073e00 2024-09-21T02:22:48.938 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:48.938 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc020073e00 msgr2=0x7fc020076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc020073e00 0x7fc020076310 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc03c06ff40 tx=0x7fc02c0091e0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 msgr2=0x7fc03c0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc03c0b1430 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fc03801ca00 tx=0x7fc038001cc0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 shutdown_connections 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc020073e00 0x7fc020076310 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc03c160320 0x7fc03c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc03c06eeb0 0x7fc03c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 --2- 172.21.15.170:0/2040444015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc028003a40 0x7fc03c0b1430 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 >> 172.21.15.170:0/2040444015 conn(0x7fc03c0b1c40 msgr2=0x7fc03c0b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:48.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 shutdown_connections 2024-09-21T02:22:48.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:48.939+0000 7fc0439ee640 1 -- 172.21.15.170:0/2040444015 wait complete. 2024-09-21T02:22:48.961 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-21T02:22:46.950816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":304688,"kb_used_data":17744,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754670032,"statfs":{"total":773094113280,"available":772782112768,"internally_reserved":0,"allocated":18169856,"data_stored":8728582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":293654065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53146,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-71,"num_write_kb":-77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002448"},"pg_stats":[{"pgid":"1.0","version":"73'79","reported_seq":403,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T02:22:43.258641+0000","last_change":"2024-09-21T02:13:16.551775+0000","last_active":"2024-09-21T02:22:43.258641+0000","last_peered":"2024-09-21T02:22:43.258641+0000","last_clean":"2024-09-21T02:22:43.258641+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:22:43.258641+0000","last_undegraded":"2024-09-21T02:22:43.258641+0000","last_fullsized":"2024-09-21T02:22:43.258641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:13:15.522950+0000","last_clean_scrub_stamp":"2024-09-21T02:13:15.522950+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:56.110144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":930144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2105344,"data_stored":930192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509557,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33836,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338004,"statfs":{"total":96636764160,"available":96602116096,"internally_reserved":0,"allocated":2711552,"data_stored":1521218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542262,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37404,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334436,"statfs":{"total":96636764160,"available":96598462464,"internally_reserved":0,"allocated":2105344,"data_stored":930212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41620,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330220,"statfs":{"total":96636764160,"available":96594145280,"internally_reserved":0,"allocated":2162688,"data_stored":982456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41628,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330212,"statfs":{"total":96636764160,"available":96594137088,"internally_reserved":0,"allocated":2170880,"data_stored":982560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42144,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329696,"statfs":{"total":96636764160,"available":96593608704,"internally_reserved":0,"allocated":2699264,"data_stored":1520793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2109440,"data_stored":931007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:22:48.962 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 -- 172.21.15.170:0/3759381646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 msgr2=0x7f87280cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 --2- 172.21.15.170:0/3759381646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f87280cb200 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f87280cc8b0 tx=0x7f872001cba0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 -- 172.21.15.170:0/3759381646 shutdown_connections 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 --2- 172.21.15.170:0/3759381646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f87280cb200 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 --2- 172.21.15.170:0/3759381646 >> v2:172.21.15.191:3300/0 conn(0x7f87280ce4c0 0x7f87280ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 -- 172.21.15.170:0/3759381646 >> 172.21.15.170:0/3759381646 conn(0x7f87280b1c40 msgr2=0x7f87280b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 -- 172.21.15.170:0/3759381646 shutdown_connections 2024-09-21T02:22:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.115+0000 7f87300aa640 1 -- 172.21.15.170:0/3759381646 wait complete. 2024-09-21T02:22:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 Processor -- start 2024-09-21T02:22:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 -- start start 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f8728151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87280ce4c0 0x7f872809ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f872815f990 0x7f872809d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8728167ff0 con 0x7f87280cadf0 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8728074f00 con 0x7f87280ce4c0 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f87300aa640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f8728168170 con 0x7f872815f990 2024-09-21T02:22:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f8728151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f8728151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40006/0 (socket says 172.21.15.170:40006) 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 -- 172.21.15.170:0/3988957456 learned_addr learned my addr 172.21.15.170:0/3988957456 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872e647640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f872815f990 0x7f872809d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872d645640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87280ce4c0 0x7f872809ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 -- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f872815f990 msgr2=0x7f872809d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f872815f990 0x7f872809d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 -- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87280ce4c0 msgr2=0x7f872809ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.119+0000 7f872de46640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87280ce4c0 0x7f872809ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f872de46640 1 -- 172.21.15.170:0/3988957456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872815f130 con 0x7f87280cadf0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f872de46640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f8728151720 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f872400b410 tx=0x7f872400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724015030 con 0x7f87280cadf0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8724004020 con 0x7f87280cadf0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872809ae40 con 0x7f87280cadf0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87240044b0 con 0x7f87280cadf0 2024-09-21T02:22:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872809b270 con 0x7f87280cadf0 2024-09-21T02:22:49.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87280cb200 con 0x7f87280cadf0 2024-09-21T02:22:49.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f8724018070 con 0x7f87280cadf0 2024-09-21T02:22:49.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f871e7fc640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f870c073dd0 0x7f870c0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:49.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f8724092a60 con 0x7f87280cadf0 2024-09-21T02:22:49.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f872d645640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f870c073dd0 0x7f870c0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:49.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.123+0000 7f872d645640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f870c073dd0 0x7f870c0762e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87180058d0 tx=0x7f8718005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:49.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.127+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f872405f2e0 con 0x7f87280cadf0 2024-09-21T02:22:49.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.299+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87280ce2a0 con 0x7f87280cadf0 2024-09-21T02:22:49.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f871e7fc640 1 -- 172.21.15.170:0/3988957456 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+13136 (secure 0 0 0) 0x7f872405ebe0 con 0x7f87280cadf0 2024-09-21T02:22:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:49.304 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":169,"fsid":"75527983-1c6c-4a94-aa07-cb5c10ea5972","created":"2024-09-21T02:12:59.967612+0000","modified":"2024-09-21T02:22:46.100888+0000","last_up_change":"2024-09-21T02:13:13.517013+0000","last_in_change":"2024-09-21T02:13:03.867885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:13:14.921556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"46247445-2c36-4916-9356-d55f137755bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6801","nonce":1810219020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6803","nonce":1810219020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6807","nonce":1810219020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1810219020},{"type":"v1","addr":"172.21.15.170:6805","nonce":1810219020}]},"public_addr":"172.21.15.170:6801/1810219020","cluster_addr":"172.21.15.170:6803/1810219020","heartbeat_back_addr":"172.21.15.170:6807/1810219020","heartbeat_front_addr":"172.21.15.170:6805/1810219020","state":["exists","up"]},{"osd":1,"uuid":"dfc90b85-149d-4f5c-8d74-883fcc23d5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6825","nonce":2831250627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6827","nonce":2831250627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6831","nonce":2831250627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":2831250627},{"type":"v1","addr":"172.21.15.170:6829","nonce":2831250627}]},"public_addr":"172.21.15.170:6825/2831250627","cluster_addr":"172.21.15.170:6827/2831250627","heartbeat_back_addr":"172.21.15.170:6831/2831250627","heartbeat_front_addr":"172.21.15.170:6829/2831250627","state":["exists","up"]},{"osd":2,"uuid":"16bebfcd-e061-4b85-b764-34d910585a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6809","nonce":2727456551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6811","nonce":2727456551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6815","nonce":2727456551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2727456551},{"type":"v1","addr":"172.21.15.170:6813","nonce":2727456551}]},"public_addr":"172.21.15.170:6809/2727456551","cluster_addr":"172.21.15.170:6811/2727456551","heartbeat_back_addr":"172.21.15.170:6815/2727456551","heartbeat_front_addr":"172.21.15.170:6813/2727456551","state":["exists","up"]},{"osd":3,"uuid":"fb2695db-07dd-4a82-a7a5-9bf70a4df160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6817","nonce":708481897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6819","nonce":708481897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6823","nonce":708481897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":708481897},{"type":"v1","addr":"172.21.15.170:6821","nonce":708481897}]},"public_addr":"172.21.15.170:6817/708481897","cluster_addr":"172.21.15.170:6819/708481897","heartbeat_back_addr":"172.21.15.170:6823/708481897","heartbeat_front_addr":"172.21.15.170:6821/708481897","state":["exists","up"]},{"osd":4,"uuid":"009a9026-b1d1-4160-b28e-fc3ccb6e12b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6803","nonce":1771265885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6805","nonce":1771265885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6809","nonce":1771265885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1771265885},{"type":"v1","addr":"172.21.15.191:6807","nonce":1771265885}]},"public_addr":"172.21.15.191:6803/1771265885","cluster_addr":"172.21.15.191:6805/1771265885","heartbeat_back_addr":"172.21.15.191:6809/1771265885","heartbeat_front_addr":"172.21.15.191:6807/1771265885","state":["exists","up"]},{"osd":5,"uuid":"0a1faa45-e320-423a-b677-6e0e691f3988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6819","nonce":2854287101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6821","nonce":2854287101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6825","nonce":2854287101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2854287101},{"type":"v1","addr":"172.21.15.191:6823","nonce":2854287101}]},"public_addr":"172.21.15.191:6819/2854287101","cluster_addr":"172.21.15.191:6821/2854287101","heartbeat_back_addr":"172.21.15.191:6825/2854287101","heartbeat_front_addr":"172.21.15.191:6823/2854287101","state":["exists","up"]},{"osd":6,"uuid":"beeea962-a5f5-4cad-a37a-f6d0f5c91199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6827","nonce":283530163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6829","nonce":283530163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6833","nonce":283530163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":283530163},{"type":"v1","addr":"172.21.15.191:6831","nonce":283530163}]},"public_addr":"172.21.15.191:6827/283530163","cluster_addr":"172.21.15.191:6829/283530163","heartbeat_back_addr":"172.21.15.191:6833/283530163","heartbeat_front_addr":"172.21.15.191:6831/283530163","state":["exists","up"]},{"osd":7,"uuid":"274c2ada-cfc8-41eb-aa4f-88f17b4adb3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6811","nonce":1521218345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6813","nonce":1521218345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6817","nonce":1521218345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1521218345},{"type":"v1","addr":"172.21.15.191:6815","nonce":1521218345}]},"public_addr":"172.21.15.191:6811/1521218345","cluster_addr":"172.21.15.191:6813/1521218345","heartbeat_back_addr":"172.21.15.191:6817/1521218345","heartbeat_front_addr":"172.21.15.191:6815/1521218345","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.408410+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.302948+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.675186+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.800476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.040032+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:10.905774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.226549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:13:11.021355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6835/1712756729":"2024-09-22T02:22:42.878916+0000","172.21.15.170:6834/1712756729":"2024-09-22T02:22:42.878916+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:22:49.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f870c073dd0 msgr2=0x7f870c0762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f870c073dd0 0x7f870c0762e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87180058d0 tx=0x7f8718005860 comp rx=0 tx=0).stop 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 msgr2=0x7f8728151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f8728151720 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f872400b410 tx=0x7f872400b8d0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 shutdown_connections 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f870c073dd0 0x7f870c0762e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f872815f990 0x7f872809d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87280ce4c0 0x7f872809ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 --2- 172.21.15.170:0/3988957456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87280cadf0 0x7f8728151720 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 >> 172.21.15.170:0/3988957456 conn(0x7f87280b1c40 msgr2=0x7f8728155240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 shutdown_connections 2024-09-21T02:22:49.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:49.303+0000 7f87300aa640 1 -- 172.21.15.170:0/3988957456 wait complete. 2024-09-21T02:22:50.326 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T02:22:50.327 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 -- 172.21.15.170:0/1298527120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 msgr2=0x7f942c06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 --2- 172.21.15.170:0/1298527120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c06f2c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f942c06dc60 tx=0x7f942401cd80 comp rx=0 tx=0).stop 2024-09-21T02:22:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 -- 172.21.15.170:0/1298527120 shutdown_connections 2024-09-21T02:22:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 --2- 172.21.15.170:0/1298527120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c06f2c0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 --2- 172.21.15.170:0/1298527120 >> v2:172.21.15.191:3300/0 conn(0x7f942c072e50 0x7f942c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 -- 172.21.15.170:0/1298527120 >> 172.21.15.170:0/1298527120 conn(0x7f942c0b1c40 msgr2=0x7f942c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 -- 172.21.15.170:0/1298527120 shutdown_connections 2024-09-21T02:22:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 -- 172.21.15.170:0/1298527120 wait complete. 2024-09-21T02:22:50.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 Processor -- start 2024-09-21T02:22:50.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.499+0000 7f9433398640 1 -- start start 2024-09-21T02:22:50.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f942c070050 0x7f942c1f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f942c072e50 0x7f942c1fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f942c169ce0 con 0x7f942c06eeb0 2024-09-21T02:22:50.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f942c169b60 con 0x7f942c070050 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f942c169e60 con 0x7f942c072e50 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c0c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40024/0 (socket says 172.21.15.170:40024) 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 -- 172.21.15.170:0/2905452278 learned_addr learned my addr 172.21.15.170:0/2905452278 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431935640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f942c072e50 0x7f942c1fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 -- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f942c072e50 msgr2=0x7f942c1fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f942c072e50 0x7f942c1fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 -- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f942c070050 msgr2=0x7f942c1f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9430933640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f942c070050 0x7f942c1f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f942c070050 0x7f942c1f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 -- 172.21.15.170:0/2905452278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c1fac50 con 0x7f942c06eeb0 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9430933640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f942c070050 0x7f942c1f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9431134640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c0c3c00 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f942800c970 tx=0x7f942800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:50.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428007b50 con 0x7f942c06eeb0 2024-09-21T02:22:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9428007de0 con 0x7f942c06eeb0 2024-09-21T02:22:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c0c60b0 con 0x7f942c06eeb0 2024-09-21T02:22:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428005290 con 0x7f942c06eeb0 2024-09-21T02:22:50.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c0c6670 con 0x7f942c06eeb0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.503+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f942c06f2c0 con 0x7f942c06eeb0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f9428005430 con 0x7f942c06eeb0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9410073e00 0x7f9410076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9430933640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9410073e00 0x7f9410076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f9428093500 con 0x7f942c06eeb0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7f9410079d10 0x7f941007c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 --> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f941007c850 con 0x7f9410079d10 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7f9428095e70 con 0x7f942c06eeb0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9431935640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7f9410079d10 0x7f941007c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9430933640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9410073e00 0x7f9410076310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f942c1faeb0 tx=0x7f941c009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9431935640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7f9410079d10 0x7f941007c180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:50.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.507+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== osd.0 v2:172.21.15.170:6800/1810219020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f941007c850 con 0x7f9410079d10 2024-09-21T02:22:50.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 --> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f942c0ccdd0 con 0x7f9410079d10 2024-09-21T02:22:50.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9421ffb640 1 -- 172.21.15.170:0/2905452278 <== osd.0 v2:172.21.15.170:6800/1810219020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f942c0ccdd0 con 0x7f9410079d10 2024-09-21T02:22:50.523 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:50.523 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:50.523 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7f9410079d10 msgr2=0x7f941007c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6800/1810219020,v1:172.21.15.170:6801/1810219020] conn(0x7f9410079d10 0x7f941007c180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9410073e00 msgr2=0x7f9410076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f9410073e00 0x7f9410076310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f942c1faeb0 tx=0x7f941c009040 comp rx=0 tx=0).stop 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 msgr2=0x7f942c0c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 --2- 172.21.15.170:0/2905452278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f942c06eeb0 0x7f942c0c3c00 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f942800c970 tx=0x7f942800ce30 comp rx=0 tx=0).stop 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9431935640 1 -- 172.21.15.170:0/2905452278 reap_dead start 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 shutdown_connections 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 >> 172.21.15.170:0/2905452278 conn(0x7f942c0b1c40 msgr2=0x7f942c0b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.519+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 shutdown_connections 2024-09-21T02:22:50.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.523+0000 7f9433398640 1 -- 172.21.15.170:0/2905452278 wait complete. 2024-09-21T02:22:50.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 -- 172.21.15.170:0/2397455771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914004810 msgr2=0x7fa914004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 --2- 172.21.15.170:0/2397455771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914004810 0x7fa914004c20 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa920009b30 tx=0x7fa92001cf70 comp rx=0 tx=0).stop 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 -- 172.21.15.170:0/2397455771 shutdown_connections 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 --2- 172.21.15.170:0/2397455771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914004810 0x7fa914004c20 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 -- 172.21.15.170:0/2397455771 >> 172.21.15.170:0/2397455771 conn(0x7fa92c0711c0 msgr2=0x7fa92c0715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 -- 172.21.15.170:0/2397455771 shutdown_connections 2024-09-21T02:22:50.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 -- 172.21.15.170:0/2397455771 wait complete. 2024-09-21T02:22:50.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 Processor -- start 2024-09-21T02:22:50.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.691+0000 7fa930fff640 1 -- start start 2024-09-21T02:22:50.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa914004810 0x7fa92c0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 0x7fa92c0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa92c0cb7e0 0x7fa92c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0ccdd0 con 0x7fa92c153c40 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa92c0ccc50 con 0x7fa92c0cb7e0 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fa92c0ccf50 con 0x7fa914004810 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 0x7fa92c0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 0x7fa92c0cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40046/0 (socket says 172.21.15.170:40046) 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 -- 172.21.15.170:0/4210607622 learned_addr learned my addr 172.21.15.170:0/4210607622 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa92ad9d640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa92c0cb7e0 0x7fa92c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa92a59c640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa914004810 0x7fa92c0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 -- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa914004810 msgr2=0x7fa92c0cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa914004810 0x7fa92c0cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 -- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa92c0cb7e0 msgr2=0x7fa92c159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa92c0cb7e0 0x7fa92c159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 -- 172.21.15.170:0/4210607622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92c0cbbf0 con 0x7fa92c153c40 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa92a59c640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa914004810 0x7fa92c0cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa929d9b640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 0x7fa92c0cb2a0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa920009b30 tx=0x7fa9200adb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9200bc020 con 0x7fa92c153c40 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa920005020 con 0x7fa92c153c40 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92c15b1b0 con 0x7fa92c153c40 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9200b2040 con 0x7fa92c153c40 2024-09-21T02:22:50.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92c15b770 con 0x7fa92c153c40 2024-09-21T02:22:50.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.695+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92c0d0750 con 0x7fa92c153c40 2024-09-21T02:22:50.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.699+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fa9200bf070 con 0x7fa92c153c40 2024-09-21T02:22:50.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.699+0000 7fa91affd640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa90c073dd0 0x7fa90c0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:50.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.699+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fa92012cf90 con 0x7fa92c153c40 2024-09-21T02:22:50.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.699+0000 7fa92a59c640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa90c073dd0 0x7fa90c0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:50.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.699+0000 7fa92a59c640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa90c073dd0 0x7fa90c0762e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa92c075850 tx=0x7fa91c006a20 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:50.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.703+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fa9200c5080 con 0x7fa92c153c40 2024-09-21T02:22:50.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa92c0b4d50 con 0x7fa90c073dd0 2024-09-21T02:22:50.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa91affd640 1 -- 172.21.15.170:0/4210607622 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa92c0b4d50 con 0x7fa90c073dd0 2024-09-21T02:22:50.868 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T02:22:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa90c073dd0 msgr2=0x7fa90c0762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa90c073dd0 0x7fa90c0762e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa92c075850 tx=0x7fa91c006a20 comp rx=0 tx=0).stop 2024-09-21T02:22:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 msgr2=0x7fa92c0cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 0x7fa92c0cb2a0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa920009b30 tx=0x7fa9200adb40 comp rx=0 tx=0).stop 2024-09-21T02:22:50.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 shutdown_connections 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fa90c073dd0 0x7fa90c0762e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa92c0cb7e0 0x7fa92c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c153c40 0x7fa92c0cb2a0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 --2- 172.21.15.170:0/4210607622 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa914004810 0x7fa92c0cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 >> 172.21.15.170:0/4210607622 conn(0x7fa92c0711c0 msgr2=0x7fa92c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 shutdown_connections 2024-09-21T02:22:50.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:50.867+0000 7fa930fff640 1 -- 172.21.15.170:0/4210607622 wait complete. 2024-09-21T02:22:50.889 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T02:22:50.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:51.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 -- 172.21.15.170:0/676370696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc62c003a60 msgr2=0x7fc62c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 --2- 172.21.15.170:0/676370696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc62c003a60 0x7fc62c003ed0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc634009b30 tx=0x7fc63401d230 comp rx=0 tx=0).stop 2024-09-21T02:22:51.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 -- 172.21.15.170:0/676370696 shutdown_connections 2024-09-21T02:22:51.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 --2- 172.21.15.170:0/676370696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc62c003a60 0x7fc62c003ed0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 -- 172.21.15.170:0/676370696 >> 172.21.15.170:0/676370696 conn(0x7fc64409a9c0 msgr2=0x7fc6440b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 -- 172.21.15.170:0/676370696 shutdown_connections 2024-09-21T02:22:51.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 -- 172.21.15.170:0/676370696 wait complete. 2024-09-21T02:22:51.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 Processor -- start 2024-09-21T02:22:51.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.051+0000 7fc64a970640 1 -- start start 2024-09-21T02:22:51.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc62c003a60 0x7fc6441f9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6440ca650 0x7fc6440b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6440af350 0x7fc6440af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440cf6a0 con 0x7fc6440af350 2024-09-21T02:22:51.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440cf520 con 0x7fc6440ca650 2024-09-21T02:22:51.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc6440cf820 con 0x7fc62c003a60 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6440af350 0x7fc6440af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6440af350 0x7fc6440af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40078/0 (socket says 172.21.15.170:40078) 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 -- 172.21.15.170:0/4024193171 learned_addr learned my addr 172.21.15.170:0/4024193171 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc643fff640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc62c003a60 0x7fc6441f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc6437fe640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6440ca650 0x7fc6440b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 -- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc62c003a60 msgr2=0x7fc6441f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc62c003a60 0x7fc6441f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 -- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6440ca650 msgr2=0x7fc6440b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6440ca650 0x7fc6440b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 -- 172.21.15.170:0/4024193171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6440afca0 con 0x7fc6440af350 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc6437fe640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6440ca650 0x7fc6440b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc648f0d640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6440af350 0x7fc6440af760 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc644073d40 tx=0x7fc63c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c018070 con 0x7fc6440af350 2024-09-21T02:22:51.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc63c003980 con 0x7fc6440af350 2024-09-21T02:22:51.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c003db0 con 0x7fc6440af350 2024-09-21T02:22:51.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6440c9140 con 0x7fc6440af350 2024-09-21T02:22:51.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6440c9890 con 0x7fc6440af350 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.055+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6440ca230 con 0x7fc6440af350 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fc63c01b070 con 0x7fc6440af350 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc630073e30 0x7fc630076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc643fff640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc630073e30 0x7fc630076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fc63c092460 con 0x7fc6440af350 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7fc630079d40 0x7fc63007c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 --> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc63c016e70 con 0x7fc630079d40 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7fc63c094dd0 con 0x7fc6440af350 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc6437fe640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7fc630079d40 0x7fc63007c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc643fff640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc630073e30 0x7fc630076340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc6340ae000 tx=0x7fc634019fd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc6437fe640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7fc630079d40 0x7fc63007c1b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.059+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== osd.1 v2:172.21.15.170:6824/2831250627 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc63c016e70 con 0x7fc630079d40 2024-09-21T02:22:51.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.071+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 --> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc6441fa560 con 0x7fc630079d40 2024-09-21T02:22:51.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.071+0000 7fc640ff9640 1 -- 172.21.15.170:0/4024193171 <== osd.1 v2:172.21.15.170:6824/2831250627 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc6441fa560 con 0x7fc630079d40 2024-09-21T02:22:51.075 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:51.075 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:51.075 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.071+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7fc630079d40 msgr2=0x7fc63007c1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.071+0000 7fc64a970640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6824/2831250627,v1:172.21.15.170:6825/2831250627] conn(0x7fc630079d40 0x7fc63007c1b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc630073e30 msgr2=0x7fc630076340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc630073e30 0x7fc630076340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc6340ae000 tx=0x7fc634019fd0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6440af350 msgr2=0x7fc6440af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 --2- 172.21.15.170:0/4024193171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6440af350 0x7fc6440af760 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc644073d40 tx=0x7fc63c00db90 comp rx=0 tx=0).stop 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc648f0d640 1 -- 172.21.15.170:0/4024193171 reap_dead start 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 shutdown_connections 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 >> 172.21.15.170:0/4024193171 conn(0x7fc64409a9c0 msgr2=0x7fc6440ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 shutdown_connections 2024-09-21T02:22:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.075+0000 7fc64a970640 1 -- 172.21.15.170:0/4024193171 wait complete. 2024-09-21T02:22:51.094 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T02:22:51.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 -- 172.21.15.170:0/3239983553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 msgr2=0x7f78cc0b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 --2- 172.21.15.170:0/3239983553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc0b25b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f78bc005fd0 tx=0x7f78bc01c730 comp rx=0 tx=0).stop 2024-09-21T02:22:51.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 -- 172.21.15.170:0/3239983553 shutdown_connections 2024-09-21T02:22:51.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 --2- 172.21.15.170:0/3239983553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78cc0b60e0 0x7f78cc0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 --2- 172.21.15.170:0/3239983553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc0b25b0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 -- 172.21.15.170:0/3239983553 >> 172.21.15.170:0/3239983553 conn(0x7f78cc09a9c0 msgr2=0x7f78cc0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 -- 172.21.15.170:0/3239983553 shutdown_connections 2024-09-21T02:22:51.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.243+0000 7f78d2006640 1 -- 172.21.15.170:0/3239983553 wait complete. 2024-09-21T02:22:51.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 Processor -- start 2024-09-21T02:22:51.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 -- start start 2024-09-21T02:22:51.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc1fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc0b60e0 0x7f78cc156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78cc0ccce0 0x7f78cc15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0b18e0 con 0x7f78cc0b21a0 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0b1760 con 0x7f78cc0b60e0 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78d2006640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc16a4f0 con 0x7f78cc0ccce0 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc1fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc1fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40094/0 (socket says 172.21.15.170:40094) 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 -- 172.21.15.170:0/2966723667 learned_addr learned my addr 172.21.15.170:0/2966723667 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cbfff640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78cc0ccce0 0x7f78cc15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 -- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78cc0ccce0 msgr2=0x7f78cc15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78cc0ccce0 0x7f78cc15f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 -- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc0b60e0 msgr2=0x7f78cc156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78caffd640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc0b60e0 0x7f78cc156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc0b60e0 0x7f78cc156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 -- 172.21.15.170:0/2966723667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc156b90 con 0x7f78cc0b21a0 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.247+0000 7f78cb7fe640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc1fc7c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f78bc01a3c0 tx=0x7f78bc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0bf070 con 0x7f78cc0b21a0 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78bc004470 con 0x7f78cc0b21a0 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1fd0a0 con 0x7f78cc0b21a0 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0048d0 con 0x7f78cc0b21a0 2024-09-21T02:22:51.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc165990 con 0x7f78cc0b21a0 2024-09-21T02:22:51.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78cc0b25b0 con 0x7f78cc0b21a0 2024-09-21T02:22:51.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f78bc0c2070 con 0x7f78cc0b21a0 2024-09-21T02:22:51.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78b3fff640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78b8073e00 0x7f78b8076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78caffd640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78b8073e00 0x7f78b8076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f78bc12d320 con 0x7f78cc0b21a0 2024-09-21T02:22:51.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.251+0000 7f78caffd640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78b8073e00 0x7f78b8076310 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0cb0f0 tx=0x7f78c0009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.255+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f78bc0c9020 con 0x7f78cc0b21a0 2024-09-21T02:22:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.419+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f78cc0b5d20 con 0x7f78b8073e00 2024-09-21T02:22:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78b3fff640 1 -- 172.21.15.170:0/2966723667 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f78cc0b5d20 con 0x7f78b8073e00 2024-09-21T02:22:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78b8073e00 msgr2=0x7f78b8076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78b8073e00 0x7f78b8076310 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0cb0f0 tx=0x7f78c0009040 comp rx=0 tx=0).stop 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 msgr2=0x7f78cc1fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc1fc7c0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f78bc01a3c0 tx=0x7f78bc005780 comp rx=0 tx=0).stop 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 shutdown_connections 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f78b8073e00 0x7f78b8076310 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78cc0ccce0 0x7f78cc15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc0b60e0 0x7f78cc156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 --2- 172.21.15.170:0/2966723667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78cc0b21a0 0x7f78cc1fc7c0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 >> 172.21.15.170:0/2966723667 conn(0x7f78cc09a9c0 msgr2=0x7f78cc15c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 shutdown_connections 2024-09-21T02:22:51.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.423+0000 7f78d2006640 1 -- 172.21.15.170:0/2966723667 wait complete. 2024-09-21T02:22:51.444 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T02:22:51.445 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:51.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/4228248456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b4d80 msgr2=0x7fdeb00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 --2- 172.21.15.170:0/4228248456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b4d80 0x7fdeb00ccae0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009900 tx=0x7fdea001c850 comp rx=0 tx=0).stop 2024-09-21T02:22:51.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/4228248456 shutdown_connections 2024-09-21T02:22:51.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 --2- 172.21.15.170:0/4228248456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b4d80 0x7fdeb00ccae0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 --2- 172.21.15.170:0/4228248456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb00b52d0 0x7fdeb00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/4228248456 >> 172.21.15.170:0/4228248456 conn(0x7fdeb00b1c40 msgr2=0x7fdeb00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/4228248456 shutdown_connections 2024-09-21T02:22:51.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/4228248456 wait complete. 2024-09-21T02:22:51.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.603+0000 7fdeb7dbe640 1 Processor -- start 2024-09-21T02:22:51.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- start start 2024-09-21T02:22:51.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb00b4d80 0x7fdeb009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b52d0 0x7fdeb009af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb00cd020 0x7fdeb01fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0169d20 con 0x7fdeb00b52d0 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0169ba0 con 0x7fdeb00cd020 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb0169ea0 con 0x7fdeb00b4d80 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b52d0 0x7fdeb009af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b52d0 0x7fdeb009af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40102/0 (socket says 172.21.15.170:40102) 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 -- 172.21.15.170:0/1775639599 learned_addr learned my addr 172.21.15.170:0/1775639599 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5b5a640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb00b4d80 0x7fdeb009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb635b640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb00cd020 0x7fdeb01fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 -- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb00b4d80 msgr2=0x7fdeb009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdeb00b4d80 0x7fdeb009a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 -- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb00cd020 msgr2=0x7fdeb01fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb00cd020 0x7fdeb01fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 -- 172.21.15.170:0/1775639599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb01f8600 con 0x7fdeb00b52d0 2024-09-21T02:22:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb635b640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdeb00cd020 0x7fdeb01fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:51.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb5359640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b52d0 0x7fdeb009af00 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009850 tx=0x7fdea0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00b2040 con 0x7fdeb00b52d0 2024-09-21T02:22:51.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdea0004040 con 0x7fdeb00b52d0 2024-09-21T02:22:51.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0004490 con 0x7fdeb00b52d0 2024-09-21T02:22:51.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb0151da0 con 0x7fdeb00b52d0 2024-09-21T02:22:51.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb01f8bc0 con 0x7fdeb00b52d0 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.607+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdeb00ccae0 con 0x7fdeb00b52d0 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fdea00bf020 con 0x7fdeb00b52d0 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fde94073dd0 0x7fde940762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fdea012dcc0 con 0x7fdeb00b52d0 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdeb5b5a640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fde94073dd0 0x7fde940762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7fde94079ce0 0x7fde9407c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 --> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde9407c820 con 0x7fde94079ce0 2024-09-21T02:22:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7fdea0130630 con 0x7fdeb00b52d0 2024-09-21T02:22:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdeb635b640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7fde94079ce0 0x7fde9407c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdeb5b5a640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fde94073dd0 0x7fde940762e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00bb00 tx=0x7fdeac005dd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdeb635b640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7fde94079ce0 0x7fde9407c150 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.611+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== osd.2 v2:172.21.15.170:6808/2727456551 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde9407c820 con 0x7fde94079ce0 2024-09-21T02:22:51.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 --> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdeb009d840 con 0x7fde94079ce0 2024-09-21T02:22:51.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdea67fc640 1 -- 172.21.15.170:0/1775639599 <== osd.2 v2:172.21.15.170:6808/2727456551 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdeb009d840 con 0x7fde94079ce0 2024-09-21T02:22:51.627 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:51.627 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:51.627 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:51.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7fde94079ce0 msgr2=0x7fde9407c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6808/2727456551,v1:172.21.15.170:6809/2727456551] conn(0x7fde94079ce0 0x7fde9407c150 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fde94073dd0 msgr2=0x7fde940762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fde94073dd0 0x7fde940762e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00bb00 tx=0x7fdeac005dd0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b52d0 msgr2=0x7fdeb009af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb7dbe640 1 --2- 172.21.15.170:0/1775639599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeb00b52d0 0x7fdeb009af00 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009850 tx=0x7fdea0005de0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.623+0000 7fdeb635b640 1 -- 172.21.15.170:0/1775639599 reap_dead start 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.627+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 shutdown_connections 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.627+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 >> 172.21.15.170:0/1775639599 conn(0x7fdeb00b1c40 msgr2=0x7fdeb00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.627+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 shutdown_connections 2024-09-21T02:22:51.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.627+0000 7fdeb7dbe640 1 -- 172.21.15.170:0/1775639599 wait complete. 2024-09-21T02:22:51.646 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/448728426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b4004810 msgr2=0x7fc1b4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/448728426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c20 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005fd0 tx=0x7fc1c001cd80 comp rx=0 tx=0).stop 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/448728426 shutdown_connections 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/448728426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b4004810 0x7fc1b4004c20 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/448728426 >> 172.21.15.170:0/448728426 conn(0x7fc1cc0b1c40 msgr2=0x7fc1cc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/448728426 shutdown_connections 2024-09-21T02:22:51.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/448728426 wait complete. 2024-09-21T02:22:51.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 Processor -- start 2024-09-21T02:22:51.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- start start 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1cc0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 0x7fc1cc0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc15c980 0x7fc1cc0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.787+0000 7fc1d1ec3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0cf760 con 0x7fc1cc0ae990 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1d1ec3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0c3a10 con 0x7fc1b4004810 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1d1ec3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0cf8e0 con 0x7fc1cc15c980 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 0x7fc1cc0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 0x7fc1cc0c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40122/0 (socket says 172.21.15.170:40122) 2024-09-21T02:22:51.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 -- 172.21.15.170:0/2946708768 learned_addr learned my addr 172.21.15.170:0/2946708768 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1cb7fe640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1cc0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 -- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc15c980 msgr2=0x7fc1cc0c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc15c980 0x7fc1cc0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 -- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 msgr2=0x7fc1cc0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1cc0ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 -- 172.21.15.170:0/2946708768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc1f8290 con 0x7fc1cc0ae990 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1caffd640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 0x7fc1cc0c7f10 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000ba60 tx=0x7fc1c0004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00be070 con 0x7fc1cc0ae990 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1c0005130 con 0x7fc1cc0ae990 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00ab050 con 0x7fc1cc0ae990 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1f7710 con 0x7fc1cc0ae990 2024-09-21T02:22:51.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc1f7cd0 con 0x7fc1cc0ae990 2024-09-21T02:22:51.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc1f7480 con 0x7fc1cc0ae990 2024-09-21T02:22:51.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fc1c00c1070 con 0x7fc1cc0ae990 2024-09-21T02:22:51.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1b3fff640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc1b8073dd0 0x7fc1b80762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:51.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fc1c012cfb0 con 0x7fc1cc0ae990 2024-09-21T02:22:51.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.791+0000 7fc1cb7fe640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc1b8073dd0 0x7fc1b80762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:51.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.795+0000 7fc1cb7fe640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc1b8073dd0 0x7fc1b80762e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc075850 tx=0x7fc1bc009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.799+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc1c0016a90 con 0x7fc1cc0ae990 2024-09-21T02:22:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.955+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc1cc0b3640 con 0x7fc1b8073dd0 2024-09-21T02:22:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.955+0000 7fc1b3fff640 1 -- 172.21.15.170:0/2946708768 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc1cc0b3640 con 0x7fc1b8073dd0 2024-09-21T02:22:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T02:22:51.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc1b8073dd0 msgr2=0x7fc1b80762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc1b8073dd0 0x7fc1b80762e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc075850 tx=0x7fc1bc009040 comp rx=0 tx=0).stop 2024-09-21T02:22:51.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 msgr2=0x7fc1cc0c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 0x7fc1cc0c7f10 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000ba60 tx=0x7fc1c0004620 comp rx=0 tx=0).stop 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 shutdown_connections 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc1b8073dd0 0x7fc1b80762e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1cc15c980 0x7fc1cc0c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1cc0ae990 0x7fc1cc0c7f10 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 --2- 172.21.15.170:0/2946708768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1b4004810 0x7fc1cc0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 >> 172.21.15.170:0/2946708768 conn(0x7fc1cc0b1c40 msgr2=0x7fc1cc0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 shutdown_connections 2024-09-21T02:22:51.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:51.959+0000 7fc1d1ec3640 1 -- 172.21.15.170:0/2946708768 wait complete. 2024-09-21T02:22:51.979 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T02:22:51.979 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:52.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- 172.21.15.170:0/3656245238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c5c005310 msgr2=0x7f3c5c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 --2- 172.21.15.170:0/3656245238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c5c005310 0x7f3c5c00b9d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009e00 tx=0x7f3c6801d170 comp rx=0 tx=0).stop 2024-09-21T02:22:52.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- 172.21.15.170:0/3656245238 shutdown_connections 2024-09-21T02:22:52.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 --2- 172.21.15.170:0/3656245238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c5c005310 0x7f3c5c00b9d0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- 172.21.15.170:0/3656245238 >> 172.21.15.170:0/3656245238 conn(0x7f3c700b2720 msgr2=0x7f3c700b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- 172.21.15.170:0/3656245238 shutdown_connections 2024-09-21T02:22:52.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- 172.21.15.170:0/3656245238 wait complete. 2024-09-21T02:22:52.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 Processor -- start 2024-09-21T02:22:52.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- start start 2024-09-21T02:22:52.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c5c005310 0x7f3c701539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70156340 0x7f3c700b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c700b1b10 0x7f3c701f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70070230 con 0x7f3c70156340 2024-09-21T02:22:52.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c700700b0 con 0x7f3c5c005310 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c77cf3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700703b0 con 0x7f3c700b1b10 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70156340 0x7f3c700b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70156340 0x7f3c700b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40144/0 (socket says 172.21.15.170:40144) 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 -- 172.21.15.170:0/1704691181 learned_addr learned my addr 172.21.15.170:0/1704691181 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c75a8f640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c5c005310 0x7f3c701539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c76290640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c700b1b10 0x7f3c701f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 -- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c700b1b10 msgr2=0x7f3c701f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c700b1b10 0x7f3c701f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 -- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c5c005310 msgr2=0x7f3c701539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c5c005310 0x7f3c701539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 -- 172.21.15.170:0/1704691181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700b1f20 con 0x7f3c70156340 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c76290640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c700b1b10 0x7f3c701f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c7528e640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70156340 0x7f3c700b15d0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00e5c0 tx=0x7f3c6c00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.131+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c015070 con 0x7f3c70156340 2024-09-21T02:22:52.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c6c010980 con 0x7f3c70156340 2024-09-21T02:22:52.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c70154410 con 0x7f3c70156340 2024-09-21T02:22:52.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c010dd0 con 0x7f3c70156340 2024-09-21T02:22:52.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701f1770 con 0x7f3c70156340 2024-09-21T02:22:52.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c700b6000 con 0x7f3c70156340 2024-09-21T02:22:52.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f3c6c018070 con 0x7f3c70156340 2024-09-21T02:22:52.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c54073dd0 0x7f3c540762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f3c6c093790 con 0x7f3c70156340 2024-09-21T02:22:52.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c75a8f640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c54073dd0 0x7f3c540762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f3c54079ce0 0x7f3c5407c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 --> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c6c024e50 con 0x7f3c54079ce0 2024-09-21T02:22:52.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c096100 con 0x7f3c70156340 2024-09-21T02:22:52.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c76290640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f3c54079ce0 0x7f3c5407c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c75a8f640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c54073dd0 0x7f3c540762e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3c60000c10 tx=0x7f3c60006050 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c76290640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f3c54079ce0 0x7f3c5407c150 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.135+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== osd.3 v2:172.21.15.170:6816/708481897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c6c024e50 con 0x7f3c54079ce0 2024-09-21T02:22:52.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.147+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 --> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c700c42b0 con 0x7f3c54079ce0 2024-09-21T02:22:52.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.147+0000 7f3c667fc640 1 -- 172.21.15.170:0/1704691181 <== osd.3 v2:172.21.15.170:6816/708481897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3c700c42b0 con 0x7f3c54079ce0 2024-09-21T02:22:52.151 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:52.151 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:52.151 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f3c54079ce0 msgr2=0x7f3c5407c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6816/708481897,v1:172.21.15.170:6817/708481897] conn(0x7f3c54079ce0 0x7f3c5407c150 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c54073dd0 msgr2=0x7f3c540762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3c54073dd0 0x7f3c540762e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3c60000c10 tx=0x7f3c60006050 comp rx=0 tx=0).stop 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70156340 msgr2=0x7f3c700b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 --2- 172.21.15.170:0/1704691181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70156340 0x7f3c700b15d0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00e5c0 tx=0x7f3c6c00ea80 comp rx=0 tx=0).stop 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c76290640 1 -- 172.21.15.170:0/1704691181 reap_dead start 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 shutdown_connections 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 >> 172.21.15.170:0/1704691181 conn(0x7f3c700b2720 msgr2=0x7f3c700b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 shutdown_connections 2024-09-21T02:22:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.151+0000 7f3c77cf3640 1 -- 172.21.15.170:0/1704691181 wait complete. 2024-09-21T02:22:52.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- 172.21.15.170:0/1850203795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 msgr2=0x7f3d6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/1850203795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d6c004810 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f3d84160550 tx=0x7f3d7401d210 comp rx=0 tx=0).stop 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d83fff640 1 -- 172.21.15.170:0/1850203795 reap_dead start 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- 172.21.15.170:0/1850203795 shutdown_connections 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- 172.21.15.170:0/1850203795 >> 172.21.15.170:0/1850203795 conn(0x7f3d840b1c40 msgr2=0x7f3d840b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- 172.21.15.170:0/1850203795 shutdown_connections 2024-09-21T02:22:52.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- 172.21.15.170:0/1850203795 wait complete. 2024-09-21T02:22:52.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 Processor -- start 2024-09-21T02:22:52.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- start start 2024-09-21T02:22:52.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d84154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8414ed40 0x7f3d8414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d840ad5e0 0x7f3d840ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840cf8e0 con 0x7f3d6c004020 2024-09-21T02:22:52.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840cf760 con 0x7f3d840ad5e0 2024-09-21T02:22:52.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.315+0000 7f3d89dd6640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8406ff70 con 0x7f3d8414ed40 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d84154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d84154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40164/0 (socket says 172.21.15.170:40164) 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 -- 172.21.15.170:0/817401514 learned_addr learned my addr 172.21.15.170:0/817401514 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d82ffd640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8414ed40 0x7f3d8414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d83fff640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d840ad5e0 0x7f3d840ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 -- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8414ed40 msgr2=0x7f3d8414e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8414ed40 0x7f3d8414e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 -- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d840ad5e0 msgr2=0x7f3d840ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d840ad5e0 0x7f3d840ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 -- 172.21.15.170:0/817401514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8414f150 con 0x7f3d6c004020 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d837fe640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d84154890 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f3d78010990 tx=0x7f3d78010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800ed80 con 0x7f3d6c004020 2024-09-21T02:22:52.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d78005020 con 0x7f3d6c004020 2024-09-21T02:22:52.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78005450 con 0x7f3d6c004020 2024-09-21T02:22:52.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8414ea10 con 0x7f3d6c004020 2024-09-21T02:22:52.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d840b0e00 con 0x7f3d6c004020 2024-09-21T02:22:52.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d840b33e0 con 0x7f3d6c004020 2024-09-21T02:22:52.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f3d780047b0 con 0x7f3d6c004020 2024-09-21T02:22:52.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d6bfff640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d70073e00 0x7f3d70076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f3d78093a70 con 0x7f3d6c004020 2024-09-21T02:22:52.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d82ffd640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d70073e00 0x7f3d70076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.319+0000 7f3d82ffd640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d70073e00 0x7f3d70076310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3d740ad000 tx=0x7f3d74019fb0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.323+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f3d78060540 con 0x7f3d6c004020 2024-09-21T02:22:52.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.483+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f3d841f7d30 con 0x7f3d70073e00 2024-09-21T02:22:52.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.483+0000 7f3d6bfff640 1 -- 172.21.15.170:0/817401514 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3d841f7d30 con 0x7f3d70073e00 2024-09-21T02:22:52.487 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d70073e00 msgr2=0x7f3d70076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d70073e00 0x7f3d70076310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3d740ad000 tx=0x7f3d74019fb0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 msgr2=0x7f3d84154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d84154890 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f3d78010990 tx=0x7f3d78010e50 comp rx=0 tx=0).stop 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 shutdown_connections 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d70073e00 0x7f3d70076310 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d8414ed40 0x7f3d8414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d840ad5e0 0x7f3d840ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 --2- 172.21.15.170:0/817401514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d6c004020 0x7f3d84154890 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 >> 172.21.15.170:0/817401514 conn(0x7f3d840b1c40 msgr2=0x7f3d840cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 shutdown_connections 2024-09-21T02:22:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.487+0000 7f3d89dd6640 1 -- 172.21.15.170:0/817401514 wait complete. 2024-09-21T02:22:52.508 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T02:22:52.508 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc25a4cc640 1 -- 172.21.15.170:0/3014578168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc23c003a10 msgr2=0x7fc23c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc25a4cc640 1 --2- 172.21.15.170:0/3014578168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc23c003a10 0x7fc23c003e80 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fc244009b60 tx=0x7fc24401d2b0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc258a69640 1 -- 172.21.15.170:0/3014578168 reap_dead start 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc25a4cc640 1 -- 172.21.15.170:0/3014578168 shutdown_connections 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc25a4cc640 1 -- 172.21.15.170:0/3014578168 >> 172.21.15.170:0/3014578168 conn(0x7fc2540ced70 msgr2=0x7fc2540cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc25a4cc640 1 -- 172.21.15.170:0/3014578168 shutdown_connections 2024-09-21T02:22:52.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.667+0000 7fc25a4cc640 1 -- 172.21.15.170:0/3014578168 wait complete. 2024-09-21T02:22:52.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 Processor -- start 2024-09-21T02:22:52.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- start start 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc23c003a10 0x7fc25415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc25415fd20 0x7fc2541fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2541f8800 0x7fc2541fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc254168520 con 0x7fc2541f8800 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541683a0 con 0x7fc23c003a10 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc2541686a0 con 0x7fc25415fd20 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2541f8800 0x7fc2541fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2541f8800 0x7fc2541fa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40184/0 (socket says 172.21.15.170:40184) 2024-09-21T02:22:52.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 -- 172.21.15.170:0/431055327 learned_addr learned my addr 172.21.15.170:0/431055327 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc253fff640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc23c003a10 0x7fc25415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc2537fe640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc25415fd20 0x7fc2541fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 -- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc25415fd20 msgr2=0x7fc2541fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc25415fd20 0x7fc2541fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 -- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc23c003a10 msgr2=0x7fc25415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc23c003a10 0x7fc25415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 -- 172.21.15.170:0/431055327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2540c5e20 con 0x7fc2541f8800 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc258a69640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2541f8800 0x7fc2541fa5c0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fc25415cf30 tx=0x7fc24c00dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c003db0 con 0x7fc2541f8800 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24c00b890 con 0x7fc2541f8800 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2540cc5e0 con 0x7fc2541f8800 2024-09-21T02:22:52.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00bce0 con 0x7fc2541f8800 2024-09-21T02:22:52.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2540c60c0 con 0x7fc2541f8800 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.671+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc254160ef0 con 0x7fc2541f8800 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fc24c006070 con 0x7fc2541f8800 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc250ff9640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc240073e00 0x7fc240076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fc24c093c70 con 0x7fc2541f8800 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc253fff640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc240073e00 0x7fc240076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc250ff9640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7fc240079d10 0x7fc24007c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 --> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc24007c850 con 0x7fc240079d10 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7fc24c096530 con 0x7fc2541f8800 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc253fff640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc240073e00 0x7fc240076310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc2440ae000 tx=0x7fc24401a050 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc2537fe640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7fc240079d10 0x7fc24007c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.675+0000 7fc2537fe640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7fc240079d10 0x7fc24007c180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.679+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== osd.4 v2:172.21.15.191:6802/1771265885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc24007c850 con 0x7fc240079d10 2024-09-21T02:22:52.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 --> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc2540cd130 con 0x7fc240079d10 2024-09-21T02:22:52.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc250ff9640 1 -- 172.21.15.170:0/431055327 <== osd.4 v2:172.21.15.191:6802/1771265885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc2540cd130 con 0x7fc240079d10 2024-09-21T02:22:52.693 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:52.693 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:52.693 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7fc240079d10 msgr2=0x7fc24007c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.191:6802/1771265885,v1:172.21.15.191:6803/1771265885] conn(0x7fc240079d10 0x7fc24007c180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc240073e00 msgr2=0x7fc240076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc240073e00 0x7fc240076310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc2440ae000 tx=0x7fc24401a050 comp rx=0 tx=0).stop 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2541f8800 msgr2=0x7fc2541fa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 --2- 172.21.15.170:0/431055327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2541f8800 0x7fc2541fa5c0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fc25415cf30 tx=0x7fc24c00dd80 comp rx=0 tx=0).stop 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc258a69640 1 -- 172.21.15.170:0/431055327 reap_dead start 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 shutdown_connections 2024-09-21T02:22:52.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 >> 172.21.15.170:0/431055327 conn(0x7fc2540ced70 msgr2=0x7fc254163020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 shutdown_connections 2024-09-21T02:22:52.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.691+0000 7fc25a4cc640 1 -- 172.21.15.170:0/431055327 wait complete. 2024-09-21T02:22:52.712 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T02:22:52.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 -- 172.21.15.170:0/3737837501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8698003a10 msgr2=0x7f8698003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 --2- 172.21.15.170:0/3737837501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8698003a10 0x7f8698003e60 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f86b0074030 tx=0x7f86a401cf60 comp rx=0 tx=0).stop 2024-09-21T02:22:52.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 -- 172.21.15.170:0/3737837501 shutdown_connections 2024-09-21T02:22:52.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 --2- 172.21.15.170:0/3737837501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8698003a10 0x7f8698003e60 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 -- 172.21.15.170:0/3737837501 >> 172.21.15.170:0/3737837501 conn(0x7f86b00b1c40 msgr2=0x7f86b00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:52.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 -- 172.21.15.170:0/3737837501 shutdown_connections 2024-09-21T02:22:52.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.859+0000 7f86b6108640 1 -- 172.21.15.170:0/3737837501 wait complete. 2024-09-21T02:22:52.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 Processor -- start 2024-09-21T02:22:52.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- start start 2024-09-21T02:22:52.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8698003a10 0x7f86b01fb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86b01f7e50 0x7f86b0152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 0x7f86b015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0070030 con 0x7f86b015e0e0 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0074f00 con 0x7f8698003a10 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f86b00701b0 con 0x7f86b01f7e50 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 0x7f86b015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 0x7f86b015e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40198/0 (socket says 172.21.15.170:40198) 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 -- 172.21.15.170:0/2552325048 learned_addr learned my addr 172.21.15.170:0/2552325048 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86af7fe640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8698003a10 0x7f86b01fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 -- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86b01f7e50 msgr2=0x7f86b0152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86b01f7e50 0x7f86b0152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 -- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8698003a10 msgr2=0x7f86b01fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8698003a10 0x7f86b01fb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 -- 172.21.15.170:0/2552325048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b015ea30 con 0x7f86b015e0e0 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86affff640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 0x7f86b015e4f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f86b0074c60 tx=0x7f86a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8011790 con 0x7f86b015e0e0 2024-09-21T02:22:52.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a8017070 con 0x7f86b015e0e0 2024-09-21T02:22:52.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b009d540 con 0x7f86b015e0e0 2024-09-21T02:22:52.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8011cc0 con 0x7f86b015e0e0 2024-09-21T02:22:52.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01f7650 con 0x7f86b015e0e0 2024-09-21T02:22:52.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.863+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b009d2b0 con 0x7f86b015e0e0 2024-09-21T02:22:52.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.867+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f86a8006b60 con 0x7f86b015e0e0 2024-09-21T02:22:52.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.867+0000 7f8697fff640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f869c073dd0 0x7f869c0762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:52.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.867+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f86a8093270 con 0x7f86b015e0e0 2024-09-21T02:22:52.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.867+0000 7f86af7fe640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f869c073dd0 0x7f869c0762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:52.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.867+0000 7f86af7fe640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f869c073dd0 0x7f869c0762e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ab000 tx=0x7f86a4019da0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:52.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:52.871+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f86b015ea30 con 0x7f86b015e0e0 2024-09-21T02:22:53.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f86b00706a0 con 0x7f869c073dd0 2024-09-21T02:22:53.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f8697fff640 1 -- 172.21.15.170:0/2552325048 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f86b00706a0 con 0x7f869c073dd0 2024-09-21T02:22:53.040 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T02:22:53.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f869c073dd0 msgr2=0x7f869c0762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f869c073dd0 0x7f869c0762e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ab000 tx=0x7f86a4019da0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 msgr2=0x7f86b015e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 0x7f86b015e4f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f86b0074c60 tx=0x7f86a800c8b0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 shutdown_connections 2024-09-21T02:22:53.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f869c073dd0 0x7f869c0762e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86b01f7e50 0x7f86b0152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b015e0e0 0x7f86b015e4f0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 --2- 172.21.15.170:0/2552325048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8698003a10 0x7f86b01fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 >> 172.21.15.170:0/2552325048 conn(0x7f86b00b1c40 msgr2=0x7f86b00b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 shutdown_connections 2024-09-21T02:22:53.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.038+0000 7f86b6108640 1 -- 172.21.15.170:0/2552325048 wait complete. 2024-09-21T02:22:53.061 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T02:22:53.061 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 -- 172.21.15.170:0/318231189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 msgr2=0x7fe9780739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 --2- 172.21.15.170:0/318231189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780739e0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fe970009c80 tx=0x7fe97001cda0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 -- 172.21.15.170:0/318231189 shutdown_connections 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 --2- 172.21.15.170:0/318231189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780739e0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 --2- 172.21.15.170:0/318231189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe9780cace0 0x7fe9780cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 -- 172.21.15.170:0/318231189 >> 172.21.15.170:0/318231189 conn(0x7fe9780b1c40 msgr2=0x7fe9780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 -- 172.21.15.170:0/318231189 shutdown_connections 2024-09-21T02:22:53.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 -- 172.21.15.170:0/318231189 wait complete. 2024-09-21T02:22:53.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 Processor -- start 2024-09-21T02:22:53.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.218+0000 7fe97dd4a640 1 -- start start 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe97806ffc0 0x7fe978165a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9780cace0 0x7fe9780c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe978168e90 con 0x7fe9780735d0 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe978168d10 con 0x7fe9780cace0 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe978169010 con 0x7fe97806ffc0 2024-09-21T02:22:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40224/0 (socket says 172.21.15.170:40224) 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 -- 172.21.15.170:0/2784009859 learned_addr learned my addr 172.21.15.170:0/2784009859 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe9777fe640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe97806ffc0 0x7fe978165a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe977fff640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9780cace0 0x7fe9780c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 -- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe97806ffc0 msgr2=0x7fe978165a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe97806ffc0 0x7fe978165a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 -- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9780cace0 msgr2=0x7fe9780c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9780cace0 0x7fe9780c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 -- 172.21.15.170:0/2784009859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97809de20 con 0x7fe9780735d0 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe9777fe640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe97806ffc0 0x7fe978165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe977fff640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9780cace0 0x7fe9780c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe976ffd640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780c9310 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c3f0 tx=0x7fe96c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c017070 con 0x7fe9780735d0 2024-09-21T02:22:53.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe96c0053c0 con 0x7fe9780735d0 2024-09-21T02:22:53.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9781f8f90 con 0x7fe9780735d0 2024-09-21T02:22:53.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c003860 con 0x7fe9780735d0 2024-09-21T02:22:53.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9780adb80 con 0x7fe9780735d0 2024-09-21T02:22:53.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9781f8d80 con 0x7fe9780735d0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fe96c01a070 con 0x7fe9780735d0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.222+0000 7fe95ffff640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe964073dd0 0x7fe9640762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fe96c092890 con 0x7fe9780735d0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe9777fe640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe964073dd0 0x7fe9640762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe95ffff640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fe964079ce0 0x7fe96407c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 --> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe96c010e50 con 0x7fe964079ce0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7fe96c095610 con 0x7fe9780735d0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe9777fe640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe964073dd0 0x7fe9640762e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe968005870 tx=0x7fe968005dd0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe977fff640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fe964079ce0 0x7fe96407c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe977fff640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fe964079ce0 0x7fe96407c150 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.226+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== osd.5 v2:172.21.15.191:6818/2854287101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe96c010e50 con 0x7fe964079ce0 2024-09-21T02:22:53.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 --> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe97809dbd0 con 0x7fe964079ce0 2024-09-21T02:22:53.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe95ffff640 1 -- 172.21.15.170:0/2784009859 <== osd.5 v2:172.21.15.191:6818/2854287101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe97809dbd0 con 0x7fe964079ce0 2024-09-21T02:22:53.244 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:53.244 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fe964079ce0 msgr2=0x7fe96407c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.191:6818/2854287101,v1:172.21.15.191:6819/2854287101] conn(0x7fe964079ce0 0x7fe96407c150 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe964073dd0 msgr2=0x7fe9640762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe964073dd0 0x7fe9640762e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe968005870 tx=0x7fe968005dd0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 msgr2=0x7fe9780c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 --2- 172.21.15.170:0/2784009859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9780735d0 0x7fe9780c9310 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00c3f0 tx=0x7fe96c00c8b0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe977fff640 1 -- 172.21.15.170:0/2784009859 reap_dead start 2024-09-21T02:22:53.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 shutdown_connections 2024-09-21T02:22:53.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 >> 172.21.15.170:0/2784009859 conn(0x7fe9780b1c40 msgr2=0x7fe9780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 shutdown_connections 2024-09-21T02:22:53.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.242+0000 7fe97dd4a640 1 -- 172.21.15.170:0/2784009859 wait complete. 2024-09-21T02:22:53.263 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T02:22:53.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 -- 172.21.15.170:0/4191300871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47880b9ba0 msgr2=0x7f47880d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 --2- 172.21.15.170:0/4191300871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47880b9ba0 0x7f47880d4e00 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4778009900 tx=0x7f477801c8b0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 -- 172.21.15.170:0/4191300871 shutdown_connections 2024-09-21T02:22:53.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 --2- 172.21.15.170:0/4191300871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47880b6600 0x7f47880cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 --2- 172.21.15.170:0/4191300871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47880b9ba0 0x7f47880d4e00 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 -- 172.21.15.170:0/4191300871 >> 172.21.15.170:0/4191300871 conn(0x7f478809d7d0 msgr2=0x7f47880d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 -- 172.21.15.170:0/4191300871 shutdown_connections 2024-09-21T02:22:53.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.398+0000 7f478d496640 1 -- 172.21.15.170:0/4191300871 wait complete. 2024-09-21T02:22:53.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 Processor -- start 2024-09-21T02:22:53.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- start start 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 0x7f478814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47880b9ba0 0x7f478815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f478814e380 0x7f47880cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f478816c790 con 0x7f478806dd80 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f478816c610 con 0x7f478814e380 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f478816c910 con 0x7f47880b9ba0 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 0x7f478814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 0x7f478814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40242/0 (socket says 172.21.15.170:40242) 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 -- 172.21.15.170:0/2308157715 learned_addr learned my addr 172.21.15.170:0/2308157715 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f47877fe640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f478814e380 0x7f47880cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f477ffff640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47880b9ba0 0x7f478815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 -- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47880b9ba0 msgr2=0x7f478815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47880b9ba0 0x7f478815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 -- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f478814e380 msgr2=0x7f47880cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f478814e380 0x7f47880cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 -- 172.21.15.170:0/2308157715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478815a840 con 0x7f478806dd80 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f4786ffd640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 0x7f478814dd50 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f47780b0040 tx=0x7f47780a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780b2040 con 0x7f478806dd80 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4778003970 con 0x7f478806dd80 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47880b5e80 con 0x7f478806dd80 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778003d80 con 0x7f478806dd80 2024-09-21T02:22:53.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4788159b00 con 0x7f478806dd80 2024-09-21T02:22:53.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.402+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47880cfc60 con 0x7f478806dd80 2024-09-21T02:22:53.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.406+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f47780bf060 con 0x7f478806dd80 2024-09-21T02:22:53.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.406+0000 7f477f7fe640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4768073dd0 0x7f47680762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.406+0000 7f477ffff640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4768073dd0 0x7f47680762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.406+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f477812d310 con 0x7f478806dd80 2024-09-21T02:22:53.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.406+0000 7f477ffff640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4768073dd0 0x7f47680762e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f478815cab0 tx=0x7f4770009040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.410+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f478815a840 con 0x7f478806dd80 2024-09-21T02:22:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f478815aa20 con 0x7f4768073dd0 2024-09-21T02:22:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f477f7fe640 1 -- 172.21.15.170:0/2308157715 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f478815aa20 con 0x7f4768073dd0 2024-09-21T02:22:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T02:22:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4768073dd0 msgr2=0x7f47680762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4768073dd0 0x7f47680762e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f478815cab0 tx=0x7f4770009040 comp rx=0 tx=0).stop 2024-09-21T02:22:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 msgr2=0x7f478814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 0x7f478814dd50 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f47780b0040 tx=0x7f47780a57f0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 shutdown_connections 2024-09-21T02:22:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f4768073dd0 0x7f47680762e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47880b9ba0 0x7f478815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f478814e380 0x7f47880cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 --2- 172.21.15.170:0/2308157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f478806dd80 0x7f478814dd50 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 >> 172.21.15.170:0/2308157715 conn(0x7f478809d7d0 msgr2=0x7f47880b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 shutdown_connections 2024-09-21T02:22:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.570+0000 7f478d496640 1 -- 172.21.15.170:0/2308157715 wait complete. 2024-09-21T02:22:53.593 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T02:22:53.594 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:53.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/238159968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140cf590 msgr2=0x7f3d140b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/238159968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140cf590 0x7f3d140b06a0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f3d140737c0 tx=0x7f3d0801ccc0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/238159968 shutdown_connections 2024-09-21T02:22:53.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/238159968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d140b4720 0x7f3d140b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/238159968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140cf590 0x7f3d140b06a0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/238159968 >> v2:172.21.15.191:3300/0 conn(0x7f3d140cf180 0x7f3d140b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/238159968 >> 172.21.15.170:0/238159968 conn(0x7f3d140cdd00 msgr2=0x7f3d140ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/238159968 shutdown_connections 2024-09-21T02:22:53.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.734+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/238159968 wait complete. 2024-09-21T02:22:53.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 Processor -- start 2024-09-21T02:22:53.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 -- start start 2024-09-21T02:22:53.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140b4720 0x7f3d14070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d140cf180 0x7f3d1406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d140cf590 0x7f3d140cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14074b90 con 0x7f3d140b4720 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14074a10 con 0x7f3d140cf590 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1c1c9640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f3d14074d10 con 0x7f3d140cf180 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140b4720 0x7f3d14070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140b4720 0x7f3d14070a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40256/0 (socket says 172.21.15.170:40256) 2024-09-21T02:22:53.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 -- 172.21.15.170:0/2269871354 learned_addr learned my addr 172.21.15.170:0/2269871354 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:53.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d1a766640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d140cf590 0x7f3d140cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19764640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d140cf180 0x7f3d1406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 -- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d140cf180 msgr2=0x7f3d1406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d140cf180 0x7f3d1406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 -- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d140cf590 msgr2=0x7f3d140cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d140cf590 0x7f3d140cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 -- 172.21.15.170:0/2269871354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d14071ee0 con 0x7f3d140b4720 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.738+0000 7f3d19f65640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140b4720 0x7f3d14070a60 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00d730 tx=0x7f3d0c00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c017070 con 0x7f3d140b4720 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d0c009a30 con 0x7f3d140b4720 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1415eeb0 con 0x7f3d140b4720 2024-09-21T02:22:53.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c010460 con 0x7f3d140b4720 2024-09-21T02:22:53.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d14071960 con 0x7f3d140b4720 2024-09-21T02:22:53.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d1415e780 con 0x7f3d140b4720 2024-09-21T02:22:53.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f3d0c01a070 con 0x7f3d140b4720 2024-09-21T02:22:53.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d00073dd0 0x7f3d000762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f3d0c093520 con 0x7f3d140b4720 2024-09-21T02:22:53.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d19764640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d00073dd0 0x7f3d000762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f3d00079ce0 0x7f3d0007c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 --> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d0c026e50 con 0x7f3d00079ce0 2024-09-21T02:22:53.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7f3d0c095e90 con 0x7f3d140b4720 2024-09-21T02:22:53.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d19764640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d00073dd0 0x7f3d000762e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3d080af000 tx=0x7f3d0801b500 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d1a766640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f3d00079ce0 0x7f3d0007c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.742+0000 7f3d1a766640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f3d00079ce0 0x7f3d0007c150 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.746+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== osd.6 v2:172.21.15.191:6826/283530163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d0c026e50 con 0x7f3d00079ce0 2024-09-21T02:22:53.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 --> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3d140b4b30 con 0x7f3d00079ce0 2024-09-21T02:22:53.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d127fc640 1 -- 172.21.15.170:0/2269871354 <== osd.6 v2:172.21.15.191:6826/283530163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3d140b4b30 con 0x7f3d00079ce0 2024-09-21T02:22:53.762 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:53.762 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:53.762 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:53.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f3d00079ce0 msgr2=0x7f3d0007c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.191:6826/283530163,v1:172.21.15.191:6827/283530163] conn(0x7f3d00079ce0 0x7f3d0007c150 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d00073dd0 msgr2=0x7f3d000762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f3d00073dd0 0x7f3d000762e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3d080af000 tx=0x7f3d0801b500 comp rx=0 tx=0).stop 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140b4720 msgr2=0x7f3d14070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 --2- 172.21.15.170:0/2269871354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d140b4720 0x7f3d14070a60 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00d730 tx=0x7f3d0c00dbf0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1a766640 1 -- 172.21.15.170:0/2269871354 reap_dead start 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 shutdown_connections 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 >> 172.21.15.170:0/2269871354 conn(0x7f3d140cdd00 msgr2=0x7f3d1414c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 shutdown_connections 2024-09-21T02:22:53.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.758+0000 7f3d1c1c9640 1 -- 172.21.15.170:0/2269871354 wait complete. 2024-09-21T02:22:53.780 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 -- 172.21.15.170:0/1403884526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 msgr2=0x7f82080b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 --2- 172.21.15.170:0/1403884526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 0x7f82080b5f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8204009840 tx=0x7f820401c820 comp rx=0 tx=0).stop 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 -- 172.21.15.170:0/1403884526 shutdown_connections 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 --2- 172.21.15.170:0/1403884526 >> v2:172.21.15.191:3300/0 conn(0x7f82080b68d0 0x7f82080b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 --2- 172.21.15.170:0/1403884526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 0x7f82080b5f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 -- 172.21.15.170:0/1403884526 >> 172.21.15.170:0/1403884526 conn(0x7f820809d870 msgr2=0x7f820808bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:53.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 -- 172.21.15.170:0/1403884526 shutdown_connections 2024-09-21T02:22:53.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 -- 172.21.15.170:0/1403884526 wait complete. 2024-09-21T02:22:53.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 Processor -- start 2024-09-21T02:22:53.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.926+0000 7f820ebc7640 1 -- start start 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 0x7f82081556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82080b64c0 0x7f8208157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 0x7f8208158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f820816c280 con 0x7f82080b68d0 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f820816c100 con 0x7f82080b64c0 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f820816c400 con 0x7f820806f430 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 0x7f8208158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 0x7f8208158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40268/0 (socket says 172.21.15.170:40268) 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 -- 172.21.15.170:0/3945456141 learned_addr learned my addr 172.21.15.170:0/3945456141 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f81fffff640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82080b64c0 0x7f8208157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820c963640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 0x7f82081556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 -- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 msgr2=0x7f82081556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 0x7f82081556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 -- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82080b64c0 msgr2=0x7f8208157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82080b64c0 0x7f8208157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 -- 172.21.15.170:0/3945456141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8208157e50 con 0x7f82080b68d0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f81fffff640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82080b64c0 0x7f8208157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820d164640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 0x7f8208158850 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f820000ef20 tx=0x7f820000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200003980 con 0x7f82080b68d0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8200003c10 con 0x7f82080b68d0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8208158110 con 0x7f82080b68d0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200004020 con 0x7f82080b68d0 2024-09-21T02:22:53.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8208155f00 con 0x7f82080b68d0 2024-09-21T02:22:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82080b5f80 con 0x7f82080b68d0 2024-09-21T02:22:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.930+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f820001d020 con 0x7f82080b68d0 2024-09-21T02:22:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.934+0000 7f81fd7fa640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f81f4073dd0 0x7f81f40762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.934+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f8200093900 con 0x7f82080b68d0 2024-09-21T02:22:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.934+0000 7f820c963640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f81f4073dd0 0x7f81f40762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.934+0000 7f820c963640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f81f4073dd0 0x7f81f40762e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f82040af000 tx=0x7f82040b2040 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:53.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:53.934+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f8200060170 con 0x7f82080b68d0 2024-09-21T02:22:54.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.106+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f820815f880 con 0x7f81f4073dd0 2024-09-21T02:22:54.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.106+0000 7f81fd7fa640 1 -- 172.21.15.170:0/3945456141 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f820815f880 con 0x7f81f4073dd0 2024-09-21T02:22:54.110 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f81f4073dd0 msgr2=0x7f81f40762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f81f4073dd0 0x7f81f40762e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f82040af000 tx=0x7f82040b2040 comp rx=0 tx=0).stop 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 msgr2=0x7f8208158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 0x7f8208158850 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f820000ef20 tx=0x7f820000c800 comp rx=0 tx=0).stop 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 shutdown_connections 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f81f4073dd0 0x7f81f40762e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82080b68d0 0x7f8208158850 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82080b64c0 0x7f8208157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 --2- 172.21.15.170:0/3945456141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f820806f430 0x7f82081556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 >> 172.21.15.170:0/3945456141 conn(0x7f820809d870 msgr2=0x7f82080b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:54.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 shutdown_connections 2024-09-21T02:22:54.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.110+0000 7f820ebc7640 1 -- 172.21.15.170:0/3945456141 wait complete. 2024-09-21T02:22:54.133 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T02:22:54.134 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T02:22:54.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 -- 172.21.15.170:0/487033195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80ca7b0 msgr2=0x7f40b80cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 --2- 172.21.15.170:0/487033195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80ca7b0 0x7f40b80cabc0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f40b8075300 tx=0x7f40b401c6c0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 -- 172.21.15.170:0/487033195 shutdown_connections 2024-09-21T02:22:54.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 --2- 172.21.15.170:0/487033195 >> v2:172.21.15.191:3300/0 conn(0x7f40b806ffc0 0x7f40b80703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 --2- 172.21.15.170:0/487033195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b80ca7b0 0x7f40b80cabc0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 -- 172.21.15.170:0/487033195 >> 172.21.15.170:0/487033195 conn(0x7f40b80b1c40 msgr2=0x7f40b80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:54.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 -- 172.21.15.170:0/487033195 shutdown_connections 2024-09-21T02:22:54.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.278+0000 7f40bffd3640 1 -- 172.21.15.170:0/487033195 wait complete. 2024-09-21T02:22:54.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 Processor -- start 2024-09-21T02:22:54.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- start start 2024-09-21T02:22:54.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b806ffc0 0x7f40b815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80ca7b0 0x7f40b815de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80cc7e0 0x7f40b813f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40b806f460 con 0x7f40b806ffc0 2024-09-21T02:22:54.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40b806f2e0 con 0x7f40b80cc7e0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f40b806f5e0 con 0x7f40b80ca7b0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b806ffc0 0x7f40b815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b806ffc0 0x7f40b815beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40292/0 (socket says 172.21.15.170:40292) 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 -- 172.21.15.170:0/3541580507 learned_addr learned my addr 172.21.15.170:0/3541580507 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 -- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80ca7b0 msgr2=0x7f40b815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40b80ca7b0 0x7f40b815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 -- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80cc7e0 msgr2=0x7f40b813f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40be570640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80cc7e0 0x7f40b813f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40b80cc7e0 0x7f40b813f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 -- 172.21.15.170:0/3541580507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b815e5f0 con 0x7f40b806ffc0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bdd6f640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b806ffc0 0x7f40b815beb0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000c10 tx=0x7f40b40a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40ac070 con 0x7f40b806ffc0 2024-09-21T02:22:54.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40b4004030 con 0x7f40b806ffc0 2024-09-21T02:22:54.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004400 con 0x7f40b806ffc0 2024-09-21T02:22:54.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b815e8d0 con 0x7f40b806ffc0 2024-09-21T02:22:54.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81513e0 con 0x7f40b806ffc0 2024-09-21T02:22:54.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.282+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f40b815e380 con 0x7f40b806ffc0 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f40b40af070 con 0x7f40b806ffc0 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40ae7fc640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f409c073e00 0x7f409c076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40bd56e640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f409c073e00 0x7f409c076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f40b412ce20 con 0x7f40b806ffc0 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40ae7fc640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f409c079d10 0x7f409c07c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 --> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40b4003e50 con 0x7f409c079d10 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=169) ==== 24+0+0 (secure 0 0 0) 0x7f40b412f790 con 0x7f40b806ffc0 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40bd56e640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f409c073e00 0x7f409c076310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f40a80058d0 tx=0x7f40a8005860 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:54.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40be570640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f409c079d10 0x7f409c07c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.286+0000 7f40be570640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f409c079d10 0x7f409c07c180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:54.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.290+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== osd.7 v2:172.21.15.191:6810/1521218345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40b4003e50 con 0x7f409c079d10 2024-09-21T02:22:54.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 --> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f40b806fb80 con 0x7f409c079d10 2024-09-21T02:22:54.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40ae7fc640 1 -- 172.21.15.170:0/3541580507 <== osd.7 v2:172.21.15.191:6810/1521218345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f40b806fb80 con 0x7f409c079d10 2024-09-21T02:22:54.304 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T02:22:54.304 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T02:22:54.304 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T02:22:54.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f409c079d10 msgr2=0x7f409c07c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.191:6810/1521218345,v1:172.21.15.191:6811/1521218345] conn(0x7f409c079d10 0x7f409c07c180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f409c073e00 msgr2=0x7f409c076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f409c073e00 0x7f409c076310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f40a80058d0 tx=0x7f40a8005860 comp rx=0 tx=0).stop 2024-09-21T02:22:54.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b806ffc0 msgr2=0x7f40b815beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 --2- 172.21.15.170:0/3541580507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b806ffc0 0x7f40b815beb0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000c10 tx=0x7f40b40a6ac0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40be570640 1 -- 172.21.15.170:0/3541580507 reap_dead start 2024-09-21T02:22:54.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 shutdown_connections 2024-09-21T02:22:54.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 >> 172.21.15.170:0/3541580507 conn(0x7f40b80b1c40 msgr2=0x7f40b80afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:54.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 shutdown_connections 2024-09-21T02:22:54.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.302+0000 7f40bffd3640 1 -- 172.21.15.170:0/3541580507 wait complete. 2024-09-21T02:22:54.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T02:22:54.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- 172.21.15.170:0/3125487831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 msgr2=0x7fd0d0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/3125487831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0d0004c20 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0016f0 tx=0x7fd0dc01c7b0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0ed574640 1 -- 172.21.15.170:0/3125487831 reap_dead start 2024-09-21T02:22:54.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- 172.21.15.170:0/3125487831 shutdown_connections 2024-09-21T02:22:54.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- 172.21.15.170:0/3125487831 >> 172.21.15.170:0/3125487831 conn(0x7fd0e80b1c40 msgr2=0x7fd0e80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:54.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- 172.21.15.170:0/3125487831 shutdown_connections 2024-09-21T02:22:54.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- 172.21.15.170:0/3125487831 wait complete. 2024-09-21T02:22:54.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 Processor -- start 2024-09-21T02:22:54.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- start start 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0e814b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0e809ab10 0x7fd0e809af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0e80c4b70 0x7fd0e80c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e80cf8e0 con 0x7fd0d0004810 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e80cf760 con 0x7fd0e809ab10 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0eefd7640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e80cd210 con 0x7fd0e80c4b70 2024-09-21T02:22:54.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.462+0000 7fd0ecd73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0e814b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0e814b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40300/0 (socket says 172.21.15.170:40300) 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 -- 172.21.15.170:0/483894054 learned_addr learned my addr 172.21.15.170:0/483894054 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ed574640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0e80c4b70 0x7fd0e80c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e7fff640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0e809ab10 0x7fd0e809af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 -- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0e80c4b70 msgr2=0x7fd0e80c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0e80c4b70 0x7fd0e80c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 -- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0e809ab10 msgr2=0x7fd0e809af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0e809ab10 0x7fd0e809af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 -- 172.21.15.170:0/483894054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e80c54c0 con 0x7fd0d0004810 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ed574640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0e80c4b70 0x7fd0e80c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:54.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e7fff640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0e809ab10 0x7fd0e809af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:54.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0ecd73640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0e814b580 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800f550 tx=0x7fd0d800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:54.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8017070 con 0x7fd0d0004810 2024-09-21T02:22:54.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d8011990 con 0x7fd0d0004810 2024-09-21T02:22:54.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e809d6c0 con 0x7fd0d0004810 2024-09-21T02:22:54.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8011de0 con 0x7fd0d0004810 2024-09-21T02:22:54.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e809dc80 con 0x7fd0d0004810 2024-09-21T02:22:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e80b3350 con 0x7fd0d0004810 2024-09-21T02:22:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fd0d801a070 con 0x7fd0d0004810 2024-09-21T02:22:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e57fa640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd0d4073dd0 0x7fd0d40762e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e7fff640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd0d4073dd0 0x7fd0d40762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fd0d8095ed0 con 0x7fd0d0004810 2024-09-21T02:22:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.466+0000 7fd0e7fff640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd0d4073dd0 0x7fd0d40762e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc006040 tx=0x7fd0dc0195a0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:54.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.474+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fd0e80c54c0 con 0x7fd0d0004810 2024-09-21T02:22:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.630+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fd0e80b2170 con 0x7fd0d4073dd0 2024-09-21T02:22:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.630+0000 7fd0e57fa640 1 -- 172.21.15.170:0/483894054 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd0e80b2170 con 0x7fd0d4073dd0 2024-09-21T02:22:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T02:22:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd0d4073dd0 msgr2=0x7fd0d40762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd0d4073dd0 0x7fd0d40762e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc006040 tx=0x7fd0dc0195a0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 msgr2=0x7fd0e814b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0e814b580 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800f550 tx=0x7fd0d800fa10 comp rx=0 tx=0).stop 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 shutdown_connections 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fd0d4073dd0 0x7fd0d40762e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd0e80c4b70 0x7fd0e80c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0e809ab10 0x7fd0e809af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 --2- 172.21.15.170:0/483894054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0d0004810 0x7fd0e814b580 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 >> 172.21.15.170:0/483894054 conn(0x7fd0e80b1c40 msgr2=0x7fd0e80c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 shutdown_connections 2024-09-21T02:22:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.634+0000 7fd0eefd7640 1 -- 172.21.15.170:0/483894054 wait complete. 2024-09-21T02:22:54.655 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T02:22:54.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 -- 172.21.15.170:0/3574876466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc17c005360 msgr2=0x7fc17c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 --2- 172.21.15.170:0/3574876466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc17c005360 0x7fc17c00b9d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fc1900cbfa0 tx=0x7fc18001ce40 comp rx=0 tx=0).stop 2024-09-21T02:22:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 -- 172.21.15.170:0/3574876466 shutdown_connections 2024-09-21T02:22:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 --2- 172.21.15.170:0/3574876466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc17c005360 0x7fc17c00b9d0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 -- 172.21.15.170:0/3574876466 >> 172.21.15.170:0/3574876466 conn(0x7fc1900b1c40 msgr2=0x7fc1900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 -- 172.21.15.170:0/3574876466 shutdown_connections 2024-09-21T02:22:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.802+0000 7fc194edf640 1 -- 172.21.15.170:0/3574876466 wait complete. 2024-09-21T02:22:55.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 Processor -- start 2024-09-21T02:22:55.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 -- start start 2024-09-21T02:22:55.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc17c005360 0x7fc19015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 0x7fc19015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc190152190 0x7fc1900b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900cbc00 con 0x7fc1900c3740 2024-09-21T02:22:55.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900cba80 con 0x7fc190152190 2024-09-21T02:22:55.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900cbd80 con 0x7fc17c005360 2024-09-21T02:22:55.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 0x7fc19015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 0x7fc19015c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40324/0 (socket says 172.21.15.170:40324) 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 -- 172.21.15.170:0/3409588148 learned_addr learned my addr 172.21.15.170:0/3409588148 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18e59c640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc17c005360 0x7fc19015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18ed9d640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc190152190 0x7fc1900b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 -- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc17c005360 msgr2=0x7fc19015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc17c005360 0x7fc19015bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 -- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc190152190 msgr2=0x7fc1900b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc190152190 0x7fc1900b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 -- 172.21.15.170:0/3409588148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1900b53d0 con 0x7fc1900c3740 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc18dd9b640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 0x7fc19015c500 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc17800e590 tx=0x7fc17800ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178015020 con 0x7fc1900c3740 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc178010980 con 0x7fc1900c3740 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19015ee40 con 0x7fc1900c3740 2024-09-21T02:22:55.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178010dd0 con 0x7fc1900c3740 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.806+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901424b0 con 0x7fc1900c3740 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.810+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190070550 con 0x7fc1900c3740 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.810+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fc178018020 con 0x7fc1900c3740 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.810+0000 7fc186ffd640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc170073e00 0x7fc170076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.810+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fc178093500 con 0x7fc1900c3740 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.810+0000 7fc18e59c640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc170073e00 0x7fc170076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.810+0000 7fc18e59c640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc170073e00 0x7fc170076310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc188006fd0 tx=0x7fc188006d80 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.814+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fc17801f020 con 0x7fc1900c3740 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.986+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 --> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc190158ae0 con 0x7fc170073e00 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.986+0000 7fc186ffd640 1 -- 172.21.15.170:0/3409588148 <== mgr.4122 v2:172.21.15.170:6836/796667214 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13373 (secure 0 0 0) 0x7fc190158ae0 con 0x7fc170073e00 2024-09-21T02:22:55.035 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc170073e00 msgr2=0x7fc170076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc170073e00 0x7fc170076310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc188006fd0 tx=0x7fc188006d80 comp rx=0 tx=0).stop 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 msgr2=0x7fc19015c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 0x7fc19015c500 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc17800e590 tx=0x7fc17800ea50 comp rx=0 tx=0).stop 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 shutdown_connections 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fc170073e00 0x7fc170076310 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc190152190 0x7fc1900b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1900c3740 0x7fc19015c500 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 --2- 172.21.15.170:0/3409588148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc17c005360 0x7fc19015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 >> 172.21.15.170:0/3409588148 conn(0x7fc1900b1c40 msgr2=0x7fc1900ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:55.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 shutdown_connections 2024-09-21T02:22:55.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:54.990+0000 7fc194edf640 1 -- 172.21.15.170:0/3409588148 wait complete. 2024-09-21T02:22:55.038 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-21T02:22:54.952688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":304792,"kb_used_data":17832,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754669928,"statfs":{"total":773094113280,"available":772782006272,"internally_reserved":0,"allocated":18259968,"data_stored":8714704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":293654065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-4,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.851263"},"pg_stats":[{"pgid":"1.0","version":"169'81","reported_seq":417,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-21T02:22:51.805032+0000","last_change":"2024-09-21T02:22:51.805032+0000","last_active":"2024-09-21T02:22:51.805032+0000","last_peered":"2024-09-21T02:22:51.805032+0000","last_clean":"2024-09-21T02:22:51.805032+0000","last_became_active":"2024-09-21T02:13:16.551101+0000","last_became_peered":"2024-09-21T02:13:16.551101+0000","last_unstale":"2024-09-21T02:22:51.805032+0000","last_undegraded":"2024-09-21T02:22:51.805032+0000","last_fullsized":"2024-09-21T02:22:51.805032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"169'81","last_scrub_stamp":"2024-09-21T02:22:51.804928+0000","last_deep_scrub":"169'81","last_deep_scrub_stamp":"2024-09-21T02:22:51.804928+0000","last_clean_scrub_stamp":"2024-09-21T02:22:51.804928+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:47.177600+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":941746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33276,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2134016,"data_stored":941746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33848,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337992,"statfs":{"total":96636764160,"available":96602103808,"internally_reserved":0,"allocated":2727936,"data_stored":1532114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37436,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334404,"statfs":{"total":96636764160,"available":96598429696,"internally_reserved":0,"allocated":2134016,"data_stored":941746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41596,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330244,"statfs":{"total":96636764160,"available":96594169856,"internally_reserved":0,"allocated":2134016,"data_stored":941746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41596,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330244,"statfs":{"total":96636764160,"available":96594169856,"internally_reserved":0,"allocated":2134016,"data_stored":941746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42168,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329672,"statfs":{"total":96636764160,"available":96593584128,"internally_reserved":0,"allocated":2727936,"data_stored":1532114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41596,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330244,"statfs":{"total":96636764160,"available":96594169856,"internally_reserved":0,"allocated":2134016,"data_stored":941746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:22:55.040 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T02:22:55.188 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:22:55.191 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:22:55.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3832496541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe014004d50 msgr2=0x7fe014005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3832496541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe014004d50 0x7fe014005160 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fe0280b61d0 tx=0x7fe02001d2c0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02d779640 1 -- 172.21.15.170:0/3832496541 reap_dead start 2024-09-21T02:22:55.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3832496541 shutdown_connections 2024-09-21T02:22:55.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3832496541 >> 172.21.15.170:0/3832496541 conn(0x7fe0280d0170 msgr2=0x7fe0280d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:55.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3832496541 shutdown_connections 2024-09-21T02:22:55.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.194+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3832496541 wait complete. 2024-09-21T02:22:55.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 Processor -- start 2024-09-21T02:22:55.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 -- start start 2024-09-21T02:22:55.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 0x7fe0281483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02814acb0 0x7fe02814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe028149e60 0x7fe02814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0280ceb90 con 0x7fe02809b470 2024-09-21T02:22:55.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0280b5bb0 con 0x7fe028149e60 2024-09-21T02:22:55.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7fe0280ced10 con 0x7fe02814acb0 2024-09-21T02:22:55.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 0x7fe0281483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 0x7fe0281483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40336/0 (socket says 172.21.15.170:40336) 2024-09-21T02:22:55.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 -- 172.21.15.170:0/3021164630 learned_addr learned my addr 172.21.15.170:0/3021164630 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:55.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe027fff640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02814acb0 0x7fe02814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02d779640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe028149e60 0x7fe02814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 -- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02814acb0 msgr2=0x7fe02814b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02814acb0 0x7fe02814b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 -- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe028149e60 msgr2=0x7fe02814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe028149e60 0x7fe02814a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 -- 172.21.15.170:0/3021164630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02814b600 con 0x7fe02809b470 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe027fff640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02814acb0 0x7fe02814b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02cf78640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 0x7fe0281483c0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fe010011a50 tx=0x7fe010011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01001f070 con 0x7fe02809b470 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe01000fe80 con 0x7fe02809b470 2024-09-21T02:22:55.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe028148900 con 0x7fe02809b470 2024-09-21T02:22:55.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010012690 con 0x7fe02809b470 2024-09-21T02:22:55.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.198+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281490a0 con 0x7fe02809b470 2024-09-21T02:22:55.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.202+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe028073c90 con 0x7fe02809b470 2024-09-21T02:22:55.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.202+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7fe010022070 con 0x7fe02809b470 2024-09-21T02:22:55.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.202+0000 7fe0257fa640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe008073e00 0x7fe008076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.202+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7fe0100938d0 con 0x7fe02809b470 2024-09-21T02:22:55.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.202+0000 7fe027fff640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe008073e00 0x7fe008076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.202+0000 7fe027fff640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe008073e00 0x7fe008076310 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe0280af5a0 tx=0x7fe0180091e0 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:55.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.206+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7fe02814b600 con 0x7fe02809b470 2024-09-21T02:22:55.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.366+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fe02815ee40 con 0x7fe02809b470 2024-09-21T02:22:55.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.374+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v54)=0 v54) ==== 123+0+0 (secure 0 0 0) 0x7fe01005fb80 con 0x7fe02809b470 2024-09-21T02:22:55.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.374+0000 7fe0257fa640 1 -- 172.21.15.170:0/3021164630 <== mon.0 v2:172.21.15.170:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fe01005f690 con 0x7fe02809b470 2024-09-21T02:22:55.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe008073e00 msgr2=0x7fe008076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe008073e00 0x7fe008076310 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe0280af5a0 tx=0x7fe0180091e0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 msgr2=0x7fe0281483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 0x7fe0281483c0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fe010011a50 tx=0x7fe010011f10 comp rx=0 tx=0).stop 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 shutdown_connections 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7fe008073e00 0x7fe008076310 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02814acb0 0x7fe02814b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe028149e60 0x7fe02814a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 --2- 172.21.15.170:0/3021164630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02809b470 0x7fe0281483c0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 >> 172.21.15.170:0/3021164630 conn(0x7fe0280d0170 msgr2=0x7fe0280b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:55.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 shutdown_connections 2024-09-21T02:22:55.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.378+0000 7fe02f1dc640 1 -- 172.21.15.170:0/3021164630 wait complete. 2024-09-21T02:22:55.400 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T02:22:55.400 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T02:22:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 -- 172.21.15.170:0/1332013782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 msgr2=0x7f54700ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 --2- 172.21.15.170:0/1332013782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54700ce4c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f5460009900 tx=0x7f546001c8e0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 -- 172.21.15.170:0/1332013782 shutdown_connections 2024-09-21T02:22:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 --2- 172.21.15.170:0/1332013782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54700ce4c0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 --2- 172.21.15.170:0/1332013782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547015f990 0x7f5470070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 -- 172.21.15.170:0/1332013782 >> 172.21.15.170:0/1332013782 conn(0x7f54700b1c40 msgr2=0x7f54700b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 -- 172.21.15.170:0/1332013782 shutdown_connections 2024-09-21T02:22:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.534+0000 7f54778e0640 1 -- 172.21.15.170:0/1332013782 wait complete. 2024-09-21T02:22:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 Processor -- start 2024-09-21T02:22:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- start start 2024-09-21T02:22:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54701fa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54700cea00 0x7f54700c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547015f990 0x7f547015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5470166160 con 0x7f5470070570 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5470165fe0 con 0x7f54700cea00 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 -- 0x7f54701662e0 con 0x7f547015f990 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54701fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54701fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36282/0 (socket says 172.21.15.170:36282) 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 -- 172.21.15.170:0/2969331691 learned_addr learned my addr 172.21.15.170:0/2969331691 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f5474e7b640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54700cea00 0x7f54700c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f5475e7d640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547015f990 0x7f547015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 -- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547015f990 msgr2=0x7f547015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547015f990 0x7f547015e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 -- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54700cea00 msgr2=0x7f54700c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54700cea00 0x7f54700c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 -- 172.21.15.170:0/2969331691 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54701faf20 con 0x7f5470070570 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f5474e7b640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54700cea00 0x7f54700c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f547567c640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54701fa760 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f546c00c940 tx=0x7f546c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c004220 con 0x7f5470070570 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f546c0079b0 con 0x7f5470070570 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54701fb200 con 0x7f5470070570 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c007e00 con 0x7f5470070570 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547009abd0 con 0x7f5470070570 2024-09-21T02:22:55.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.538+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547009a9c0 con 0x7f5470070570 2024-09-21T02:22:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.542+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 29) ==== 98305+0+0 (secure 0 0 0) 0x7f546c005020 con 0x7f5470070570 2024-09-21T02:22:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.542+0000 7f5465ffb640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5454073e00 0x7f5454076310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:22:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.542+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4971+0+0 (secure 0 0 0) 0x7f546c093120 con 0x7f5470070570 2024-09-21T02:22:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.542+0000 7f5474e7b640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5454073e00 0x7f5454076310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:22:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.542+0000 7f5474e7b640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5454073e00 0x7f5454076310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f54600af000 tx=0x7f5460019680 comp rx=0 tx=0).ready entity=mgr.4122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:22:55.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.546+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+189041 (secure 0 0 0) 0x7f546c014030 con 0x7f5470070570 2024-09-21T02:22:55.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.738+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54700cd8a0 con 0x7f5470070570 2024-09-21T02:22:55.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.738+0000 7f5465ffb640 1 -- 172.21.15.170:0/2969331691 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+3928 (secure 0 0 0) 0x7f546c05fb90 con 0x7f5470070570 2024-09-21T02:22:55.742 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T02:22:55.743 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":123,"btime":"2024-09-21T02:22:43:881714+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6713,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3666073786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":3666073786},{"type":"v1","addr":"172.21.15.191:6835","nonce":3666073786}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":7267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6801/3405016841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3405016841},{"type":"v1","addr":"172.21.15.191:6801","nonce":3405016841}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3091879017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3091879017},{"type":"v1","addr":"172.21.15.170:6833","nonce":3091879017}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114},{"gid":9612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1294158404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1294158404},{"type":"v1","addr":"172.21.15.170:6835","nonce":1294158404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[]} 2024-09-21T02:22:55.743 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 123 2024-09-21T02:22:55.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5454073e00 msgr2=0x7f5454076310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5454073e00 0x7f5454076310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f54600af000 tx=0x7f5460019680 comp rx=0 tx=0).stop 2024-09-21T02:22:55.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 msgr2=0x7f54701fa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:22:55.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54701fa760 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f546c00c940 tx=0x7f546c00ce00 comp rx=0 tx=0).stop 2024-09-21T02:22:55.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 shutdown_connections 2024-09-21T02:22:55.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:6836/796667214,v1:172.21.15.170:6837/796667214] conn(0x7f5454073e00 0x7f5454076310 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547015f990 0x7f547015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54700cea00 0x7f54700c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 --2- 172.21.15.170:0/2969331691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5470070570 0x7f54701fa760 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:22:55.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 >> 172.21.15.170:0/2969331691 conn(0x7f54700b1c40 msgr2=0x7f5470075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:22:55.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 shutdown_connections 2024-09-21T02:22:55.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T02:22:55.742+0000 7f54778e0640 1 -- 172.21.15.170:0/2969331691 wait complete. 2024-09-21T02:22:55.763 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T02:22:55.764 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T02:22:55.764 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T02:22:55.764 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:55.777 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T02:22:55.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T02:22:55.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:55.831 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T02:22:55.832 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T02:22:55.832 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:55.848 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T02:22:55.848 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T02:22:55.848 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:55.909 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T02:22:55.910 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T02:22:55.910 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T02:22:55.910 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:55.999 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T02:22:55.999 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T02:22:55.999 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.089 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T02:22:56.089 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T02:22:56.089 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.159 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T02:22:56.159 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T02:22:56.159 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.250 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T02:22:56.250 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T02:22:56.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.302 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T02:22:56.302 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T02:22:56.302 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.375 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T02:22:56.375 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T02:22:56.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.469 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T02:22:56.469 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T02:22:56.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.557 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T02:22:56.557 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T02:22:56.558 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T02:22:56.558 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.595 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T02:22:56.595 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T02:22:56.595 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.626 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T02:22:56.626 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T02:22:56.627 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T02:22:56.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.656 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T02:22:56.656 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T02:22:56.656 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.684 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T02:22:56.684 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T02:22:56.684 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T02:22:56.752 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T02:22:56.752 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T02:22:56.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-21T02:22:56.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:22:56.774 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T02:22:57.224 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:22:57.224 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T02:22:57.462 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:22:57.463 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T02:22:57.655 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:22:57.655 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T02:22:57.829 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:22:57.830 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T02:22:58.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:22:58.073 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T02:22:58.307 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:22:58.308 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T02:22:58.505 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:22:58.506 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T02:22:58.700 INFO:tasks.ceph:Archiving mon data... 2024-09-21T02:22:58.701 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/data/mon.a.tgz 2024-09-21T02:22:58.701 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-09-21T02:22:58.707 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.FLH0TLHcZA 2024-09-21T02:22:58.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FLH0TLHcZA 2024-09-21T02:22:58.977 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.FLH0TLHcZA 2024-09-21T02:22:59.039 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.FLH0TLHcZA is 1MB 2024-09-21T02:22:59.109 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.FLH0TLHcZA 2024-09-21T02:22:59.115 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/data/mon.b.tgz 2024-09-21T02:22:59.115 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-09-21T02:22:59.122 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.uZpbVjoN91 2024-09-21T02:22:59.122 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.uZpbVjoN91 2024-09-21T02:22:59.390 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.uZpbVjoN91 2024-09-21T02:22:59.457 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.uZpbVjoN91 is 1MB 2024-09-21T02:22:59.523 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.uZpbVjoN91 2024-09-21T02:22:59.528 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/data/mon.c.tgz 2024-09-21T02:22:59.529 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-09-21T02:22:59.576 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.kbqiQSrCqa 2024-09-21T02:22:59.577 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.kbqiQSrCqa 2024-09-21T02:22:59.854 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.kbqiQSrCqa 2024-09-21T02:22:59.920 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.kbqiQSrCqa is 1MB 2024-09-21T02:23:00.002 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.kbqiQSrCqa 2024-09-21T02:23:00.012 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T02:23:00.012 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T02:23:00.026 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T02:23:00.087 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T02:23:00.135 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T02:23:00.135 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T02:23:00.149 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/remote/smithi170/crash 2024-09-21T02:23:00.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T02:23:00.206 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/remote/smithi191/crash 2024-09-21T02:23:00.213 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T02:23:00.230 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T02:23:00.230 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:23:00.266 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T02:23:00.314 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T02:23:00.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T02:23:00.388 INFO:tasks.ceph:Compressing logs... 2024-09-21T02:23:00.389 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:23:00.420 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T02:23:00.449 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34584.log 2024-09-21T02:23:00.449 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T02:23:00.450 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.34584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29032.log 2024-09-21T02:23:00.451 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28953.log 2024-09-21T02:23:00.451 INFO:teuthology.orchestra.run.smithi191.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.1.34584.log.gz 2024-09-21T02:23:00.451 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T02:23:00.451 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.29032.log: /var/log/ceph/ceph-client.admin.28953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34995.log 2024-09-21T02:23:00.452 INFO:teuthology.orchestra.run.smithi191.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29032.log.gz 2024-09-21T02:23:00.452 INFO:teuthology.orchestra.run.smithi191.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28953.log.gz 2024-09-21T02:23:00.452 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35714.log 2024-09-21T02:23:00.452 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34795.log 2024-09-21T02:23:00.452 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.34995.log: /var/log/ceph/ceph-client.1.35714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35541.log 2024-09-21T02:23:00.453 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.34795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28977.log 2024-09-21T02:23:00.453 INFO:teuthology.orchestra.run.smithi191.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.1.34995.log.gz 2024-09-21T02:23:00.453 INFO:teuthology.orchestra.run.smithi191.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.1.35714.log.gz 2024-09-21T02:23:00.454 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.35541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29128.log 2024-09-21T02:23:00.454 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.28977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34318.log 2024-09-21T02:23:00.454 INFO:teuthology.orchestra.run.smithi191.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.1.34795.log.gz 2024-09-21T02:23:00.454 INFO:teuthology.orchestra.run.smithi191.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28977.log.gz 2024-09-21T02:23:00.455 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.29128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35196.log 2024-09-21T02:23:00.456 INFO:teuthology.orchestra.run.smithi191.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.1.35541.log.gz 2024-09-21T02:23:00.456 INFO:teuthology.orchestra.run.smithi191.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29128.log.gz 2024-09-21T02:23:00.456 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.34318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29001.log 2024-09-21T02:23:00.456 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.35196.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T02:23:00.457 INFO:teuthology.orchestra.run.smithi191.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.1.34318.log.gz/var/log/ceph/ceph-client.admin.29001.log: 2024-09-21T02:23:00.457 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35375.log 2024-09-21T02:23:00.457 INFO:teuthology.orchestra.run.smithi191.stderr: 86.8% 92.1% -- replaced with /var/log/ceph/ceph-client.admin.29001.log.gz -- replaced with /var/log/ceph/ceph-client.1.35196.log.gz 2024-09-21T02:23:00.457 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-09-21T02:23:00.457 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33951.log 2024-09-21T02:23:00.458 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.35375.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T02:23:00.458 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.3.33951.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T02:23:00.458 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40983.log 2024-09-21T02:23:00.458 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44537.log 2024-09-21T02:23:00.459 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51884.log 2024-09-21T02:23:00.460 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40983.log: /var/log/ceph/ceph-client.admin.44537.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40983.log.gz 2024-09-21T02:23:00.460 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-09-21T02:23:00.460 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44537.log.gz 2024-09-21T02:23:00.460 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-09-21T02:23:00.461 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51884.log: /var/log/ceph/ceph-client.admin.42901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40905.log 2024-09-21T02:23:00.463 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47745.log: gzip 87.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51884.log.gz 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49563.log 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44142.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40905.log.gz 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-09-21T02:23:00.464 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.49563.log.gz 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47399.log 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34385.log 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54880.log 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34385.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47399.log.gz 2024-09-21T02:23:00.465 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52808.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34385.log.gz 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38764.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54880.log.gz 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44194.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52808.log.gz 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38764.log.gz 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38097.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44194.log.gz 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48527.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34610.log 2024-09-21T02:23:00.475 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40034.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38097.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48527.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45011.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34610.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43375.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40034.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40301.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45011.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49049.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40957.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43375.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40301.log.gz 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42869.log 2024-09-21T02:23:00.476 INFO:teuthology.orchestra.run.smithi170.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49049.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35924.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40957.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42869.log: gzip -5/var/log/ceph/ceph-client.admin.35924.log: --verbose -- /var/log/ceph/ceph-client.admin.34402.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42869.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48421.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35924.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45117.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34402.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45117.log: /var/log/ceph/ceph-client.admin.53628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53941.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45117.log.gz 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose --/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.37648.log 2024-09-21T02:23:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53941.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38920.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53941.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40272.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38920.log: /var/log/ceph/ceph-client.admin.40272.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38920.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47547.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40272.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46968.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47547.log: /var/log/ceph/ceph-client.admin.37518.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38410.log 2024-09-21T02:23:00.478 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-09-21T02:23:00.479 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.3.33951.log.gz 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.1.35375.log.gz 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29080.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35868.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.29080.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29080.log.gz 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.35868.log: 88.2% -- replaced with /var/log/ceph/ceph-client.1.35868.log.gz 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29104.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.29104.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29104.log.gz 2024-09-21T02:23:00.480 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33832.log 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29056.log 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.33832.log: 89.0% -- replaced with /var/log/ceph/ceph-client.1.33832.log.gz 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.29056.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29056.log.gz 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mds.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T02:23:00.481 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34110.log 2024-09-21T02:23:00.493 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46968.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38410.log: /var/log/ceph/ceph-client.admin.45917.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38410.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40112.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48079.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52046.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48079.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52046.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40112.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54244.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52046.log.gz 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44773.log 2024-09-21T02:23:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43589.log: 87.1%/var/log/ceph/ceph-client.admin.54244.log: -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41328.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54244.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34790.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44773.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52390.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41328.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34790.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42221.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52390.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34348.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46349.log: /var/log/ceph/ceph-client.admin.42221.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-09-21T02:23:00.495 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42221.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52834.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34348.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28971.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%/var/log/ceph/ceph-client.admin.52834.log: -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39244.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52834.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35599.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.28971.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39739.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39244.log: /var/log/ceph/ceph-client.admin.35599.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39244.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48735.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35599.log.gz 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38334.log 2024-09-21T02:23:00.496 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39739.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51584.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38334.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48735.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36605.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 87.8% 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51584.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36605.log: /var/log/ceph/ceph-osd.0.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46730.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36605.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gz 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49124.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40524.log 2024-09-21T02:23:00.497 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49124.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35794.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49124.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39053.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40524.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46890.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35794.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39053.log: /var/log/ceph/ceph-client.admin.46890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39616.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39053.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44220.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.46890.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36841.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39452.log 2024-09-21T02:23:00.498 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6%/var/log/ceph/ceph-client.admin.36841.log: -- replaced with /var/log/ceph/ceph-client.admin.39616.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39452.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36841.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39956.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44427.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39956.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29516.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50610.log: /var/log/ceph/ceph-client.admin.44427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37421.log 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44427.log.gz 2024-09-21T02:23:00.499 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29516.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46401.log 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29516.log.gz 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34270.log 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37421.log: /var/log/ceph/ceph-client.admin.46401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34972.log 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37421.log.gz 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46401.log.gz 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51558.log 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37937.log 2024-09-21T02:23:00.500 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34270.log.gz 2024-09-21T02:23:00.502 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36023.log 2024-09-21T02:23:00.502 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.34110.log: 89.1% -- replaced with /var/log/ceph/ceph-client.1.34110.log.gz 2024-09-21T02:23:00.508 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T02:23:00.508 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.36023.log: 88.4% -- replaced with /var/log/ceph/ceph-client.1.36023.log.gz 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34972.log: /var/log/ceph/ceph-client.admin.51558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29131.log 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51558.log.gz 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34972.log.gz 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40060.log 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40748.log 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37937.log.gz 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29131.log.gz 2024-09-21T02:23:00.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43915.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40060.log.gz 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40748.log: /var/log/ceph/ceph-client.admin.43915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40748.log.gz 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43915.log.gz 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52152.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53299.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gz 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36496.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41654.log 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53299.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52152.log.gz 2024-09-21T02:23:00.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36496.log: gzip 87.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.53299.log.gz -- 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr: /var/log/ceph/ceph-client.admin.49966.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41654.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36496.log.gz 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36815.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41654.log.gz 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38201.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49966.log.gz 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36815.log.gz 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53362.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38201.log.gz 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52364.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2024-09-21T02:23:00.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53362.log: /var/log/ceph/ceph-client.admin.52364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53362.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49784.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52364.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34192.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34296.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34296.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34192.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41300.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34296.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.40644.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48709.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54270.log 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-09-21T02:23:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48709.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41300.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40550.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48709.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39926.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54270.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53414.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40550.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40617.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53414.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39926.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54594.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53414.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49324.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54594.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40617.log.gz 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53388.log 2024-09-21T02:23:00.533 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54594.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50345.log 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49324.log: 84.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.49324.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53388.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28910.log 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50345.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53388.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42083.log 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50345.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5/var/log/ceph/ceph-client.admin.28910.log: --verbose -- /var/log/ceph/ceph-client.admin.54516.log 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr: 79.1% -- replaced with /var/log/ceph/ceph-client.admin.28910.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53270.log 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54516.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48579.log 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54516.log.gz 2024-09-21T02:23:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38816.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53270.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46610.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38816.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48579.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38816.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37264.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46610.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37142.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38149.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37264.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33653.log 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38149.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37142.log.gz 2024-09-21T02:23:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34868.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38149.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38572.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33653.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49199.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34868.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34868.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49386.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38572.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37291.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49386.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50850.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49386.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37291.log.gz 2024-09-21T02:23:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50850.log.gz 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34946.log 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35488.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46375.log 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%/var/log/ceph/ceph-client.admin.45867.log: -- replaced with /var/log/ceph/ceph-client.admin.34946.log.gz 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37700.log 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46375.log.gz 2024-09-21T02:23:00.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44483.log 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35738.log 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37700.log: /var/log/ceph/ceph-client.admin.44483.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37700.log.gz 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45374.log 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44483.log.gz 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35738.log.gz 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29129.log 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45374.log: /var/log/ceph/ceph-client.admin.43641.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45374.log.gz 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40008.log 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2024-09-21T02:23:00.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44019.log 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29129.log.gz 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51428.log 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44019.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40008.log.gz 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44019.log.gz 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46453.log 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51428.log.gz 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5/var/log/ceph/ceph-client.admin.37317.log: --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46453.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42531.log 2024-09-21T02:23:00.539 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46453.log.gz 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40696.log 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41970.log/var/log/ceph/ceph-client.admin.42531.log: 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40696.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42531.log.gz -5 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49538.log 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40696.log.gz 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48921.log 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41970.log.gz 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35711.log 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.49538.log.gz 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39765.log 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48921.log.gz 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44849.log 2024-09-21T02:23:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35155.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35711.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39765.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.44849.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.35155.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44849.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54192.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37116.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.35155.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39374.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39479.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43110.log 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39374.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54192.log.gz 2024-09-21T02:23:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44116.log 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39479.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39374.log.gz 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34395.log 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39479.log.gz 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43110.log: /var/log/ceph/ceph-client.admin.44116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41838.log 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43110.log.gz 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49361.log 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44116.log.gz 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0%/var/log/ceph/ceph-client.0.41838.log: -- replaced with /var/log/ceph/ceph-client.admin.34395.log.gz 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38710.log 2024-09-21T02:23:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54750.log 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49361.log.gz 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54776.log 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.41838.log.gz 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46089.log 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%/var/log/ceph/ceph-client.admin.54750.log: -- replaced with /var/log/ceph/ceph-client.admin.38710.log.gz 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45221.log 2024-09-21T02:23:00.561 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54750.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46089.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54776.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42195.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53524.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42195.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45221.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48605.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42195.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53524.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39166.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28848.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48605.log: /var/log/ceph/ceph-client.admin.39166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48683.log 2024-09-21T02:23:00.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48605.log.gz 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39166.log.gz 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.28848.log.gz 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43665.log 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48683.log.gz 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39557.log 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46427.log 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.43665.log.gz --verbose 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr: -- /var/log/ceph/ceph-client.admin.52020.log 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39557.log: 88.0%/var/log/ceph/ceph-client.admin.46427.log: -- replaced with /var/log/ceph/ceph-client.admin.39557.log.gz 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T02:23:00.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52020.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46427.log.gz 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41782.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52020.log.gz 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53550.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54114.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41782.log: /var/log/ceph/ceph-client.admin.53550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41782.log.gz 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53550.log.gz 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38253.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54114.log: /var/log/ceph/ceph-client.admin.42706.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54114.log.gz 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40472.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38253.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51201.log 2024-09-21T02:23:00.564 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38253.log.gz 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29054.log 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.40472.log.gz 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34218.log 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51201.log.gz 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49513.log 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29054.log.gz 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44905.log 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34218.log.gz 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46486.log 2024-09-21T02:23:00.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.44905.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.49513.log.gz 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46010.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44246.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46486.log: /var/log/ceph/ceph-client.admin.46010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50883.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46486.log.gz 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46010.log.gz 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44246.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44246.log.gz 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47515.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43615.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50883.log.gz 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.47515.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43695.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43615.log: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41892.log 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47515.log.gz 2024-09-21T02:23:00.566 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43615.log.gz 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43695.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.43695.log.gz 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38282.log 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42758.log 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41892.log.gz 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50282.log 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50796.log 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38282.log.gz/var/log/ceph/ceph-client.admin.42758.log: 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50282.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42758.log.gz 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50282.log.gz 2024-09-21T02:23:00.567 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53734.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50796.log.gz/var/log/ceph/ceph-client.0.53734.log: 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41680.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53576.log: /var/log/ceph/ceph-client.admin.45943.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53244.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41680.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41680.log.gz 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36196.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54568.log 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53244.log: 87.8%/var/log/ceph/ceph-client.admin.36196.log: -- replaced with /var/log/ceph/ceph-client.admin.53244.log.gz 2024-09-21T02:23:00.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36275.log 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36196.log.gz 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54568.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54568.log.gz 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34036.log 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36275.log: /var/log/ceph/ceph-client.admin.40498.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36275.log.gz 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.53734.log.gz 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45979.log 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43018.log 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34036.log: /var/log/ceph/ceph-client.admin.45979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35300.log 2024-09-21T02:23:00.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43018.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34036.log.gz 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45979.log.gz 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46141.log 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43018.log.gz 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.2.35300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41550.log 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45247.log 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% 90.2% -- replaced with /var/log/ceph/ceph-client.admin.46141.log.gz 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41550.log: -- replaced with /var/log/ceph/ceph-client.2.35300.log.gz 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2024-09-21T02:23:00.570 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47824.log 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49862.log 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45247.log.gz/var/log/ceph/ceph-client.admin.47824.log: 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49810.log 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47824.log.gz 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49862.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49862.log.gz 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41602.log 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54367.log 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49810.log.gz/var/log/ceph/ceph-client.admin.41602.log: 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39531.log 2024-09-21T02:23:00.571 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41602.log.gz 2024-09-21T02:23:00.572 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54367.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54367.log.gz 2024-09-21T02:23:00.572 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40086.log 2024-09-21T02:23:00.572 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54056.log 2024-09-21T02:23:00.595 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2%/var/log/ceph/ceph-client.admin.40086.log: -- replaced with /var/log/ceph/ceph-client.admin.39531.log.gz 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51766.log 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40086.log.gz 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54056.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54056.log.gz 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50310.log 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51766.log.gz 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38652.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40853.log 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-09-21T02:23:00.596 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%/var/log/ceph/ceph-client.admin.40853.log: -- replaced with /var/log/ceph/ceph-client.admin.50310.log.gz 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52507.log 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51610.log 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40853.log.gz 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51506.log: /var/log/ceph/ceph-client.admin.52507.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52507.log.gz 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39081.log 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55036.log 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6%/var/log/ceph/ceph-client.admin.39081.log: -- replaced with /var/log/ceph/ceph-client.admin.51610.log.gz 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36522.log 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39081.log.gz 2024-09-21T02:23:00.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.55036.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55036.log.gz 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49274.log 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51992.log 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36522.log.gz/var/log/ceph/ceph-client.admin.49274.log: 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.49274.log.gz 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51992.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46323.log 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29136.log 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz/var/log/ceph/ceph-client.admin.46323.log: 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49758.log 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46323.log.gz 2024-09-21T02:23:00.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29136.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29136.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39426.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%/var/log/ceph/ceph-client.admin.39426.log: -- replaced with /var/log/ceph/ceph-client.admin.49758.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28959.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39426.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37908.log: /var/log/ceph/ceph-client.admin.46219.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50136.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51084.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.28959.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39000.log 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50136.log.gz 2024-09-21T02:23:00.599 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51331.log 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51084.log: /var/log/ceph/ceph-client.admin.39000.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51084.log.gz 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39000.log.gz 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34609.log 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.51331.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36407.log 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%/var/log/ceph/ceph-client.admin.34609.log: -- replaced with /var/log/ceph/ceph-client.admin.51331.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43271.log 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34609.log.gz 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36407.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36407.log.gz 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39982.log 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47652.log 2024-09-21T02:23:00.600 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%/var/log/ceph/ceph-client.admin.39982.log: -- replaced with /var/log/ceph/ceph-client.admin.43271.log.gz 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39982.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52100.log 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47652.log: /var/log/ceph/ceph-client.admin.47284.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47652.log.gz 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45703.log 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%/var/log/ceph/ceph-client.admin.45703.log: -- replaced with /var/log/ceph/ceph-client.admin.52100.log.gz 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50453.log 2024-09-21T02:23:00.601 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45703.log.gz 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34599.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45037.log 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41128.log 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%/var/log/ceph/ceph-client.admin.45037.log: -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40722.log 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38045.log 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45037.log.gz 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41128.log: /var/log/ceph/ceph-client.admin.40722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41128.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52202.log 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40722.log.gz 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38045.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38045.log.gz 2024-09-21T02:23:00.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52957.log 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43993.log 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8%/var/log/ceph/ceph-client.admin.52957.log: -- replaced with /var/log/ceph/ceph-client.admin.52202.log.gz 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35205.log 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43993.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43993.log.gz 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53153.log 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49024.log 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35205.log: /var/log/ceph/ceph-client.admin.53153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49024.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35205.log.gz 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53153.log.gz 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.49024.log.gz 2024-09-21T02:23:00.603 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47204.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37674.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42680.log: /var/log/ceph/ceph-client.0.47204.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24023.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37674.log.gz 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.tmp-client.admin.24023.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24023.log.gz 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52437.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37622.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46584.log: /var/log/ceph/ceph-client.admin.52437.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52437.log.gz 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54802.log 2024-09-21T02:23:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38625.log 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37622.log: /var/log/ceph/ceph-client.admin.54802.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37622.log.gz 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54802.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38625.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38625.log.gz 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40162.log 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48631.log: /var/log/ceph/ceph-client.admin.40162.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.40162.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.54698.log 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50018.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-09-21T02:23:00.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34894.log 2024-09-21T02:23:00.625 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54698.log: /var/log/ceph/ceph-client.admin.34894.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54698.log.gz 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34894.log.gz 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49436.log 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50532.log: /var/log/ceph/ceph-client.admin.49436.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.49436.log.gz 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51849.log 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43535.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28923.log 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35820.log 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51849.log: /var/log/ceph/ceph-client.admin.28923.log: 78.4% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51849.log.gz 2024-09-21T02:23:00.626 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28923.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48209.log 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35820.log: /var/log/ceph/ceph-client.admin.48209.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35820.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48209.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36891.log 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35350.log 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50397.log: /var/log/ceph/ceph-client.admin.36891.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36891.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.44905.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- 92.7% /var/log/ceph/ceph-client.admin.46063.log 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.0.47204.log.gz 2024-09-21T02:23:00.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35350.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47624.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35350.log.gz 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34140.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47624.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46063.log.gz 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.47624.log.gz 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55008.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44985.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34140.log.gz 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.55008.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50921.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55008.log.gz 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54828.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44985.log: /var/log/ceph/ceph-client.admin.50921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-09-21T02:23:00.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54828.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50921.log.gz 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44985.log.gz 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.54828.log.gz --verbose 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr: -- /var/log/ceph/ceph-client.admin.36631.log 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39900.log 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36631.log: /var/log/ceph/ceph-client.admin.35462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48973.log 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39900.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36631.log.gz 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39900.log.gz 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50824.log 2024-09-21T02:23:00.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48973.log: /var/log/ceph/ceph-client.admin.47928.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29604.log 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50824.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50824.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50505.log 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44799.log 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29604.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41219.log 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44799.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50505.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48785.log 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47798.log 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41219.log: /var/log/ceph/ceph-client.admin.48785.log: 91.2% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41219.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44799.log.gz 2024-09-21T02:23:00.630 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48785.log.gz 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53009.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47798.log.gz/var/log/ceph/ceph-client.admin.54672.log: 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40586.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53009.log: /var/log/ceph/ceph-client.admin.40586.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53009.log.gz 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40586.log.gz 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41472.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42165.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48366.log: /var/log/ceph/ceph-client.admin.41472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42165.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-21T02:23:00.631 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41472.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40353.log 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42165.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53444.log 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46693.log: /var/log/ceph/ceph-client.admin.40353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36470.log 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%/var/log/ceph/ceph-client.admin.53444.log: -- replaced with /var/log/ceph/ceph-client.admin.40353.log.gz 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53444.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53970.log 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36170.log 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36470.log: /var/log/ceph/ceph-client.admin.53970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50479.log 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36170.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36470.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53970.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36170.log.gz 2024-09-21T02:23:00.632 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40388.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44747.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50479.log: /var/log/ceph/ceph-client.admin.40388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46942.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44747.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50479.log.gz 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.40388.log.gz 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50044.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44747.log.gz 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46942.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46942.log.gz 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35683.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50044.log: /var/log/ceph/ceph-client.admin.49836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50044.log.gz 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35683.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54854.log 2024-09-21T02:23:00.633 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35683.log.gz 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40827.log 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45143.log 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40827.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54854.log.gz 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42057.log 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40827.log.gz 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40218.log 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45143.log: /var/log/ceph/ceph-client.admin.42057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43192.log 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.40218.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45143.log.gz 2024-09-21T02:23:00.634 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42057.log.gz 2024-09-21T02:23:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43719.log 2024-09-21T02:23:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41272.log 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43192.log: /var/log/ceph/ceph-client.admin.43719.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43192.log.gz 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.43719.log.gz 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38466.log 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41272.log: /var/log/ceph/ceph-client.admin.35540.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41272.log.gz 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36763.log 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37447.log 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.38466.log: /var/log/ceph/ceph-client.admin.36763.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36763.log.gz 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.0.38466.log.gz 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T02:23:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37447.log: /var/log/ceph/ceph-osd.1.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37447.log.gz 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45077.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38974.log 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37473.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.40218.log.gz 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49888.log 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53678.log 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38974.log: /var/log/ceph/ceph-client.admin.49888.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38974.log.gz 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49888.log.gz 2024-09-21T02:23:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51305.log 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53678.log: /var/log/ceph/ceph-client.admin.51305.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53678.log.gz 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51305.log.gz 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51005.log 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44090.log 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51454.log: /var/log/ceph/ceph-client.admin.51005.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51005.log.gz 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52338.log 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43837.log 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44090.log: /var/log/ceph/ceph-client.admin.52338.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44090.log.gz 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52338.log.gz 2024-09-21T02:23:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36947.log 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34583.log 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43837.log: /var/log/ceph/ceph-client.0.36947.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43837.log.gz 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35376.log 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34583.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34583.log.gz 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29135.log 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35376.log.gz 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49992.log 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29135.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.29135.log.gz 2024-09-21T02:23:00.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39296.log 2024-09-21T02:23:00.648 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.4.log: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T02:23:00.653 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49992.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49992.log.gz 2024-09-21T02:23:00.653 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54906.log 2024-09-21T02:23:00.653 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39296.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39296.log.gz 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37395.log 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54906.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54906.log.gz 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38684.log 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37395.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.36947.log.gz 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37395.log.gz 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38175.log 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45169.log 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38684.log: /var/log/ceph/ceph-client.admin.38175.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38684.log.gz 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38175.log.gz 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-09-21T02:23:00.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45169.log: /var/log/ceph/ceph-client.admin.51148.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45169.log.gz 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48392.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35976.log 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35566.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50253.log 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35976.log.gz/var/log/ceph/ceph-client.admin.43349.log: 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-09-21T02:23:00.655 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36435.log 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50253.log: /var/log/ceph/ceph-client.admin.41576.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50253.log.gz 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38842.log 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36435.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36435.log.gz 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54542.log 2024-09-21T02:23:00.658 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38842.log: /var/log/ceph/ceph-client.admin.37171.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38842.log.gz 2024-09-21T02:23:00.659 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2024-09-21T02:23:00.659 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-09-21T02:23:00.659 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54542.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54542.log.gz 2024-09-21T02:23:00.659 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-21T02:23:00.660 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49627.log 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43889.log: /var/log/ceph/ceph-client.admin.36222.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39848.log 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43425.log 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49627.log: /var/log/ceph/ceph-client.admin.39848.log: 87.5% 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49627.log.gz 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39848.log.gz 2024-09-21T02:23:00.661 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46115.log 2024-09-21T02:23:00.663 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53087.log 2024-09-21T02:23:00.663 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43425.log: /var/log/ceph/ceph-client.admin.46115.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43425.log.gz 2024-09-21T02:23:00.663 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46115.log.gz 2024-09-21T02:23:00.663 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49224.log 2024-09-21T02:23:00.663 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51279.log 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53087.log: /var/log/ceph/ceph-client.admin.49224.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53087.log.gz 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49224.log.gz 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41096.log 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29071.log 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51279.log: /var/log/ceph/ceph-client.admin.41096.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51279.log.gz 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-09-21T02:23:00.665 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41096.log.gz 2024-09-21T02:23:00.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29071.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29071.log.gz 2024-09-21T02:23:00.667 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47336.log 2024-09-21T02:23:00.667 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2024-09-21T02:23:00.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37596.log: /var/log/ceph/ceph-client.admin.47336.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-09-21T02:23:00.667 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47336.log.gz 2024-09-21T02:23:00.668 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2024-09-21T02:23:00.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46193.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gz 2024-09-21T02:23:00.669 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53327.log 2024-09-21T02:23:00.669 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39400.log 2024-09-21T02:23:00.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47310.log: /var/log/ceph/ceph-client.admin.53327.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz 2024-09-21T02:23:00.670 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53327.log.gz 2024-09-21T02:23:00.670 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-09-21T02:23:00.671 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47954.log 2024-09-21T02:23:00.671 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39400.log: /var/log/ceph/ceph-client.admin.44563.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39400.log.gz 2024-09-21T02:23:00.671 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-09-21T02:23:00.671 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29133.log 2024-09-21T02:23:00.672 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47678.log 2024-09-21T02:23:00.673 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47954.log: /var/log/ceph/ceph-client.admin.29133.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47954.log.gz 2024-09-21T02:23:00.673 INFO:teuthology.orchestra.run.smithi170.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.29133.log.gz 2024-09-21T02:23:00.673 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42109.log 2024-09-21T02:23:00.673 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47678.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47678.log.gz 2024-09-21T02:23:00.673 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41997.log 2024-09-21T02:23:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51643.log 2024-09-21T02:23:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42109.log: /var/log/ceph/ceph-client.admin.41997.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42109.log.gz 2024-09-21T02:23:00.675 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48183.log 2024-09-21T02:23:00.675 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41997.log.gz 2024-09-21T02:23:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51643.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51643.log.gz 2024-09-21T02:23:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47902.log 2024-09-21T02:23:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50976.log 2024-09-21T02:23:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48183.log: /var/log/ceph/ceph-client.admin.47902.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48183.log.gz 2024-09-21T02:23:00.677 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-09-21T02:23:00.677 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47902.log.gz 2024-09-21T02:23:00.677 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50976.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50976.log.gz 2024-09-21T02:23:00.678 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-09-21T02:23:00.678 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35027.log 2024-09-21T02:23:00.679 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54464.log: /var/log/ceph/ceph-client.admin.36073.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-09-21T02:23:00.680 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz --verbose 2024-09-21T02:23:00.680 INFO:teuthology.orchestra.run.smithi170.stderr: -- /var/log/ceph/ceph-client.admin.37570.log 2024-09-21T02:23:00.680 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35027.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35027.log.gz 2024-09-21T02:23:00.680 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43967.log 2024-09-21T02:23:00.681 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-09-21T02:23:00.681 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37570.log: /var/log/ceph/ceph-client.admin.43967.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37570.log.gz 2024-09-21T02:23:00.681 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43967.log.gz 2024-09-21T02:23:00.682 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38520.log 2024-09-21T02:23:00.682 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-09-21T02:23:00.682 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35514.log: /var/log/ceph/ceph-client.admin.38520.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-09-21T02:23:00.682 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38520.log.gz 2024-09-21T02:23:00.683 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53814.log 2024-09-21T02:23:00.683 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37882.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-09-21T02:23:00.683 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46167.log 2024-09-21T02:23:00.684 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53035.log 2024-09-21T02:23:00.684 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53814.log: /var/log/ceph/ceph-client.admin.46167.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53814.log.gz 2024-09-21T02:23:00.684 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46167.log.gz 2024-09-21T02:23:00.684 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42583.log 2024-09-21T02:23:00.685 INFO:teuthology.orchestra.run.smithi191.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T02:23:00.686 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52782.log 2024-09-21T02:23:00.686 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53035.log: /var/log/ceph/ceph-client.admin.42583.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53035.log.gz 2024-09-21T02:23:00.686 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-09-21T02:23:00.686 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42583.log.gz 2024-09-21T02:23:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52782.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52782.log.gz 2024-09-21T02:23:00.687 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50371.log 2024-09-21T02:23:00.688 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2024-09-21T02:23:00.688 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42390.log: /var/log/ceph/ceph-client.admin.50371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-09-21T02:23:00.688 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T02:23:00.688 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50371.log.gz 2024-09-21T02:23:00.689 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45676.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2024-09-21T02:23:00.689 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49174.log 2024-09-21T02:23:00.690 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36553.log 2024-09-21T02:23:00.690 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.49174.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.49174.log.gz 2024-09-21T02:23:00.692 INFO:teuthology.orchestra.run.smithi170.stderr: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T02:23:00.692 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51792.log 2024-09-21T02:23:00.693 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49488.log 2024-09-21T02:23:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36553.log: /var/log/ceph/ceph-client.admin.51792.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36553.log.gz 2024-09-21T02:23:00.694 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51792.log.gz 2024-09-21T02:23:00.694 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52678.log 2024-09-21T02:23:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52126.log 2024-09-21T02:23:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49488.log: /var/log/ceph/ceph-client.admin.52678.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.49488.log.gz 2024-09-21T02:23:00.695 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52678.log.gz 2024-09-21T02:23:00.695 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34244.log 2024-09-21T02:23:00.696 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52126.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52126.log.gz 2024-09-21T02:23:00.696 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52625.log 2024-09-21T02:23:00.696 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52730.log 2024-09-21T02:23:00.697 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34244.log: /var/log/ceph/ceph-client.admin.52625.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34244.log.gz 2024-09-21T02:23:00.698 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52625.log.gz 2024-09-21T02:23:00.698 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49249.log 2024-09-21T02:23:00.698 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39107.log 2024-09-21T02:23:00.700 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52730.log: /var/log/ceph/ceph-client.admin.49249.log: 84.9% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52730.log.gz 2024-09-21T02:23:00.700 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49249.log.gz 2024-09-21T02:23:00.700 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39026.log 2024-09-21T02:23:00.700 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44669.log 2024-09-21T02:23:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39107.log: /var/log/ceph/ceph-client.admin.39026.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39107.log.gz 2024-09-21T02:23:00.701 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51055.log 2024-09-21T02:23:00.701 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39026.log.gz 2024-09-21T02:23:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44669.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44669.log.gz 2024-09-21T02:23:00.702 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44337.log 2024-09-21T02:23:00.702 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54393.log 2024-09-21T02:23:00.702 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51055.log: /var/log/ceph/ceph-client.admin.44337.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51055.log.gz 2024-09-21T02:23:00.702 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.44337.log.gz 2024-09-21T02:23:00.702 INFO:teuthology.orchestra.run.smithi170.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44695.log 2024-09-21T02:23:00.703 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54393.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54393.log.gz 2024-09-21T02:23:00.703 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-09-21T02:23:00.704 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39322.log 2024-09-21T02:23:00.704 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44695.log: /var/log/ceph/ceph-client.admin.34602.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-09-21T02:23:00.705 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-09-21T02:23:00.705 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-09-21T02:23:00.705 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39322.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39322.log.gz 2024-09-21T02:23:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44641.log 2024-09-21T02:23:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48657.log 2024-09-21T02:23:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41918.log: /var/log/ceph/ceph-client.admin.44641.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-09-21T02:23:00.707 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44641.log.gz 2024-09-21T02:23:00.707 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48131.log 2024-09-21T02:23:00.708 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40801.log 2024-09-21T02:23:00.708 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48657.log: /var/log/ceph/ceph-client.admin.48131.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48657.log.gz 2024-09-21T02:23:00.709 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48275.log 2024-09-21T02:23:00.709 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48131.log.gz 2024-09-21T02:23:00.709 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40801.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40801.log.gz 2024-09-21T02:23:00.709 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37079.log 2024-09-21T02:23:00.710 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39218.log 2024-09-21T02:23:00.711 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48275.log: /var/log/ceph/ceph-client.admin.37079.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48275.log.gz 2024-09-21T02:23:00.711 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37079.log.gz 2024-09-21T02:23:00.711 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-09-21T02:23:00.712 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39648.log 2024-09-21T02:23:00.712 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39218.log.gz 2024-09-21T02:23:00.713 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36327.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-09-21T02:23:00.713 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42468.log 2024-09-21T02:23:00.713 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51740.log 2024-09-21T02:23:00.713 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39648.log.gz/var/log/ceph/ceph-client.admin.42468.log: 2024-09-21T02:23:00.713 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42468.log.gz 2024-09-21T02:23:00.714 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-09-21T02:23:00.714 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-09-21T02:23:00.714 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51740.log: /var/log/ceph/ceph-client.admin.38868.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51740.log.gz 2024-09-21T02:23:00.715 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47072.log 2024-09-21T02:23:00.715 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.716 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35410.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-09-21T02:23:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43140.log 2024-09-21T02:23:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51910.log 2024-09-21T02:23:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47072.log: /var/log/ceph/ceph-client.admin.43140.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47072.log.gz 2024-09-21T02:23:00.717 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-09-21T02:23:00.718 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43140.log.gz 2024-09-21T02:23:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51910.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51910.log.gz 2024-09-21T02:23:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36709.log 2024-09-21T02:23:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38123.log 2024-09-21T02:23:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52475.log: /var/log/ceph/ceph-client.admin.36709.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-09-21T02:23:00.719 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36709.log.gz 2024-09-21T02:23:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37726.log 2024-09-21T02:23:00.721 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48841.log 2024-09-21T02:23:00.721 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38123.log: /var/log/ceph/ceph-client.admin.37726.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38123.log.gz 2024-09-21T02:23:00.721 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37726.log.gz 2024-09-21T02:23:00.721 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-09-21T02:23:00.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37343.log 2024-09-21T02:23:00.722 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.48841.log: /var/log/ceph/ceph-client.admin.52074.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-09-21T02:23:00.722 INFO:teuthology.orchestra.run.smithi170.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.0.48841.log.gz 2024-09-21T02:23:00.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45760.log 2024-09-21T02:23:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41009.log 2024-09-21T02:23:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45760.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gz 2024-09-21T02:23:00.724 INFO:teuthology.orchestra.run.smithi170.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45760.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47597.log 2024-09-21T02:23:00.724 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41009.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41009.log.gz 2024-09-21T02:23:00.726 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50770.log 2024-09-21T02:23:00.726 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43743.log 2024-09-21T02:23:00.726 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47597.log: /var/log/ceph/ceph-client.admin.50770.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47597.log.gz 2024-09-21T02:23:00.726 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41061.log 2024-09-21T02:23:00.726 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50770.log.gz 2024-09-21T02:23:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43743.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43743.log.gz 2024-09-21T02:23:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34598.log 2024-09-21T02:23:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29140.log 2024-09-21T02:23:00.728 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41061.log.gz 2024-09-21T02:23:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34598.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34598.log.gz 2024-09-21T02:23:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36657.log 2024-09-21T02:23:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36579.log 2024-09-21T02:23:00.730 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29140.log: /var/log/ceph/ceph-client.admin.36657.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29140.log.gz 2024-09-21T02:23:00.730 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50558.log 2024-09-21T02:23:00.730 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36657.log.gz 2024-09-21T02:23:00.730 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36579.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36579.log.gz 2024-09-21T02:23:00.732 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35872.log 2024-09-21T02:23:00.732 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40443.log 2024-09-21T02:23:00.732 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50558.log: /var/log/ceph/ceph-client.admin.35872.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50558.log.gz 2024-09-21T02:23:00.733 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40931.log 2024-09-21T02:23:00.733 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35872.log.gz 2024-09-21T02:23:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40443.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40443.log.gz 2024-09-21T02:23:00.733 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-09-21T02:23:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38071.log 2024-09-21T02:23:00.734 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40931.log: /var/log/ceph/ceph-client.admin.42628.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40931.log.gz 2024-09-21T02:23:00.735 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47850.log 2024-09-21T02:23:00.735 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-09-21T02:23:00.735 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38071.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38071.log.gz 2024-09-21T02:23:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51936.log 2024-09-21T02:23:00.736 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49596.log 2024-09-21T02:23:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47850.log: /var/log/ceph/ceph-client.admin.51936.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47850.log.gz 2024-09-21T02:23:00.737 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51936.log.gz 2024-09-21T02:23:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48895.log 2024-09-21T02:23:00.737 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53903.log 2024-09-21T02:23:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49596.log: /var/log/ceph/ceph-client.admin.48895.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49596.log.gz 2024-09-21T02:23:00.739 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48895.log.gz 2024-09-21T02:23:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-09-21T02:23:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46812.log 2024-09-21T02:23:00.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53903.log: /var/log/ceph/ceph-client.admin.41732.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53903.log.gz 2024-09-21T02:23:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37993.log 2024-09-21T02:23:00.740 INFO:teuthology.orchestra.run.smithi170.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-09-21T02:23:00.740 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46812.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46812.log.gz 2024-09-21T02:23:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43481.log 2024-09-21T02:23:00.741 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52931.log 2024-09-21T02:23:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37993.log: /var/log/ceph/ceph-client.0.43481.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37993.log.gz 2024-09-21T02:23:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44272.log 2024-09-21T02:23:00.742 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52931.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.43481.log.gz 2024-09-21T02:23:00.742 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52931.log.gz 2024-09-21T02:23:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28854.log 2024-09-21T02:23:00.743 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44272.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44272.log.gz 2024-09-21T02:23:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45297.log 2024-09-21T02:23:00.744 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49940.log 2024-09-21T02:23:00.745 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28854.log: /var/log/ceph/ceph-client.admin.45297.log: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.28854.log.gz 2024-09-21T02:23:00.745 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45297.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51357.log 2024-09-21T02:23:00.745 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49940.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49940.log.gz 2024-09-21T02:23:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54932.log 2024-09-21T02:23:00.746 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37856.log 2024-09-21T02:23:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51357.log: /var/log/ceph/ceph-client.admin.54932.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51357.log.gz 2024-09-21T02:23:00.748 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54932.log.gz 2024-09-21T02:23:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-09-21T02:23:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2024-09-21T02:23:00.749 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37856.log: /var/log/ceph/ceph-client.admin.48553.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37856.log.gz 2024-09-21T02:23:00.749 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52258.log 2024-09-21T02:23:00.749 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-09-21T02:23:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47489.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2024-09-21T02:23:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37791.log 2024-09-21T02:23:00.750 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48501.log 2024-09-21T02:23:00.751 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.52258.log: /var/log/ceph/ceph-client.admin.37791.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37791.log.gz 2024-09-21T02:23:00.759 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52983.log 2024-09-21T02:23:00.760 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48501.log: 92.6% -- replaced with /var/log/ceph/ceph-client.0.52258.log.gz 2024-09-21T02:23:00.760 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48501.log.gz 2024-09-21T02:23:00.762 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51675.log 2024-09-21T02:23:00.762 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53498.log 2024-09-21T02:23:00.762 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%/var/log/ceph/ceph-client.admin.51675.log: -- replaced with /var/log/ceph/ceph-client.admin.52983.log.gz 2024-09-21T02:23:00.762 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51675.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-09-21T02:23:00.762 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53498.log.gz 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54984.log 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3%/var/log/ceph/ceph-client.admin.45812.log: -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47148.log 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43245.log 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54984.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54984.log.gz 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47148.log.gz 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42416.log 2024-09-21T02:23:00.766 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6%/var/log/ceph/ceph-client.admin.41944.log: -- replaced with /var/log/ceph/ceph-client.admin.43245.log.gz 2024-09-21T02:23:00.767 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 2024-09-21T02:23:00.767 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40879.log 2024-09-21T02:23:00.768 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42416.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42416.log.gz 2024-09-21T02:23:00.768 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47437.log 2024-09-21T02:23:00.768 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38227.log 2024-09-21T02:23:00.768 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40879.log: /var/log/ceph/ceph-client.admin.47437.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40879.log.gz 2024-09-21T02:23:00.769 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34401.log 2024-09-21T02:23:00.769 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2024-09-21T02:23:00.769 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51175.log 2024-09-21T02:23:00.769 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34401.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38227.log.gz 2024-09-21T02:23:00.770 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50947.log 2024-09-21T02:23:00.770 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34401.log.gz 2024-09-21T02:23:00.770 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51175.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51175.log.gz 2024-09-21T02:23:00.771 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38546.log 2024-09-21T02:23:00.771 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-09-21T02:23:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50947.log: /var/log/ceph/ceph-client.admin.38546.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50947.log.gz 2024-09-21T02:23:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54023.log 2024-09-21T02:23:00.772 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38546.log.gz 2024-09-21T02:23:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43166.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-09-21T02:23:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38948.log 2024-09-21T02:23:00.773 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47098.log 2024-09-21T02:23:00.774 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54023.log: /var/log/ceph/ceph-client.admin.38948.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54023.log.gz 2024-09-21T02:23:00.774 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35950.log 2024-09-21T02:23:00.774 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38948.log.gz 2024-09-21T02:23:00.775 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47098.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47098.log.gz 2024-09-21T02:23:00.775 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-09-21T02:23:00.776 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35950.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T02:23:00.776 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35950.log.gz 2024-09-21T02:23:00.776 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49099.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-09-21T02:23:00.776 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T02:23:00.777 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39192.log 2024-09-21T02:23:00.780 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.c.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T02:23:00.781 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42810.log 2024-09-21T02:23:00.782 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39192.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39192.log.gz 2024-09-21T02:23:00.782 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29132.log 2024-09-21T02:23:00.783 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42810.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42810.log.gz 2024-09-21T02:23:00.783 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-09-21T02:23:00.784 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29132.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.29132.log.gz 2024-09-21T02:23:00.784 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49411.log 2024-09-21T02:23:00.785 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45412.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-09-21T02:23:00.786 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42139.log 2024-09-21T02:23:00.786 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49411.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49411.log.gz 2024-09-21T02:23:00.787 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29104.log 2024-09-21T02:23:00.787 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42139.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42139.log.gz 2024-09-21T02:23:00.788 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29126.log 2024-09-21T02:23:00.789 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29104.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29104.log.gz 2024-09-21T02:23:00.789 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2024-09-21T02:23:00.790 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29126.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29126.log.gz 2024-09-21T02:23:00.790 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42024.log 2024-09-21T02:23:00.791 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45468.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2024-09-21T02:23:00.792 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T02:23:00.792 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42024.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42024.log.gz 2024-09-21T02:23:00.793 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50660.log 2024-09-21T02:23:00.807 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29145.log 2024-09-21T02:23:00.808 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50660.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50660.log.gz 2024-09-21T02:23:00.812 INFO:teuthology.orchestra.run.smithi191.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T02:23:00.823 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54087.log 2024-09-21T02:23:00.824 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29145.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29145.log.gz 2024-09-21T02:23:00.831 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2024-09-21T02:23:00.831 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54087.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54087.log.gz 2024-09-21T02:23:00.839 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34062.log 2024-09-21T02:23:00.839 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50070.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2024-09-21T02:23:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38894.log 2024-09-21T02:23:00.847 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34062.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34062.log.gz 2024-09-21T02:23:00.855 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44401.log 2024-09-21T02:23:00.855 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38894.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38894.log.gz 2024-09-21T02:23:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29086.log 2024-09-21T02:23:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44401.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44401.log.gz 2024-09-21T02:23:00.864 INFO:teuthology.orchestra.run.smithi191.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T02:23:00.871 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47463.log 2024-09-21T02:23:00.871 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29086.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29086.log.gz 2024-09-21T02:23:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34390.log 2024-09-21T02:23:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47463.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47463.log.gz 2024-09-21T02:23:00.887 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37967.log 2024-09-21T02:23:00.887 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34390.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34390.log.gz 2024-09-21T02:23:00.895 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39133.log 2024-09-21T02:23:00.895 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37967.log.gz 2024-09-21T02:23:00.903 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48025.log 2024-09-21T02:23:00.903 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39133.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39133.log.gz 2024-09-21T02:23:00.911 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43941.log 2024-09-21T02:23:00.911 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48025.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48025.log.gz 2024-09-21T02:23:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-09-21T02:23:00.919 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43941.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43941.log.gz 2024-09-21T02:23:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49706.log 2024-09-21T02:23:00.927 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42442.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-09-21T02:23:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41524.log 2024-09-21T02:23:00.935 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49706.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49706.log.gz 2024-09-21T02:23:00.943 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44453.log 2024-09-21T02:23:00.943 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41524.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41524.log.gz 2024-09-21T02:23:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-21T02:23:00.951 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44453.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44453.log.gz 2024-09-21T02:23:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42966.log 2024-09-21T02:23:00.959 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35768.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-21T02:23:00.960 INFO:teuthology.orchestra.run.smithi170.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T02:23:00.961 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54620.log 2024-09-21T02:23:00.961 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42966.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42966.log.gz 2024-09-21T02:23:00.962 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41446.log 2024-09-21T02:23:00.962 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54620.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54620.log.gz 2024-09-21T02:23:00.962 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33980.log 2024-09-21T02:23:00.963 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41446.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41446.log.gz 2024-09-21T02:23:00.963 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49149.log 2024-09-21T02:23:00.964 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33980.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33980.log.gz 2024-09-21T02:23:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49914.log 2024-09-21T02:23:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49149.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49149.log.gz 2024-09-21T02:23:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53121.log 2024-09-21T02:23:00.966 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49914.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49914.log.gz 2024-09-21T02:23:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34114.log 2024-09-21T02:23:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53121.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53121.log.gz 2024-09-21T02:23:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38790.log 2024-09-21T02:23:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34114.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34114.log.gz 2024-09-21T02:23:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52652.log 2024-09-21T02:23:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38790.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38790.log.gz 2024-09-21T02:23:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39505.log 2024-09-21T02:23:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52652.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52652.log.gz 2024-09-21T02:23:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47714.log 2024-09-21T02:23:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39505.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39505.log.gz 2024-09-21T02:23:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52756.log 2024-09-21T02:23:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47714.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47714.log.gz 2024-09-21T02:23:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28891.log 2024-09-21T02:23:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52756.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52756.log.gz 2024-09-21T02:23:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33809.log 2024-09-21T02:23:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28891.log: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.28891.log.gz 2024-09-21T02:23:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37759.log 2024-09-21T02:23:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33809.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33809.log.gz 2024-09-21T02:23:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44511.log 2024-09-21T02:23:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37759.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37759.log.gz 2024-09-21T02:23:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-09-21T02:23:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44511.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44511.log.gz 2024-09-21T02:23:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-09-21T02:23:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34586.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-09-21T02:23:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39270.log 2024-09-21T02:23:00.979 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37200.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-09-21T02:23:00.980 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-09-21T02:23:00.980 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39270.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39270.log.gz 2024-09-21T02:23:00.981 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47999.log 2024-09-21T02:23:00.981 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42557.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-09-21T02:23:00.982 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34816.log 2024-09-21T02:23:00.983 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47999.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47999.log.gz 2024-09-21T02:23:00.983 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54341.log 2024-09-21T02:23:00.984 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34816.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34816.log.gz 2024-09-21T02:23:00.984 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50227.log 2024-09-21T02:23:00.985 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54341.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54341.log.gz 2024-09-21T02:23:00.985 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46864.log 2024-09-21T02:23:00.986 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50227.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50227.log.gz 2024-09-21T02:23:00.986 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52905.log 2024-09-21T02:23:00.986 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46864.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46864.log.gz 2024-09-21T02:23:00.987 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37053.log 2024-09-21T02:23:00.987 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52905.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52905.log.gz 2024-09-21T02:23:00.988 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-09-21T02:23:00.989 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37053.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37053.log.gz 2024-09-21T02:23:00.989 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48243.log 2024-09-21T02:23:00.990 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44305.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz 2024-09-21T02:23:00.990 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43219.log 2024-09-21T02:23:00.991 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48243.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48243.log.gz 2024-09-21T02:23:00.992 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39874.log 2024-09-21T02:23:00.992 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43219.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43219.log.gz 2024-09-21T02:23:00.993 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51117.log 2024-09-21T02:23:00.993 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39874.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39874.log.gz 2024-09-21T02:23:00.994 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-09-21T02:23:00.995 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51117.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51117.log.gz 2024-09-21T02:23:00.995 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54140.log 2024-09-21T02:23:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51402.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-09-21T02:23:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54218.log 2024-09-21T02:23:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54140.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54140.log.gz 2024-09-21T02:23:00.997 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2024-09-21T02:23:00.998 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54218.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54218.log.gz 2024-09-21T02:23:00.998 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44721.log 2024-09-21T02:23:00.999 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46994.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz 2024-09-21T02:23:00.999 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34400.log 2024-09-21T02:23:01.000 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44721.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44721.log.gz 2024-09-21T02:23:01.001 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44589.log 2024-09-21T02:23:01.001 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34400.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34400.log.gz 2024-09-21T02:23:01.002 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35898.log 2024-09-21T02:23:01.003 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44589.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44589.log.gz 2024-09-21T02:23:01.003 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41035.log 2024-09-21T02:23:01.004 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35898.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35898.log.gz 2024-09-21T02:23:01.004 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46756.log 2024-09-21T02:23:01.005 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41035.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41035.log.gz 2024-09-21T02:23:01.005 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49732.log 2024-09-21T02:23:01.006 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46756.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46756.log.gz 2024-09-21T02:23:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53840.log 2024-09-21T02:23:01.007 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49732.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49732.log.gz 2024-09-21T02:23:01.008 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52594.log 2024-09-21T02:23:01.009 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53840.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53840.log.gz 2024-09-21T02:23:01.009 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46518.log 2024-09-21T02:23:01.010 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52594.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52594.log.gz 2024-09-21T02:23:01.010 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33919.log 2024-09-21T02:23:01.011 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46518.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46518.log.gz 2024-09-21T02:23:01.011 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46916.log 2024-09-21T02:23:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33919.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33919.log.gz 2024-09-21T02:23:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36353.log 2024-09-21T02:23:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46916.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46916.log.gz 2024-09-21T02:23:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51253.log 2024-09-21T02:23:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36353.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36353.log.gz 2024-09-21T02:23:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42784.log 2024-09-21T02:23:01.016 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51253.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51253.log.gz 2024-09-21T02:23:01.016 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40417.log 2024-09-21T02:23:01.017 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42784.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42784.log.gz 2024-09-21T02:23:01.017 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-09-21T02:23:01.018 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40417.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40417.log.gz 2024-09-21T02:23:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36735.log 2024-09-21T02:23:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48105.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-09-21T02:23:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-09-21T02:23:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36735.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36735.log.gz 2024-09-21T02:23:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-09-21T02:23:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50104.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-09-21T02:23:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-09-21T02:23:01.023 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42505.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-09-21T02:23:01.023 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35846.log 2024-09-21T02:23:01.024 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43810.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-09-21T02:23:01.024 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39822.log 2024-09-21T02:23:01.025 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35846.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35846.log.gz 2024-09-21T02:23:01.025 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47772.log 2024-09-21T02:23:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39822.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39822.log.gz 2024-09-21T02:23:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49461.log 2024-09-21T02:23:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47772.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47772.log.gz 2024-09-21T02:23:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50427.log 2024-09-21T02:23:01.029 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49461.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49461.log.gz 2024-09-21T02:23:01.029 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42992.log 2024-09-21T02:23:01.030 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50427.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50427.log.gz 2024-09-21T02:23:01.030 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50716.log 2024-09-21T02:23:01.031 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42992.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42992.log.gz 2024-09-21T02:23:01.031 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34398.log 2024-09-21T02:23:01.036 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.50716.log: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T02:23:01.037 INFO:teuthology.orchestra.run.smithi170.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.50716.log.gz 2024-09-21T02:23:01.037 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43075.log 2024-09-21T02:23:01.038 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34594.log 2024-09-21T02:23:01.039 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34398.log.gz/var/log/ceph/ceph-client.admin.43075.log: 2024-09-21T02:23:01.039 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43075.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-09-21T02:23:01.039 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:01.041 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34594.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2024-09-21T02:23:01.041 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34594.log.gz 2024-09-21T02:23:01.041 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54646.log 2024-09-21T02:23:01.041 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48475.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-09-21T02:23:01.041 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2024-09-21T02:23:01.041 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33954.log 2024-09-21T02:23:01.042 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.admin.54646.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-09-21T02:23:01.042 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54646.log.gz 2024-09-21T02:23:01.042 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33954.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33954.log.gz 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51480.log 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33710.log 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz/var/log/ceph/ceph-client.admin.51480.log: 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51480.log.gz 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53218.log 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36789.log 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33710.log.gz/var/log/ceph/ceph-client.admin.53218.log: 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53218.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-09-21T02:23:01.045 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:01.046 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37027.log 2024-09-21T02:23:01.046 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36789.log.gz 2024-09-21T02:23:01.052 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37027.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41193.log 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37027.log.gz 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34006.log 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41193.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41193.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54419.log 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54419.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34006.log.gz 2024-09-21T02:23:01.053 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54419.log.gz 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33891.log 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33835.log: /var/log/ceph/ceph-client.admin.33891.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38598.log 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33891.log.gz 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36683.log 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38598.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38598.log.gz 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39583.log 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43563.log 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36683.log: /var/log/ceph/ceph-client.admin.39583.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36683.log.gz 2024-09-21T02:23:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:gzip 87.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-09-21T02:23:01.055 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39583.log.gz 2024-09-21T02:23:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47876.log 2024-09-21T02:23:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41628.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43563.log.gz 2024-09-21T02:23:01.063 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-09-21T02:23:01.063 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34166.log 2024-09-21T02:23:01.063 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52879.log 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.47876.log: /var/log/ceph/ceph-client.admin.34166.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47876.log.gz 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44959.log 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34166.log.gz 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28947.log 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52879.log.gz 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44959.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44959.log.gz 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43047.log 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.28947.log.gz 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36047.log: gzip 87.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz -- 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr: /var/log/ceph/ceph-client.admin.52704.log 2024-09-21T02:23:01.064 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43047.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43047.log.gz 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42289.log 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52704.log.gz/var/log/ceph/ceph-client.admin.42289.log: 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42289.log.gz 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38738.log 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37233.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39348.log 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51227.log 2024-09-21T02:23:01.065 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38738.log.gz 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39348.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39348.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53788.log 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51227.log.gz 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41498.log 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz/var/log/ceph/ceph-client.admin.53788.log: 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53788.log.gz 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49299.log 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33861.log 2024-09-21T02:23:01.066 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41498.log: /var/log/ceph/ceph-client.admin.49299.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41498.log.gz 2024-09-21T02:23:01.078 INFO:teuthology.orchestra.run.smithi170.stderr: 84.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50201.log 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49299.log.gz 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.50201.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33861.log.gz 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50201.log.gz 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39713.log 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36379.log 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49654.log: /var/log/ceph/ceph-client.admin.39713.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29463.log 2024-09-21T02:23:01.079 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48053.log 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36379.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36379.log.gz/var/log/ceph/ceph-client.admin.29463.log: 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29463.log.gz 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54490.log 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53866.log 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48053.log.gz/var/log/ceph/ceph-client.admin.54490.log: 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54490.log.gz 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36002.log 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28935.log 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53866.log.gz 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36002.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36301.log 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36002.log.gz 2024-09-21T02:23:01.080 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.28935.log: 80.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46037.log 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28935.log.gz 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39794.log 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36301.log.gz 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45195.log 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46037.log.gz 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52312.log 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.39794.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39794.log.gz 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45195.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45195.log.gz 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz/var/log/ceph/ceph-client.admin.46636.log: 2024-09-21T02:23:01.081 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54958.log 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43863.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43779.log 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54958.log.gz 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43779.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43779.log.gz 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29439.log 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.49074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.29439.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29439.log.gz 2024-09-21T02:23:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34842.log 2024-09-21T02:23:01.083 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34322.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-09-21T02:23:01.083 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53472.log 2024-09-21T02:23:01.083 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48999.log 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34842.log.gz/var/log/ceph/ceph-client.admin.53472.log: 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53472.log.gz 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54296.log 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.48999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46838.log 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.48999.log.gz 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54296.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54296.log.gz -5 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46838.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46838.log.gz 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35626.log 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.46245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53061.log 2024-09-21T02:23:01.094 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz/var/log/ceph/ceph-client.admin.35626.log: 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35626.log.gz 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51532.log 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.53061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37001.log 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53061.log.gz 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.51532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35657.log 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37001.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51532.log.gz 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34088.log 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37001.log.gz 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34088.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35657.log.gz 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33743.log 2024-09-21T02:23:01.095 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34088.log.gz 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34920.log 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33743.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33743.log.gz 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48157.log 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41706.log 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34920.log: /var/log/ceph/ceph-client.admin.48157.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34920.log.gz 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44615.log 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48157.log.gz 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45498.log 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.44615.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41706.log.gz 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44615.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54166.log 2024-09-21T02:23:01.096 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.45498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43297.log 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.54166.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45498.log.gz 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54166.log.gz 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54724.log 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40327.log 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43297.log: /var/log/ceph/ceph-client.admin.54724.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43297.log.gz 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54724.log.gz 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.40327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34396.log 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41415.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40327.log.gz 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41245.log 2024-09-21T02:23:01.097 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-21T02:23:01.098 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38360.log 2024-09-21T02:23:01.098 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.41245.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34396.log.gz 2024-09-21T02:23:01.101 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41245.log.gz 2024-09-21T02:23:01.101 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44168.log 2024-09-21T02:23:01.101 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43323.log 2024-09-21T02:23:01.101 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.38360.log: /var/log/ceph/ceph-client.admin.44168.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44168.log.gz 2024-09-21T02:23:01.101 INFO:teuthology.orchestra.run.smithi170.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.38360.log.gz 2024-09-21T02:23:01.101 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36249.log 2024-09-21T02:23:01.102 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.43323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36105.log 2024-09-21T02:23:01.102 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43323.log.gz 2024-09-21T02:23:01.102 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36249.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-09-21T02:23:01.102 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36249.log.gz 2024-09-21T02:23:01.102 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36105.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36105.log.gz 2024-09-21T02:23:01.102 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.42836.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-09-21T02:23:01.123 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T02:23:01.156 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T02:23:01.239 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T02:23:01.297 INFO:teuthology.orchestra.run.smithi191.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T02:23:01.308 INFO:teuthology.orchestra.run.smithi170.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T02:23:01.335 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T02:23:01.336 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-09-21T02:23:01.336 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m0.901s 2024-09-21T02:23:01.336 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m4.709s 2024-09-21T02:23:01.337 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.193s 2024-09-21T02:23:01.402 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T02:23:01.413 INFO:teuthology.orchestra.run.smithi170.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T02:23:01.423 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T02:23:01.500 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T02:23:01.504 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T02:23:01.506 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-09-21T02:23:01.506 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m1.084s 2024-09-21T02:23:01.507 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m6.267s 2024-09-21T02:23:01.507 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m0.407s 2024-09-21T02:23:01.507 INFO:tasks.ceph:Archiving logs... 2024-09-21T02:23:01.507 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/remote/smithi170/log 2024-09-21T02:23:01.508 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:23:02.600 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/remote/smithi191/log 2024-09-21T02:23:02.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T02:23:03.076 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T02:23:03.085 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T02:23:03.085 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:23:03.088 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T02:23:03.155 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T02:23:03.164 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T02:23:03.164 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:23:03.167 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:23:03.175 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:23:03.175 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-09-21T02:23:03.176 INFO:teuthology.orchestra.run.smithi170.stdout:*hv01.front.sepi 67.205.162.81 3 u 12 64 377 0.065 -2.944 1.586 2024-09-21T02:23:03.176 INFO:teuthology.orchestra.run.smithi170.stdout:+hv02.front.sepi 63.231.80.2 3 u 7 64 377 0.095 +0.463 2.373 2024-09-21T02:23:03.176 INFO:teuthology.orchestra.run.smithi170.stdout:+hv03.front.sepi 74.6.168.72 3 u 11 64 377 0.058 -3.666 1.482 2024-09-21T02:23:03.176 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-21T02:23:03.179 INFO:teuthology.orchestra.run.smithi191.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T02:23:03.179 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================== 2024-09-21T02:23:03.179 INFO:teuthology.orchestra.run.smithi191.stdout:+hv01.front.sepi 67.205.162.81 3 u 10 64 377 0.088 -2.594 1.567 2024-09-21T02:23:03.180 INFO:teuthology.orchestra.run.smithi191.stdout:+hv02.front.sepi 63.231.80.2 3 u 5 64 377 0.085 +0.154 1.458 2024-09-21T02:23:03.180 INFO:teuthology.orchestra.run.smithi191.stdout:*hv03.front.sepi 74.6.168.72 3 u 11 64 377 0.098 -1.286 1.397 2024-09-21T02:23:03.180 INFO:teuthology.orchestra.run.smithi191.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-21T02:23:03.181 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T02:23:03.190 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T02:23:03.190 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T02:23:03.198 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T02:23:03.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T02:23:03.215 INFO:teuthology.task.internal:Duration was 1256.615139 seconds 2024-09-21T02:23:03.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T02:23:03.224 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T02:23:03.224 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:23:03.226 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T02:23:03.264 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T02:23:03.264 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T02:23:03.264 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:23:03.312 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-09-21T02:23:03.312 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T02:23:03.322 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T02:23:03.322 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:23:03.356 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T02:23:03.397 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T02:23:03.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:23:03.428 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T02:23:03.510 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T02:23:03.520 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T02:23:03.520 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:23:03.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T02:23:03.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T02:23:03.553 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:23:03.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:23:03.583 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-09-21T02:23:03.594 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-09-21T02:23:03.607 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:23:03.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:23:03.643 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T02:23:03.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:23:03.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T02:23:03.666 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T02:23:03.666 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/remote/smithi170 2024-09-21T02:23:03.667 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:23:03.708 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913409/remote/smithi191 2024-09-21T02:23:03.708 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T02:23:03.729 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T02:23:03.729 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:23:03.748 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T02:23:03.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T02:23:03.783 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T02:23:03.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T02:23:03.858 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T02:23:03.858 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:23:03.861 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T02:23:03.865 INFO:teuthology.orchestra.run.smithi170.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:23 /home/ubuntu/cephtest 2024-09-21T02:23:03.867 INFO:teuthology.orchestra.run.smithi191.stdout: 658615 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 02:23 /home/ubuntu/cephtest 2024-09-21T02:23:03.868 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T02:23:03.878 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T02:23:03.900 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1256.615139245987 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T02:23:03.900 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:23:05.146 INFO:teuthology.run:pass